Recognizing the need for such a system, our team has chosen to undertake the project "Design and Fabrication of a Visual Inspection Workstation for Manual Arc Welding Applied in Welding
INTRODUCTION
Thesis background
In manual metal arc welding (MMAW), the quality of welds significantly impacts the structural integrity and durability of products Achieving high-quality welds demands skill, experience, and careful attention to detail Traditional welding training often depends on subjective evaluations by instructors, leading to inconsistent assessments and challenging learning experiences for students This subjectivity can impede effective feedback, making it difficult for students to recognize and rectify their mistakes.
The advancement of manufacturing technologies and the increasing complexity of welding tasks have led to a heightened demand for precise and reliable weld quality assessment methods Automation and digitalization in welding processes can significantly improve training methodologies, enhancing their efficiency and objectivity.
Despite the potential advantages of advanced technologies, there is a significant gap in specialized systems for welding training that effectively leverage these tools Most current solutions focus on industrial quality control rather than educational needs This highlights the urgent requirement for a dedicated system that can capture and analyze weld bead images from manual arc welding, assess weld quality, and deliver constructive feedback to students in a training setting.
Figure 1.1 Metal welding class at vocational college
The combination of image processing and machine learning technologies presents an effective solution for assessing weld quality in manual arc welding By utilizing advanced algorithms to analyze images of weld beads, this method achieves high accuracy in classification It ensures consistent and objective evaluations while offering immediate feedback, which significantly enhances students' learning and skill development in welding.
To address the challenges and opportunities in welding education, we developed an advanced workstation that combines weld image capture, processing, quality classification, and defect detection This innovative system enhances training for students in manual metal arc welding by incorporating modern technologies that offer recommendations for improving weld quality based on identified defects.
Related works
The integration of artificial intelligence (AI) in welding is well-documented, with more than 5,400 articles available on SpringerLink Despite this extensive research, there remains a significant lack of relevant literature specifically addressing the focus of our thesis.
Moinuddin et al (2021) conducted a study on weld defect classification in Gas Metal Arc Welding (GMAW) using machine learning techniques, focusing on tube-to-tube butt joints and varying process parameters to achieve quality welds and identify three types of defects While this research aligns with our project in weld quality classification and defect detection through machine learning, it diverges in key areas, including its focus on tube welding rather than butt welding, the use of electrical signals instead of images as input data, and the absence of image processing and recommendation features that are integral to our proposed system.
Ho et al (2023) created an Artificial Neural Network (ANN) model designed to predict welding defects in Metal Inert Gas (MIG) welding This innovative two-stage ANN utilizes welding current, travel speed, and gas flow as input parameters to accurately forecast four specific types of weld defects Remarkably, the model demonstrated an 85% accuracy rate during testing, contributing significantly to the field of weld defect classification through machine learning techniques.
3 learning and focusing on MIG, it differs from our project by using process parameters as inputs rather than weld images
Ajmi et al (2020) utilized a VGG-based deep learning model to classify three weld defect classes in X-ray images, achieving an impressive 97% accuracy through data augmentation and channel substitution In contrast, our project aims to classify specific defects in Manual Metal Arc Welding (MMAW) by analyzing raw weld surface images and offering tailored recommendations.
Gantala and Balasubramaniam (2021) focus on automating weld defect recognition using AI, specifically targeting butt welds and two defect types: porosity and slag In contrast, our project addresses manual metal arc welding and encompasses a broader spectrum of defects Additionally, our system offers improvement recommendations, a feature absent in their approach.
In summary, unlike other studies that assess weld quality in industrial environments, our project prioritizes the enhancement of weld training Our innovative system provides instant feedback and personalized guidance, greatly advancing the training experience for students and equipping them for practical applications in the field.
Scientific and practical significances
This study highlights the importance of researching and developing artificial intelligence applications in the welding industry, focusing on enhancing the quality of welding training The self-collected dataset created during this research serves as a valuable resource for future studies on AI applications in diagnosing and evaluating Manual Metal Arc Welding (MMAW) products.
The integration of AI in welding education significantly improves training quality, allowing students to acquire knowledge and skills more effectively By offering immediate and precise feedback on weld quality, AI accelerates the skill enhancement process for learners.
Objectives
We aim to address the challenge of visual inspection of MMAW samples from students To achieve this, the following objectives have been established:
• Design and construct a workstation for the precise capture of MMAW imagery with integrated real-time system feedback mechanisms
• Collect a dataset consisting of MMAW product images from students to serve the project’s objectives
• Analyze and categorize the weld quality, identifying and visualizing any defects present Subsequently, generate recommendations to address the specific defects observed in the weld.
Research methods
• Investigate the characteristics of weld defects in MMAW based on established literature
• Analyze MMAW products to determine the necessary components and features for the workstation design, aligning with the project's objectives
• Research and evaluate various deep learning (DL) models to identify the most suitable one that meets the thesis requirements for accuracy, efficiency, and adaptability to MMAW weld image analysis.
Scope of the study
Within the scope of the thesis constraints, we concentrate on accomplishing the following tasks:
• Gaining a comprehensive understanding of the landscape of quality classification and defect detection from MMAW images and investigating different methodologies
• Developing a workstation for capturing MMAW image and interacting with the software
• Developing a dataset of MMAW images categorized into four classes (A, B,
C, D) and annotating each image with its respective defects
• Testing models such as Vision Transformer for weld classification, YOLOv8 for defect detection, and Llama 3 with RAG for text (advice) generation.
Structure of the report
The report includes six chapter:
• CHAPTER 1 INTRODUCTION: Brief introduction to the study
• CHAPTER 2: THEORETICAL BASIS: The theoretical knowledge related to and applied during the implementation will be presented in this section Regarding welding theory, the main topics explored are welding parameters,
This article explores five welding techniques and identifies common defects associated with Manual Metal Arc Welding It also delves into the application of deep learning models, including classification, object detection, and Retrieval-Augmented Generation (RAG) within Large Language Models (LLM) Furthermore, it discusses the theoretical aspects of cloud servers in relation to these technologies.
• CHAPTER 3: DESIGN MECHANICAL SYSTEM: In the Design
Mechanical System section, the research analysis and selection of the approach will be presented first Following that, topics such as design, material selection and manufacturing will be discussed
• CHAPTER 4: DESIGN ELECTRONICS-CONTROL SYSTEM: Explain the block diagram in the electronics-control system and select devices for the system
• CHAPTER 5: DESIGN AI ALGORITHM: Objectives of the AI algorithm, establishing input and output criteria for AI models, build the AI models, deploy the models to cloud
• CHAPTER 6: CONCLUSION AND RECOMMENDATION: Evaluate the system's performance and the resulting outcomes.
THEORETICAL BASIS
Theoretical basis of manual metal arc welding (MMAW)
Manual metal arc welding (MMAW), commonly referred to as stick welding, utilizes a consumable electrode coated in flux to create a weld This process involves an electric current, either alternating (AC) or direct (DC), from a welding power supply to generate an electric arc between the electrode and the metals being joined As the welding occurs, the flux coating disintegrates, releasing vapors that act as a shielding gas and forming a layer of slag, which protects the weld area from atmospheric contamination.
Figure 2.1 Manual metal arc welding process
• Diameter of the welding rod (electrode) 𝒅 𝒘𝒓 affects the amount of heat input, the penetration depth, and the overall quality of the weld
Formula of welding rod diameter:
- With corner-joint or T-joint welding:
Where: S is the thickness of the welded material
K is the leg length of the weld
In practice, only welding rods with diameters from 2.0 to 5mm are used Therefore, the diameter of the welding rod can be determined according to the table below:
Table 2.1 Welding rod diameter reference
Welding current (Iw) is influenced by the thickness of the material being welded, the diameter of the welding rod, and the spatial position of the weld For flat position steel welding, Iw can be estimated using a specific formula.
Where: 𝑑 𝑤𝑟 is the diameter of the welding rod (mm)
𝛽 and 𝛼 Experimental coefficients, 𝛽 = 20, 𝛼 = 6 when welding with carbon steel rods
In production, we can calculate 𝐼 𝑤 using formula:
Increasing the welding current enhances the penetration depth of the weld; however, excessively high current can overheat the rod and compromise weld quality On the other hand, if the current is too low, the arc weakens, resulting in minimal penetration depth.
- Welding rod specified for DC welding may not be suitable for use with
• Arc length 𝒍 𝒂 as the distance between the tip of the electrode and the surface of the workpiece being welded There are different types:
A long arc length in welding can lead to increased travel distance for molten metal droplets from the electrode to the weld pool, making them more vulnerable to atmospheric contamination This extended arc also raises arc voltage, reduces penetration depth, and heightens metal loss from spatter and evaporation, ultimately resulting in an uneven weld surface and a greater risk of undercut defects.
Short arc lengths can compromise arc stability, causing frequent short circuits and reducing both arc voltage and weld bead width, which leads to an uneven weld surface In contrast, when using direct current (DC) for welding, the arc exhibits less deflection, promoting a more stable and consistent weld.
Welding speed (𝑉𝑤) refers to the rate at which the welding rod travels along the weld axis An excessively high welding speed can result in a narrow weld, reduced penetration depth, an uneven surface, and potential interruptions in the weld On the other hand, a welding speed that is too low may cause burn-through, overheating of the base metal, an enlarged heat-affected zone, and increased weld width and penetration depth.
To maintain the arc length and the width of the weld, 3 basic movements must be simultaneously performed:
To achieve a stable arc during welding, it's essential to adjust the length of the welding rod by moving it along its axis at a speed that matches the melting rate of the rod For optimal weld quality, the welding rod should be positioned at a tilt of 75-85° in the direction of the weld.
• Movement along the axis of the weld to weld the entire length of the weld This movement has a significant impact on the quality of the weld and labor productivity
• Oscillating movement to ensure the width of the weld, ensure penetration of both weld edges, and even heating to allow the weld to cool slowly
Figure 2.2 Movement of the welding rod
The manual arc welding process can result in numerous defects In this section, we will discuss some basic defects, their concepts, causes, and remedies
• Crack: A tear, fracture or fissure in the weld or base metal appearing as a broken, jagged or straight line This is the most serious defect
Crack formation can be attributed to various factors, including:
- High thermal gradients can cause contraction stresses that exceed the material's tensile strength, leading to cracks
- Using a high current or a slow travel speed can cause the weld to overheat, increasing the risk of cracking
- Materials with high carbon content are more prone to cracking due to increased hardness and brittleness
- Improper joint design or fit-up can create high stress concentrations, leading to cracks
- Remove contaminants from the joint (rust, grease, moisture, etc.) prior to welding
- Apply and maintain required preheat
- Do not allow the base material to cool too quickly
- Maintain filler metal control requirements
- Use correct filler metal type for the joint
- Apply proper bead size and sequencing to eliminate excessive distortion and/or stress in the base material
- Repair in accordance with local procedures
• Burn Through: Excessive heat and/or penetration that results in a hole completely through the backing ring or strip, fused root, or adjacent base material
Burn through can be attributed to various factors, including:
- Using a too high current can generate excessive heat, which can cause the weld pool to become too large and burn through the base metal
- Welding on metal material that is too thin for the chosen electrode and current setting can easily result in burn through because the heat is concentrated over a small area
- Using an electrode that is too large for the thickness of the metal can result in excessive heat input and burn through
- Improper welding technique, such as lingering too long in one spot, can lead to localized overheating and burn through
Preventive action for burn through:
- Maintain appropriate arc length/wire stick-out
- Use ceramic tape or approved metal backing strap on areas with root gap Corrective action for burn through:
- Place ceramic tape or approved metal backing strap on the bottom side of the hole
- Weld repair the first side of the hole from the easiest side to weld
- Once sufficient weld metal has been deposited on the easiest top side, grind or carbon arc the other side of the hole to sound metal
- Weld the other side of the hole to the appropriate size or height
- A situation where the weld metal does not fuse or completely bond with the base metal or previously deposited weld metal
• Incomplete Fusion: A situation where the weld metal does not fuse or completely bond with the base metal or previously deposited weld metal
Figure 2.5 Incomplete fusion (infusion) defect
Infusion can be attributed to various factors, including:
- Low welding current or voltage can result in insufficient heat, preventing proper melting and fusion of the base metals and filler material
Inconsistent welding angles, improper speed, and incorrect electrode manipulation can result in insufficient fusion For example, traveling too quickly can hinder the heat from properly penetrating the base material.
Using an inappropriate electrode for the base material or welding position can lead to incomplete fusion It is essential to ensure that the electrode's size and type align with the specified welding requirements.
- Incorrect settings for parameters such as arc length, polarity, and amperage can negatively affect the welding process, causing incomplete fusion
- Welding in positions that are not suited for the electrode type or that the welder is not skilled at can result in incomplete fusion
- Maintain appropriate arc length/wire stickout
- Ensure previous beads are free of overlap (bead roll-over) and slag prior to welding additional passes
- Grind or carbon arc the weld to sound metal
- Weld repair the affected area
• Roughness: Sharp ridges (irregularities) or deep valleys be- tween weld beads
The angle formed between the adjacent beads of the weld must be 90° or greater
Roughness can be attributed to various factors, including:
- Using too high or too low of a current can affect the weld bead's appearance
- Moving the electrode too fast or too slow can cause roughness
- Holding the electrode at an incorrect angle can cause irregularities
- Maintain appropriate arc length/wire stickout
- Sequence weld passes so that the toes of the beads sufficiently cover one another, minimizing valleys
- Consult local Welding Engineering in cases where the base material is magnetized
- Grind or carbon arc the weld to sound metal
- Weld repair the affected area, if needed
• Porosity: Open holes formed by gas that was trapped when the weld cooled
Porosity can be attributed to various factors, including:
- Dirt, oil, grease, moisture, and other contaminants on the surface of the metal can cause porosity These contaminants can vaporize during welding and create gas pockets
- Electrodes that are not stored properly can absorb moisture from the air When these electrodes are used, the moisture can turn into gas during welding, causing porosity
- Arc blow, caused by magnetic fields in the welding area, can lead to instability in the welding arc, causing defects like porosity
- Wind or drafts in the welding environment can disturb the shielding gas coverage, leading to porosity
- Remove contaminants from the joint (rust, grease, moisture, etc.) prior to welding
- Maintain filler metal control requirements
- Maintain appropriate arc length/wire stickout
- Use the largest size gas cup possible and keep it free of spatter
- Position wind screens between the welding operation and any heavy flow of air
- Completely remove porosity from all intermediate weld areas
- Grind or carbon arc the affected area until the unacceptable porosity is removed from the weld
- Weld repair the affected area, if needed
• Spatter: The metal particles expelled during welding that do not form a part of the weld
Spatter can be attributed to various factors, including:
- If the electrode coating contains a high level of certain elements like potassium or sodium, they can vaporize and contribute to spatter formation
- Incorrect voltage or current settings can lead to excessive spatter Higher currents and voltages can increase spatter formation
- Different electrode types and diameters can affect spatter Larger diameter electrodes generally produce more spatter due to higher deposition rates Preventive action for spatter:
- Remove contaminants from the joint (rust, grease, moisture, etc.) prior to welding
- Maintain filler metal control requirements
- Use refrasil to protect surrounding surfaces from secondary weld spatter
- Maintain appropriate arc length/wire stickout
- Use ceramic tape or approved metal backing strap on areas with root gap
- Consult local Welding Engineering in cases where the base material is magnetized
- Completely remove spatter from all intermediate weld areas
- Remove all loose spatter with a needle gun
- Grind all tightly adhering, unacceptable spatter until it blends smoothly into the base material or weld
• Slag: The nonmetallic layer that forms on top of the molten metal
Slag can be attributed to various factors, including:
The electrode utilized in Shielded Metal Arc Welding (SMAW) features a flux coating that melts during the welding process This flux generates a protective gas shield around the weld pool, safeguarding it from atmospheric contaminants, while also forming a slag layer on the surface of the weld bead.
- The flux coating contains materials called deoxidizers, such as manganese oxide and silicon dioxide These react with impurities in the weld metal and help remove them as slag
- The molten flux reacts with the metal and other impurities at high temperatures, leading to the formation of slag
- The slag layer acts as a barrier to protect the weld metal from oxidation and contamination until it solidifies
Slag is a byproduct of the welding process that cannot be prevented Below are some actions that make slag removal easier
- Maintain an appropriate arc length/wire stick- out
- Sequence weld passes so that the toes of the beads sufficiently overlap one another, minimizing valleys
- Completely remove slag from all intermediate weld areas
- Remove all loose slag with a needle gun
- Grind all tightly adhering, unacceptable slag from the surface of the base material or weld.
Artificial intelligence (AI) and related terms
2.2.1 Artificial intelligence, machine learning (ML) and deep learning (DL)
Artificial Intelligence (AI) is a groundbreaking technology that emulates human intelligence by utilizing algorithms, data analysis, and computer systems Central to AI are techniques like machine learning, which allows systems to enhance their task performance through experience, and deep learning, which replicates the neural networks of the human brain to handle complex data.
AI's applications span across multiple domains, including natural language processing that enables machines to understand human language and computer vision that allows them to interpret visual data These advancements are transforming industries like healthcare by enhancing diagnostic precision and personalizing treatment; transportation through the innovation of autonomous vehicles; and customer service with AI-powered chatbots delivering immediate assistance.
Machine learning (ML) is a subset of artificial intelligence (AI) and computer science that emphasizes the utilization of data and algorithms to allow AI systems to replicate human learning processes, progressively enhancing their accuracy over time.
Machine learning models can be categorized based on the degree of human intervention in the raw dataset, which includes methods that involve rewards, feedback, or labeled data.
• Supervised Learning: Trained on labeled data (e.g., classification and regression tasks)
• Unsupervised Learning: Trained on unlabeled data to find hidden patterns
• Semi-Supervised Learning: Uses both labeled and unlabeled data
• Reinforcement Learning: Learns by interacting with an environment and receiving feedback
Deep learning, a specialized branch of machine learning, employs multi-layered neural networks to identify intricate patterns within extensive datasets This approach excels in handling large-scale data and demands considerable computational power.
Deep learning (DL) utilizes artificial neural networks that mimic the human brain's structure and function, distinguishing it from traditional machine learning (ML), which often relies on simpler algorithms This approach requires large datasets and significant computational power, typically provided by GPUs.
AI is a comprehensive field that includes all intelligent systems, while machine learning (ML) is a specific subset that concentrates on data-driven learning and decision-making Within ML, deep learning represents a more specialized area that focuses on advanced techniques for processing and analyzing data.
18 deep neural networks to model complex patterns in large datasets Together, these fields drive the advancements in creating intelligent systems capable of performing a wide range of tasks
Figure 2.11 Comparing Artificial Intelligence related terms
2.2.2 Computer vision and Convolution Neural Network (CNN)
Computer vision, a vital subset of artificial intelligence (AI), enables computers to interpret and make decisions based on visual data from their environment By mimicking human visual processing, computer vision empowers machines to detect and analyze images with enhanced accuracy and speed.
Below are some the key components of computer vision which are involved in this project:
Image acquisition involves gathering visual data using various devices, including cameras, sensors, and scanners The quality and resolution of the captured images are essential for effective subsequent processing.
• Preprocessing: Before analysis, images often undergo preprocessing to enhance their quality
Feature extraction is a crucial process in image analysis that involves identifying key characteristics, such as edges, corners, and textures By emphasizing essential information and eliminating irrelevant details, feature extraction effectively simplifies images, making it easier to analyze and interpret visual data.
Computer vision systems utilize algorithms and machine learning models for object detection and recognition in images These systems can perform tasks from basic shape identification to complex applications such as face recognition and tumor detection in medical images.
A neural network is a computational model inspired by the human brain, consisting of interconnected nodes arranged in layers It is highly effective at learning patterns from data during training, which enables it to perform tasks such as image recognition with remarkable accuracy.
Our project employs a convolutional neural network (CNN) to analyze welding images, enabling the detection of defects and improving the accuracy of our AI system in identifying butt welding issues.
Each neural network has 3 components:
The input layer of a neural network is the initial stage where the network receives data Each node in this layer signifies a specific feature or attribute of the input data, such as representing individual pixels in image analysis.
The output layer of a neural network generates the final predictions based on the processed data The number of nodes in this layer varies according to the specific task at hand; for binary classification, a single node typically represents the output class, whereas in multi-class classification, multiple nodes are utilized, each representing a distinct class.
Data transmission and processing via cloud server
The client-server model is a framework that enables computers to communicate and exchange data with each other.
In a client-server model, clients do not share resources with other computers; rather, they request and utilize resources offered by the server Notably, a client can also operate as a server in different contexts, depending on the user's needs.
A server is a powerful computer designed to deliver resources and services to clients within a network, significantly improving the efficiency of operations on client machines.
The client-server model defines how software components communicate, with the client making requests and the server providing data or services in response Application Programming Interfaces (APIs) act as the standardized language facilitating this interaction.
A REST API, also known as a RESTful API, adheres to the principles of the representational state transfer architectural style, offering a flexible and lightweight solution for integrating applications and connecting components within microservices architectures Unlike traditional methods that rely on URLs for processing information, REST utilizes HTTP methods such as GET, PUT, POST, and DELETE to manage data effectively This technology is often favored over alternatives due to its lower bandwidth usage, resulting in greater efficiency for internet applications.
Details of the HTTP method is:
• PUT to change or update the state of a Resource A Resource can be an object, a file, an image, or a block
REST APIs are usually created and tested locally, but to make them accessible over the internet, they must be deployed on servers with public IP addresses or domain names Although on-premises servers can host APIs, cloud platforms such as Google Cloud, AWS, and Azure are preferred for their scalability and reliability.
Google Cloud is a robust cloud computing platform that utilizes the same infrastructure as Google's consumer products, offering a wide range of services including compute, storage, networking, big data analytics, machine learning, and IoT capabilities It also provides essential cloud management, security, and developer tools, enabling businesses to scale and adapt effectively while maintaining high performance, reliability, and security One of its key services, Cloud Run, allows users to deploy servers directly on Google's scalable infrastructure.
To deploy a server on Cloud Run, it must be packaged as a container image, which is a portable software package that includes all essential components for service execution, such as build artifacts, assets, and system packages, along with an optional runtime environment This portability allows containerized applications to function reliably across various compatible container runtime platforms.
Cloud Run simplifies the deployment of server applications by automatically managing container serving in response to HTTP requests, including load balancing and scaling based on demand It provides a public URL for easy internet access to services, ensuring high availability and scalability typical of cloud hosting, all without the need for direct infrastructure management.
DESIGN MECHANICAL SYSTEM
Hardware objectives
• Compact and convenient model: The model should have an appropriate size for easy movement and installation in the workspace without taking up too much space
• Camera mounting location: There must be a specific location to mount the camera, allowing for easy adjustment of the viewing angle and ensuring accurate observation of the welds
• Weld fixing location for observation: There must be a place to securely fix the weld, ensuring that the camera can clearly and accurately observe any defects on the weld
To ensure stable and consistent lighting conditions during the camera's observation process, the design should incorporate an enclosed chamber that effectively blocks external light sources.
For optimal user experience, the screen must be large and have a high resolution to ensure that information and images are displayed clearly, facilitating easy monitoring and analysis of results.
• Convenient workbench: The workbench should be designed to allow users to work comfortably and efficiently, with tools and equipment that are easy to access and use.
Mechanical Design Process
The distance between the camera and the object is a direct line connecting both points, allowing for various camera and object mounting orientations The two primary orientations for this setup are vertical and horizontal.
The advantages and disadvantages of each approach are compared below:
Criteria Horizontal axis Vertical axis
Vision Wide, suitable for large areas Vertical limitation
Adjust the rotation angle Easy Difficult to adjust
Observation details Difficult with small details Clear, good for small details
Suitable space Wide space Narrow space
Obstruction Prone to obstruction Minimize obstruction
Vision Obstructed by objects Limitation in large space
Light Influenced by horizontal light Influenced by overhead light
The vertical camera mounting option is ideal for model design, particularly in narrow spaces, as it allows for detailed and clear observation while minimizing occlusion By utilizing mechanical solutions, the drawbacks of this mounting method can be effectively addressed, ensuring optimal observation quality and compliance with technical requirements.
In the model design objectives, an enclosed chamber is essential to eliminate complex lighting conditions However, this enclosed environment can lead to darkness, making it necessary to incorporate lighting for the camera to effectively capture clear images of the weld inside the chamber.
Currently, there are many types of lighting for cameras:
Table 3.2 Comparison of common lighting types used for cameras
Criteria LED light Infrared light (IR) Ultraviolet light (UV) Spectrum of light
Night observation, low-light conditions
Visibility Visible to the human eye
Invisible to the human eye
Invisible to the human eye
Energy consumption Low Low Low
Good detail in normal light
Safe, but avoid direct exposure to intense light
Requires protection, can cause harm to skin and eyes
White LED lights are the optimal choice for our experimental model due to their ability to produce realistic color images, adjustable lighting for various observation conditions, and energy efficiency They also boast a long lifespan and are safe for the eyes, making them a superior option compared to medium and higher-cost alternatives like IR lights.
The weld placement block is where the weld is placed and secured in the correct position for the camera to clearly observe and capture images
The components of the block include:
Three ball-end locating pins effectively restrict three degrees of freedom, specifically translation along the Z-axis and rotation around the X and Y axes By limiting these movements, the pins establish three contact points in a plane, which stabilizes the weld and ensures evenness during the welding process.
• Two locating pins on the side: Restricting two degrees of freedom, translation along the X-axis and rotation around the Z-axis
• One support pin on the top edge: Restricts one degree of freedom, translation along the Y-axis
By limiting three degrees of freedom, we establish a defined space in the weld inspection chamber This space is bounded on the left by two positioning elements and at the top by a positioning element along the weld's upper edge This design ensures that when the weld is placed into the inspection chamber, it is positioned accurately and efficiently within this designated area.
For welds with uneven surfaces, our team utilizes locating pins as the positioning mechanism to accurately mount the weld on the model Given that actual weld surfaces often feature convex or concave areas rather than being perfectly flat, purchasing standard pins is preferred However, if standard options are unavailable, we can efficiently machine custom pins, which helps reduce costs and accelerates material sourcing and model completion.
Our team evaluated two types of locating pins for positioning the bottom surface: ball-end locating pins and knurled pins We opted for ball-end support pins due to their widespread use in mechanics, availability, and cost-effectiveness.
The team utilized flat-end locating pins to effectively position the edges of the weld, ensuring precise alignment while restricting the space available for the weld within the light shield.
An effective display of content and weld evaluation results is crucial; therefore, it is essential to position the display screen in a highly visible area This strategic placement, combined with user-friendly actions for conducting weld inspections, ensures a seamless and convenient inspection process.
19-inch monitors are increasingly popular in manufacturing plants due to their optimal display size and exceptional durability Recognizing this demand, manufacturers have developed robust monitors tailored to meet the specific needs of businesses and industrial environments Furthermore, these monitors feature a versatile design, equipped with both a stand and four screws on the back, allowing for easy customization and installation in various settings.
42 a) The display screen is placed separately from the model b) The display screen is mounted directly onto the model via a mounting bracket Figure 3.4 Monitor setup options
In the design process, we have two options for screen installation: keeping it separate outside or utilizing the four screw holes on the back to mount it directly onto the workbench To enhance the integration of components, we will opt for a mounting bracket, allowing the screen to be affixed directly to the model This approach not only facilitates easier mobility but also conserves space by eliminating the need for an external screen placement.
Based on the design requirements of the workstation, we can use several materials for the model's frame
Aluminum profile Density 7.85 g/cm³ 2.70 g/cm³ 2.70 g/cm³
Tensile strength 400 - 550 MPa 70 - 700 MPa 200 - 400 MPa
Easy to machine, especially in shaping
Machine parts Aircraft, automobiles, food packaging
Framework models, medical equipment, industrial machinery
From the comparison table above, the team has chosen to use extruded aluminum (specifically the 20x20mm type) for the model's frame due to the following factors:
- Lightweight: With a low density, extruded aluminum reduces the overall weight of the frame, making it easy to transport and assemble
- High Durability: Extruded aluminum has good durability, enough to withstand force without the need for complex structures or additional reinforcement
- Corrosion Resistance: Unlike steel, extruded aluminum does not rust and has good corrosion resistance, making the frame more durable in humid or chemical environments
Extruded aluminum offers exceptional machinability, allowing for easy cutting, drilling, and assembly This efficiency significantly reduces both time and costs in the manufacturing process of frames.
- High Aesthetics: Extruded aluminum has a shiny, beautiful surface that can be finished with various coatings and paints, creating a frame with a professional and attractive appearance
- Recyclability and Environmental Friendliness: Extruded aluminum can be recycled multiple times without losing its physical properties, helping to reduce negative impacts on the environment
About the frame structure, we have designed is divided into two main areas:
- Weld inspection chamber dimensions: 250x210x400mm
Extruded aluminum comes in long bars, so we only need to use a specialized extruded aluminum cutting machine to cut it to the correct size
Extruded aluminum has many accessories for assembly In this case, we have chosen to use right-angle brackets to assemble the bars together
Figure 3.8 Using right-angle brackets and bolts to assemble the model frame
The extruded frame for the weld inspection chamber Eletrical Cabinet
External cover panels for industrial machines are essential for safeguarding internal components against dust, moisture, and environmental factors, while also prioritizing user safety.
Currently, there are many types of cover materials that we can choose for the model:
- Materials: Stainless steel and aluminum are two common choices because they are corrosion resistant and have high durability
- Advantages: High strength and durability help protect machines from strong impacts
- Disadvantages: Heavier weight and higher production costs compared to other materials
- Applications: Often used in harsh industrial environments where high durability and corrosion resistance are required
Figure 3.9 Metal covers for machines
- Materials: Plastics such as Polycarbonate, ABS, and PVC are all lightweight and easy to process
- Advantages: Plastic is lightweight, water-resistant, and easily formed into complex shapes
- Disadvantages: Plastic cannot withstand high temperatures and is easily scratched when in contact with hard objects
- Applications: Often used in light industry and electronic equipment
Figure 3.10 Plastic cover for mini fan
- Materials: Fiberglass and carbon fiber are common composite materials with good strength properties
- Advantages: Composites are lightweight and corrosion-resistant, making them ideal for applications that require high strength but low weight
- Disadvantages: High cost and difficult to repair if damaged
- Applications: Often used in aerospace and automotive industries where high mechanical performance and light weight are required
Figure 3.11 Composite mudguards for cars
Type Material Advantages Disadvantages Applications
Heavy, expensive, can rust (if not using stainless steel), not heat- resistant, easy to scratch
Lightweight, easy to process, waterproof
Not as strong as metal, can be brittle, not as heat- resistant
In a controlled factory setting characterized by stable temperatures and minimal dust, our team has opted for plastic covers for the model The benefits of plastic covers surpass those of alternative materials, as they notably lower both material and processing costs Furthermore, these covers improve the model's visual appeal while effectively fulfilling their protective role.
• Choose aluminum composite panels (Alu panels or ACP) as covers for the model:
Figure 3.12 Aluminum composite panel’s structure
• The cover for the weld inspection chamber consists of:
- One top cover panel a front cover (door) b left cover c right cover d back cover e top cover
Figure 3.13 The cover panels for the light-blocking chamber of the model
• The cover for the electrical cabinet consists of:
50 a) front cover b) back cover c) side cover d) top cover
Figure 3.14 The cover for the electrical cabinet
Mechanical system summary
Figure 3.15 Completed mechanical system design
Figure 3.16 Completed mechanical system fabrication
During the model design process, the team carefully selected materials and applied optimal design solutions to ensure feasibility, durability, and cost- effectiveness for the project
We selected a 20x20mm extruded aluminum frame for our model because of its exceptional benefits, including lightweight construction, high durability, excellent corrosion resistance, ease of machining and assembly, as well as its recyclability and eco-friendly properties.
The external cover panels of the model are crafted to safeguard internal components from dust, moisture, and environmental factors, prioritizing user safety The design team opted for aluminum composite panels (Alu panels) for their lightweight nature, ease of machining, water resistance, and robust strength properties.
The display screen is seamlessly integrated into the model using a mounting bracket, allowing for easy movement A durable 19-inch monitor, widely utilized in manufacturing plants, was selected for its flexibility and robustness.
The model design successfully fulfills key criteria such as weight, durability, corrosion resistance, and ease of machining and assembly, while also utilizing environmentally friendly materials These strategic choices enhance the model's functionality and promote long-term sustainability and economic efficiency.
DESIGN ELECTRONICS-CONTROL SYSTEM
System electrical requirements
When designing and implementing the electrical system, it is necessary to comply with the following requirements:
• Ensure safety within the system and eliminate all hazards
• The system must operate continuously and stably without interruptions during use
• The electrical designs must be compatible with the mechanical and software components of the system
• Ensure compliance with technical specifications.
Electrical System Connection Diagram
The block diagram below illustrates the connection of devices within the system The diagram specifically shows power requirements and necessary connections of the devices
Figure 4.1 Block diagram of electrical device connection
The manual metal arc welding evaluation system is designed with a primary microcontroller unit responsible for managing communication and data reception from various devices Additionally, it integrates peripherals such as cameras, LCD screens, a mouse, and LEDs to enhance functionality and user interaction.
53 main processor unit Additionally, devices like cameras and LEDs will integrate relays for easy on/off control via the microcontroller
The compact system features a user-friendly plug for power supply and includes a convenient on/off switch with an integrated signal light This signal light clearly indicates the operational status, illuminating when the system is powered on and turning off when it is off, ensuring users can easily monitor its functionality.
The system utilizes a microcontroller to manage devices like cameras and LED lights, with integrated relays enabling this control These relays respond to signals from the microcontroller's GPIO pins, allowing for efficient on/off operation of the connected devices.
Figure 4.2 Electronics-control system fabrication
The selection of devices
The Jetson Nano Developer Kit B01 has been selected as the platform for deploying the optical welding seam evaluation system, where it will manage device control during the evaluation and facilitate data transmission.
54 to a server to execute system functionalities This microcontroller will run the Linux Ubuntu operating system and function as a standalone computer within the system
The Jetson Nano, powered by NVIDIA JetPack™, includes essential software components such as a board support package (BSP), Linux OS, NVIDIA CUDA®, cuDNN, and TensorRT™ libraries These tools facilitate deep learning, computer vision, GPU computing, and multimedia processing With a readily available flashable SD card image, users can enjoy a quick and hassle-free setup experience.
With NVIDIA Jetson Nano having a robust development community, abundant learning resources, sample projects, and supportive forums, developers find it easy to start and deploy their projects
Figure 4.3 Jetson Nano Developer kit Table 4.1 Jetson Nano Specification
CPU Quad-core ARM A57 @ 1.43 GHz
Memory 4 GB 64-bit LPDDR4 25.6 GB/s
Display HDMI and Video Decoder display port
Power Options Micro-USB 5V 2A or DC Power Adapter 5V 4A
Mechanical 69 mm x 45 mm, 260-pin edge connector
To evaluate manual metal arc welding, a high-quality camera is essential for capturing images The camera should possess adjustable parameters to ensure optimal performance, prioritizing image quality to facilitate accurate assessment.
The team selected the Flir Grasshopper2 GigE camera (model GS2-GE-20S4M-C) for capturing footage during the evaluation of manual metal arc welding This camera is highly efficient, featuring high resolution, fast frames per second (FPS), a compact design, and user-friendly software that simplifies parameter adjustments.
Figure 4.4 GigE Grasshopper 2 camera Table 4.2 Camera specification
Image Sensor Sony ICX274 CCD
Max FPS at Max Resolution 29 FPS (1600x1200 at 30FPS)
The team selected the HP V193 LCD display to offer users a visual interface for viewing camera images and ensuring precise timing for image capture This LCD will also function as the output interface for displaying evaluation results and facilitating system interaction, meeting essential criteria such as image quality, screen size, and weight.
Figure 4.5 LCD Table 4.3 Screen specification
The team has decided to use the NXB-63 MCB 2P 10A C10 in the system to ensure safety during operational incidents
Table 4.4 Miniature Circuit Breaker specification
The team selected the S-120-12 AC-DC power supply for the manual metal arc welding evaluation system, ensuring it meets essential requirements like delivering adequate current to all downstream devices and maintaining suitable output voltage levels.
Figure 4.7 AC-DC Power Supply Table 4.5 AC-DC power supply specification
• Voltage Converter IC (Buck Converter):
The XL4015 DC-DC Buck Converter is designed to efficiently step down voltage from an AC-DC power supply, catering to the diverse power supply needs of various devices within the system.
Figure 4.8 Buck DC-DC Converter Table 4.6 Buck Converter specification
The relay module is used to control functions such as turning on/off cameras or lights based on logic signals from the microcontroller
Figure 4.9 Relay Module Table 4.7 Relay module specification
The LC-P800 LED has been chosen for the manual metal arc welding evaluation system to provide optimal lighting for capturing welding images, in accordance with the criteria specified in Section 3.
Figure 4.10 LED Table 4.8 LED specification
A two-position toggle switch ZB2-BE101 ED25
The AD16-22DS indicator light, green, 22mm, 220V
DESIGN AI SYSTEM
Criteria for evaluating weld quality
Drawing from the theoretical foundations outlined in Chapter 2 and the analysis of data gathered from student exercises, we, alongside a welding expert, have classified MMAW products into four primary categories.
A Perfect weld bead with small or no surface defects
B The weld bead has been disrupted by appearing of several surface defects than class A
C Weld bead created but the welding line has been disrupted continuously by appearing of more surface defects than class B
D Not creating a weld or burn through
• A class: The weld bead is straight and continuous, with consistent shape and dimensions, and also has a defect-free surface
• B Class: The weld bead is straight and continuous, and the weld surface has a few minor defects
Figure 5.2 B class weld with some spatters
This class of welds approaches Class A quality, yet exhibits minor imperfections, including uneven and non-uniform metal ripples, spatter, and slag inclusions on the surface, resulting in an aesthetically unpleasing appearance.
• C class: The weld bead is not straight or continuous, the shape and dimensions are inconsistent, and the weld surface has multiple defects
Figure 5.3 C class weld with multiple defects
This class demonstrates various welding defects, including uneven weld beads, inconsistent sizing, absence of metal ripples, and the presence of porosity, spatter, and slag inclusions, all of which compromise the reliability and joint strength of the weld.
• D class: The weld bead has not formed, or the weld is burned through
Figure 5.4 D class weld which is burned through
Figure 5.5 D class weld which has not form a weld
In this class, the welder lacks mastery of the welding technique, resulting in inappropriate welding parameters Consequently, this can cause interruptions in the weld bead or even lead to burn-through issues.
After thorough analysis and discussion with a specialized expert, the team identified six common defects in manual arc welds made by students, based on numerous images reviewed These defects include burn through, infusion, porosity, roughness, slag, and spatter, which will be the focus of further detection efforts as detailed in Section 2.1.4 An annotated image from our dataset is provided for reference.
AI process overview
The Arc Welding Evaluation System offers a user-friendly interface for quick and efficient operation, requiring users to simply power it on It analyzes arc weld samples placed in a photography chamber, capturing images for evaluation Key features include weld classification, defect detection, and identification of root causes and corrective actions, all displayed clearly on the screen.
The system operates through a defined workflow that begins with setting up the shooting chamber environment to ensure optimal image quality by adjusting lighting and positioning the weld It first verifies if the captured image is indeed a weld; if not, it prompts for reconfiguration and a retake Upon confirming the image as a weld, the system immediately classifies it, identifying class D welds and determining their causes and solutions For welds in classes A, B, and C, the system employs a defect detection feature, using the results to identify causes and propose solutions.
Developing weld quality classification model
Our system's weld quality grading feature delivers an automated, consistent, and objective evaluation of weld quality through captured images of weld beads It classifies welds into four quality grades: A, B, C, and D Furthermore, these classification results will support additional functionalities, including weld defect detection and suggestions for correcting welding issues.
The process of building a manual metal arc welding classification model involves three main stages: data preparing, data processing, and model evaluation
• Data preparing involves preparing the data for feature engineering
• Data processing involves applying various techniques to improve the quality of the dataset
• Model evaluation: Following the training process with the processed dataset, models are generated and assessed for their performance and practicality
A dataset comprising around 1,200 MMAW images was gathered from student exercises and meticulously annotated by a welding expert into four quality classes: A, B, C, and D The labeled data was then divided into training and testing sets to effectively train the model and assess its performance throughout the training process.
Figure 5.14 Weld Quality Class Balance
The data processing pipeline for the Weld Quality Classification Feature consists of multiple steps aimed at preparing weld bead images for analysis using a Vision Transformer (ViT) model This process includes essential tasks such as image augmentation, normalization, and resizing, all of which are crucial for enhancing the ViT model's performance.
Before feeding the images into the ViT model, various augmentation techniques are applied to enhance the robustness and generalization capability of the model These augmentations include:
- Horizontal/Vertical Flip: Randomly flipping the images horizontally/vertically to introduce variability and help the model learn features invariant to orientation
- Change Brightness: Adjusting the brightness of the images to simulate different lighting conditions and improve the model's ability to handle varying illumination
Random Gaussian blur is utilized to apply a Gaussian blur with varying parameters, enabling the model to concentrate on the critical features of welds while minimizing sensitivity to camera focus This augmentation technique enhances the training dataset, allowing the model to learn more generalized patterns and ultimately boosting its performance on unseen data.
- These augmentation methods help in creating a more comprehensive training set, which aids the model in learning more generalized patterns and improves its performance on unseen data
Once the images are augmented, they undergo a series of preprocessing steps using the ViT Image Processor The parameters and operations involved are as follows:
- Resize: The images are resized to a standard dimension of 224x224 pixels, which is the required input size for the ViT model This resizing ensures uniformity and compatibility with the model
- Rescale: The pixel values of the images are rescaled by a factor of
0.00392156862745098 (1/255) This normalization step converts the pixel values from the range [0, 255] to [0, 1], facilitating better convergence during training
The images are normalized with a mean of [0.5, 0.5, 0.5] and a standard deviation of [0.5, 0.5, 0.5] This crucial normalization step stabilizes the learning process and ensures that the input data maintains a consistent scale.
- Resample: The images are resampled using a bilinear interpolation method
(indicated by the value 2), which helps in maintaining the quality of the images during resizing
Processed images are formatted into tensors, which serve as the necessary input for the ViT model This conversion transforms the images into a data structure that allows for efficient processing by the model.
After preprocessing and formatting the images, they are input into the ViT model for classification This model assigns a quality grade of A, B, C, or D, utilizing the features it has learned from the training data.
When evaluating classification model performance, various metrics like precision, recall, and F1 score are available However, this report focuses on accuracy as it offers a simple and intuitive measure of overall model effectiveness Accuracy clearly indicates the frequency of correct predictions regarding weld bead quality, making it an appropriate metric for an initial assessment of the model's performance.
Accuracy is a key metric for assessing the performance of a classification model, calculated as the ratio of correct predictions to the total number of predictions made.
• TP (True Positives): The number of instances correctly classified as a certain class
• TN (True Negatives): The number of instances correctly classified as not being a certain class
• FP (False Positives): The number of instances incorrectly classified as a certain class
• FN (False Negatives): The number of instances incorrectly classified as not being a certain class
Accuracy serves as a fundamental metric that offers a concise assessment of a model's performance, reflecting its overall effectiveness in making correct predictions In the realm of weld bead quality classification (A/B/C/D), accuracy reveals the percentage of weld beads accurately categorized into their respective quality levels.
Figure 5.15 Testing classify model on real weld image
The Vision Transformer model outperforms traditional CNN architectures with an impressive accuracy of 89% Among CNNs, InceptionV3 follows with a strong performance of 82.60%, while Resnet-50 and VGG-16 show moderate effectiveness at 72.1% and 70.2%, respectively These findings indicate that attention mechanisms and multi-scale processing significantly enhance performance for this task.
Table 5.2 Classify model accuracy comparison
Figure 5.16 depicts the training and validation accuracy, loss of the quality classification model Validation accuracy followed a similar upward trend, peaking at approximately 89%
5.1a) Classification accuracy curve 5.1b) Classification loss curve
Developing weld defect detection model
The weld defect detection feature, powered by the YOLOv8 (You Only Look Once) model, is designed to identify and classify specific defects in student-produced welds Its primary goals are to enhance the educational experience, improve weld quality, and ensure safety and reliability in welding practices.
• Detailed Defect Identification: To automatically detect and classify various types of weld defects, such as cracks, porosity, undercuts, and spatter, providing a comprehensive analysis of the weld quality.
Providing students with detailed feedback on the specific defects in their welds enables them to understand the types and locations of these imperfections This insight allows students to enhance their welding techniques and effectively learn how to rectify their mistakes.
• Enhanced Learning: To support the educational curriculum by providing instructors with detailed information on common defects This data can be
70 used to tailor instructional content, address common issues, and focus on areas where students struggle the most
Enhancing the quality of welds involves early identification of defects, enabling students to make crucial adjustments before the final evaluation This proactive approach to quality control not only improves overall weld quality but also fosters a culture of continuous improvement in welding practices.
Similar to classification, defect features are also developed in three main stages: data preparing, data processing, and model evaluation
Using the same dataset of the Weld Quality classification feature but labeled with bounding boxes around defects (porosity, spatters, etc.)
The data augmentation technique for Weld Defect Detection mirrors the approach used in Weld Quality Classification It includes methods such as horizontal and vertical flipping, brightness adjustment, and the addition of random Gaussian noise, all aimed at ensuring weld bead images are optimally prepared for analysis by the YOLOv8 model.
After augmentation, the images undergo preprocessing steps to prepare them for the YOLOv8 model:
- Resize: The images are resized to a standard dimension of 640x640 pixels
This resizing ensures uniformity and compatibility with the YOLOv8 model, which requires a fixed input size
Normalizing image pixel values is essential for enhancing the model's learning efficiency This process involves scaling the pixel values to a specific range, commonly between 0 and 1, or adjusting them based on the mean and standard deviation By normalizing the data, we ensure that the model can effectively interpret the input, leading to improved performance in image processing tasks.
In YOLOv8, key output parameters are essential for assessing the model's performance in object detection, offering valuable insights into its precision, recall, and F1-score, among other effectiveness metrics.
The YOLOv8 model excels in detecting objects, specifically welding defects, by analyzing output parameters to assess its accuracy This thorough evaluation process allows the team to refine the model's configuration for optimal performance, ensuring precise and reliable object detection in images related to welding defects.
Precision measures the accuracy of positive predictions by calculating the ratio of true positives to the total number of positive predictions, which includes both true positives and false positives It effectively addresses the question: "Of all instances identified as positive, how many are genuinely positive?"
Recall, often referred to as sensitivity or true positive rate, measures the proportion of true positive predictions relative to the total number of actual positives, which includes both true positives and false negatives This metric addresses the question: "How many of the actual positive instances were accurately identified?"
The F1-score is a crucial metric that represents the harmonic mean of precision and recall, effectively balancing these two important measures This makes it particularly valuable in scenarios where it is essential to manage the trade-off between precision and recall.
The YOLOv8 defect detection model exhibits impressive accuracy in identifying defects in arc welding images This section will analyze its performance using key metrics, including mean Average Precision (mAP) and F1-score, to offer a detailed evaluation of its effectiveness across different types of defects.
Figure 5.17 Testing detection model on real weld image
The accuracy of an object detection model is measured by its F1-score, which considers both precision and recall, serving as their harmonic mean The F1-Confidence Curve is utilized to plot the F1-score against five different confidence thresholds A higher F1-score indicates better performance, and the confidence level that optimally maximizes the F1-score is often regarded as the ideal threshold for predictions.
Figure 5.18 F1-Confidence curve of defect detection model
The Precision-Recall Curve illustrates the trade-off between precision and recall across different threshold values Recall, or sensitivity, measures the proportion of true positive predictions against the total actual positives, which includes both true positives and false negatives In contrast, precision calculates the ratio of true positive predictions to the overall number of positive predictions, comprising true positives and false positives.
Figure 5.19 Precision-Recall curve of defect detection model
The model exhibits impressive performance across multiple defect categories, achieving a peak F1-score of 0.8 at a confidence threshold of 0.253, showcasing a strong balance between precision and minimizing false positives Additionally, it records a mean Average Precision of 0.844 at an Intersection over Union (IoU) threshold of 0.5, highlighting its effectiveness in accurately localizing and classifying defects within images.
Developing weld feedback generating model
To address the issues found in our system's processing of real-world MMAW images from student exercises, we created a RAG system that utilizes the document mentioned in section 2.1.4 This document is vectorized and subsequently analyzed by a large language model (LLM) to generate feedback.
Running large language models (LLMs) necessitates considerable hardware resources, especially high-performance GPUs, to ensure real-time inference and optimal performance Recognizing this demand, NVIDIA has developed innovative solutions, including NVIDIA Inference Microservices (NIMs), which offer a powerful platform for the efficient deployment and management of LLMs.
In order to give the advice for the weld defects, we followed these steps:
To deploy the selected LLM (Llama 3) on NVIDIA's cloud, utilize NVIDIA Inference Microservices (NIMs) for model deployment This process includes configuring the model for inference and ensuring its accessibility through the hosted endpoints.
• Create Vector Stores from PDF: PDFs are converted into vector format and stored in a vector database e This step ensures that the system can retrieve relevant information efficiently
Model embedding transforms textual data into a vector format for storage in a vector database, facilitating operations such as vector similarity comparisons This process enhances data analysis efficiency, especially in natural language processing and machine learning applications.
Integrating the deployed model with the LangChain framework enhances the RAG system's capabilities, enabling it to query the model for generating language outputs while efficiently retrieving relevant information from vector stores.
• Return output: based on the pdf file content, the system will give the correct advice for each weld defect.
Developing data transmission and processing feature
Building upon the theoretical foundation laid out in section 2.3, we have developed a system that leverages a microcontroller as the client and Google Cloud as the server.
The microcontroller architecture facilitates data collection and transmission to the robust Google Cloud platform for advanced processing and analysis This strategy supports the development of advanced functionalities, including real-time monitoring, remote diagnostics, and machine learning-driven assessments of weld quality, while minimizing the computational demands on the client device.
Figure 5.21 The data transmission and processing workflow
The data transmission and processing system operates by allowing the Client to capture an image and send a request to the Server The Server first executes an image classification request to determine if the image depicts a weld If it is classified as a weld, the Server returns the results to the Client; otherwise, it prompts the Client to retake the picture Upon successful classification, if the image falls under class D, the system identifies the cause and result For images classified as A, B, or C, the system detects defects, specifies the defects present in the weld, and offers both the cause and potential solutions.
Developing graphical user interface
The primary objective of this project is to create an intuitive and efficient GUI that allows users to:
• Capture images of welds using a camera
• Analyze and classify the weld images to identify defects
• Provide corrective action advice based on the detected defects
The GUI consists of the following key components:
• Camera Control: Allows the user to turn the camera on or off to avoid overheat
• Light Control: Allows the user to turn the light source on or off to enhance image quality base on environment
• Image Capture: Captures the current view from the camera
• Defect Detection and Classification: Analyzes the captured image to detect and classify weld defects
• Result Display: Shows the original and analyzed images side by side, highlighting the detected defects and providing classification results
The user-friendly GUI features a clean layout that enhances usability, clearly distinguishing between the camera feed and the processed image results for easy comparison Additionally, it includes all functions outlined in section 4.7.
Overall System Result
The integrated camera captures high-resolution images of the weld bead when the weld product is placed in the inspection booth These images are transmitted to the Jetson Nano central controller, which evaluates the weld quality through advanced models, ensuring consistency and accuracy in the assessment process.
The system classifies welds into different classes based on the severity and type of defects For example:
• Class B: The weld bead has been disrupted by the appearance of surface defects
The corrective actions are provided based on the detected defects For instance:
• For Roughness: Grind or carbon arc the weld to sound metal and repair the affected area if needed
The developed GUI for weld defect detection enhances quality control by delivering real-time analysis and classification of weld images Its intuitive interface and dependable defect detection features significantly improve the efficiency and accuracy of weld inspections.
CONCLUSION AND RECOMMENDATION
Upon project completion, we successfully met all objectives by researching, designing, and implementing a hardware system for evaluating Manual Metal Arc Welding This system was integrated with AI-driven evaluation software, ensuring highly accurate results Furthermore, our team developed and deployed APIs on the Cloud, allowing for the execution of AI models and providing real-time results These APIs can be accessed and utilized by multiple systems simultaneously, enhancing overall efficiency.
The project faces challenges, particularly due to a limited and varied dataset derived from student assignments, which results in inconsistencies caused by differing completion times.
The project has successfully demonstrated that the system can effectively evaluate students' welding skills by assessing the quality of their welds, categorizing them as good or better With these promising results, the team aims to further develop the system into a widely adopted tool for assessing welding skills in educational settings To facilitate this expansion, the team has established specific improvement goals for the system's future.
• Enhance the evaluation methods so that the system can provide more accurate results for students
• Developing a feature to collect data from experimental images on the machine
• Expanding the evaluation to cover more types of welds
• Transitioning to more modern models with higher accuracy
In this section, we will discuss about how to operate the workstation
• Place the workstation in a flat position
• It's advisable to place the machine on an elevated surface like a desk to ensure a good viewing angle with the LCD
• Plug the machine's power cord into a 220V AC power source
• Switch on the power toggle switch to the ON position When it's switched to
ON, the green indicator light will illuminate
Figure 7.1 LED indicator when power on
3 Checking devices: After powering on, check all connected devices
• LCD screen: Ensure it is switched to the ON state (default: ON)
• The mouse should be connected to a USB port (default: connected)
The welding seam is fixtured in the inspection chamber
When fixing the welding seam in place, ensure it is securely positioned in the fixture:
- The orientation of the weld sample should face upwards, opposite the light and camera
- The weld should be oriented perpendicular to the line connecting the two adjacent corner pins
- The weld seam should be positioned perpendicular to the line that connects the two adjacent corner pins
The software will automatically start when the machine boots up
The workflow with the software:
- Firstly, capture an image of the weld seam in the inspection chamber using the
The "Capture" button allows users to take an image of a weld seam, which will then appear in the "Image result" frame If the system identifies that the captured image is not of a weld seam, it will prompt the user with a message like "xxx" and ask for a new capture.
After capturing the image, the weld seam is evaluated for severity by clicking the "Classification" button, with the results appearing in the "Class" field, along with additional troubleshooting recommendations.
"Advice" if the result is Class D) You can capture a new image to retry this step, but error detection cannot proceed without classification beforehand
After classification, users can detect defects in the optical welding seam by clicking the "Detect Defect" button The results will be shown in the "Image result" field, along with troubleshooting advice in the "Advice" section Additionally, users have the option to capture a new image to repeat the process without needing to reaccess the functionality.
Figure 7.4 Displaying results on GUI
- Once completed, if you wish to evaluate another weld, you can fixture a different weld seam and repeat the process
- After completing the evaluation, click the "Exit" button to shut down the system
[1] Chuyên trang Giáo dục nghề nghiệp (2019, April 17) Hướng nghiệp: Nghề hàn (Công nghệ hàn) YouTube https://www.youtube.com/watch?v=8_Dvod oeKiI
[2] Moinuddin, S Q., Hameed, S S., Dewangan, A K., Kumar, K R., & Kumari,
A S (2021) A study on weld defects classification in gas metal arc welding process using machine learning techniques Materials Today: Proceedings, 43, 623-628
[3] Ho, M P., Ngai, W K., Chan, T W., & Wai, H W (2022) An artificial neural network approach for parametric study on welding defect classification The International Journal of Advanced Manufacturing Technology, 120(1), 527-
[4] Ajmi, C., Zapata, J., Martínez-Álvarez, J J., Doménech, G., & Ruiz, R
(2020) Using deep learning for defect classification on a small weld X-ray image dataset Journal of Nondestructive Evaluation, 39, 1-13
[5] Gantala, T., & Balasubramaniam, K (2021) Automated defect recognition for welds using simulation assisted TFM imaging with artificial intelligence Journal of Nondestructive Evaluation, 40(1), 28
[6] Hồ Viết, B., & Phan Minh, T (2013) Giáo Trình Công Nghệ Chế Tạo Máy
Viet Nam National University Ho Chi Minh City
[7] Hunt, E B (2014) Artificial Intelligence In Google Books Academic Press https://books.google.com/books?hl=en&lr=&id=9y2jBQAAQBAJ& oi=fnd
&pg=PP1&dq=artificial+intelligence&ots=uQSCZo5Vxy&sig=udCmMOr- 0yZjPKHg7iRi8TidcT0
[8] IBM (2023b) What Is Machine Learning? IBM https://www.ibm.com/topics
[9] Technology, T (2023, November 28) Light GBM Light and Powerful
Gradient Boost Algorithm Medium https://medium.com/@turkishtechnolog y/light-gbm-light-and-powerful-gradient-boost-algorithm-eaa1e804eca8
[10] IBM (2023a) What is Deep Learning? Www.ibm.com; IBM https://www.ibm.com/topics/deep-learning
[11] Crabtree, M (2023, June) What is Machine Learning? Definition, Types,
Tools & More Datacamp https://www.datacamp.com/blog/what-is-machine- learning
[12] O’shea, K., & Nash, R (2015) An introduction to convolutional neural networks arXiv preprint arXiv:1511.08458
[13] Wu, J (2017) Introduction to convolutional neural networks National Key
Lab for Novel Software Technology Nanjing University China, 5(23), 495
[14] Yi, J., Kang, H K., Kwon, J H., Kim, K S., Park, M H., Seong, Y K., &
Bang, W C (2021) Technology trends and applications of deep learning in ultrasonography: image quality enhancement, diagnostic support, and improving workflow efficiency Ultrasonography, 40(1), 7
[15] Altan, G., & Kutlu, Y (2021) Superiorities of deep extreme learning machines against convolutional neural networks arXiv preprint arXiv:2101.10265
[16] What is Image Classification? - Hugging Face (n.d.) Huggingface.co https://huggingface.co/tasks/image-classification
[17] Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A N.,
& Polosukhin, I (2017) Attention is all you need Advances in neural information processing systems, 30
[18] Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X.,
Unterthiner, T., & Houlsby, N (2020) An image is worth 16x16 words: Transformers for image recognition at scale arXiv preprint arXiv:2010.11929
[19] Bi, J., Zhu, Z., & Meng, Q (2021, September) Transformer in computer vision In 2021 IEEE International Conference on Computer Science, Electronic Information Engineering and Intelligent Control Technology (CEI) (pp 178-188) IEEE
[20] Object detection (n.d.) Huggingface.co https://huggingface.co/docs/transfor mers/en/tasks/object_detection
[21] Jocher, G., Chaurasia, A., & Qiu, J (2023) Ultralytics YOLO (Version 8.0.0)
[Computer software] https://github.com/ultralytics/ultralytics
[22] Redmon, J., Divvala, S., Girshick, R., & Farhadi, A (2016) You only look once: Unified, real-time object detection In Proceedings of the IEEE conference on computer vision and pattern recognition (pp 779-788)
[23] Zhao, X (2023) Deep learning based visual perception and decision-making technology for autonomous vehicles
[24] Kerner, S M (2023, September) What is a large language model (LLM)? –
TechTarget Definition WhatIs.com https://www.techtarget.com/whatis/defini tion/large-language-model-LLM
[25] Large Language Models (LLMs): Challenges, Predictions, Tutorial (n.d.) www.v7labs.com https://www.v7labs.com/blog/large-language-models-llms
[26] ChatGPT: The Tech Behind the Hype (and What It Means for Your Support)
(2023a, January 17) www.ultimate.ai https://www.ultimate.ai/blog/ai- automation/chatgpt-the-tech-behind-the-hype-and-what-it-means-for-your- support
[27] Merritt, R (2023, November 15) What Is Retrieval-Augmented Generation?
NVIDIA Blog https://blogs.nvidia.com/blog/what-is-retrieval-augmented-ge neration/
[28] Lewis, P., Perez, E., Piktus, A., Petroni, F., Karpukhin, V., Goyal, N., &
Kiela, D (2020) Retrieval-augmented generation for knowledge-intensive nlp tasks Advances in Neural Information Processing Systems, 33, 9459-9474
[29] J, A (2024, March 9) Understanding RAG: Making Language Models
Smarter Medium https://medium.com/@jcarun28/understanding-rag-makin g-language-models-smarter-f45bb1c66773
[30] Ngo, A (2020, October 31) 5 Phút Tìm Hiểu Về Mô Hình Client - Server
Codelearn.io https://codelearn.io/sharing/tim-hieu-ve-mo-hinh-client-server
[31] Benharosh, J (2018, July 31) What Is REST API | PHPenthusiast
Phpenthusiast.com https://phpenthusiast.com/blog/what-is-rest-api
[32] Google (n.d.) What is Cloud Run | Cloud Run Documentation Google Cloud https://cloud.google.com/run/docs/overview/what-is-cloud-run
[33] Fischer ASIA (n.d.) Fischer ASIA | Sản phẩm dập kim loại, dập kim loại tấm
Fischer Asia Retrieved July 1, 2024, from https://www.fischer- asia.com/portfolio/san-pham-dap-kim-loai/?lang=vi
[34] IMART, C T C P Đ T T M (n.d.) Nhận gia công phụ kiện quạt làm mát động cơ xe máy chất lượng giá tốt số #1 Www.plasticsaigon.com Retrieved July 1,
2024, from https://www.plasticsaigon.com/nhan-gia-cong-phu-kien-quat-lam- mat-dong-co-xe-may-chat-luong-gia-tot-so-1
[35] Amazon (n.d.) Toyota MR2 Turbo SW20 91-99 Authentic Stage21 Front Fenders
Set Widebody RARE EBay Retrieved July 1, 2024, from https://www.ebay.com/i tm/285687176510
[36] NVIDIA (2024, February) Technical Brief NVIDIA Docs https://docs.nvidi a.com/ai-enterprise/workflows-generative-ai/0.1.0/technical -brief.html
[37] Goutte, C., & Gaussier, E (2005, March) A probabilistic interpretation of precision, recall and F-score, with implication for evaluation In European conference on information retrieval (pp 345-359) Berlin, Heidelberg:
[38] Rezatofighi, H., Tsoi, N., Gwak, J., Sadeghian, A., Reid, I., & Savarese, S
(2019) Generalized intersection over union: A metric and a loss for bounding box regression In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (pp 658-666).