HO CHI MINH CITY UNIVERSITY OF TECHNOLOGY AND EDUCATIONGRADUATION THESIS MAJOR: ROBOTICS AND ARTIFICIAL INTELLIGENCE INSTRUCTOR: NGUYEN MINH TRIET VONG HONG SANG NGUYEN QUOC VIET DESIGN
INTRODUCTION
Background
Handwriting has been a vital form of communication for centuries, serving as both a medium for expressing thoughts and an art that showcases individual personality and cultural heritage Despite the prevalence of digital communication, handwriting remains an essential skill in education It continues to be taught in schools, significantly contributing to children's cognitive development by enhancing motor skills, memory retention, and creative expression.
The industrial sector has increasingly focused on automating handwriting processes for applications such as personalized printing, secure document signing, and customized product labeling This growing demand for robotic systems that can replicate human handwriting is fueled by the need for enhanced efficiency and precision in these tasks.
The integration of robotic systems and machine learning techniques for handwriting simulation represents a captivating blend of mechanical engineering, computer science, and cognitive psychology Recent advancements in machine learning, especially deep learning and attention mechanisms, enable robots to accurately replicate human handwriting while adapting to various styles and nuances.
Handwriting simulation technology plays a crucial role not only in industrial applications but also in education and cultural preservation In educational settings, robotic handwriting systems can enhance writing instruction by offering personalized feedback and aiding children with disabilities who face challenges with conventional writing methods Furthermore, these systems contribute to the preservation of cultural heritage by digitizing and reproducing historical manuscripts, ensuring their accessibility and longevity for future generations.
Motivation
This project is motivated by the increasing convergence of robotics and machine learning, highlighting their potential to address real-world challenges One significant area poised for improvement is handwriting simulation and reproduction, which can greatly benefit from advancements in these technologies.
- Automation of Personalized Writing: In many industries, there is a need for personalized writing, such as in customized product labeling,
2 personalized printing, and secure document signing Automating these tasks can save time and reduce human error, enhancing efficiency and accuracy
Traditional handwriting teaching methods can be labor-intensive and may not address individual student needs A robotic system that mimics human handwriting offers an effective educational tool, delivering personalized instruction and feedback This technology can significantly assist children with learning difficulties or disabilities, enhancing their learning experience.
The preservation of cultural heritage is crucial, as numerous historical manuscripts and documents face the threat of degradation Implementing robotic systems capable of accurately replicating handwriting styles can facilitate the digitization and preservation of these valuable documents, ensuring their accessibility for future generations.
This project harnesses the latest advancements in robotics and machine learning, focusing on deep learning and attention mechanisms, to develop a system that achieves high-fidelity handwriting simulation By integrating these technologies, it demonstrates their potential and expands the possibilities for automated handwriting reproduction.
The development of a five-bar robotic arm for handwriting focuses on enhancing robotic precision and flexibility This project aims to improve the control and adaptability of robotic systems, enabling them to perform delicate tasks with greater accuracy and versatility.
The project focuses on advanced image processing techniques that transform digital text into coordinates for robotic arm manipulation Key processes include converting images to grayscale, applying filters, and skeletonizing images to effectively extract stroke paths These advancements play a significant role in enhancing the fields of image processing and computer vision.
This project simulates human handwriting to offer insights into the cognitive and motor processes involved in writing By enhancing our understanding of these processes, we can improve educational tools and therapeutic methods for individuals with motor impairments.
- Adapting to Different Handwriting Styles: A key aspect of this project is the ability of the robotic system to adapt to various handwriting styles
This adaptability is crucial for personalized applications and can lead to more natural and human-like writing outputs.
Objective
The main aim of this project is to design and develop a five-bar robotic system that utilizes machine learning techniques to generate handwriting This project includes several specific objectives, which are outlined in detail below.
1.3.1 Design and Development of the Robotic System
Mechanical Design: Develop a robust and precise mechanical structure for the five-bar robotic arm This includes:
- Creating a detailed Computer-Aided Design (CAD) model of the robotic arm
- Ensuring the mechanical components are capable of achieving the necessary range of motion and accuracy required for handwriting
- Implementing kinematic and positional analysis to optimize the design for smooth and accurate movements
Hardware Integration: Integrate various hardware components necessary for the functioning of the robotic arm This includes:
- Selecting and configuring appropriate motors, motor drivers, and controllers
- Designing a control system that can efficiently manage the movements of the robotic arm
- Ensuring the hardware setup is capable of real-time operation and can handle the computational demands of handwriting simulation.
1.3.2 Implementation of Machine Learning Techniques
Image Processing and Preprocessing: Develop algorithms for processing input text and converting it into a format suitable for the robotic system This includes:
- Converting RGB images of text into grayscale and binary images
- Applying filters and skeletonizing the images to extract the stroke paths needed for handwriting
- Ensuring the preprocessing techniques are efficient and accurate
Handwriting Simulation Model: Create a machine learning model capable of generating handwriting coordinates from input text This includes:
- Developing and training a neural network model, such as a Convolutional Neural Network (CNN) or a Recurrent Neural Network (RNN) with attention mechanisms, to simulate handwriting
- Fine-tuning the model to handle different handwriting styles and nuances, ensuring it can adapt to various inputs
- Validating the model's performance to ensure high fidelity in the generated handwriting.
Control Algorithm: Design and implement control algorithms to convert the output of the handwriting simulation model into precise movements of the robotic arm This includes:
- Translating the (x, y) coordinates generated by the model into motor commands
- Implementing feedback mechanisms to correct errors and ensure the accuracy of the written output
- Ensuring the control system is responsive and capable of real-time adjustments
User Interface and Communication: Develop an intuitive user interface for inputting text and managing the robotic system This includes:
- Creating a software interface that allows users to input text and visualize the output
- Implementing communication protocols for seamless interaction between the user interface, the handwriting simulation model, and the robotic hardware.
Performance Evaluation: Conduct thorough testing of the robotic system to evaluate its performance in terms of accuracy, speed, and adaptability This includes:
- Testing the system with various handwriting styles and evaluating the fidelity of the written output
- Measuring the system's response time and accuracy in real-time operation
- Identifying and addressing any limitations or areas for improvement
Practical Applications: Demonstrate the practical applications of the system in real-world scenarios This includes:
- Showcasing the system's ability to automate personalized writing tasks in industrial and educational settings
- Exploring potential uses in assisting individuals with disabilities and preserving cultural heritage through the reproduction of historical manuscripts.
Limitations
While the project shows great promise, it is essential to recognize its limitations, which include both technical and non-technical factors These limitations may affect the overall performance and effectiveness of the five-bar robotic system designed for handwriting creation Acknowledging these challenges is vital for establishing realistic expectations and pinpointing opportunities for future enhancements.
The precision and stability of a five-bar robotic arm are essential for achieving accurate handwriting simulation Mechanical imperfections, including joint backlash, friction, and component flexibility, can significantly impact the accuracy of the written output.
- Ensuring that all mechanical parts are perfectly aligned and minimizing mechanical play can be challenging, potentially leading to deviations in the handwriting quality
- Therefore, the 97% precision is expected
- The performance of the motors and actuators, including their resolution, speed, and torque, can limit the robotic arm’s ability to replicate intricate handwriting details
- High-resolution motors may be required to achieve the necessary fine movements, which can increase the cost and complexity of the system
- In this thesis, we apply the resolution of 1/16 step for the motor to balance the accuracy and the cost
Real-time Processing and Latency:
Real-time processing and low-latency communication between the control system and the robotic arm are essential for seamless operation Any delays or lag can lead to inaccuracies and interruptions in the handwriting process.
- Computational limitations of the control hardware may impact the speed and responsiveness of the system
- By using arduino Uno, the latency of 0.5 second are expected
Image Processing and Conversion Accuracy:
The precision of image processing algorithms, such as filtering, skeletonization, and stroke path extraction, significantly influences the quality of the resulting handwritten output Any flaws in these algorithms can result in inaccuracies in the generated coordinates.
- Variations in input image quality, such as resolution and noise, can impact the effectiveness of the preprocessing techniques Hence, the error of 5% are expected
Adaptability to Different Handwriting Styles:
Machine learning models can be trained to recognize diverse handwriting styles, but they may struggle to generalize effectively to entirely new or highly intricate styles, particularly in Vietnamese.
- The model’s performance may vary based on the diversity and quality of the training data used 80% of accuracy are expected
- Developing and maintaining a high-precision robotic system can be expensive, requiring significant investment in high-quality components, advanced sensors, and powerful computing hardware
- Limited budget and resources may restrict the scope of the project and the ability to implement all desired features and improvements
Designing an intuitive and user-friendly interface for managing robotic systems and text input is crucial, as usability challenges can significantly impact the user experience and the system's effectiveness in practical applications.
- Ensuring compatibility with various devices and platforms can add complexity to the development process
Adhering to regulatory standards and safety requirements is crucial for robotic systems, especially in industrial and educational environments This compliance can complicate the development process, highlighting the importance of integrating these standards from the outset.
- Safety mechanisms must be implemented to prevent accidents or malfunctions, especially when the system is operated by non-experts.
Thesis Organization
The thesis is organized into multiple chapters, each dedicated to various elements of designing and controlling a five-bar robot specifically for handwriting generation.
LITERATURE REVIEW
Drawing Robot
The history of drawing robots dates back to the 18th century, initiated by innovators like Jaquet-Droz and Maillardet, who created remarkable drawing automata These mechanical marvels utilized intricate cam systems to generate detailed drawings Jaquet-Droz's automaton was designed to produce four distinct images, while Maillardet's creation could not only draw elaborate pictures but also write poetry, showcasing the advanced engineering of cams and levers in that era.
In the 19th and early 20th centuries, significant progress was achieved with the invention of drawing machines, such as John Nevil Maskelyne's "Zoe" automaton and Phillip Vielmetter's mechanical clown artist These innovative devices utilized advanced mechanical systems to produce drawings, mesmerizing audiences with their realistic outputs.
In the mid-20th century, artists like Jean Tinguely and Raymond Auger revolutionized drawing robots with their innovative painting machines Tinguely's Méta-matics created random patterns, examining the interplay between art and machinery, while Auger's automatic painting machine utilized perforated paper rolls to guide its actions, akin to a player piano.
The emergence of computer graphics in the 1960s significantly advanced the creation of drawing robots, exemplified by Harold Cohen's AARON, one of the earliest computer-controlled plotters designed for generating artistic images Since its inception in 1968, AARON has undergone continual evolution, showcasing the potential of technology in the art world.
30 years, creating both drawings and paintings, and is considered a significant milestone in the field of robotic art
In the 21st century, drawing robots have evolved significantly, showcasing advanced manipulation, control, and computing abilities Notable projects such as MEART, which utilizes a pneumatically actuated robotic arm guided by a network of rat neurons, and Leonel Moura's ArtSwarm robots, which generate artwork inspired by ant-like behavior, highlight the growing creativity and autonomy of contemporary drawing robots.
This thesis focuses on developing a five-bar writing robot, leveraging a parallel manipulator for precise and flexible movements, making it well-suited for writing tasks To fully grasp the design and operation of this innovative writing robot, it's crucial to examine the components and mechanisms that form the basis of a drawing robot.
Components of Drawing Robot
A drawing robot consists of several critical components that work together to achieve its function:
- Mechanical Structure: Includes the frame, motors, belts, pulleys, and linkages that enable movement
- Drawing Head: Comprises the tool holder and the drawing instrument itself
- Control System: The microcontroller or computer that processes input data and sends commands to the motors, along with motor drivers
- Power Supply: Provides the necessary electrical power
- Sensors: Such as limit switches and encoders, which ensure precise control
- Software: Firmware and control software that converts digital images or text into executable commands
- Communication Interface: For wired or wireless communication between the robot and the control device
- Drawing Surface: The area where the robot performs the drawing
- Miscellaneous: Includes cables, connectors, and mounting hardware.
Mechanism of Drawing Robot
X-Y writing robots, also known as Cartesian plotters, operate on a two- dimensional Cartesian coordinate system These robots move a drawing instrument along the X and Y axes to create images, text, or patterns on a flat surface This configuration is simple yet effective, making X-Y writing robots popular for various applications, including educational tools, artistic endeavors, and industrial processes
Figure 1 CAD model of XY Plotter
The above CAD model shows main components of the plotter The following table summarizes the operation of each and every component of mechanical structure
Table 1 Mechanical components and operation
X axis motor Plotter operation in x direction
Y axis motor Plotter operation in y direction
Gear wheels Guide the belt for moving
Linear bearing Guide the circular rods
Timing Belt Support the pen manipulator movement
DC Solenoid Moves pen in z direction
Rod support bearing Keep the rods stable
Base Support the entire structure
The main electrical components associated with the project are Stepper motor driver, AC-DC converter, Arduino controller, Solenoid control circuit etc Using AC-
DC converter, 230VAC input is converted to two 12VDC outputs The current rating of the power supply is 15A
- Precision: X-Y writing robots can achieve high levels of accuracy, making them suitable for detailed and intricate designs
- Simplicity: The straightforward design and operation make these robots easy to build, maintain, and troubleshoot
- Flexibility: They can be adapted for various tasks, from simple writing and drawing to complex CNC milling and laser cutting
- Mechanical Alignment: Ensuring that the X and Y axes are perfectly perpendicular and that the drawing surface is level
- Vibration and Stability: Minimizing vibrations and maintaining stability to achieve smooth and accurate lines
- Software Integration: Developing or selecting software that can efficiently convert designs into precise motor commands
Polar writing robots utilize a polar coordinate system to determine the position of their drawing tools through an angle and radius Featuring a rotational base and an extendable arm, these robots can effectively cover circular or semi-circular work areas Their distinct motion sets them apart from Cartesian systems, making them ideal for various artistic and industrial applications.
- Rotary plate to adjust the angle phi
- Linear axis to adjast the radius r
- Pen lift up / down mechanism
- Ready-to-use microcontroller, based on an Arduino Mega
The drawing paper is affixed to a rotary plate, enabling the adjustment of angle phi through rotation This rotary plate is powered by a NEMA 17 stepper motor, ensuring rapid and precise positioning.
The linear axis is predominantely built of fischertechnik parts This measure saves 3D printing costs and time Moreover, fischertechnik parts can be reused in other projects
The linear axis operates using a NEMA 14 stepper motor, which, despite having lower torque compared to the NEMA 17 motor, offers a higher rotational speed crucial for effective performance due to significant gear reduction at the screw While the stepper motor enables precise positioning, it is essential to incorporate a limit switch to establish a reference point for calibration.
The pen lift mechanism operates efficiently by quickly raising and lowering the pen using a simple DC motor, avoiding the jitter and instability associated with servo motors This mechanism employs a rotating cam disk that either presses the pen down or allows a return spring to lift it back up, with a limit switch ensuring precise positioning at the end of the movement.
The microcontroller serves as a standalone project, featuring a plug-and-play design with built-in motor drivers It is specifically designed to control fischertechnik hardware, including DC motors and various sensors.
- Efficiency in Circular Designs: Polar robots are particularly effective for designs that are circular or radial in nature
- Unique Motion: The polar coordinate system allows for smooth, continuous curves and circular motions that are difficult to achieve with Cartesian robots
- Compact Design: Polar robots can cover a large work area with a relatively small footprint, as the arm and rotational base do not require extensive linear travel
- Complexity in Path Planning: Converting designs from Cartesian to polar coordinates can be complex and requires sophisticated software algorithms
- Mechanical Precision: Ensuring the radial arm and rotational base move precisely to avoid distortions in the drawing
- Calibration: Accurate calibration of the robot is essential to maintain consistency and accuracy across the entire work area.
The five-bar writing robot is a type of parallel robot characterized by its two fixed base joints and three moving joints that form a closed-loop linkage system This
The five-bar mechanism enables precise and rapid movement of the end-effector that holds the drawing instrument, making it ideal for applications demanding high speed and accuracy, such as handwriting and detailed drawing tasks.
Figure 4 Five-bar writing robot
- Base Joints: Two fixed joints that anchor the robot to its base These joints are the pivot points for the linkages
- Linkages: Two pairs of links (arms) that connect the base joints to the end- effector The lengths of these links are critical for the robot's kinematics
The end-effector is the robotic component that carries the drawing instrument, linked to the robot's armature It accurately follows a predetermined path, guided by the movements of the base joints, ensuring precise execution of tasks.
- Motors: Typically two motors are used, each controlling one of the base joints These motors are responsible for moving the linkages and, consequently, the end- effector
- Control System: A microcontroller or computer that processes input commands and sends control signals to the motors Common choices include Arduino, Raspberry Pi, and other robotic controllers
- Motor Drivers: Interface devices that enable the control system to drive the motors with the required precision
- Power Supply: Provides the necessary electrical power to the motors and control system
- High Speed and Acceleration: The parallel structure allows for rapid movements, making it suitable for tasks that require quick and smooth handwriting
- Precision and Rigidity: The closed-loop linkage system provides high rigidity and precision, minimizing deflection and ensuring accurate drawings
- Reduced Inertia: Since the actuators are typically fixed at the base, the moving parts have lower inertia, contributing to better dynamic performance
- Compact and Efficient Design: The five-bar mechanism maximizes the workspace relative to its footprint, making it efficient for space-constrained environments
- Complex Motion Capabilities: Capable of producing complex, smooth, and continuous motions, ideal for detailed and intricate writing or drawing tasks
The kinematic equations and control algorithms for complex robotic systems are significantly more intricate than those used in simpler mechanisms such as X-Y or polar robots, necessitating a deeper understanding of robotics and control systems for effective implementation.
The work envelope of a five-bar robot is generally constrained and often features an irregular shape, which can limit the range of tasks it is capable of executing effectively.
Different types of writing robots, including five-bar, X-Y, and polar models, each offer unique advantages and disadvantages The X-Y robot features a straightforward design with a consistent work area but is slower and less rigid compared to others In contrast, the polar robot is ideal for circular designs and is compact, yet it faces challenges with path planning and mechanical precision Ultimately, the five-bar robot stands out for its superior speed, precision, and ability to perform complex motions, making it the preferred choice for advanced applications.
Kinematics of Five-bar robot
Different types of writing robots, including five-bar, X-Y, and polar, each have unique advantages and disadvantages The X-Y robot features a straightforward design and consistent work area but is slower and less rigid In contrast, the polar robot is effective for circular designs and is compact, yet it faces challenges with path planning and mechanical precision Consequently, the five-bar robot is preferred for its superior speed, precision, and ability to handle complex motions.
The five link planar manipulative system (MS), shown in Figure 1, contains only rotational joints[1]
Figure 5 Model of Five-Bar MS
The mechanism consists of both passive and active components, with body 5 (l_5) serving as a stationary support, as illustrated in Figure 2 Bodies 1 and 4 function as the driving elements, enabling the mechanism to achieve motion By appropriately rotating the actuating bodies, the characteristic point C of the mechanism can accurately trace the desired planar trajectory within the defined working zone.
This essay analyzes the lengths of five links, denoted as l_1, l_2, l_3, l_4, and l_5, and the angles between these links, represented by θ_1, θ_2, θ_3, and θ_4 Additionally, it incorporates two extra angles, θ_6 and θ_7, which are essential for conducting a comprehensive kinematic analysis based on the established link angles.
The five-bar manipulative system offers high efficiency, significant payload capacity, and versatile application options Nonetheless, designing and controlling such robots necessitates a comprehensive understanding of kinematics and dynamics.
This thesis aims to create an efficient design methodology for a 5-bar parallel robot, focusing on optimizing accuracy and performance for writing tasks The methodology incorporates kinematic and dynamic modeling, utilizing numerical and geometric methods to address challenges related to forces, speeds, energy, positions, and torques.
This thesis is structured to first introduce the modeling methods, followed by a detailed simulation section Lastly, it will discuss the experiments and results to assess the robot's performance.
Figure 6 Schematic Diagram of Five-Bar Manipulative System
The primary goal of the dynamic analysis of a five-bar mechanism system (MS) is to extract the direct equations for the actuating angles based on the end-effector coordinate equations The initial positions of the links can be determined through kinematic analysis, and by applying the geometric relationships of the mechanism, the coordinates of point C can be derived, as illustrated in Figure 2.
From (1) and (2), it can be found that 𝜃 1 and 𝜃 4 are independent in the system, and 𝜃 2 and 𝜃 3 can be determined by 𝜃 1 and 𝜃 4 as follows: [1]
From (2) and (3) above, we get,
Equations (3) and (4) provide an indirect method for determining the actuating angles in a manipulative system based on specific end-effector coordinates To utilize these equations effectively, simulation through SimMechanics or MATLAB software is required.
This approach may not be ideal for industrial robots that require precise and dynamic control of the end-effector To enhance control, modifications are applied to existing equations, establishing a direct relationship between the end-effector coordinates, link lengths, and the actuating angles 𝜃 1 and 𝜃 4.
By combining the equations (1) and (2) and eliminating the secondary angles 𝜃 3 and 𝜃 2 the following equations are obtained.[4]
Equations (5) and (6) enable direct control of the actuating angles without needing to know the dependent angles 𝜃 2 and 𝜃 3 Given that the link lengths are constant for a predefined robot, these equations can be simplified Consequently, the end-effector coordinates (𝑥 𝑐 and 𝑦 𝑐) become the sole inputs necessary for controlling the mechanism This streamlines the semi-automated and automated control processes, making them significantly easier compared to traditional methods.
However, the dependent and additional angles are also derived for future references in kinematic analysis is shown below:
Equations (5) and (6) are verified by applying sample link lengths and end- effector coordinates in the parametric software CREO and found to be correct
The Figure 3 shows the placement of the above mentioned link angles in the five- bar manipulative system
Figure 7 Auto CAD Generated Five-bar Manipulative System
The velocity 𝑉 = [𝑉 𝐶 𝑥 , 𝑉 𝐶 𝑦 ] 𝑇 of the characteristic point C is determined through the angular velocities 𝜃̇ = [𝜃̇ 1 𝜃̇ 4 ] 𝑇 of the bodies 1 and 4 and depends on transfer function of the mechanism
Usually the transfer function is described by the Jacoby matrix J
𝑉 = 𝐽𝜃̇(7) This expression is known as forward kinematics problem and for the considered
MS could be solved by using different approaches The analytic symbolic solution could
19 be particularly useful for making several conclusions concerning the singular configurations of the MS as well as MS metric [2]
Figure 8 Representation of MS with Two Open Structures
The classical approach for solving such kind of problems requires the solution of standard position task (forward kinematics) 𝑓(𝜃 𝑖 ) = 𝑋; 𝑥 = 1, 4; 𝑉 = [𝑥 𝑐 , 𝑦 𝑐 ] 𝑇 or of the inverse kinematics.[6]
The results are differentiated concerning the general coordinates 𝜃 = [𝜃 1 , 𝜃 4 ] 𝑇 In the standard position task of forward kinematics, there are two solutions, while the inverse has four This highlights the need to explore alternative methods for analytically solving the forward kinematics position task.
It is assumed that the MS is divided into two pairs as shown in Figure 4 representing two open planar kinematics chain with two links
The matrix of Jacoby J 1,2 for each of them is known For the left (J 1 ) MS we obtain,
Analogously we can obtain for the right system:
When both systems are positioned at a distance of l1 and converge at point B, achieving the same velocity V at that point, the resulting system can be analyzed effectively.
Eliminating the angular velocities 𝜃̇ 6 and 𝜃̇ 7 in the passive joints for the forward kinematics problem it is obtained that,
The equations (8), (9), (10), (11), and (12) provide a method for determining the angular velocities of joints A, B, C, and D, as well as the linear velocity of the end-effector C However, solving these equations is complex and demands significant resources to accurately calculate the angular velocities of the active links, 𝜃̇ 1 and 𝜃̇ 4.
From these equations it is modified into simpler formulae which can directly give the values of active link’s angular velocities
In this analysis, the required end-effector velocity is broken down into its x and y components (𝑉 𝐵 𝑥 and 𝑉 𝐵 𝑦) to align with the movement direction Prior to utilizing equations (13) and (14), it is essential to determine these components based on the provided constants.
The speed of the individual motors 1 and 4 controlling the links A and B can be found out easily using the following relations:
For path movements of the end-effector the speed and angles can be simultaneously varied using the proposed equations
From these modified equations the real time variable speed can be achieved at the end-effector by changing the speed of the controller motors.
Image Processing
2.5.1 Theory of images and related issues
An image serves as a visual representation of objects, landscapes, or people, created through various methods such as photography, drawing, printing, and digital means Images can be found in both digital formats, like computer files, and physical forms, such as paintings or printed photographs.
Images can be categorized into two-dimensional formats, like flat paintings, and three-dimensional representations, such as sculptures or holograms They can be captured using optical devices, including cameras, mirrors, lenses, telescopes, and microscopes, or through natural means like the human eye and water surfaces.
Images can be used in a broad sense, representing maps, graphs, and abstract art
Images can be entirely newly created rather than merely recorded, utilizing methods such as drawing, sculpting, printing, or computer graphics These imaginative visuals arise in human thought, akin to the process of memory.
A digital image is a collection of points (pixels) when digitized The amount of information in digital images is huge
An image consists of a continuous range of space and brightness To effectively process this image, it is essential to convert the analog signal into a discrete signal through sampling and quantization This process creates a set of image elements, commonly referred to as digitization.
Digital images are stored in a numerical format that computers can process, allowing for easy manipulation and storage These images can be captured using digital cameras and smartphones or created by digitizing traditional physical photographs.
2.5.2.2 Structure of a Digital Image File
An image file is a collection of many pixels:
Figure 9 2-dimensional array of an image file
A pixel, the fundamental unit of a digital image, holds essential information about color and brightness Digital images are structured as a coordinate matrix, with each matrix element representing an individual pixel.
So, the image can be viewed as a coordinate matrix Let f(x,y) be the function representing the matrix of a digital image In there:
+ x and y are the coordinates of the pixel in 2-dimensional space
+ f(x,y) is the intensity value of the pixel at coordinates (x,y)
We can represent the function f(x,y) as follows:
+ M is the number of rows (height) of the matrix
+ N is the number of columns (width) of the matrix
+ f(x,y) is the intensity or color value of the pixel at coordinates (x,y)
RGB images, often referred to as "true color" images, utilize the RGB (Red, Green, Blue) color model to depict colors These images are structured as a three-dimensional matrix with dimensions "m x n x 3," where "m x n" indicates the pixel dimensions Each pixel's color is created by combining the three primary color components—Red (R), Green (G), and Blue (B)—with each color represented by a numeric value ranging from 0 to 255 in an 8-bit system.
R (Red): Represents the intensity of the red color
G (Green): Represents the intensity of the green color
B (Blue): Represents the intensity of the blue color
Each pixel in an RGB image has three values corresponding to the intensity of these three primary colors
The RGB color model is a widely utilized system in digital photography and electronic display devices, essential for capturing and displaying images It plays a crucial role in graphic design and image processing, enabling the representation of virtual environments and objects effectively.
An RGB image is composed of three two-dimensional matrices that represent the intensity values of the primary colors: red, green, and blue These matrices, denoted as IR(x,y), IG(x,y), and IB(x,y), correspond to the intensity levels of red, green, and blue at specific coordinates (x,y) within the image.
(20) Combining these three matrices, we get a complete RGB image
Example of RGB color encoding
Figure 11 RGB image and RGB image matrix
Each color can be represented by a combination of R, G, and B values For example:
Grayscale images are monochrome visuals where each pixel conveys brightness information without any color The gray level is determined by encoding the light intensity of each pixel into a specific value, typically ranging from 0 (black) to 255 (white) in an 8-bit system, or from 0 to 1 in a normalized format This encoding process results in the creation of a data matrix.
Grayscale imaging has many applications in different fields:
Grayscale images play a crucial role in image processing algorithms, including edge detection, image segmentation, and feature extraction Their simplicity and lower computational demands compared to color images make them a preferred choice for these applications.
- Photography: Grayscale photography is often used to create artistic photographs or to emphasize geometric and textural details
- Medical: Medical images such as X-rays, MRIs, and CT scans are often presented in grayscale to show internal body structures clearly
- Pattern recognition: Grayscale images are used in pattern recognition, optical character recognition (OCR), and face recognition systems
A Grayscale image can be represented as a two-dimensional matrix, where each element of the matrix represents the intensity value of a pixel at coordinates (x,y)
Calling I(x,y) a function representing the matrix of a Grayscale image, we have:
+ M is the number of rows (height) of the matrix
+ N is the number of columns (width) of the matrix
+ I(x,y) is the gray level intensity value at coordinates (x,y)
A binary image consists of pixels that can represent only two values, typically 0 and 1, corresponding to black and white These images are widely utilized in various applications, including image recognition, classification, and processing algorithms.
Convolution is a fundamental operation in image processing, particularly within Convolutional Neural Networks (CNNs) This process involves applying a kernel or filter that slides over the original image to generate a new output image It is primarily utilized for various mathematical operations on images, including image derivatives, smoothing, and edge extraction.
According to mathematics, convolution is a linear operation, producing the result as a function by calculating based on two existing functions
Given two functions f(x) and g(x), the convolution of these two functions is defined as:
In the context of digital image processing, the above formula is converted to discrete form:
+ f is the input image matrix
+ i,j are the coordinates of pixels in the output image
Figure 14 Illustration of convolution of 2 matrices
Convolution is defined as an operation on the integrable space of linear functions, so it has commutative, associative and distributive properties
Convolution in image processing involves multiple successive operations with different filters (kernels) represented as f * g * h To enhance efficiency and reduce computational complexity, we can precompute the kernel matrix as k = v * h, given its smaller size compared to the image This approach allows us to streamline the convolution process, enabling us to compute the result as r = f * (v * h) = f * k, rather than following the traditional order of r = (f * g) * h.
A kernel, or filter, is a small matrix of numerical values utilized in various image transformations, including blurring, sharpening, and edge detection By applying this kernel to an image through a convolution operation, a new image is generated with altered features.
The kernel is usually a small square matrix such as 3x3, 5x5, or 7x7 Each element in the kernel matrix is a coefficient used to calculate the new value of the pixel
2.5.6 Methods of digital image processing in the project
2.5.6.1 Convert RGB color image to Gray-level image
MECHANICAL SIMULATION
Problem Definition and Constraints
The initial phase for the designer involves clearly defining the robot's intended task, including the required timeframe and accuracy level This thesis focuses on a five-bar parallel robot designed for writing tasks, where specific points in a plane are established for the two-degree-of-freedom robot to reach, thereby outlining an operational area and trajectory Additionally, various constraints must be identified, such as maximum payload capacity, end effector type, budget, materials, and access to necessary equipment and tools.
This thesis showcases a design methodology for a parallel robot featuring five links and two degrees of freedom The robot is engineered to maneuver pens weighing under 100 g, achieving a minimum displacement of 15 cm parallel to the actuator's center of rotation It operates at a speed of at least 0.5 m/s while maintaining an accuracy of ± 0.5 mm.
HARDWARE DESIGN
Machine component calculation and selection
This section outlines the calculations and rationale behind the selection of machine components for a five-bar linkage robot designed for handwriting The goal is to create a robot that excels in performance while being cost-effective, compact, and lightweight.
For optimal handwriting quality, the robot must achieve high precision, ensuring that its output closely resembles traditional handwriting To maintain both aesthetic appeal and accuracy, the positional error of the pen tip should not exceed ±0.5mm.
Durability: The machine components and structure of the robot must be durable enough to operate continuously for a long time without failure
The robot should be designed to be lightweight and compact, ensuring effortless movement and installation across diverse workspaces Its total weight must remain under 2kg, with dimensions capped at 50x50x50cm.
When designing a robot, it is crucial to prioritize ease of maintenance and repair, ensuring that machine components are readily accessible and easily replaceable This approach not only enhances the robot's longevity but also minimizes downtime, as components should be widely available in the market for quick and efficient replacements.
Torque at the Joints: Determine the required torque at the joints to maintain the movement of the pen We use the following formula:
- F is the force acting on the pen (due to its weight)
- r is the radius from the joint center to the point of force application
Assume 𝑟 max 0 mm and the force due to the pen's weight is:
When selecting motors, it is crucial to choose those with the appropriate torque and speed based on prior calculations For optimal performance, select motors with a torque exceeding 0.1962 N·m to guarantee adequate force for pen movement A suitable choice would be the Nema 17 motor, which offers a torque of 0.55 N·m.
Bearings and Joints: Select bearings and joints with high durability and low friction => choose Ball bearings
Materials: Select materials for the links and frame, choose Aluminum fiber to ensure stiffness and light weigh
Figure 27 Detail drawing (full version is attached)
IMAGE PROCESSING INTERFACE
Structure of the software
A block diagram visually represents the operation of a program or algorithm, making it easier to understand and follow its processing steps This method simplifies the comprehension of complex processes by breaking them down into clear, sequential components.
Figure 35 Diagram of blocks in the program
+ Capture: necessary photo-taking equipment
+ Sending file: images are sent over the internet to the server
+ Image processing: the server processes images and extracts coordinates or necessary information
+ Send information: the server sends coordinates information to the robot
+ Take action: robot receives information and takes corresponding action
Web interface of the project
Figure 36 Web interface on computers (left) and phones (right)
5.1.2 Communication between devices over the internet
Capture a photo of handwritten text or upload an image containing handwriting, which is then sent to the internet and processed by a server Once the processing is complete, the results are returned and displayed on your device or phone via the internet.
Image filtering and preprocessing algorithm
The main task of this function is to process the input image, filter noise and process it to produce an output image that is a processed binary image
The input image is an unprocessed image, that is, an image taken with a camera or an image uploaded from a device
The image will be converted from rgb color space to gray-scale
Next, through the DOG set, the details of the writing will be further highlighted, helping them separate from the color of the paper background
The image will then be converted from gray space to binary with black as (1) and white as (0)
Next, proceed to fix, correct, and invert the color of some white pixels between the black pixels, helping the writing lines become uniform, without holes
To enhance the robot hand's trajectory, the image's skeleton is extracted, reducing its width from many pixels to just 1 pixel.
5.2.1 Difference of Gaussians filter (DoG)
Figure 39 Difference of Gaussians filter flowchart
The input image is provided for processing
Two different values for the standard deviation of the Gaussian filter, named Sigma1 and Sigma2, are chosen These values determine the amount of blurring applied to the image
Gaussian Blur1 Filter: the image is passed through a Gaussian blur filter using Sigma1 This filter smooths the image by reducing high-frequency noise and detail
The Gaussian Blur2 Filter applies a second layer of smoothing to the image, utilizing a different Sigma2 value This filter enhances the image's softness, varying the degree of blurriness based on Sigma2, resulting in a distinct visual effect.
The Difference of Gaussians (DoG) method involves subtracting two images processed with different levels of Gaussian blur This subtraction accentuates the edges and features within the image by emphasizing the differences between the two blurred versions, effectively highlighting important details.
Filtered Image: the result of the subtraction is the filtered image This image typically has enhanced edges and reduced noise
Figure 40 Convert image to binary flowchart
The initial step involves validating the first sigma value (Sigma1) for Gaussian blur; if this validation fails, the process is halted If Sigma1 is valid, the next step is to verify the second sigma value (Sigma2).
The Gaussian blur is considered valid when the specified value is 68; if this value is invalid, the process will halt Upon validation, the system will then verify whether the threshold value for color filtering is also specified and valid.
If all the parameters is suitable with conditions, image will be process by DOG filter
Then, image will be convert to invert color to better perform the handwritting before apply the threshold to change to binary
Apply threshold to convert color finish, we will have a binary image of handwriting which better and more clear for next step
5.2.3 Fix white spots between black borders
Figure 41 Fix white spots between black borders flowchart
In this process, a binary image undergoes convolution using a 3x3 kernel, where the center point is assigned a value of 0 and the surrounding seven points are set to 1 This technique is employed to determine the presence of black pixels in proximity to the pixel under consideration.
When the value is 7 or higher, it indicates that at least 7 out of the 8 surrounding pixels are black in the original image Consequently, to eliminate small holes, you should set that pixel to 0 (white) in the “fix image.”
If value