INTRODUCTION
O BJECTIF
This internship focuses on the challenge of image segmentation, specifically seeking a simple yet effective method for processing graphic document images to distinctly separate the background from the object In essence, the goal is to develop a binarization technique that can automatically and efficiently determine the threshold for each pixel in the image.
S TRUCTURE DU RAPPORT
My report consists of five sections The first section provides an introduction to my internship The second section focuses on a general overview of image segmentation techniques, particularly binarization, along with brief descriptions of several methods The third section is dedicated to a detailed description of the proposed method The fourth section addresses the analysis of results and evaluation metrics Finally, the report concludes with a summary in the fifth section.
L IEU DE STAGE
Le LORIA, or the Lorraine Laboratory for Research in Computer Science and its Applications, is a Joint Research Unit (UMR7503) formed by several institutions, including the National Centre for Scientific Research (CNRS), the National Polytechnic Institute of Lorraine (INPL), the National Institute for Research in Computer Science and Automation (INRIA), and the Universities of Henri Poincaré Nancy 1 (UPH) and Nancy 2.
My six-month internship took place within the QGAR (Querying Graphics through Analysis and Recognition) team at INRIA Lorraine, under the supervision of Salvatore Tabbone, a lecturer at Nancy 2 University This team specializes in the analysis of documents with a strong graphical component, focusing on indexing and information retrieval in the context of technical documentation.
For more detailed information about the laboratory, visit http://www.loria.fr You can find a comprehensive presentation of the team at http://www.loria.fr/equipes/qgar and http://www.inria.fr/recherche/equipes/qgar.en.html.
ETAT DE L’ART
M ETHODES DE SEUILLAGE GLOBAL
An image's grayscale histogram is defined as a function h: [0 L - 1] → Ν, which maps each grayscale level between 0 and L-1 to the number of pixels in the image that exhibit that particular light intensity [Braviano, 1995].
An image's histogram can be represented as a vector, where each component indicates the number of pixels corresponding to a specific gray level This representation effectively estimates the probability density of pixel values within the observed image.
1 , 0 , ) (i =n i = L− h i , ó n i le nombre de pixels de niveau de gris i dans l’image
The Otsu method aims to determine an optimal threshold that maximizes the difference between two classes, utilizing variance as its foundation The optimal threshold, denoted as s optimal, is identified by maximizing one of the specified functions.
Où δ T 2 ,δ B T ,δ W 2 sont successivement la variance totale de l’image, la variance inter- classes (between-class variance) et la variance intra-classes (within-class variance)
: la moyenne totale de tous les points dans l’image
• p i : la probabilité d’occurrence du niveau de gris i dans l’image
• P fond (t),P objet (t): la somme des probabilités d’occurrence des niveaux de gris des pixels du fond et celle de l’objet en prenant le seuil t
• m fond ,m objet : la moyenne des pixels appartenant au fond et celle des pixels de l’objet
N M i h image l dans pixels de nombre i gris de niveau le dont pixels de nombre p i
• δ 2 fond (t),δ objet 2 (t) : la variance de la classe fond et la variance de la class objet
• [min, max] est l’intervalle dynamique de l’image
This method is easy to implement and generally yields good results However, when it comes to images of documents, the outcomes may lack clarity, leading to the potential confusion of two different objects.
2.2.2 Méthodes se basant sur l’entropie
According to information theory, entropy quantifies the amount of information within a system For a finite set S = {s1, s2, , sk} of independent events, where pi represents the probability of occurrence for each element si, entropy is defined accordingly.
Higher entropy leads to greater information retrieval Various image segmentation methods have leveraged this principle to enhance the quality of the final output.
Si l’ensemble S est considéré comme un ensemble flou avec le degré d’appartenance correspondant à chaque ộlộment dans S à S (s i ),i=1,k , appelộ la fonction d’appartenance (membership function), l’entropie floue de l’ensemble S est définie par :
The principle behind these methods is to identify an image partition that maximizes entropy Various techniques have been developed that focus on the variations in entropy of a partition Below, we will explore three examples specifically related to binarization.
( fond t i i fond fond objet t i i objet objet
Le seuil est choisi tel que la fonction H =H objet +H fond est maximale óp i est la probabilité d’occurrence du niveau de gris i dans l’image
Dans ce cas, la distribution de probabilité de l’objet P t et la distribution de probabilité du fond (1 - P t ) sont prises en compte en déterminant l’entropie de la partition
Méthode de Cheng et Chen [Cheng,1998b]:
Unlike the previous two methods, the entropy of a partition is calculated by considering the occurrence probabilities of subsets, including both the object and the background Additionally, fuzzy subset theory is incorporated into this calculation Therefore, in this context, the entropy of a partition is defined as follows:
/ (i et à objet fond est toujours la fonction d’appartenance du niveau de gris i à la classe objet / fond
Méthode de Mello et Lins [Mello,2000] :
C’est une méthode qui se spécialise pour l’image de documents historiques
Assuming that t is the most frequently occurring color in the image, we take this value as the initial threshold The entropy values for the object Hb and the background Hw are determined as outlined in [Pun, 1980] A pixel i, with the color color(i), will be classified as background if:
(i mw Hw mb Hb couleur ≥ + s inon il sera classé comme l’objet
Les deux facteurs mw et mb sont déterminés par expériences en évaluant l’entropie de l’image entière et dédiés particulièrement à un type d’images observé
Described in Gadi (2000), this method exemplifies the adaptive local approach It operates on a hierarchical principle to address the issue of non-uniform lighting Assuming that the image consists of only two classes—object and background—the core objective of this method is to maximize the retrieval of pixels belonging to the object class.
La méthode se compose de 2 étapes :
The construction of a quadtree involves sequentially dividing the original image into four increasingly smaller sub-images based on a homogeneity criterion Each sub-image corresponds to a node in the quadtree If a sub-image meets the homogeneity criterion, it becomes a terminal node, eliminating the need for further division If not, the sub-image is further divided into four smaller sections This process continues until all nodes in the quadtree are terminal nodes.
To satisfy the homogeneity criterion in a region, there must be no significant differences between that region and its four sub-regions This condition is assessed using Fisher's statistical test.
>F k f : sous-image est non homogène f : l’estimation du critère d’homogénéité sur la sous-image évaluée (voir partie 3.3 pour plus détaillé) α
F : la valeur prédéfinie de la distribution F avec 3 et 4(k-1) degrés de liberté
The membership degrees of all pixels are calculated at each level of the tree, where the mean and standard deviation are determined for the region containing the pixel (x, y) at level k.
Decision: After conducting various evaluations of each pixel's membership in one of two classes, Zadeh's t-conorm aggregation function is applied to determine the final membership measure for the object class.
( x y moyenne ecart type moyenne moyenne ecart type ) k S y x , ) ( , ); _ , , _
( = − + à et le degré d’appartenance final au fond :
- Défuzzification : il s’agit de mettre au point des pixels à deux classes
This method is designed to address the issue of uneven lighting in images However, it is most effective when the background is truly uniform; otherwise, background pixels may inadvertently affect the object being highlighted.
S EGMENTATION HIERARCHIQUE FLOUE
The image histogram of a document features two modes: a strong mode representing the background and a weak mode corresponding to the object However, the focus should be on the object's mode While a global thresholding method can effectively eliminate the background mode, it does not guarantee the accurate extraction of the object, which consists of lines and characters The distinction between the object and the background is often ambiguous, particularly in areas where characters and lines are in close proximity Therefore, the ultimate goal of our method is to achieve a clear and sharp representation of the object.
The proposed method integrates both global and local approaches, consisting of two main steps The first step employs global thresholding to eliminate the dominant background in the observed image while retaining the essential part containing the object The second step refines the results from the previous stage to enhance the clarity of the object, utilizing a variation of the adaptive local binarization method [Gadi, 2000].
Figure 3.1 : Principe de la méthode proposée
In the following sections, we will use the following notations: g(x, y) represents the grayscale level of the pixel (x, y) in the original image I, g I (x, y) denotes the grayscale level of the pixel (x, y) in the intermediate image I I, and g F (x, y) indicates the grayscale level of the pixel (x, y) in the final result image I F.
Une méthode de seuillage global nous aide à chercher un seuil pour toute l’image
In principle, any global thresholding method can be applied at this stage; however, opting for a simple method is often a wise choice Therefore, we have selected Otsu's method as a viable solution for this process.
METHODE PROPOSEE
E TAPE DE SEUILLAGE GLOBAL
Une méthode de seuillage global nous aide à chercher un seuil pour toute l’image
In principle, any global thresholding method can be applied at this stage; however, opting for a simple method is often a wise initial choice Therefore, we have selected Otsu's method as a viable solution in this context.
Instead of converting the result of this step into a black and white image, we will retain the original pixel values belonging to the object to create an intermediate image If g I (x, y) represents the brightness intensity of the pixel (x, y) in this image, then:
Figure 3.2 : Image originale – jaures_patie1.tif Figure 3.3 : Image intermédiaire de jaures_patie1.tif
Figure 3.4 : Image binaire de jaures_patie1.tif après la première étape
E TAPE DE RAFFINAGE
The image obtained in the first step successfully retains the area of interest However, the object lacks clarity, as the distinct parts of the object are not clearly defined due to a small number of pixels that should have belonged to the background.
In the image processing workflow, the initial step often results in unwanted pixels along the object-background boundaries To address this issue, a second treatment is necessary to eliminate these extraneous pixels This operation focuses solely on the object derived from the first step, manipulating the intermediate image while excluding background pixels that have a gray level of 255.
When attempting to apply a global threshold on an image, there is a risk of losing parts of the object with lower intensity values due to inconsistent illumination across the image To address this issue, it is essential to implement an adaptive thresholding method that considers local information to mitigate this effect This approach is grounded in the principles of quadtree structures and fuzzy subset theory The image will be progressively decomposed into smaller sub-images, assessing the homogeneity criterion If an image does not meet this criterion, it will be divided into four sub-images for further analysis.
Le processus appliqué, afin de re-affecter un pixel qui est déjà classé comme l’objet dans I I à la classe fond ou à la classe objet, se compose de 3 sous-étapes :
♦ Calcul de degrés d'appartenance de chaque pixel à chaque niveau de l'arbre
♦ Décision de degré final d'un pixel pour le classer au fond ou à l'objet
Let O Ri represent the set of pixels that maintain the original value of a specific rectangular region (sub-image) R i within the intermediate image I I In this context, R i can be viewed as a node in a tree structure, with R 0 serving as the root of the image I I.
From now on, all concepts and formulas related to the region O Ri will only be applied to the pixels within O Ri, as we focus solely on this area.
La hiérarchie associée à l'image I I de taille M x N est construite en divisant successivement cette image en sous-images de taille de plus en plus petite
- L'image I I est pris e comme la racine de l'arbre qu'on va construire Elle correspondant à un noeud au niveau 0
- Les noeuds au niveau k sont créés par des noeuds décomposables au niveau k-
1 Les noeuds décomposables sont ceux qui ne satisfont pas le critère d'homogénéité Un noeud décomposable au niveau k est divisé en 4 noeuds au
O = ( , )∈ ( , )≠255, ⊂ niveau k + 1 Ceux qui ne sont pas décomposables représentent des noeuds terminaux (des feuilles) de l'arbre Ce processus est répété jusqu'à ce qu’il n'y a plus de noeuds décomposables
Quand le processus de subdivision s'arrête, l'image originale est représentée par des noeuds terminaux
Figure 3.5 : Structure tridimensionnelle de l’arbre quaternaire
In the image segmentation process, it is essential to determine when to stop subdividing an image into smaller segments A region \( R_i \) is not further decomposed if all its pixels are classified as background, resulting in an empty set \( O_{R_i} \), thus designating \( R_i \) as a terminal node For other regions, the decision to stop depends on the relationship between the parent region and its four corresponding child regions Specifically, subdivision ceases when there is no significant difference in the means and variances between the parent region and its children To establish a threshold for this significant difference, a predefined value \( e \) is set To avoid issues with threshold selection, Fisher's statistical test is employed to assess the stopping criterion, enabling a comparison of means and standard deviations between the parent region \( R_i \) and its child regions \( R_{i1}, R_{i2}, R_{i3}, \) and \( R_{i4} \).
R i4 o ó σ σ j , j∈{1,2,3,4} et sont successivement les écarts-types calculés sur les données de 4 filles et de la mère m et j m j , ∈{1,2,3,4} sont les moyennes correspondantes m et j m j j , , {1,2,3,4} σ, σ ∈ sont calculées surO Rij , j∈{1,2,3,4} et O Ri
H e alternativ Hypothèse m m m m m et H null Hypothèse j j σ σ σ σ σ σ σ
Assuming that the four child sub-images of the parent image are independent and exhibit identical normal distributions of gray levels, the Fisher's F-test for homogeneity follows an F-distribution with parameters α, p; n - p - 1.
K : le nombre de pixels dans chaque sous image
In the context of image processing, Xjk represents the gray level of the k-th pixel in the sub-image j The parameter p denotes the degrees of freedom, where p equals 3, indicating the number of subsets minus one Additionally, n refers to the total number of pixels in the original image, which is 4K Lastly, α signifies the confidence level associated with the analysis.
Les valeurs de la distribution F sont indiquées dans un tableau de Fisher La décision d'homogénéité d'une région dépend de la comparaison f et F α p; n -p -1 α
≤F p n p f : L’hypothốse H 0 est ô vrai ằ La rộgion est homogốne α
>F p n p f : L’hypothốse H 1 est ô vrai ằ La rộgion est hộtộrogốne
A statistical test only holds significance if the sample size is sufficiently large Therefore, it is essential to determine the minimum sample size required for the test to be applicable This approach helps avoid the issue of over-segmentation.
En bref, la décomposition d’une région R i s’arrête si une des deux conditions suivantes est satisfaite Ri deviendra un nœud terminal
1)Card(O Ri )≤taille min ou bien
The cardinality of the set O Ri is crucial for determining the minimum sample size required for the application of statistical tests The necessary sample size is influenced by several parameters, including the confidence level α and the sample variance For our implementation, we have selected 40 as a detailed minimum experimental value.
Figure 3.6 : Quadrillage de l’image intermédiaire
3.3.2 Calcul des degrés d'appartenance de chaque pixel À chaque nœud de l’arbre, si la région correspondante n’est pas homogène, la théorie de l’ensemble flou sera appliquée pour la classification de ses données en deux sous-ensembles flous F (fond) & O (objet) en évaluant leurs degrés d’appartenance Cela signifie que ces degrộs d’appartenance à la classe objet à O k (x,y) et à la classe fond
(x y k à F de chaque pixel sont calculộs pour chaque niveau k de l’arbre
Etant une fonction la plus souvent utilisée, la fonction S de Zadeh est prise à calculer le degrộ d’appartenance à la classe fond d’un pixel Supposons que à F (x,y)et
(x y à O sont successivement le degrộ d’appartenance à la classe objet et celui à la classe fond du pixel (x, y) ayant le niveau de grisg I (x,y), ils sont déterminés comme suivant :
For estimating the parameters a, b, and c, local properties of the regions (or nodes) are utilized The dynamic interval of the region defines the uncertainty band as the range between the mean minus the standard deviation and the mean plus the standard deviation.
The degree of membership is absolute across the entire grayscale range, except for the fuzzy interval The membership degrees of a pixel (x, y) at level k are determined by the mean (m) and standard deviation (s) calculated from a sub-image corresponding to a node at that level k.
3.3.3 Décision de degré d'appartenance final
Après avoir calculé les degrés d'appartenance d'un pixel à tous les niveaux, il faut prendre une décision : parmi eux quelle est la valeur qui va décider la classification du pixel ?
O à à à à On s’intéresse tout d’abord au degré d’appartenance du pixel à l’objet
To ensure the quality of the final object obtained from processed data primarily consisting of object pixels, it is crucial to choose the appropriate membership function If the min function (Zadeh's t-norm) is used, the lowest value across all levels is rendered as the pixel's membership measure, which may result in losing parts of the object, especially when the gray level intensity is high Conversely, employing the max function (Zadeh's t-conorm) effectively preserves the object pixels while eliminating false pixels at the object-background boundary This approach allows each pixel to maintain its maximum potential membership to the object, leading to a more accurate final degree of membership Therefore, the t-conorm function of Zadeh, which selects the highest value, is preferred for determining the final degree of membership to the object.
O à à à à à à à = Et le degré d’appartenance d’un pixel au fond sera :
Alors, un pixel (x, y) va appartenir à la classe fond F si à F f (x,y) >à O f (x,y)et sinon il est mis comme un pixel de l'objet
Figure 3.7 : Résultat final de la méthode proposée sur l’image jaures_partie1.tif
EVALUAT IONS
M ESURES DE PERFORMANCE
To evaluate the effectiveness of the proposed method, we assess it based on two qualitative criteria for the obtained results, which are fundamental in image segmentation The first criterion is the contrast between classes, while the second criterion to consider is homogeneity.
In this section, we will compare the proposed method with other existing techniques, including Otsu's method, the Gadi and Benslimane method, and the Trier and Taxt method [Trier, 1995a] The performance metrics introduced by Levine & Nazif will be utilized to quantify contrast and homogeneity [Tabbone, 2003].
Où : m F : la moyenne des valeurs des pixels appartenant au fond m O : la moyenne des valeurs des pixels appartenant à l’objet
R i m : la moyenne des valeurs des pixels dans la région R i
#Régions : le nombre de régions dans l’image sans compter le fond
La valeur de C I indique le contraste entre les deux classes fond et objet Plus la valeur de C I est grande, plus le contraste entre deux régions est élevé
4.2.2 Mesure d’homogénéité ó : Card (objet) est le nombre des pixels dans la classe objet
The H I value signifies the level of homogeneity within regions, specifically reflecting the uniformity of pixels within the object class It is observed that a smaller H I value indicates a higher degree of homogeneity in the region.
Table 4.1 presents the contrast and homogeneity measurements for the proposed method, alongside results from Otsu, Gadi & Benslimane, and Trier & Taxt.
Mesure de contraste Mesure d'homogénéité
Méthode de Otsu Méthode de Benslimane Méthode de Trier & Taxt Méthode proposée Méthode de Otsu Méthode de Benslimane Méthode de Trier & Taxt Méthode proposée
Jaures.tif 96.5696 92.8239 108.962 23.4738 29.9696 14.7079 hachures.tif 145.458 135.453 150.088 29.1459 30.6727 14.7696 plan2.tif 130.191 124.822 118.183 149.366 39.1952 44.264 48.8416 25.5288
Ce tableau a montré des avantages de notre méthode
Through testing, our method has proven effective in the binarization of graphic document images It offers a solid solution for addressing the pixel classification issue in the blurred area at the foreground-background boundary.
De plus, la complexité temporelle de la méthode proposée n’est pas trop grande On ne peut pas faire des comparaisons avec les méthodes de seuillage global comme Otsu
There is a significant difference in execution time between the proposed method and the method by Gadi and Benslimane [Gadi, 2000] Table 4.2 illustrates the execution times of both methods when run on the same machine using identical images.
Tailles d’images Méthode proposée Méthode de Gadi et Benslimane
Tableau 4.2 : Comparaison du temps d’exécution
The final outcome of this method relies heavily on the success of the global thresholding step If the initial stage fails to retain the entire object within the image, no subsequent operations will yield a satisfactory final result.
Upon examining the original image and its result in figure 4.8a), it is evident that the background of the original image is quite uniform, making the outcome of the first step already satisfactory The binarization of this image does not require a second step, as the final result is inferior to the intermediate one, risking the loss of object details Unfortunately, a solution to verify the necessity of this step has yet to be identified In principle, checking the homogeneity of the set O Ri, in terms of low variance, could serve as a potential solution However, the automatic homogeneity testing for a region of the image remains an unresolved issue.
A VANTAGES ET INCONVENIENTS
During my six-month internship, I focused on image segmentation techniques, particularly binarization, and developed a binarization method for graphic document images based on my supervisor's suggestions After some initial unsuccessful tests, I proposed a cooperative two-step binarization method The experimental results demonstrated promising object-background separation for graphic document images Additionally, I conducted comparative studies of my method against others, such as Otsu's method, Cheng and Chen's entropy-based approach, and Gadi and Benslimane's method Notably, my method offers lower computational complexity compared to Gadi's method, which it builds upon.
When the image from the initial step exhibits excessive homogeneity (i.e., very low variance), the outcome of the subsequent step deteriorates, as the stopping condition for constructing the quaternary tree is no longer met In such cases, it is essential to assess the pixel homogeneity within each sub-region Ri before verifying the splitting condition to prevent false binarization While a predefined threshold for variance may serve as a solution for testing the homogeneity of a set, the challenge lies in selecting the appropriate threshold Therefore, we will seek a method that can either automatically define the threshold for each set or verify its homogeneity automatically.
Currently, the majority of computational time is dedicated to calculating the averages and standard deviations of all nodes in the tree By enhancing the efficiency of these calculations, we can achieve significant reductions in execution time Therefore, selecting a more suitable data structure will be one of our top priorities in our research perspective.
CONCLUSIONS
During my six-month internship, I focused on image segmentation techniques, particularly binarization, and aimed to develop a binarization method for graphic document images based on my supervisor's suggestions After several unsuccessful tests, I proposed a cooperative two-step binarization method The experimental results demonstrated promising object-background separation for graphic document images Additionally, we conducted comparative studies of our method against others, including Otsu's method, Cheng and Chen's entropy-based method, and the approach by Gadi and Benslimane Furthermore, the computational complexity of our method is relatively low compared to Gadi's method, which served as the foundation for our proposed technique.
When the initial image is too homogeneous, resulting in very low variance, the outcome of the subsequent processing step deteriorates because the stopping condition for constructing the quaternary tree is no longer satisfied To address this, it is essential to assess the pixel homogeneity in each sub-region before verifying the splitting condition to prevent false binarization While a predefined threshold for variance may serve as a solution for testing the homogeneity of a set, the challenge lies in selecting the appropriate threshold Therefore, a method that can automatically determine the threshold for each set or verify its homogeneity is sought.
Currently, the majority of computational time is dedicated to calculating the averages and standard deviations of all nodes within the tree By improving the efficiency of these calculations, we can significantly reduce execution time Therefore, selecting a more suitable data structure will be one of our top research priorities.
[Braviano,1995] Gilson Braviano ô Logique floue en segmentation d'images: seuillage par entropie et structures pyramidales irrộguliốres ằ, thốse de doctorat à l'Université Joseph Fourrier-Grenoble 1 Octobre 1995
[Cheng,1998a] H.D Cheng, C H Chen, H.H Chiu and Huijuan Xu ô Fuzzy Homogeneity
Approach to Multilevel Thresholding ằ IEEE Transactions on Image
[Cheng,1998b] H.D Cheng, Jim-Rong Chen and Jiguang Li ô Threshold selection based on fuzzy c-partition entropy approach ằ Pattern Recognition, vol 31, No 7, pp 857-870, 1998
[Cheng,1999a] H D Cheng, Yen-Hung Chen, Ying Sun ô A novel fuzzy entropy approach to image enhancement and thresholding ằ Signal Processing 75, pp.277-301,
[Cheng,1999b] H.D Cheng, Yen-Hung Chen ô Fuzzy partition of two -dimensional histogram and its application to thresholding ằ Pattern Recognition, vol.32, pp.825-843,
[Cheriet,1998] M Cheriet, J N Said, C.Y Suen ô A recursive thresholding technique for image segmentation ằ IEEE Transactions on Image Processing, vol 7, n.6,
[Gadi, 2000] T.Gadi, R Benslimane ô Fuzzy hierarchical segmentation ằ Traitement du signal, vol.7, n°1, 2000
[Horaud, 1993] R Horaud, O Monga Vision par ordinateur – outils fondamentaux Editions
[Kapur, 1985] J.N Kapur, P.K Sahoo, A.K.C Wong ô A New method for gray-level picture threshold using the entropy of the histogram ằ.Graphical Models and Image
[Kunt, 1993] Murat Kunt, Goesta Grandlund, Michel Kocher ô Traitement numộrique des images ằ, vol.2, 1993
[Mello, 2000] Carlos A.B Mello and Rafael D Lins ô Image segmentation of historical documents ằ http://www.upe.poli.br/dsc/recpad/site_hist/visual2000.pdf
[Otsu, 1978] N.Otsu ô A threshold selection method from grey-level histograms ằ IEEE
Trans Syst Man Cybern., vol.SMC-8, 1978
[Pun, 1980] T.Pun ô A New method for gray-level picture threshold using the entropy of the histogram ằ Signal processing, vol.2, n°3, 1980.