Digital watermarking robust to print and scan process

DIGITAL WATERMARKING ROBUST TO PRINT-AND-SCAN PROCESS Zhou Zhicheng NATIONAL UNIVERSITY OF SINGAPORE 2004 DIGITAL WATERMARKING ROBUTS TO PRINT-AND-SCAN PROCESS Zhou Zhicheng (B.Eng.(Hon.) in Computer Science, USTC) A THESIS SUBMITTED FOR THE DEGREE OF MASTER OF SCIENCE DEPARTMENT OF COMPUTER SCIENCE SCHOOL OF COMPUTING NATIONAL UNIVERSITY OF SINGAPORE 2004 Acknowledgement The writing of this paper has received deep concerns and valuable help from my advisors Dr. Qibin Sun and A/P Ee-chien Chang, whose kind encouragement, generous patience, and pertinent criticisms have guided me throughout the whole research work and the formation of this thesis. Their broad knowledge, deep insight, serious attitude and research enthusiasm benefited me a lot in conducting my research work and will surely guide me for the future challenge. I also want to show my sincere gratitude to my friends: Dajun He, Shuiming Ye, Zhishou Zhang, Zixiang Yang, Xinglei Zhu, and Rong Zhang, for their continuous encouragement, valuable discussion and kind advice. I will always remember the time we spent together with joy and harmony like a big family. Finally, allow me to show my deep thanks to my family for their consistent encouragement and support. Their deep love is the incentive force for my progress. Special thanks go to my wife for sharing my joy and sorrows all the time. i Publications Shuiming Ye, Zhicheng Zhou, Qibin Sun, and Qi Tian, A Quantization-Based Image Authentication System, Fourth International Conference on Information, Communications & Signal Processing, Fourth IEEE Pacific-Rim Conference On Multimedia, Singapore, December 2003. Qibin Sun, Dajun He, Zhicheng Zhou, and Shuiming Ye, Feature Selection for Semi-fragile Signature-based Authentication Systems, International Conference on Information Technology: Research and Education, Newark, New Jersey, August, 2003. ii Table of Content Acknowledgement..........................................................................i Publications ...................................................................................ii Summary ......................................................................................vi List of Figure..............................................................................viii List of Table .................................................................................xi Chapter 1 Introduction ................................................................1 1.1 Digital Watermarking .........................................................................................3 1.1.1 Overview of Digital Watermarking ...................................................................3 1.1.2 Models of Digital Watermarking .......................................................................7 1.2 Print-and-Scan Process .....................................................................................10 1.3 Motivation.........................................................................................................12 1.4 Organization of thesis .......................................................................................13 Chapter 2 Printer and Scanner .................................................15 2.1 Working Principles of Printer ...........................................................................15 2.2 Working Principles of Scanner .........................................................................21 2.3 Related Work on Watermarking .......................................................................23 2.4 Summary ...........................................................................................................26 Chapter 3 Noise of Print-and-Scan Process .............................28 iii 3.1 Introduction.......................................................................................................28 3.2 Properties in Pixel Domain ...............................................................................32 3.3 Properties in DCT Domain ...............................................................................37 3.4 Summary ...........................................................................................................44 Chapter 4 Watermarking Algorithm........................................46 4.1 Original DEW ...................................................................................................47 4.1.1 Watermark Embedding ....................................................................................48 4.1.2 Watermark Extracting......................................................................................52 4.2 Our Watermarking Scheme ..............................................................................53 4.2.1 Watermark Embedding ....................................................................................56 4.2.2 Watermark Extracting......................................................................................61 Chapter 5 Experiments and Discussions ..................................64 5.1 Side Effect of RST Distortion...........................................................................66 5.2 Optimal Parameters for Watermarking.............................................................69 5.2.1 Energy Threshold for Watermark Extracting ..................................................70 5.2.2 Quality Factor for Watermark Embedding ......................................................73 5.2.3 Energy threshold for watermark embedding ...................................................76 5.2.4 Cutting Index Cmin .........................................................................................77 5.2.5 Number of blocks in one GOB ........................................................................80 5.3 Performance Comparison .................................................................................84 5.3.1 Fidelity .............................................................................................................85 5.3.2 Robustness .......................................................................................................87 5.4 Summary ...........................................................................................................93 iv Chapter 6 Conclusion and Future Work..................................94 6.1 Conclusion ........................................................................................................94 6.2 Future Work ......................................................................................................95 v Summary After decades of development, the research of digital watermarking has advanced greatly in many ways. Many digital watermarking systems are robust to the ordinary image processing techniques like lossy compression, cropping, scaling, and rotation. However, digital watermarking that is robust to D\A and A\D transformation, such as print-and-scan process, still lacks full investigations. In general, the print-and-scan distortion includes two types of distortions: Rotation, Scaling and Transformation (RST) distortion and pixel value distortion. However, many existing research only focuses on the former distortion because the pixel value distortion is difficult to be perfectly modeled. In this thesis, we explore the property of pixel value distortion and present a watermarking scheme robust to this distortion. From the investigation on the noise property of the print-and-scan process, we find that the smooth blocks get less distorted during this process. Watermark embedded in these blocks is probably more robust to the print-and-scan distortion. However, protecting the texture blocks from distortion is also necessary because the watermark extractor within this blind watermarking scheme also uses these texture blocks. For this reason, we design the noise adding operation to compensate the print-and-scan distortion to the texture blocks. In addition, the watermark extractor also affects the whole system performance, so we use the multiple-vote operation to enhance the robustness of water extractor. The rules of the multiple-vote operation are specially designed based on the property of the print-and-scan distortion. The experiment results show that this watermarking scheme is robust to print-and-scan distortion for a variety of images having different texture contents. vi Furthermore, the comparison between this scheme and the differential energy watermarking scheme (DEW) shows that our special designed operations are effective for improving the watermark robustness to print-and-scan distortion. vii List of Figure Figure 1.1 Classification of Digital Watermarking ............................................................ 6 Figure 1.2 Watermarking Model I: Watermark as Noise ................................................... 7 Figure 1.3 Watermarking Model II: Watermark as Noise and Side Information ............... 8 Figure 1.4 Watermarking Model III: Watermark as Transmitted Messages ...................... 9 Figure 2.1 Basic Components of Laser Printers ............................................................... 17 Figure 2.2 Dot Gain Effect of Printers.............................................................................. 18 Figure 2.3 Gamma Correction. TOP: Gamma Error, BELOW: Gamma Correction ....... 20 Figure 2.4 Basic Components of Flatbed Scanners .......................................................... 22 Figure 3.1 The Print-and-scan Copy of 'Lena' with Supplemental Marks....................... 29 Figure 3.2 'Lena' and 'Baboon' .......................................................................................... 31 Figure 3.3 Projection Map Between Original and Scanned Images ................................. 33 Figure 3.4 Distortion Map Between Original and Scanned Images ................................. 35 Figure 3.5 Projection Maps of Different Copies of Scanned 'Lena' ................................. 36 Figure 3.6 Pixel Distortions Between Original Lena and Gaussian-Noised 'Lena' .......... 36 Figure 3.7 Zig-zag Scan Order of DCT Coefficients........................................................ 38 Figure 3.8 Absolute Distortions in Low & High Frequency ............................................ 40 Figure 3.9 Distortions of DC Coefficients, TOP: increased; BOTTOM: decresed.......... 42 Figure 3.10 The Blocks Whose Energies are Decreased by Print-and-Scan .................... 42 Figure 3.11 Magnitudes of Different Types of Distortions .............................................. 44 Figure 4.1 DEW Watermark Embedding Scheme............................................................ 48 Figure 4.2 Shuffled Images: 'Lena'(LEFT) & 'Baboon'(RIGHT) ..................................... 48 viii Figure 4.3 Creation of GOB in DEW ............................................................................... 49 Figure 4.4 DEW Watermark Extracting Scheme.............................................................. 52 Figure 4.5 Modified Watermark Embedding Scheme ...................................................... 57 Figure 4.6 Slices of the Shuffled Images: 'Lena' (LEFT) & 'Baboon' (RIGHT) .............. 58 Figure 4.7 PSNR of Different Print-and-Scan Images ..................................................... 59 Figure 4.8 Print-and-Scan Copies of 'Lena': Original(LEFT); Noise Adding(RIGHT) ... 60 Figure 4.9 Modified Watermark Extracting Scheme........................................................ 62 Figure 5.1 Bit Error Rates for Different RST ................................................................... 68 Figure 5.2 Watermarked 'Lena' and 'Baboon' ................................................................... 71 Figure 5.3 Bit Error Rates for Different G........................................................................ 72 Figure 5.4 PSNR of Watermarked Images for Different Q .............................................. 74 Figure 5.5 Watermarked Images for Q=45....................................................................... 74 Figure 5.6 Watermarked Images for Q=65....................................................................... 75 Figure 5.7 Bit Error Rates for Different Q........................................................................ 76 Figure 5.8 Bit Error Rates for Different E ........................................................................ 77 Figure 5.9 PSNR of Watermarked Images for Different Cmin ........................................ 79 Figure 5.10 Bit Error Rates for Different Cmin................................................................ 80 Figure 5.11 Watermarked Images for Different n. UP: M=8, BELOW: M=0.12 ............ 82 Figure 5.12 PSNR of Watermarked Images for Different n ............................................. 83 Figure 5.13 Bit Error Rates for Different n ...................................................................... 84 Figure 5.14 PSNRs for All Test Images ........................................................................... 86 Figure 5.15 Watermarked 'Flower's of DEW (LEFT) and Our Scheme (RIGHT)........... 87 Figure 5.16 Robustness Comparison for DEW and our Scheme on 'Lena' ...................... 88 ix Figure 5.17 Robustness Comparison for DEW and our Scheme on 'Baboon'.................. 89 Figure 5.18 Robustness Comparison for DEW and our Scheme on 'Fruit'....................... 90 Figure 5.19 Robustness Comparison for DEW and our Scheme on 'Flower' ................... 91 Figure 5.20 Robustness Comparison for DEW and our Scheme on 'Girl'........................ 92 Figure 5.21 Robustness Comparison for DEW and our Scheme on 'Women'.................. 92 x List of Table Table 3.1 Means of Different High Frequency Distortions .............................................. 43 Table 5.1 Parameters for Experiment of RST Side Effect................................................ 67 Table 5.2 Bit Error Rate for Different RST ...................................................................... 67 Table 5.3 Parameters for Experiment 1 ............................................................................ 70 Table 5.4 Parameters for Experiment 2 ............................................................................ 73 Table 5.5 Parameters for Experiment 3 ............................................................................ 76 Table 5.6 Parameters for Experiment 4 ............................................................................ 77 Table 5.7 Bit Error Rates for Different Cmin................................................................... 79 Table 5.8 Parameters for Experiment 5 ............................................................................ 80 Table 5.9 PSNR of Images in Different n......................................................................... 81 Table 5.10 Parameters for experiment 6 ........................................................................... 85 xi Chapter 1 Introduction The great development of the computer network in the past decades, including LAN, WAN, Internet and wireless network, enables fast and convenient communications between people. At the same time, the duplicate or edit of the multimedia objects in high quality and fidelity becomes easier as the digital multimedia technologies advance. However, this ease of distributing and modifying the media objects introduces new problems to the copyright protection. Cryptography is the first technology applied to protect the transmitted digital multimedia content. The basic idea behind cryptography is to encrypt the plaintext to the ciphertext, and then send it to the receiver. Only the receiver having the corresponding decryption function can get the plaintext back from the ciphertext. However, this framework only protects the information during transmission. After the multimedia object is decrypted, it may still be freely distributed by the receiver. Therefore, this object is not under protection any more. Digital watermarking is the technology for complementing the shortage of the cryptographic framework. Its general idea is to embed the identification 1 information into the multimedia content and associate them tightly. By this means, the distributed multimedia content always carries the embedded information. Digital watermarking and cryptography have very different requirements, as they have different goals. For cryptography, robustness refers to security, which evaluates how good the algorithm is for protecting the encrypted information from being known by the attackers who have no decryption keys. Nevertheless, robustness in digital watermarking has different meanings: protecting the embedded watermarks from being removed when the image content is not changed. In other words, the attackers to the digital watermarking have no interest in knowing the content of watermark, but want to make it fail to work properly. Research on the robustness of the watermarking system has been greatly investigated for all multimedia objects such as audio, image and video. For image watermarking, many schemes guarantee that the watermarking is robust to ordinary image processing operations such as scaling, cropping, rotation, transformation or lossy compression. However, little work has done on the image watermarking robustness to print-and-scan process. In some cases, the print-and-scan is also thought as the acceptable image processing techniques as it does not change the image content greatly. This thesis tries to understand the print-and-scan process and aims to design a watermarking scheme robust to the distortion comes from this process. This chapter begins with an overview of digital watermarking, and then introduces the mathematical models of watermarking. After that, the related research on digital watermarking robust to print-and-scan process is introduced. Finally the organization of the whole thesis is given. 2 1.1 Digital Watermarking 1.1.1 Overview of Digital Watermarking The history of watermarking can be traced back to several hundred years ago, when it was used mainly for trademark or decoration [1]. At that time, the watermark was created using chemical or physical methods during the process of paper manufacturing. It was not until 1954 when Emil Hembrooke invented the first digital presentation of the watermark [2]; and Komatsu and Tominaga used the term digital watermarking for the first time in 1988[1]. Then in the 1990's, interest on this research topic began to boost because the easy distribution of the media objects in computer networks introduces new problems on the copyright protection. After the decade of development, the application of digital watermarking is no longer restricted on the copyright protection. More and more applications of the digital watermarking are proposed, such as ownership protection, data monitoring and tracking, data authentication, and copy control etc. Although digital watermarking uses the term "watermarking", it only shares the invisible property with the traditional watermarking. The digital watermarking has more requirements than the traditional watermarking as it is used digitally. In fact, digital watermarking uses the similar techniques as the information hiding [3], which were utilized for secret communication between armies in wars thousands of years ago. Although Information hiding, steganography and digital watermarking are similar techniques in embedding information into multimedia objects, they have different goals and applications, resulting in different requirements. For example, commonly robustness is the main focus of digital watermarking, capacity is the main focus of information hiding 3 and secret communication is the main focus of steganography. [1, 3, 4] give the detailed explanation and comparison of these terms. Note that the requirement of a watermarking system is application-oriented; and some digital watermarking applications may take other factors into consideration but not only restrict in the robustness. In this thesis, we mainly focus on the robustness while keeping the appropriate watermark capacity and visual image quality. There are many perspectives to category the digital watermarking technologies. Figure 1.1 gives a coarse classification. Different types of media require different digital watermarking technologies. For instance, the real-time videos and audios need the fast watermark extracting, while images have less requirements on the speed of watermark extraction. Authors of [5] and [6] has detailed a comparison on the technologies for these media types. Though watermarking technologies can be classified by the property of perception, perceptible watermarking is only used in image and video applications but never used in audio. The reason comes from the different perception properties between the Human Visual System (HVS) and the Human Auditory System (HAS). HVS samples the multimedia object by a scalable means, perceives the information globally and neglects the locally visual distortion. But HAS samples every local signal instantaneously in a detailed scale, thus it is very sensitive to even locally audible distortion. Perceptible watermarking, or, visible watermarking, simply overlays a visual pattern on the image or video for watermark. This pattern generally is a trademark or a copyright logo, which claims the copyright ownership. The disadvantage of perceptible watermarking is that it distorts the original image or video content greatly. For this reason, imperceptible techniques become more and more popular. Imperceptible watermarking embeds the 4 watermark to the locations to which HVS or HAS is not sensitive. Then the modification introduced by the watermark embedding probably cannot be perceived by HVS or HAS. Spatial domain and frequency domain watermarking schemes differ in their embedding domains. In general cases, the frequency domain watermarking is more robust than the spatial domain watermarking. Embedding a watermarking bit in frequency domain is similar to distributing the energy of this bit uniformly to different pixels within spatial domain. Then the local spatial distortion only damages the watermark bit partially; and the information of this bit hidden in other locations is probably enough for correct watermark extraction. We will narrow our scheme within the imperceptible image watermarking in frequency domain. For different applications, the requirements of digital watermarking are different. The most common discussed properties are fidelity, capacity, robustness, and security. Fidelity refers to the perceptual similarity between the watermarked version and the original version of the host data; capacity refers to the amount of information that can be embedded within the host data without unacceptable distortion; robustness refers to the correct watermark detection even after the acceptable signal processing operations are done to the watermarked media; and security refers to the resistance to the malicious attack methods like the unauthorized removal, embedding, and detection. Although the tradeoff between these properties varies for different applications, robustness is fairly a common requirement for most of the watermarking applications. We are mainly concerned with the robustness, as well as the capacity and fidelity in this thesis. 5 Figure 1.1 Classification of Digital Watermarking After decades of research on digital watermarking, more and more applications are proposed and investigated. In general, these applications include ownership protection, data monitoring and tracking, data authentication, fingerprinting, copy/access control and media indexing etc [1, 2, 7]. Ownership protection, or copyright protection, aims to prove the ownership for a multimedia object by embedding the owner's information into it. How to identify the true watermark if more than two watermarks are embedded in the media is a problem for this application. The purpose of data monitoring and tracking is to find out whether the data is transmitted or broadcasted as required. For example, the total time of the advertisement broadcasted by a radio station or a television station must be summed for billing purpose. Data authentication determines whether the host media has been tampered. In authentication, the main concern is to prevent the tampered media object with a forged watermark to pass the authentication. Fingerprinting embeds different watermarks in the copies of a media object and distributes them to different customers. When any customer distributes his/her copy illegally, the owner of the multimedia object 6 can find out this pirate. Access control aims to restrict the user's access to the protected data according to his or her privilege. For example, the access control detector in the Video Cassette Recorder (VCR) can prohibit the VCR to record a watermarked video, when the embedded watermark declares this video as "never-copy". Indexing embeds distinct labels into the video contents, to facilitate the finding process of these media objects. 1.1.2 Models of Digital Watermarking In general, digital watermarking is modeled mathematically as a kind of digital communication [1, 8, 9, 10]. Watermarking system itself is a communication channel and the watermark is the information that is transmitted through this channel. To well understand the prototype of the watermarking systems, the discussion on their common models is necessary. Cox gives three communication models for digital watermarking in [1]. Among these models, the cover work plays different roles as pure noise, side information or transmitted message; watermark embedder and watermark detector play the same roles as modulator and demodulator. Figure 1.2 Watermarking Model I: Watermark as Noise 7 In the first model (Figure 1.2), the media object, i.e., the cover work C0, is thought of as the noise to the communication channel. Then the watermarked media Cw is the addiction of the encrypted watermark m and the cover work C0. The signal processing imposed on the watermarked media Cw is considered as another adding noise n to the communication channel. Watermark detector tries to eliminate the noise and get an estimate message mn of the input message m. If the cover work is known to the detector, this detector is called informed detector, otherwise it is called blind detector. The watermarking system having a blind detector is called blind watermarking system. Figure 1.3 Watermarking Model II: Watermark as Noise and Side Information In the second model (Figure 1.3), cover work C0 is regarded as the side information known to the transmitter of a communication channel. The embedding and detecting procedures of this model are similar to the first one. Similarly to the first model, the cover work C0 is still considered as the noise, but its noise effect can be minimized because the encoder also takes it as the side information. The encoder can expect and invert its noise effect in the watermark embedder side before it is added to the channel [1, 8]. 8 Figure 1.4 Watermarking Model III: Watermark as Transmitted Messages In the third model (Figure 1.4), cover work is no long considered as the noise of the channel, but another message to be transmitted along with the watermark message through the channel. This watermarking model multiplex watermark and cover work into the same transmission channel, and de-multiplex them at the receiver. Traditional communication model only has one de-multiplexing module, which separates the messages by different values of a specific parameter. The common parameters used in the multiplexing communication are time, frequency, or code sequence. Nevertheless, watermarking model uses two independent modules, i.e. the human perception and the watermark detector, to de-multiplex the transmitted information. Besides these communication models, Cox suggests a geometric model for digital watermarking [1], in which the watermark and cover work are regarded as vectors in high-dimensional spaces. The watermarked object is the cumulative vector of the watermark vector and the cover work vector. That is, watermarking can be considered as moving some vector to a specific space. In this thesis, I would like to model watermarking as a multiplexed communication. 9 1.2 Print-and-Scan Process As we are concerned with the digital watermarking robust to print-and-scan process, we need to know what the print-and-scan process is, and the property of the noise it adds to the image. The whole print-and-scan process is: printing the image of digital format onto a paper, and scanning this paper to get back the digital image. Intuitively, this process is a kind of D/A A/D transformation, and the noise of this process comes mainly from the transformation between digital format and analogue format. Nevertheless, because many uncontrolled and unexpected distortion is introduced in the print-and-scan process, the model of print-and-scan noise is more complex than the simple D/A A/D transformation. In general, the print-and-scan process includes two kinds of distortions: geometric distortion and pixel value distortion [17]. Geometric distortion is a severe attack to the synchronization of a blind digital watermarking system. Typical geometric distortion includes rotation, scale, and translation (RST) applied globally to the whole image. These operations do not change the image content, but only change the image's orientation, scale or position. Besides these global geometric distortions, Khun and Petitcolas proposed a benchmark StirMark to attack the watermark by the local geometric distortions while keep the image content with no globally visual alternations [11]. Much research work has been performed on global RST distortions, and these techniques generally fall into two categories. The first category inverts the geometric distortion and then extracts the watermark [12, 13, 14]. Pereira et al proposed to embed a template into the middle-frequency FFT spectrum of the image for reference; then inverted the geometrical transformation according to the change of this 10 template and extracted the watermark [13]. Kutter proposed to repeat the watermark bits to different locations in a pattern; then localized this pattern by a correlation function, identified and inverted the affine transformation, and extracted the watermark [14]. The second category finds a transform space that is invariant to the RST distortion and embeds the watermark directly within this space for better synchronization [15, 16]. Ruanaidh and Pun proposed to embed watermark within the Fourier-Mellin transform [15]. To get this transform space for an image, Fourier transform (FFT), log-polar mapping (LPM), and Fourier transform (FFT) are taken in sequence. Nevertheless, the basic idea of this technique is based on the continuous FFT, which is hard to calculate for discrete format images. Lin et al proposed another method to get the weaker but simpler invariant domain for watermarking in [16]. Similarly to Pun's scheme, this scheme also takes the FFT and LPM in the first stage, but after that a projection to the angle axis in log-polar coordinate rather than another FFT is taken. In this domain, the image is invariant to translation and scaling, and cyclic-shift to rotation. Although the image in this domain is not invariant to rotation operation, the watermarking detector in his scheme knows the original watermark, and an exhaustive search can find the translation angle to compensate the rotation operation. In contrast to the plenty of research on RST, there is less research investigating the properties of pixel value distortion and the watermarking robust to it. Lin gave a general model for the pixel value distortion of print-and-scan process in [17]. Although the detailed formulas of this model are not given, all types of noise involved in the print-and-scan process have been taken into consideration. Nevertheless, this model needs the calibration for different setting of printer and scanner. Thus it is not common for all the 11 printers and scanners. Zhu et al [18] proposed to use the pixel distortion of print process as the signature for authentication of the printed document, because the distortions to the same pattern in different print process are unique [18]. This reveals that the print-and-scan distortion is time-variant. The security of their schemes is based on truth that the resolution of the microscope used for watermark extraction can be very high, while the printer can not provide such resolution to counterfeit the unique printed pattern. This asymmetry guarantees that the extraction of the print signature is easy to implement while the reproduction is almost impossible. In general, the pixel value distortion of print-and-scan is a nonlinear, content dependent and time-variant function. To reduce or invert its affect like what was done to geometric distortion in detailed granularity is difficult. Nevertheless, as the human eyes can recognize an original copy and a print-and-scan copy of the same image to be similar, it is intuitive that some feature space within an image can be robust to print-and-scan. In other words, in the print-and-scan context, feature space should be found in a coarse granularity. 1.3 Motivation There are many applications for watermarking robust to print-and-scan process. The authentication of documents needs this technique for some cases, such as authentication of passports, cheques, E-tickets or ID-cards. These types of images are simple in content, providing more space for embedding. The copyright protection for documents also needs this technique. For example, the photographers publishing their pictures of high quality in magazines might hope to claim their ownership. Or, the publishers purchasing the whole copyright of the photos might also hope to protect their copyrights and guarantee that these 12 photos are published under their permission. But there are few techniques to support these applications currently. On the other hand, the research on this topic still needs further investigations. Currently, the research that is related to the print-and-scan distortion mainly focuses on the rotation, scaling and transformation (RST) distortion, but not the pixel value distortion. However, a watermarking system supporting the above applications can not ignore this type of distortion. Thus the research on this topic is important and valuable. In this thesis, we intend to understand the properties of the pixel value distortion, and design a blind watermarking system that is robust to this print-and-scan distortion. As RST distortion is not our concern, we assume it is erased by other means before watermark extracting. For the design of watermarking system, our strategy is to find the stable feature space in the images for the print-and-scan process, and design a method to blindly embed the watermark in this space. To focus our research on the print-and-scan distortion, we only consider the watermarking for grayscale images. 1.4 Organization of thesis The whole thesis is organized as follows. Chapter 2 gives a detail introduction on the working principles of the printers and the scanners. In addition, the printer model and scanner model are discussed in this chapter. In chapter 3, we will discuss the property of the print-and-scan noise, based on the printer and scanner models. The results of the chapter 2 and chapter 3 give the basis to design a robust watermarking system. Chapter 4 discusses our watermarking system. This watermarking system is a variation of the Differential Energy Watermarking (DEW). Chapter 5 presents the experiment results to 13 evaluate the robustness of our watermarking system to print-and-scan distortion. The results include the robustness performance of our watermarking system for different images, and the comparison on the robustness between the original DEW and our system. 14 Chapter 2 Printer and Scanner To design a watermarking scheme robust to print-and-scan process, we need to know the working principles and the mathematical models of printers and scanners. In this chapter, we first introduce the working principles of the printers and scanners, then their mathematical models, and finally the previous research related to our desired watermarking scheme. Printers and scanners are the image input and output systems respectively. The optical principles of these imaging systems are presented in [19]. And the structural components and their functionalities are described in [20, 21]. To understand how they work, we need to know their internal components and how they coordinate. Based on the distortions introduced by different components within printers or scanners, we can get the distortion models. 2.1 Working Principles of Printer Printers can be categorized in several ways. For example, impact printers work by hitting a head or a needle against an ink ribbon to make a mark on the paper; while non-impact 15 printers exploit other techniques like the physical theories in optics and electronics to avoid these mechanical strikes[22]. Dot-matrix printers, daisy-wheel printers, and line printers belong to the impact printers, which are noisier than the non-impact printers like inkjet printers and laser printers. On the other hand, printers can be classified as contone printers and halftone printers [19]. Contone printers can print the continuous tone images directly, but they are more expensive. Halftone printers change a continuous tone image into a binary image (i.e. halftone image) and then print this binary image. Halftone printers are more popular because they are cheaper than contone printers while providing a similar quality. Because of their popularity, we will choose the non-impact half tone printers for experiment. In particular, we will use the laser halftone printers. Figure 2.1 illustrates the major components of a laser printer and its six-step print process [20]. Firstly, the photoreceptor drum is uniformly charged by the corona wires (1); then partially discharged by the laser beam (2). The laser beam is generated based on the image or the document that is printed, and the partially discharged area on the drum represents the printing area of the binary image. After that, the toner hopper coats the drum surface with the positively charged toners, which only adhere to the discharged area (3). The drum rolls over a paper that is negatively charged by another set of corona wires below it (4). As this charge is stronger than the negative charge on the drum in step (1), the toners are pulled away from the drum and adhered to the paper. The paper carrying the adhered black toners is then sent to a fuser, which melts the toners and fuses them with the paper tightly (5). In the last step, the charge on the surface of the drum is cleared up by a discharge lamp (6). The drum is then prepared well for the next printing. The optical and electronic devices, including the mirror, the laser generator and the corona wires, are the 16 delicate equipments. They do not generate noise during the printing process. But they are very sensitive to small movements. Even a very small dithering of the mechanical devices could introduce distortion to the printed images. This mechanical distortion is the random noise independent to the printed images. Figure 2.1 Basic Components of Laser Printers Besides the hardware distortion from mechanical devices, a printer also introduces the software noise of the digital-to-analog conversion. The laser printer uses digital halftoning technique to convert continuous-tone images to the half-tone images, which are represented by black and white dots only, before they are sent for printing. The laser beam is modulated based on this binary image, but not the original image of continuous tone. The image quality of a halftone image is similar to its original image, because the Human Visual System (HVS) tends to spatially integrate the intensities of the neighboring pixels 17 within an image. Haltoning represents the gray intensity of a pixel within the original image by a group of small dots. The gray intensity decides how the dark dots and white dots are distributed in this group. Details about choosing the pattern of black and white dots can be found in [23]. Viewing a halftone image, the HVS then integrates all the dots within this small group to recover the original gray level information of this pixel. Thus we perceive an average reflectance value R, which can be modeled by the Murray-Davies equation: R = F ⋅ RB + (1− F ) ⋅ RW (2.1) where F is the weighting factor, calculated as the fractional area coverage of the black dots, RB and RW is the reflectance values of black dots and the white paper respectively. The function of (2.1) is also called the tone transfer function. Given the pixels having the same gray intensity, F varies with respect to other surrounded pixels of different gray levels. The halftoning noise then is dependent on the image content, which is different to the independent noise of the mechanical devices. Figure 2.2 Dot Gain Effect of Printers If the printing process is perfect, the tone transfer function R in (2.1) is a linear function of the weighting factor F. However, in practice the tone transfer function is a nonlinear function, and the printed image is darker than expected (Figure 2.3 up). This is caused by another distortion called dot gain, which is defined as the dot-size increase when 18 the image is printed (Figure 2.2). In general, there are two types of dot gains: mechanical dot gain and optical dot gain. Mechanical dot gain (Figure 2.2 middle) comes from the physical spreading of the toners as they are fused into paper. Optical dot gain (Figure 2.2 right) refers to the optical growth of the dot-size, which comes from the trapping of the reflecting lights beneath the printed dot. The light (e.g. Light A) projected to the dot directly will be absorbed entirely and get no reflections. However, the reflections of some lights projected near the dot are also possible to be absorbed. For instance, light B in Figure 2.2 enters the paper, scatters laterally, and emerges under the dot. Thus, the dot has a higher probability to absorb the light than one would expect from the physical dot-size. It has an optical size larger than the physical size. Compensation of the dot gain is similar to gamma correction in CRT display device, because the transfer function of dot gain resembles the voltage-to-light curve, which is a power-law transformation, of a CRT [24]. The power-law transformation has the common form [25]: s = cr α (2.2) where c and α are positive constants. For common gamma errors of CRTs, 2.5 > α > 1.8 (Figure 1.1 Plot2), which will darken the image to lose the details. To invert the gamma errors, gamma correction calculate r ' = r device. Then s = cr 'α = cr 1 ⋅α α 1 α (Figure 1.1 Plot3) and sends r ' to the display = cr , the transfer function changes to a linear one after gamma correction. Figure 1.1 illustrates effect of gamma and gamma correction employed to the example image. In such cases, the tone transfer equation taking dot gain into account is modified as: R1 α = F ⋅ RB1 α + (1 − F ) ⋅ RW 1 α (2.3) 19 which is called Yule-Nielsen equation. Although this model seems perfect for correcting the dot gain, the calibration is needed for printers to find the different suitable parameter α . The noise introduced by gamma correction is not only content-dependent, but also nonlinear. Figure 2.3 Gamma Correction. TOP: Gamma Error, BELOW: Gamma Correction In [26], a noise model is proposed, which takes all the noise of halftoning, dot gain and mechanical dithering into consideration. However, this noise model is complicated and needs the calibration before it is employed to a printer. The calibration requires that the original image is available. This requirement, however, conflicts with our goal of designing a blind watermarking system. 20 2.2 Working Principles of Scanner There are also many types of available scanners, such as flying-spot scanner, drum scanner, and flatbed scanner, all of which are different from their mechanical sub-systems [27]. Most desktop scanners use charge-coupled device (CCD) arrays, photo-multiplier tubes (PMT) or Contact Image Sensor (CIS) as the light sensors. As CCD-based flatbed scanner is popular, we only discuss the principles and models of this kind of printer. The basic components of the flat bed scanners are the Charge-Coupled Device (CCD), lamp, and the analog-to-digital converter (ADC) (Figure 2.4). During the scanning process, a scan head inside the scanner moves across the flatbed slowly from one side to the other side. The moving scan head then samples the image to reconstruct its digital format. The scan head is made up with cathode fluorescent lamp, mirrors, lens and CCD array. The lamp illuminates the lights to the scanned image, and the reflection light is led to the CCD array by the mirrors and the lens. CCD is a collection of tiny diodes, which are able to convert photons (light) into electrons (electrical charge). The brighter the light that hits on a single diode, the greater the electrical charge will be generated at that diode. After the CCD array absorbs the lights reflected by the images, it generates the corresponding electrical voltage signals and sends them to an analog-to-digital converter (ADC). ADC then converts the analog signals into digital signals. The image processor processes the received digital signals to reconstruct the image and then send it to the PC for further processing. 21 Figure 2.4 Basic Components of Flatbed Scanners From the scanning process, we can find that the scanners noise comes from the following sources: vibration or movement of mechanical sub-system, dirt or bias of the optical sub-system, illuminant unbalance of the lamp, sensitive distortion of the CCD and sampling distortion of ADC. [19] gives a scanner sensor model for the response at a single spatial location: tis = ∫−∞ f i (λ ) d (λ ) r (λ ) ls (λ ) d λ + εi ∞ (2.4) where f (λ ) is the spectral transmittance of the color filters, d (λ ) is the sensitivity of the detector (i.e. CCD) used in the measurement, ls (λ ) is the spectral radiance of the illuminant(i.e. lamp), r (λ ) is the spectral reflectance of the area being scanned, εi is the measurement noise, and tis denotes the value obtained from the ith channel. This model concerns the light and color parameters in the high degree of details. In [27], the author gives a coarser model for the CDD array output: 22 g ( x, y ) = ⎡⎣ f ( x, y ) ∗ h ( x, y )⎤⎦ ⋅ s ( x, y ) (2.5) where f ( x , y ) is the input image, h ( x , y ) is the sensor aperture response, and s ( x, y ) is the sampling function. Actually, g ( x, y ) is the convolution of f ( x , y ) and h ( x , y ) , which is then multiplied by the sampling function s ( x, y ) . Both models suggest that the scanning process is similar to convolution, where the high components of the images are erased. That is, the scanner produces a blurred copy for the printed document. These models indicate that the scanner noise is content-dependent and nonlinear, which is similar to the printer noise. 2.3 Related Work on Watermarking Due to the variations in the print-and-scan process, the research in literature only partially solve the problems related to the watermarking that is robust to print-and-scan process. As previous research on the rotation, scaling and transformation (RST) has been introduced in chapter 1. In this section, we will review the related work exploring the pixel value distortion of print-and-scan process. Cox [10] proposes a watermarking scheme exploiting the idea of spread spectrum idea used in digital communication. The watermark bits wi are embedded to the coefficients ci in the frequency domain with the strength α by: ci ' = ci (1 + αwi ) i = 1,..., n (2.6) To enhance the robustness of the watermarking scheme, ci are selected from the n most significant coefficients of the frequency domain. The watermark then is extracted by 23 wi * = (ci * − ci ) α (2.7) where ci* is the coefficients at the same location as ci . Note that it may be distorted by other sources before the watermark extraction. This method is claimed to be robust to many attacks including the pixel value distortion of the print-and-scan process. However, the assumption of this watermarking is very strong: the original image must be available when the watermark is extracted. Under this assumption, the unwanted noise can be erased by referring to the original image. Thus this assumption is impractical for many applications. The techniques similar to Cox's scheme are proposed in [28, 29]. To develop a watermarking system robust to print-and-scan process, Lin models the pixel distortion in print-and-scan process in [17]: I '( x , y ) = K ( I ( x , y ) *τ ps ( x, y ) + ( I ( x, y ) *τ h ( x, y )) ⋅ N 1 ) ⋅ s( x, y ) (2.8) I(x,y) is the intensity value at the position (x, y) of the original image I, I'(x,y) is the intensity value of the distorted image. τ ps ( x, y ) is the global point spread function, which is the convolution product of the printer spread function τ p ( x, y ) and the scanner spread function τ s ( x, y ) : τ ps ( x, y ) = τ p ( x, y ) *τ s ( x, y ) (2.9) τ h ( x, y ) is the high-pass filter, representing the high noise variance near the edges and N1 is a white Gaussian random noise. Lin claims that τ h ( x, y ) is not symmetric in the vertical and horizontal directions, since the scan head moved by a step motor will introduce the directional jitter noise. s ( x, y ) is the sampling function, K(I) is the responsive function: 24 K ( I ) = α ⋅ ( I − β I )γ + β K + N 2 ( I ) (2.10) This function models the noise of A/D, D/A and the gamma adjustments in the printer and scanner. This model is similar to the printer and scanner models mentioned in section 2.1. Lin proposes to estimate the parameters α , γ , βx and βk in equation (2.10) by the optimal MSE(Mean Square Error). The watermarking scheme in this paper is designed mainly for the robustness to RST distortion. It is similar to O'Ruanaidh 's in [15], which embeds the watermark in the Fourier-Mellin transform domain of the image. It differs from O'Ruanaidh 's scheme in using the discrete-fourier transform instead of the continuous fourier transform when calculating the Fourier-Mellin transform. However, this method also needs the original image for reference to calibrate the estimated parameters of the print-and scan models. Lin’s experiment results mainly focus on the rotation, scaling and cropping (RSC) distortion of print-and-scan, and lack the evaluation on the system robustness to the pixel value distortion. In [18], Zhu et al design an authentication system by utilizing the random property of the printing process. Though this watermarking system is not designed for the robustness to print-and-scan, it demonstrates the time-variant property of the print-and-scan process. It is claimed that the randomness in the printing process of the same content is non-repeatable, and propose to use this unique randomness as the robust feature to embed watermark for document authentication. To embed the watermark, an predefined print pattern is firstly printed out on the document; then a digital microscope is used to extract the print signature from this printed pattern; the print feature is then combined with the critical information about the document to generate the specific digital signature (specific print and specific document); this digital signature is finally printed as 25 the barcode on the document finally. To authenticate the document, we need to repeat the above feature extraction on the printed pattern to get the new extracted feature, which is then combined with critical information to generate the new digital signature. This new digital signature is then compared with the old one that is printed as the barcode to get the final authentication decision. The security of their digital watermarking framework is provided by the asymmetry of examining and reproducing the printed pattern. It is easy to get the shape profiles (i.e. extracted signature) of the printed pattern by the cheap microscope with high resolution. But a printer that is able to reproduce the same printed pattern needs extremely high resolution, which can not be achieved even by the best printer at present. Therefore, the leakage of security leak seems impossible in a practical sense. However, this method is very sensitive as a high resolution microscope is used to get the extracted signature. Every unintentional distortion to the printed pattern on an authenticated document would lead to authentication failure. For example, if the E-ticket is folded just across the printed pattern, then its new extracted feature is probably different from the original one and the authentication fails. Furthermore, the printed pattern needs to be protected during watermark embedding. The second print that generates the bar code of the digital signature would probably distort it. This paper reveals the time-variant property of the print-and-scan noise. This noise is not integrated into the printer and scanner models introduced in the previous sections of this chapter. 2.4 Summary In this section, we introduce the working principles of the laser-printers and the flatbed CCD-based scanners and highlight their mathematical models suggested by different 26 researchers. We also discuss some other digital watermarking schemes related to the print-and-scan process. From the above discussion, we can find that the noise model of the print-and-scan process is complex. It is a type of content-based, nonlinear and time-variant noise. It is impossible to totally erase this noise if the original image is not available. Even the print-and-scan model can only estimate the noise, and needs the original image for calibration before it is put into use. Therefore, to design a blind watermarking system, we can not use these models directly. At least, the strategy to erase the noise or to calibrate the models is unacceptable. We choose another strategy by looking for the stable feature space within the images for print-and-scan noise and them embedding the watermark into this space. This type of feature space should get the least distortion from the print-and-scan process, and the watermark embedded in this space can be preserved well and robust to print-and-scan distortion. 27 Chapter 3 Noise of Print-and-Scan Process As discussed in the chapter 2, print-and-scan process is a content-based, nonlinear, and time-variant noise channel. The complex print-and-scan model needs the original image for calibrating its parameter values before it can be applied to estimate the noise. But this requirement can not be achieved as we aim to design a blind watermarking system. That is, the mathematical models of printers and scanners can not be used directly in our watermarking system. These models can only guide us to find more common property of the print-and-scan distortion for different images. In this chapter, we try to find this common property by the experimental means. To fully understand the print-and-scan noise, we analyze it in both pixel domain and frequency domain in the following sections. 3.1 Introduction As we do not take the rotation, scaling and transformation (RST) distortion into considerations in this thesis, we need to avoid it by other methods. We register the print-and-scan image to its original image manually. For example, we add some supplemental marks for reference near the edges or the corners of an image before it is 28 printed (see Figure 3.1); and register the distorted image to the original one by the relative positions of these marks. The registration parameters then are used to adjust the print-and-scan image back to the initial position and size. In the chapter 5, we will check that this method is effective to erase RST distortion and almost introduces no side effects to the watermarking system. Figure 3.1 The Print-and-scan Copy of 'Lena' with Supplemental Marks We use two commercial printers (HP Laser 4000 and HP Laser 4100) and one scanner (HP Scanjet 4570) to collect the experiment results for analysis. The images are printed and scanned at different resolutions (300dpi to 600dpi for printing, 75dpi to 1200dpi for scanning). Comparing the images from different printers, we find that besides the global RST distortion, the printers also introduce local affine distortion that is similar to the attacks of StirMark proposed in [11]. For instance, the HP Laser 4000 always 29 distorts the image under the same pattern, which extends and only extends the right up corner of the image horizontally. We believe that this noise comes from the mechanical defect of the paper roller or the toner drum. The unbalance of lamp illumination in scanners is also a type of local affine distortion, but this distortion is time-variant and varies during different scans. This noise distorts the large area with the same gray level in different ways, introducing the visual gray difference in the smooth area. Furthermore, when we impose the rotation and scaling operations on the image to get it back to the original position, the pixel values are inevitably interpolated. Since the interpolation noise is small compared to the print-and-scan noise, we model the interpolation as parts of the print-and-scan noise. To find common properties in print-and-scan process, different natural images are chosen for testing. These test images are illustrated in the appendix. However, we only illustrate the experiment results on the 512 × 512 'Lena'(Figure 3.2(a)) and 512 × 512 'Baboon'(Figure 3.2(b)), as they have very different properties for their image contents. From the histograms in Figure 3.2 (c) and (d), we can find that 'Lena' contains more low-intensity pixels while 'Baboon' contains more high-intensity pixels. Furthermore, there are many smooth regions and clear cut edges in 'Lena' while many texture areas in 'Baboon'. HVS is very sensitive to the local distortion within the smooth areas, thus embedding a watermark in 'Lena' without introducing visually perceived distortion is more difficult than in 'Baboon'. Experiments on these two images can help to find whether the pint-and-noise is sensitive to the intensity or texture of images. 30 (a) Original 'Lena' (b) Original 'Baboon' (a) Histogram of 'Lena' (b) Histogram of ''Baboon'' Figure 3.2 'Lena' and 'Baboon' In the following experiments, the images are considered as matrixes of pixel intensities. For example, within an image I, I ( x, y ) is the gray intensity at the location ( x, y ) , where x and y are indices of rows and columns respectively. Without loss of generality, we assume that ( x, y ) is scanned from left to right and top to bottom within the image. 31 3.2 Properties in Pixel Domain The properties of the print-and-scan in pixel domain are explored by the statistical analysis on the intensity value of every pixel within the test image. The distortion map is defined as the absolute difference of the original image and the print-and-scan image: Dx , y =| I x , y − Iˆx , y | (3.1) where I x , y represents the original image, Iˆx , y represents the print-and-scan image, and | ⋅ | is the absolute value function. The projection map is to project of the distorted image according to the gray levels within the original image: { ^ P( g ) = I x , y I x , y = g , for all x, y } (3.2) P( g ) contains all the pixels that have the same gray intensity g in the original image, and gets a set of their possible gray intensities in the distorted image. In the following experiments, when the projection map is plotted, the x-axis represents the gray levels g within the original image, and the y-axis represents the gray levels within the set of P( g ) . Distortion map records the distortions and the locations of these distortions, while the projection map only records the distortions for the same gray level and ignores the place where the distortions occur. The relation between projection map and distortion map is very similar to the relation between the histogram and its original image. Notice that if the projection map is calculated between the same images, its plot is a y = x line. 32 (a) project map of 'Lena' (b) project map of 'Baboon' Figure 3.3 Projection Map Between Original and Scanned Images Property I. The projection map of the print-and-scan image to the original image is a spindle-like mapping. In Figure 3.3(a) and Figure 3.3(b), the dispersed small grey dots plot the projection map between the print-and-scan image and the original image; and the dark dots along the line y = x are the projection map between the same images, say, I ( x, y ) and I ( x, y ) . For every single intensity value within the original image, there are many possible intensity values in the distorted image, and vice versa. The spindle-shape mapping also shows that the noise is related to the intensity of the original pixels. The low-intensity (near 50) pixels or high-intensity (near 200) pixels have a smaller number of possible values after print-and-scan; but the middle intensity (near 125) pixels have more possible intensity values. This shows that the middle-value intensity gets more noise from the print-and-scan attack. If all the grey and dark dots are projected to x-axis or y-axis, we can find that the dark dots are wider in range than the grey dots. This shows that there is a loss of bright pixels and dark pixels after the image is print-and-scanned, and the contrast within the image is degraded. As the gray intensity mapping between every original-distorted pixel pair is a many-to-many mapping, it is impossible to determine ever value of the original 33 gray intensity for every distorted pixel. In other words, a function exactly inverting the print-and-scan process does not exist. Property II. The print-and-scan distortion is content-dependent. Property I has shown that the noise at some location depends on both the original intensity of that location and those of the nearby locations. We seek for clearer relationship between the noise and the content in this experiment. Figure 3.4(a) and Figure 3.4(b) depict the distortion maps (equation (3.1) ) of 'Lena' and 'Baboon'. The darker pixel shows the larger distortion. Note that the distortions are small if they are compared to the original intensities ranging in 256 grays levels; therefore both distortion maps are scaled to grays level 64 to show more details on the distortion with low intensities. Besides the distortion related to the gray values discussed in the Property I, we may also find that the pixel distortion is closely related to the image content. The large distortion is more likely to occur in the texture areas of the original image; and a smaller distortion appears in the smoother areas. This phenomenon can be explained by the convolution function within the print-and-scan model introduced in chapter 2. The convolution function can be considered as a low-pass filter, thus it introduces less distortion to the smoother areas but more distortion to the areas with more textures. In addition, we examine these distortion maps carefully and find that some directional noise exists. We may see that there are some white bands of pixels near the noise of 'Baboon' and the hat of 'Lena'. That is, the distortion strength is almost the same along the horizontal direction but is different along the vertical direction. The different noise strength along the vertical direction should come from the dithering of the step motor to move the scan head from one side to the other side of the flatbed. 34 (a) distortion map of 'Lena' (b) distortion map of 'Baboon' Figure 3.4 Distortion Map Between Original and Scanned Images Property III. The print-and-scan noise is time-variant. The noise of different print-and-scan process varies, even though the same printer and scanner are used. Figure 3.5(a) show the projection map between two different print-and-scan copies of 'Lena', p1s1 ( Lena , t1 ) and p1s1 ( Lena , t2 ) , under the setting of the same printer and scanner. p1s1 ( Lena , t1 ) is the print-and-scan image of 'Lena' at time t1 ; p1 and s1 indicates that the used printer and scanner are printer 1 and scanner 1. Figure 3.5 (b) plots the projection map between p1s1 ( Lena , t1 ) and p2 s1 ( Lena , t2 ) . Both print-and-scan images are generated by different printers p1 , p2 and the same scanner s1 at time t1 and t2 . From both plots, we may find that the projection map (Figure 3.5 (a)) is unstable, though its distortion is less than those generated by different settings. The reason for this time variation should be the variation of mechanical dithering, tone roller temperature, and tone dissolution for different printings. Figure 3.6(a) shows that 'Lena' is noised by an independent Gaussian noise with zero mean and 0.001 variance, Figure 3.6(b) shows a distortion map between the Gaussian noised 'Lena' and the original 'Lena'. Comparing Figure 3.5(a) Figure 3.6(b), we can find that the projection map between 35 images under the same printer and scanner setting is very similar to that between a Gaussian independent noised image and its original version. In other words, the difference between the print-and-scan copies of image under the same printer and scanner setting is almost independent to the image content. (a) between images p-s by the same printer (b) between images p-s by different printers Figure 3.5 Projection Maps of Different Copies of Scanned 'Lena' (a) ‘Lena’ with Gaussian Noise (b) projection map of Lena with Gaussian noise Figure 3.6 Pixel Distortions Between Original Lena and Gaussian-Noised 'Lena' Property IV. The print-and-scan noise in nonlinear. From the above discussion and the distortion maps in Figure 3.3(a), (b), we can find that the function used to estimate the projection map of the print-and-scan image and the 36 original image must be a nonlinear function. As discussed in the preceding chapter, the spread functions of printers and scanner, the gamma correction, and the dot gain phenomenon introduces the nonlinearity to the image. The many-to-may spindle projection distortion can be considered as a nonlinear distortion plus other random noise. From the above properties in the pixel domain, we can find that it is impossible to completely erase the nonlinear, content-based and time variant print-and-scan noise if the distorted images are available only. The above experiments also show that the most robust feature space to print-and-scan is the pixels within the smooth areas. Therefore, a watermarking method referencing to the smooth areas probably is more robust to the print-and-scan process. Before design such a watermarking system, we should first find an evaluation function to determine whether some area is smooth or not. A simple method is to evaluate the energy of high frequency components in a frequency domain, because they represent the texture details within an image. In the next section, we define this evaluation function in the frequency domain and investigate the property of print-and-scan noise based on this function. 3.3 Properties in DCT Domain We choose the block-based DCT as the frequency domain to explore the properties of print-and-scan noise. In this domain, the block-partition preserves the spatial information about the image; while the coefficients within every block record the local frequency information only. The available spatial information within this frequency domain provides more flexibility for analyzing the image content. 37 It is well known that the high frequency coefficients in the smooth area have small values while those in the texture area have large values. Based on this observation, we define an energy function of DCT coefficients Fj to distinguish texture and smooth blocks: i2 E (i1 , i2 ) = ∑ Fj 2 1 ≤ i1 , i2 ≤ 64 (3.3) j = i1 where Fj is the DCT coefficient in an 8x8 block. The coefficient Fj is indexed by the zigzag scan order (Figure 3.7) mentioned in JPEG standard. The parameters i1 and i2 are used to decide the frequency band for calculating the energy. Figure 3.7 Zigzag Scan Order of DCT Coefficients In the following experiments, the low frequency energy and high frequency energy are defined as E (2,20) and E (21,64) respectively. The DCT coefficient of i = 1 , i.e. DC, is the most significant coefficient, whose magnitude is much larger than other coefficients. If this coefficient is counted into the low frequency energy, it will hide the energy of other coefficients. Therefore, the low frequency energy function starts from index 2. The energy 38 difference D between the print-and-scan image and the original image of a block is defined as: D(i, j ) = Escanned (i, j ) − Eoriginal (i, j ) (3.4) for 1 ≤ i, j ≤ 64 . Property V. After print-and-scan, the low frequency coefficients are preserved better than high frequency coefficients. This is an obvious property as the image content is not changed by the print-and-scan distortion. Figure 3.8 illustrates the absolute values of both the low and high frequency distortions for ‘Lena’ and ‘Baboon’. These 64 × 64 distortion maps illustrate the magnitudes of the energy distortions for all the blocks within the image. The distortions are scaled to 64 gray levels for better view, with the dark dots representing the large distortion magnitudes. Comparing the image pairs of ‘Lena’ and ‘Baboon’ in Figure 3.8, we can find that the low frequency distortions are mainly located on the strong textures such as clear-cut edges along Lena’s hat and Baboon’s face, while the high frequency distortions are uniformly distributed. From this Figure, we can also find that print-and-scan noise have all frequency components, i.e., it distributes to every frequency spectrum of the original image. The techniques to eliminate the noise for some frequency band are probably not useful. Notice that some horizontally distributed distortion near the hat of Lena in Figure 3.8(c) is apparently the dithering noise come from scanner. We have found this type of noise in property II, but it is more perceivable here in DCT domain. 39 (a) 'Lena': |D(2,20)|>0 (b)'Baboon': |D(2,20)|>0 (c) 'Lena': |D(21,64)|>0 (d)'Baboon': |D(21,64)|>0 Figure 3.8 Absolute Distortions in Low & High Frequency Property VI. After print-and-scan, most of the DC coefficients tend to change into larger values. Figure 3.9 illustrates all the DC distortions. The top pair in Figure 3.9(a) shows the blocks whose DC coefficients change to larger values and the bottom pair in Figure 3.9(b) shows those change to smaller values. Comparing these distortion maps, we can find that most DC coefficients change to larger values except the areas with high intensity. These areas are marked with dark dots in Figure 3.9(b). To investigate this property quantitatively, we plot the comparison of DCs between the original image and the print-and-scan image in Figure 3.9(c). The grey dots indicate the relations between the DCs of the original image and the distorted image, and the dark dots along y = x represent the expected relations 40 without distortions. Then the DCs changing to larger values are plotted above the line y = x , while the DCs changing to smaller values are plotted below this line. From this Figure, we can find that most DC coefficients changes to larger values after they are distorted; only those with large magnitudes change to smaller values. This property is consistent with the Property I, which indicates that the dark pixels tend to change brighter while the bright pixels tend to change darker. As the DC is the arithmetic mean of the gray values within a block, the result indicates that the image globally changes lighter after print-and-scan. This tendency mainly comes from the strong lights generated by the lamp in the scanners. (a) D(1,1)>0 (b) D(1,1) 0 , the texture blocks get more distortions; while if T < 0 , the smooth blocks get more. The experiment data for all the test images is listed in Table 3.1 and plotted in Figure 3.11. This data shows that the distortion of texture blocks is always more significant than that of smooth blocks for a variety of images. The results confirm the Property I again in the DCT domain. Images DT DS T ‘Lena’ -632.5 ‘Baboon’ -7678.7 ‘Girl’ -2927.1 ‘Fruit’ -2774.4 ‘Flower’ -5654.3 ‘Women’ -8731.5 580.7 674.4 403.1 351.4 835.4 365.2 265.1 8821.4 2962.5 3734.7 8276.2 11143.8 Table 3.1 Means of Different High Frequency Distortions 43 Figure 3.11 Magnitudes of Different Types of Distortions This is a very important property for guiding us to design the watermarking scheme, as we find the robust feature space within the images that is comparably stable. The watermark embedded in the smooth blocks will probably robust to print-and-scan distortion. 3.4 Summary Because the print-and-scan model can not be used directly in our watermarking system, we discuss the properties of print-and-scan noise by experiments in this chapter. This property is explored in both the pixel domain and the frequency domain. The results show that no models are able to erase the print-and-scan noise completely if no additional information like the original image, the original watermark or the calibration parameters for the used printers and scanners is available. The results also show that the smooth blocks get far less 44 distortion during the print-and-scan process. This important property implies that the smooth areas compose a robust feature space in different images. Watermarking in this feature space may probably be more robust to print-and-scan distortion. However, HVS is sensitive to even small distortion in smooth areas. Thus the watermark can not be embedded by directly introducing modification to the smooth blocks. In the next chapter, we will discuss how to embed the watermark into this feature space indirectly, by applying the smooth blocks as a reference rather than directly modifying them. 45 Chapter 4 Watermarking Algorithm In chapter 2 and chapter3, we have discussed the working principles, the mathematical models and the distortion properties of printers and scanners, from both theoretical and experimental aspects. We found that perfectly erasing the print-and-scan distortion impractical, if neither the original image nor the watermark is known to watermark detector. Using the present image processing techniques, the watermarking system can only estimate this random distortion. However, the estimation process itself also introduces noise to the images because the applied models are not perfect. Therefore, we are not concerned with modeling the complex function of print-and-scan distortion. We embed the watermark into the feature space of the image that is robust to print-and-scan. Previous experimental investigations have shown that the smooth blocks compose this feature space. As we all know, the DCT quantization would erase the high frequency coefficients within the blocks and make them smooth. Thus DCT quantization would help us to create this robust feature space. In this chapter, we discuss how to design our watermarking system, by taking all the experimental results into consideration. 46 As we have mentioned before, HVS is sensitive to even the slight distortion in the smooth areas; while watermarking directly in this feature space will inevitably introduce distortion to the smooth blocks. For this reason, watermarking in the smooth blocks should be done indirectly. We choose to optimize the Differential Energy Watermarking (DEW) [30], because it meets our basic requirements by embedding watermark without distorting the smooth blocks but by referring to them indirectly. DEW embeds watermark by deleting high frequency coefficients in the quantized DCT domain, provides the robustness by distributing the watermark bits to different locations, and keeps the image content with little distortion by adaptive embedding. We incorporate the preprocessing techniques to improve the robustness of watermark in the feature space and use double votes to provide more robust watermark extraction scheme. In the following sections, we introduce the original DEW first, then discuss our methods on how to optimize this system based on the properties of print-and-scan distortion. 4.1 Original DEW Differential Energy Watermarking (DEW) was first proposed by Langelaar et al in [30]. The main idea of this scheme is to selectively discard high frequency DCT coefficients for watermark embedding. The pattern about how to delete the coefficients is decided by the watermark bit value. This watermarking scheme provides flexibility to balance the robustness and fidelity by many control parameters. In this section, we roughly introduce the embedding scheme and the extracting scheme of this watermarking system. 47 4.1.1 Watermark Embedding Figure 4.1 depicts the general procedure of the embedding scheme: take block-based DCT for the image, shuffle all the DCT blocks, embed one watermark bit to a GOB, and finally de-shuffle the DCT blocks and take the IDCT transform. Here, a GOB is defined as a number of blocks that are clustered together as a square. For instance, white grids in Figure 4.2 illustrate boundaries of the GOBs of 16 blocks. Figure 4.1 DEW Watermark Embedding Scheme The calculation of the block-based DCT and IDCT is addressed in details in the JPEG standard. The image is divided into bocks of 8 × 8 size, and the 2-dimension discrete cosine transform (DCT) is calculated for every block. To reconstruct the image from the DCT coefficients, an inverse discrete cosine transform (IDCT) is performed block-wise. Figure 4.2 Shuffled Images: 'Lena'(LEFT) & 'Baboon'(RIGHT) 48 Shuffle operation re-arranges all the DCT blocks randomly to spatially randomize the modifications to the DCT blocks and hide the watermark locations. The watermark key K is a symmetric key and the watermark extractor also needs the key to extract the embedded watermark. In fact, K is the seed to initialize the pseudo-random number generator within the shuffle operation module. Figure 4.2 shows the randomly shuffled blocks of 'Lena' and 'Baboon', with the white grids to indicate the boundary of the GOBs containing 4 × 4 blocks. Figure 4.3 Creation of GOB in DEW How to embed a single bit into a GOB is the core of DEW. The basic idea is to selectively delete the high frequency coefficients of the upper half or the lower half of GOB. Which half of GOB's coefficients should be deleted depends on the embedded watermark bit. And how many coefficients are deleted is determined by the expected image quality. As illustrated in Figure 4.3, a GOB is divided horizontally into two sub-groups A and B by and the energy functions of these two halves are calculated as: n/2 E A (c, n, Q jpeg ) = ∑ Ek (c, Q jpeg ) k =1 (4.1) 49 EB (c, n, Q jpeg ) = n ∑ k =n 2+1 Ek (c, Q jpeg ) (4.2) where Ek (c, Q jpeg ) is the high frequency energy function of the kth block within the GOB, n is the total number of blocks within a GOB, c is the cutting index for the calculation of the high frequency energy, and Q jpeg is the quality factor in JPEG standard. The Ek (c, Q jpeg ) in the equation (4.2) is defined as: 64 Ek (c, Q jpeg ) = ∑ ([ Fk (i )]Q )2 i =c (4.3) jpeg In equation (4.3), Fk (i ) is the ith DCT coefficient within kth block and [ ⋅ ]Q jpeg is the JPEG quantization with the quality factor Q jpeg . Based on the above definitions, the energy difference D of every GOB between the upper sub-group and the lower sub-group is defined as: D = E A − EB (4.4) The watermark embedding adapts E A and E B to modify the energy difference D, which is used to indicate the watermark bit embedded in this GOB. In particular, when watermark bit "0" is embedded, all the high frequency coefficients after the index c in the DCT-blocks of lower sub-group B are eliminated, resulting in: D = E A − EB = E A − 0 > 0 (4.5) Similarly, when a watermark bit "1" is embedded; the high frequency coefficients of upper sub-group A are erased, resulting in D = 0 − EB < 0 . Thus the watermark bit can be extracted as "0" if D > 0 , and "1" if D < 0 50 In equations (4.1) and(4.2), c is the cutting index for high frequency coefficients, which determines how many coefficients are deleted during watermark embedding. To protect the image visual quality from great distortion, this cutting index must be greater than a predefined threshold index cmin so that the important low frequency coefficients are never erased. In other words, those coefficients with indices c < cmin are never deleted. This requirement indirectly sets an upper bound for the absolute energy difference | D | if the number of blocks within a GOB, i.e. n , is fixed. Nevertheless, | D | should be large enough to tolerate errors to watermark extraction and enhance the system robustness. Therefore, DEW requires both E A and EB is larger than a predefined energy threshold E . Because | D | equals to E A or EB after watermarking, this requirement sets the bottom bound for | D | indirectly. All these requirements are combined to determine the optimal cutting index c : c( n, Q jpeg , E , cmin ) = max{ cmin , max{i ∈ {0,63} | E A (i, n, Q jpeg ) > E ∩ E B (i, n, Q jpeg ) > E}} (4.6) In other words, the optimal c is the largest index such that both E A and E B is larger than the designed energy threshold E under the constraint cmin . From equation(4.6), we can see the relation between c and other parameters: c is a decrease, increase and decrease function for Q jpeg , n and E respectively, c will be decreased. Note that even though an energy threshold E is imposed, | D | could be always smaller than E in some special cases because | D | also depends on the image content. Under the constraint cmin , the maximum energy function of a specific sub-group is E ( cmin , n, Q ) . This maximum value jpeg is still possible to be smaller than E as long as most blocks within this sub-group are smooth blocks. In these special cases, the extracted watermark is possibly wrong even 51 there are no attacks on it. The energy threshold E enables the adaptive discarding of DCT coefficients for different blocks. The index c is larger for a sub-group containing texture blocks than that containing smooth blocks. This soft threshold guarantees the robustness of the watermarking and deletes the DCT coefficients as less as possible for keeping the visual quality of the watermarked images. 4.1.2 Watermark Extracting Figure 4.4 illustrates the general procedure of DEW extracting scheme. The watermark key K used during watermark embedding is needed here. Only those who know K can reverse the shuffle sequence for the DCT blocks and extract the correct watermark. This mechanism prevents the information leakage of the watermark, which is valuable in some scenarios where the watermark also needs to be protected. Figure 4.4 DEW Watermark Extracting Scheme Because the cutting indices c for different blocks are adaptively selected in the watermarking scheme, the watermark extractor has no information about their real values. DEW uses an indirect method to find the embedded pattern of energy difference between both sub-groups. The idea is to compare the smallest indices c A and cB for sub-group A and B that enable the energy functions larger than a small energy threshold G . More formally, c A and cB are calculated for both sub-groups as: 52 c A = min{c | E A (c, n, Q jpeg ) < G} (4.7) cB = min{c | EB (c, n, Q jpeg ) < G} (4.8) where G is the predefined small energy threshold. The energy threshold G for extracting is set as a value far smaller than the energy threshold E for embedding. Comparing the results of equation (4.7) and (4.8), we can find the embedded patter of energy difference. If ( cB < c A ) , the lower sub-group B needs more frequency coefficients to reach the threshold G. In this case, B is probably the sub-group whose high frequency coefficients are eliminated for watermark embedding. Similarly, A is the modified sub-group if (c A < cB ) . In case of c A = cB , the watermark bit cannot be determined solely by a comparison between both indexes, thus a comparison between the energy functions EB (cB ) and E A ( c A ) is needed. However, it is still possible that cB = c A and EB (cB ) = E A ( c A ) . In this case, either '0' or '1' is possible; thus DEW chooses '1' as the extracted bit. In summary, the extracted watermark bit from this GOB is determined by: { Wi = 0, 1, if ( cB < c A ) or ( cB = c A and E B ( cB ) < E A ( c A )) otherwise (4.9) 4.2 Our Watermarking Scheme As we have discussed in chapter 3, the smooth blocks in the image get less print-and-scan distortion than the texture blocks. However, embedding watermark directly in this feature space is impossible because even the slight distortion to the smooth blocks can be perceived by HVS. DEW embeds watermark by selectively smoothing some blocks, which actually enlarges the robust feature space for print-and-scan process. However, the 53 discussion of the original DEW in the previous section shows that the constraints of cmin will decrease the system robustness even though the energy threshold E is also imposed to the watermarking system. In addition, DEW utilizes different patterns of energy difference to encode the watermark bits, so the distortion to the texture blocks also affects the correctness of the extracted watermark. For a blind watermarking scheme, the only information available for watermark extraction is the watermarked image itself. Therefore, the embedded watermark bit ‘1’ and ‘0’ must be mapped to different patterns within the image. The smooth blocks can only create one pattern so it is inevitable to take the texture blocks into account. We need to find a method to decrease the print-and-scan distortion to the texture blocks for better robustness. Furthermore, the DEW extracts the watermark bits by a simple comparison between the cutting indices of two sub-groups. The watermark bit extracted by this method is not always correct even there is no noise distorting the watermarked image [31]. The existence of error is because of the constraint cmin used in watermark embedding. As we know, all the blocks in the image are shuffled randomly, so the energy functions of the two sub-groups would be fairly close to each other before watermarking. If the original image I contains many smooth areas, it is possible that the high frequency energy above the cutting index cmin for some sub-group is still smaller than E, e.g., E (cmin,Q jpeg ) < E . In that case, even all the DCT coefficients larger than cmin in the other sub-group are deleted, the energy difference between these two sub-groups is not sufficiently large. Without the loss of generality, suppose the high frequency coefficients of sub-group B are deleted during 54 watermark embedding and E A (cmin,Q jpeg ) < E . Then after watermark embedding, the energy difference between the two sub-groups is: D = E A ( cmin , Q jpeg ) − EB ( cmin , Q jpeg ) = E A ( cmin , Q jpeg ) < E (4.10) E A ( cmin , Q jpeg ) then has no bottom bound as expected, and E A ( cmin , Q jpeg ) < G is possible. In this case, the equation (4.9) for watermark extraction can not guarantee c A > cB and thus the extracted watermark is probably wrong. Langelaar et al gives the theoretical analysis on calculating this type of error probability in [31]. When the print-and-scan distortion exists, this simple watermark extraction method is more fragile. Following the above example of deleting high frequency coefficients of sub-group B to embed the watermark bit "0", D = E A − EB = E A − 0 > 0 is expected after watermarking. The discussion in the chapter 3 shows that print-and-scan distortion increases the energy of smooth blocks while decreases that of texture blocks. Given the energy decrease of sub-group A is ∆ A ( ∆ A > 0 ) and the energy increase of sub-group B is ∆ B ( ∆ B > 0 ) , the energy function for the two sub-groups then changes to E A ' = E A − ∆ A and EB ' = EB + ∆ B . Then, the energy difference will change to: D ' = E A '− EB ' = ( E A − EB ) − ( ∆ A + ∆ B ) < D (4.11) If the distortion is large enough, the distortion D ' < 0 is possible, and the extracted watermark is wrong. To abandon the use of cmin in the watermark embedding is a solution to this problem, but the shortage of protection on the image quality is risky. Solving these problems needs a more robust watermark extracting method. 55 DEW requires that every group contains an even square number of blocks. This strategy lacks the flexible control on the watermark capacity, as the gap between square numbers is large. In fact, the number of blocks in a group is only need to be an even integer so that it can be divided by two sub-groups. In DEW, many even numbers within every gap of square numbers can never be used as the group size. This parameter also affects the robustness of the watermarking system indirectly. When a larger number is chosen as the group size, the watermark is more robust as every watermark bit will be distributed to more locations. More flexible choices on this value provide more levels of robustness to the watermarking system and more possible tradeoffs between the robustness and the capacity. In following sections, we show how to modify the original DEW for better robustness to print-and-scan distortion. 4.2.1 Watermark Embedding The original DEW proposes a method to create the smooth blocks, which get less distorted by the print-and-scan process. However, we still need to protect the texture blocks because the correct watermark extraction also depends on these blocks. As it is impractical to erase the print-and-scan distortion, we protect the texture blocks by compensating the expected distortion. Besides deleting DCT coefficients from one sub-group, our method adds DCT coefficients in the other sub-group to increase the magnitude of energy difference. The print-and-scan distortion has been classified into two parts in (4.11): ∆ A is the distortion to the texture blocks and ∆ B is the distortion to the smooth blocks. To compensate ∆ B , we can erase more coefficients than we need. Similarly, we can also add coefficients into the 56 texture blocks to compensate ∆ A . If the added noise to the image is controlled carefully, the image quality may be reserved without great changes. Furthermore, we provide more choices on the number of blocks within a group, by exploiting a slice structure rather than the original square structure to the group of blocks. A slice structure is formed by a row of 2n blocks, and it can be divided into two sub-slices from the middle. Choosing the slice structure enables a flexible tradeoff between the robustness and the capacity of a watermarking system. The general procedure of our watermarking scheme is similar to the original DEW in Figure 4.1 of section 4.1.1. The only difference is the operation within the embedding block. Referring to Figure 4.1, the input to this block includes the shuffled DCT blocks and the watermark bit stream, and the output is the watermarked blocks. Figure 4.5 illustrates the details of our new operations within this block. Figure 4.5 Modified Watermark Embedding Scheme A slice of blocks is defined as a row vector of 2n blocks, which can be divided into two n-size sub-slices. To create a slice, we scan the image in a raster order, collect 2n blocks and arrange them into a row vector. Formally, the kth slice of an image I is defined as: 57 S k ( I ) = {block j (k −1) ⋅ M + 1 ≤ j ≤ k ⋅ M −1 and j ≤N } (4.12) where block j is the jth block ordered in a raster scan within the I , N is the number of blocks in image I and M = 2n is the number of blocks in a slice. In Figure 4.6, the white grids mark all the slices containing 12 blocks in the shuffled ‘Lena’ and ‘Baboon’. When the roster scan fails to get enough blocks in this scan line to create the slice, the blocks in the next scan line will be selected. Therefore a slice may contain blocks from different scan lines of the image. The creation of slice continues, until there are no enough blocks in the last scan line for a slice. Figure 4.6 Slices of the Shuffled Images: 'Lena' (LEFT) & 'Baboon' (RIGHT) After the image is sliced, it is divided in the left sub-slice A and the right sub-slice B from the middle for differential energy watermarking. The watermark embedding is defined as: addDCT ( A), B ' = eraseDCT ( B ), if w = 1 { AA '' == eraseDCT ( A), B ' = addDCT ( B ), if w = 0 (4.13) where functions addDCT (⋅) and eraseDCT (⋅) intend to add or delete the high frequency DCT coefficients of the sub-slice to encode the watermark bit w . The 58 function eraseDCT (⋅) is the same as what the original DEW does in watermark embedding, i.e., deleting all the coefficients above the optimal cutting index c . In addDCT (⋅) , we add high frequency coefficients in the other sub-slice. But if we directly add the coefficients in DCT domain, we have no idea about the visual effect of the noise to the pixel domain. For this reason, we add the high frequency DCT coefficients indirectly by the white Gaussian noise in the pixel domain. By this means, we know the visual effect of noise in pixel domain while increase the energy of this sub-slice. Figure 4.7 PSNR of Different Print-and-Scan Images The strength of this noise must be large enough to compensate the print-and-scan distortion; but it must also be small enough to protect the image content from distortion. To achieve a good tradeoff between watermark robustness and image quality, we need to 59 know the strength of the print-and-scan distortion. The PSNR values for different print-and-scan images are calculated and illustrated in Figure 4.7. From this figure, we find that the print-and-scan distortion is so great that the PSNR for different original images drop to smaller than 30. Then if the PSNR of the noisy image is larger than 40, the image quality is still better than the print-and-scan image. Based on this observation, we add noise into the image such that its PSNR is larger to 40 before watermarking. Figure 4.8 shows the print-and-scan copy of the original "Lena" and that of the noisy "Lena". We can hardly perceive their difference, even in the smooth areas such Lena’s shoulder. The other images contain more texture blocks than "Lena" and hide the noise distortion more easily, thus they are not illustrated here for comparison. Figure 4.8 Print-and-Scan Copies of 'Lena': Original(LEFT); Noise Adding(RIGHT) In addDCT (⋅) , we use the binary search to find the appropriate variance for the Gaussian noise, calculate its DCT coefficients, and add its high frequency coefficients to the DCT domain of original image. To protect the low frequency coefficients of the image, only the 60 coefficients with indices larger than cmin within the Gaussian noise are used for calculation. The details of addDCT (⋅) is described below: Iˆ = addDCT ( I , psnrThreshold , cmin ) Step 1. mean=0; var=1; varMin=0; varMax =1; while( psnr ( Iˆ , I ) - psnrThreshold > 0.01 ){ var = 0.5(varMin + varMax); Iˆ = I + GaussianNoise ( m, v ) ; If ( psnr ( Iˆ, I ) > psnrThreshold ) varMin = var; else varMax = var; } Step 2. NoiseDCT [...]... vector to a specific space In this thesis, I would like to model watermarking as a multiplexed communication 9 1.2 Print- and- Scan Process As we are concerned with the digital watermarking robust to print- and- scan process, we need to know what the print- and- scan process is, and the property of the noise it adds to the image The whole print- and- scan process is: printing the image of digital format onto... image watermarking robustness to print- and- scan process In some cases, the print- and- scan is also thought as the acceptable image processing techniques as it does not change the image content greatly This thesis tries to understand the print- and- scan process and aims to design a watermarking scheme robust to the distortion comes from this process This chapter begins with an overview of digital watermarking, ... theories in optics and electronics to avoid these mechanical strikes[22] Dot-matrix printers, daisy-wheel printers, and line printers belong to the impact printers, which are noisier than the non-impact printers like inkjet printers and laser printers On the other hand, printers can be classified as contone printers and halftone printers [19] Contone printers can print the continuous tone images directly,... distortion The results include the robustness performance of our watermarking system for different images, and the comparison on the robustness between the original DEW and our system 14 Chapter 2 Printer and Scanner To design a watermarking scheme robust to print- and- scan process, we need to know the working principles and the mathematical models of printers and scanners In this chapter, we first... eyes can recognize an original copy and a print- and- scan copy of the same image to be similar, it is intuitive that some feature space within an image can be robust to print- and- scan In other words, in the print- and- scan context, feature space should be found in a coarse granularity 1.3 Motivation There are many applications for watermarking robust to print- and- scan process The authentication of documents... on Watermarking Due to the variations in the print- and- scan process, the research in literature only partially solve the problems related to the watermarking that is robust to print- and- scan process As previous research on the rotation, scaling and transformation (RST) has been introduced in chapter 1 In this section, we will review the related work exploring the pixel value distortion of print- and- scan. .. watermarking robust to it Lin gave a general model for the pixel value distortion of print- and- scan process in [17] Although the detailed formulas of this model are not given, all types of noise involved in the print- and- scan process have been taken into consideration Nevertheless, this model needs the calibration for different setting of printer and scanner Thus it is not common for all the 11 printers and scanners... format onto a paper, and scanning this paper to get back the digital image Intuitively, this process is a kind of D/A A/D transformation, and the noise of this process comes mainly from the transformation between digital format and analogue format Nevertheless, because many uncontrolled and unexpected distortion is introduced in the print- and- scan process, the model of print- and- scan noise is more complex... based on the printer and scanner models The results of the chapter 2 and chapter 3 give the basis to design a robust watermarking system Chapter 4 discusses our watermarking system This watermarking system is a variation of the Differential Energy Watermarking (DEW) Chapter 5 presents the experiment results to 13 evaluate the robustness of our watermarking system to print- and- scan distortion The results... an overview of digital watermarking, and then introduces the mathematical models of watermarking After that, the related research on digital watermarking robust to print- and- scan process is introduced Finally the organization of the whole thesis is given 2 1.1 Digital Watermarking 1.1.1 Overview of Digital Watermarking The history of watermarking can be traced back to several hundred years ago, when ... the digital watermarking robust to print-and-scan process, we need to know what the print-and-scan process is, and the property of the noise it adds to the image The whole print-and-scan process. .. tries to understand the print-and-scan process and aims to design a watermarking scheme robust to the distortion comes from this process This chapter begins with an overview of digital watermarking, ... techniques similar to Cox's scheme are proposed in [28, 29] To develop a watermarking system robust to print-and-scan process, Lin models the pixel distortion in print-and-scan process in [17]: