DSpace at VNU: Blur estimation for barcode recognition in out-of-focus images

Lecture Notes in Computer Science Commenced Publication in 1973 Founding and Former Series Editors: Gerhard Goos, Juris Hartmanis, and Jan van Leeuwen Editorial Board David Hutchison Lancaster University, UK Takeo Kanade Carnegie Mellon University, Pittsburgh, PA, USA Josef Kittler University of Surrey, Guildford, UK Jon M Kleinberg Cornell University, Ithaca, NY, USA Alfred Kobsa University of California, Irvine, CA, USA Friedemann Mattern ETH Zurich, Switzerland John C Mitchell Stanford University, CA, USA Moni Naor Weizmann Institute of Science, Rehovot, Israel Oscar Nierstrasz University of Bern, Switzerland C Pandu Rangan Indian Institute of Technology, Madras, India Bernhard Steffen TU Dortmund University, Germany Madhu Sudan Microsoft Research, Cambridge, MA, USA Demetri Terzopoulos University of California, Los Angeles, CA, USA Doug Tygar University of California, Berkeley, CA, USA Gerhard Weikum Max Planck Institute for Informatics, Saarbruecken, Germany 6744 Sergei O Kuznetsov Deba P Mandal Malay K Kundu Sankar K Pal (Eds.) Pattern Recognition and Machine Intelligence 4th International Conference, PReMI 2011 Moscow, Russia, June 27 – July 1, 2011 Proceedings 13 Volume Editors Sergei O Kuznetsov National Research University Higher School of Economics School for Applied Mathematics and Information Science 11 Pokrovski Boulevard, 109028 Moscow, Russia E-mail: skuznetsov@hse.ru Deba P Mandal Malay K Kundu Sankar K Pal Indian Statistical Institute, Machine Intelligence Unit 203, B.T Road, Kolkata 700108, India E-mail: {dpmandal, malay, sankar}@isical.ac.in ISSN 0302-9743 e-ISSN 1611-3349 e-ISBN 978-3-642-21786-9 ISBN 978-3-642-21785-2 DOI 10.1007/978-3-642-21786-9 Springer Heidelberg Dordrecht London New York Library of Congress Control Number: 2011929642 CR Subject Classification (1998): I.4, F.1, I.2, I.5, J.3, H.3-4, K.4.4, C.1.3 LNCS Sublibrary: SL – Image Processing, Computer Vision, Pattern Recognition, and Graphics © Springer-Verlag Berlin Heidelberg 2011 This work is subject to copyright All rights are reserved, whether the whole or part of the material is concerned, specifically the rights of translation, reprinting, re-use of illustrations, recitation, broadcasting, reproduction on microfilms or in any other way, and storage in data banks Duplication of this publication or parts thereof is permitted only under the provisions of the German Copyright Law of September 9, 1965, in its current version, and permission for use must always be obtained from Springer Violations are liable to prosecution under the German Copyright Law The use of general descriptive names, registered names, trademarks, etc in this publication does not imply, even in the absence of a specific statement, that such names are exempt from the relevant protective laws and regulations and therefore free for general use Typesetting: Camera-ready by author, data conversion by Scientific Publishing Services, Chennai, India Printed on acid-free paper Springer is part of Springer Science+Business Media (www.springer.com) Preface This volume contains the proceedings of the 4th International Conference on Pattern Recognition and Machine Intelligence (PReMI-2011) which was held at the National Research University Higher School of Economics (HSE), Moscow, Russia, during June 27 - July 1, 2011 This was the fourth conference in the series The first three conferences were held in December at the Indian Statistical Institute, Kolkata, India, in 2005 and 2007 and at the Indian Institute of Technology, New Delhi, India, in 2009 PReMI has become a premier international conference presenting the state-ofart research findings in the areas of machine intelligence and pattern recognition The conference is also successful in encouraging academic and industrial interaction, and in promoting collaborative research and developmental activities in pattern recognition, machine intelligence and other allied fields, involving scientists, engineers, professionals, researchers and students from India and abroad The conference is scheduled to be held every alternate year making it an ideal platform for sharing views, new results and experiences in these fields in a regular manner PReMI-2011 attracted 140 submissions from 21 different countries across the world Each paper was subjected to at least two reviews; the majority had three reviews The review process was handled by the PC members with the help of additional reviewers These reviews were analyzed by the PC Co-chairs Finally, on the basis of reviews, it was decided to accept 65 papers for oral and poster sessions We are grateful to the PC members and reviewers for providing critical reviews This volume contains the final version of these 65 papers after incorporating reviewers’ suggestions These papers have been organized under nine thematic sections For PReMI-2011, we had a distinguished panel of keynote and plenary speakers We are grateful to Rakesh Agrawal for agreeing to deliver the keynote talk We are also grateful to John Oommen, Mikhail Roytberg, Boris Mirkin, Santanu Chaudhury, and Alexei Chervonenkis for delivering the plenary talks Our Tutorial Co-chairs arranged an excellent set of pre-conference tutorials We are thankful to all the tutorial speakers We would like to take this as an opportunity to thank the host institute, National Research University Higher School of Economics, Moscow, for providing all facilities to organize this conference We are grateful to the co-organizer Laboratoire Poncelet (UMI 2615 du CNRS, Moscow) We are also grateful to Springer, Heidelberg, for publishing the volume and the National Centre for Soft Computing Research, ISI, Kolkata, for providing the necessary support The success of the conference is also due to the funding received from different VI Preface agencies and industrial partners, among them ABBYY, the Russian Foundation for Basic Research, Yandex, and Russian Association for Artificial Intelligence (RAAI) We are thankful to all of them for their active support We are grateful to the Organizing Committee for their endeavor in making this conference a success The volume editors would like to especially thank our Organizing Chair Dmitry Ignatov for his enormous contributions toward the organization of the conference and publication of these proceedings Our special thanks are also due ´ ezak for his kind co-operation, co-ordination and help, and for to Dominik Sl being involved in one form or other with PReMI since its first edition in 2005 And last, but not least, we thank the members of our Advisory Committee who provided the required guidance and sponsors PReMI-2005, PReMI-2007 and PReMI-2009 were successful conferences We believe that you will find the proceedings of PReMI-2011 to be a valuable source of reference for your ongoing and future research activities April 2011 Sergei O Kuznetsov Deba P Mandal Malay K Kundu Sankar K Pal Organization General Chair Conference Chair Program Co-chairs Organizing Chair Tutorial Co-chairs Publicity Co-chairs Coordination Chair Sankar K Pal, ISI Kolkata, India Sergei O Kuznetsov, Higher School of Economics, Russia Malay K Kundu, ISI, Kolkata, India Deba P Mandal, ISI, Kolkata, India Dmitry I Ignatov, Higher School of Economics, Russia Chris Cornelis, Ghent University, Belgium Sanghamitra Bandyopadhyay, ISI, Kolkata, India Goutam Chakraborty, Iwate Prefectural University, Japan Joydeep Ghosh, University of Texas, USA Simon C K Shiu, HK Polytechnical University, Hong Kong Advisory Committee Lotfi Zadeh, USA Michael Brady, UK Anil Jain, USA Josef Kittler, UK Rama Chellappa, USA Gennady S Osipov, Russia Witold Pedrycz, Canada Andrzej Skowron, Poland Brian C Lovell, Australia Dwijesh Dutta Majumdar, India Arun Majumder, India Konstantin V Rudakov, Russia Konstantin Anisimovich, Russia Gabriella Sanniti di Baja, Italy B Yegnanarayana, India B.L Deekshatulu, India Program Committee Tinku Acharya Aditya Bagchi Sanghamitra Bandyopadhyay Roberto Baragona Andrzej Bargiela Jayanta Basak Tanmay Basu Dinabandhu Bhandari Bhargab B Bhattacharya Intelectual Ventures, Kolkata, India Indian Statistical Institute, Kolkata, India Indian Statistical Institute,Kolkata, India Sapienza University of Rome, Rome, Italy University of Nottingham, Selangor Darul Ehsan, Malaysia IBM Research, Bangalore, India Indian Statistical Institute, Kolkata, India Indian Statistical Institute, Kolkata, India Indian Statistical Institute, Kolkata, India VIII Organization Pushpak Bhattacharyya Kanad Biswas Prabir Kumar Biswas Sambhunath Biswas Smarajit Bose Lorenzo Bruzzone Roberto Cesar Partha P Chakrabarti Mihir Chakraborty Bhabatosh Chanda Subhasis Chaudhuri Santanu Chaudhury Sung-Bae Cho Sudeb Das Sukhendu Das B.S Dayasagar Rajat K De Kalyanmoy Deb Lipika Dey Sumantra Dutta Roy Utpal Garain Ashish Ghosh Hiranmay Ghosh Kuntal Ghosh Sujata Ghosh Susmita Ghosh Phalguni Gupta C.V Jawahar Grigori Kabatianski Vladimir F Khoroshevsky Indian Institute of Technology Bombay, Mumbai, India Indian Institute of Technology Delhi, New Delhi, India Indian Institute of Technology Kharagpur, Kharagpur, India Indian Statistical Institute, Kolkata, India Indian statistical Institute, Kolkata, India University of Trento, Italy University of S˜ ao Paulo, S˜ ao Carlos, Brazil Indian Institute of Technology Kharagpur, Kharagpur, India Indian Statistical Institute, Kolkata, India Indian Statistical Institute, Kolkata, India Indian Institute of Technology Bombay, Mumbai, India Indian Institute of Technology Delhi, New Delhi, India Yonsei University, Seoul, Korea Indian Statistical Institute, Kolkata, India Indian Institute of Technology Madras, Chennai, India Indian Statistical Institute, Bangalore, India Indian Statistical Institute, Kolkata, India Indian Institute of Technology Kanpur, Kanpur, India Tata Consultancy Services Ltd., New Delhi, India Indian Institute of Technology Delhi, New Delhi, India Indian Statistical Institute, Kolkata, India Indian Statistical Institute, Kolkata, India Tata Consultancy Services Ltd., New Delhi, India Indian Statistical Institute, Kolkata, India University of Groningen, Netherlands Jadavpur University, Kolkata, India Indian Institute of Technology Kanpur, Kanpur, India IIIT, Hyderabad, India Institute for Information Transmission Problems of Russian Academy of Sciences, Moscow, Russia Computing Centre of Russian Academy of Sciences, Moscow, Russia Organization Ravi Kothari Malay K Kundu Sergei O Kuznetsov Yan Li Lucia Maddalena Pradipta Maji Deba P Mandal Anton Masalovitch Francesco Masulli Pabitra Mitra Suman Mitra Sushmita Mitra Dipti P Mukherjee Jayanta Mukherjee C.A Murthy Narasimha Murty Musti Sarif Naik Tomaharu Nakashima B.L Narayana Ben Niu Sergei Obiedkov Nikhil R Pal Pinakpani Pal Sankar K Pal Swapan K Parui Gabriella Pasi Leif Peterson Alfredo Petrosino Arun K Pujari Ganesh Ramakrishnan Shubhra S Ray Siddheswar Roy Suman Saha P.S Sastry Debashis Sen Srinivasan Sengamedu Rudy Setiono B Uma Shankar Roberto Tagliaferri IX IBM Research, New Delhi, India Indian Statistical Institute, Kolkata, India Higher School of Economics, Moscow, Russia The Hong Kong Polytechnic University, Hong Kong, China National Research Council, Naples, Italy Indian Statistical Institute, Kolkata, India Indian Statistical Institute, Kolkata, India ABBYY, Moscow, Russia Universita’ di Genova, Genova, Italy Indian Institute of Technology Kharagpur, Kharagpur, India DAIICT, Gandhinagar, India Indian Statistical Institute, Kolkata, India Indian Statistical Institute, Kolkata, India Indian Institute of Technology Kharagpur, Kharagpur, India Indian Statistical Institute, Kolkata, India Indian Institute of Science, Bangalore, India Philips India, Bangalore, India University of Osaka Prefecture, Osaka, Japan Yahoo India, Bangalore, India The Hong Kong Polytechnic University, Hong Kong, China Higher School of Economics, Moscow, Russia Indian Statistical Institute, Kolkata, India Indian Statistical Institute, Kolkata, India Indian Statistical Institute, Kolkata, India Indian Statistical Institute, Kolkata, India Universita’ di Milano Bicocca, Milano, Italy The Methodist Hospital Research Institute, Houston, USA University of Naples, Italy LNM IIT, Jaipur, India Indian Institute of Technology Bombay, Mumbai, India Indian Statistical Institute, Kolkata, India Monash University, Melbourne, Australia Indian Statistical Institute, Kolkata, India Indian Institute of Science, Bangalore, India Indian Statistical Institute, Kolkata, India Yahoo! Labs, Bangalore, India National University of Singapore, Singapore Indian Statistical Institute, Kolkata, India Universita’ di Salerno, Italy X Organization Tieniu Tan Yuan Y Tang Dmitri V Vinorgadov Yury Vizliter Konstantin V Vorontsov Guoyin Wang Jason Wang Narahari Yadati Ning Zhong Chinese Academy of Sciences, Beijing, China Hong Kong Baptist University, Hongkong, China All-Russian Institute for Scientific and Technical Information of Russian Academy of Sciences, Moscow, Russia State Research Institute of Aviation Systems, Moscow, Russia Computing Centre of Russian Academy of Sciences, Moscow, Russia Chongqing University of Posts and Telecommunications, China New Jersey Institute of Technology, USA Indian Institute of Science, Bangalore, India Maebashi Institute of Technology, Japan Additional Reviewers Bhadra, Tapas Dhara, Bibhas Gupta, Lalit Halder, Anindya Jayaraman, Umarani Khan, Aquil Kumar, Rajesh M., Arunkumar Makkapati, Vishnu Marrara, Stefania Nigam, Aditya Prakash, Surya Saha, Sanjoy Kumar Samanta, Syamal Sen, Jayanta Sengupta, Debarka Vajinepalli, Pallavi 460 R Gupta and S Chaudhury However, this approach is slow, since a large number of computations and comparisons are carried out for every pixel Noting that background pixels generally have very low values of saliency, computation of saliency for these pixels is superfluous Hence, we first run a SIFT algorithm and locate the keypoints in the image, which are salient not only spatially but also across different scales We take one keypoint at a time and compute its saliency using [3] If the saliency of this point is above a threshold (0.4 here, required since a keypoint may lie on a cluttered background), we start growing a region from that point The saliency value of neighboring pixels is used as region membership criterion and all pixels visited are marked so that they are not re-visited when a different seed point is chosen We stop when the distance between the new pixel and region mean exceeds a threshold (0.2 here) This feature map is also normalized to [0,1] 2.3 Rarity Conspicuity: Pulse Discrete Cosine Transform A biologically plausible, real time model simulating lateral inhibition in the receptive field has been proposed in [6] It has also been shown to outperform other transform domain approaches like [5] both in terms of speed as well as accuracy over psychological patterns We apply the pulse DCT algorithm to smoothened images to produce our rarity feature map A Gaussian blurred image simulates the scene viewed from a distance and thus finer edge details in a cluttered background are not noticed, leading to a sparser feature map We normalize it to the range [0,1] 2.4 Learning to Integrate the Feature Maps The steps followed for combining the feature maps are as follows First, we selected 30 images, of size 300×400, encompassing the failure cases of each of the feature maps viewers were asked to mark each part of the image they considered salient In accordance with [1], our images (mostly taken from [10]) had well-defined salient regions and hence the markings turned out to be exactly the same for almost all images Then, an MB level, dimensional training data (total 450×30 points) was prepared taking average values of each of the feature maps over each MB of size 16×16 A target class label ’1’ was assigned to an MB if more than half of the pixels of that MB were marked salient; else class label ’0’ was assigned Next, we trained an RVM over this training data as a binary classification problem Here we must point out that we are not really interested in a binary label (salient/non-salient) but the relative saliency value of each MB which will later be used for bit allocation A potential advantage of RVM over SVM, which is desired here, is that it provides posterior probabilities Also, RVM has better generalization ability and its sparser kernel function leads to faster decisions The probabilistic outputs of the RVM formed our final saliency map To test the machine, we generated a testing data from 120 images (450×120 points) and evaluated the saliency maps obtained against ground truth Various authors like Bruce et al [11] have used area under the ROC curves to quantify the quality of their algorithms The ROC curve obtained on our own ground A Scheme for Attentional Video Compression 461 truth data is shown in Fig Also shown in the same figure is a comparison of our result with another leading graph based visual saliency approach [12], which has been shown to outperform various other approaches like [2] We obtained a 0.90048 (s.e 0.00136) area under the curve compared to 0.87009 (s.e 00161) for [12] In the context of application of saliency to video compression, an FN (actually salient but classified non-salient) is costlier compared to an FP A very low FN rate, less than 2%, at the cut-off point reflects the potential of our algorithm for such applications Some results and comparisons with [12] and [11] are shown in Fig A comparison with [3] and [6] is inherent in these results as our local and rarity feature maps respectively It is apparent that our approach is better or at least at par with these other high-ranking approaches Fig ROC curves for our approach and [12] obtained by varying thresholds on saliency values Fig (a) Input image, (b) global, (c) local [3], (d) rarity [6] feature maps, (e) our resized saliency map, (f) saliency map obtained from [12] and (g) [11] 462 R Gupta and S Chaudhury Video Compression Architecture We wish to employ saliency for the purpose of video compression However, computation of feature maps for each video frame can prove to be computationally very expensive if we rely on techniques such as those proposed in [5,8,9] as they necessitate calculation of saliency map of each frame We propose here the use of temporal redundancy inherent in videos to propagate saliency values Ideally the saliency map should be re-calculated only when there is a large change in saliency However, to measure this change, we require the saliency for the next frame which is unavailable Hence, we also propose a workaround to detect the frames for which re-computation of saliency map is indispensable A block diagram of the architecture is shown in Fig which is discussed in detail in the following subsections Fig Our video compression architecture incorporating saliency propagation 3.1 Propagation of Saliency Values Firstly, we describe the need for the mutual information (MI) computation unit The idea is that we perform a re-calculation of saliency map on the basis of MI between successive frames A concise information theoretic shot detection algorithm has been proposed by Cernekova et al in [13] and an improved version of the same using motion prediction in [14] The authors compute the MI between consecutive frames and argue that a small value of MI indicates existence of a cut We experimented with this method over some video sequences, with saliency map of each frame pre-computed, and plotted the MI distributions for color as well as saliency MI for an Airtel ad sequence with scene changes is plotted in Fig It is apparent that not only does this method effectively capture changes in saliency as shown in Fig 4(a), but also, that the RGB and saliency plots follow a very similar distribution (Fig 4(b)) Figure 4(b) implies that we can detect the frames requiring re-computation of saliency maps by calculating MI A Scheme for Attentional Video Compression 463 over the color channels The frame where a large change is detected should be coded as an I frame (or I MBs in H.264) and saliency re-computed for this frame and stored The method has been found to work best on natural video sequences Fig (a) MI plot for saliency maps, (b) MI plots of RGB and saliency overlaid An Airtel ad sequence with cuts is used here For P frames, we make use of motion vectors to approximate saliency values We select an MB in the current frame and look for the best match in the reference frame This best match may or may not exactly overlap an MB in the reference frame, but we have the saliency values for only non overlapping 16×16 MBs Therefore, we take a weighted average of the saliency values of each of the MBs under the best match region the in reference frame, as the saliency value for the MB in current frame The weights correspond to the amount of area overlap as shown in Fig Fig Image illustrating a weighted averaging of saliency values, the orange, blue, yellow, green colors denote the amount of overlap and hence weights 464 3.2 R Gupta and S Chaudhury Selection of Quantization Parameters Once the saliency map is obtained, bits may be non-uniformly distributed across a frame We require a function which can optimally tune the quantization parameters of salient and non-salient MBs to achieve compression, i.e, reduce rate (R), without any significant loss of perceptual quality, i.e, constant distortion (D) In [9], this is posed as a global optimization problem and solved using the method of Lagrange multipliers The final result for quantization step Qistep for the ith MB having a saliency value wi is given as: Qistep = Ws Qstep wi S (1) where W is the sum of saliency values over all MBs, s is the area of M Bi (16×16 here), S is the area of entire frame and Qstep is a fixed value depending on the amount of distortion tolerable This formula implies that the quantization step size should be inversely proportional to the saliency value which is completely justified We present here a short verification of how this formulation achieves compression without compromising on perceptual quality Assuming a R-D function [15] for an M Bi is given by: Di = σi2 e−γRi or Ri = log γ σi2 Di (2) where σi2 is variance of encoding signal and γ is a constant coefficient Ignoring the constant term γ and taking σi2 = 1/α we get: Ri = log αDi (3) Now, the average rate R is calculated as N i=1 sRi /S, where N is the number of MBs Noting that Di ∝ Qistep , we get after replacing Qistep by (1): R= Ns log S αQstep + log (w1 w2 wN ) N w1 + w2 + + wN + log S s (4) From the above equation it is clear that the first term denotes the rate if every MB was quantized with the same parameter Qstep , the second term is always ≤ by the AM-GM inequality and the third term is a constant Thus R is reduced It can also be readily observed from (1) that overall D ( wi Di /W ) remains constant We limit the Qistep to minimum and maximum values of max(0.5 × Qstep , Qistep ) and min(1.5 × Qstep , Qistep ) respectively Also, we smoothen our saliency map using a Gaussian filter before computing the quantization step This serves two purposes, firstly, it ensures that the salient object/region is covered completely and secondly, it ensures a smooth transition from salient to non-salient regions A Scheme for Attentional Video Compression 465 Conclusion A vast amount of research has gone into modelling of the human visual system with each model having its own merits and shortcomings The potential which lies in an integration of these models has been demonstrated by the accuracy of our results A simple and effective learning based approach for such a unification has been presented Though we make use of only features, this model is easily extendible to more features if desired We computed saliency at MB level to save computation, however our model is equally applicable at pixel level The compression framework proposed, to approximate saliency of P frames, can save a lot of computation, speeding-up compression We plan to integrate our it into the H.264 coding system which remains a challenge owing to the complex mode decision metrics and hybrid coding structures in this standard [16] References Engelke, U., Maeder, A., Zepernick, H.J.: Analysing Inter-observer Saliency Variations in Task-Free Viewing of Natural Images In: ICIP, pp 1085–1088 (2010) Itti, L., Koch, C., Niebur, E.: A Model of Saliency-Based Visual Attention for Rapid Scene Analysis IEEE Trans PAMI 20(11), 1254–1259 (1998) Huang, R., Sang, N., Liu, L., Tang, Q.: Saliency Based on Multi-scale Ratio of Dissimilarity In: ICPR, pp 13–16 (2010) Hou, X., Zhang, L.: Saliency Detection: A Spectral Residual Approach In: CVPR, pp 1–8 (2007) Guo, C., Zhang, L.: A Novel Multiresolution Spatiotemporal Saliency Detection Model and Its Applications in Image and Video Compression IEEE Trans Image Proc 19(1), 185–198 (2010) Yu, Y., Wang, B., Zhang, L.: Pulse Discrete Cosine Transform for Saliency-Based Visual Attention In: ICDL, pp 1–6 (2009) Chiang, J., Hsieh, C., Chang, G., Jou, F., Lie, W.: Region-of-Interest Based Rate Control Scheme with Flexible Quality on Demand In: ICME, pp 238–242 (2010) Itti, L.: Automatic Foveation for Video Compression Using a Neurobiological Model of Visual Attention IEEE Trans Image Proc 13(10), 1304–1318 (2004) Li, Z., Qin, S., Itti, L.: Visual Attention Guided Bit Allocation in Video Compression Image and Vision Computing 29(1), 1–14 (2011) 10 Liu, T., Sun, J., Zheng, N.-N., Tang, X., Shum, H.-Y.: Learning to Detect a Salient Object In: CVPR, pp 1–8 (2007) 11 Bruce, N.D.B., Tsotsos, J.K.: Saliency Based on Information Maximization In: NIPS, pp 155–162 (2006) 12 Harel, J., Koch, C., Perona, P.: Graph-Based Visual Saliency In: NIPS, pp 545– 552 (2006) 13 Cernekova, Z., Pitas, I., Nikou, C.: Information Theory-Based Shot Cut/Fade Detection and Video Summarization IEEE Trans CSVT 16(1), 82–91 (2006) 14 Krulikovska, L., Pavlovic, J., Polec, J., Cernekova, Z.: Abrupt Cut Detection Based on Mutual Information and Motion Prediction In: ELMAR, pp 89–92 (2010) 15 Bhaskaran, V., Konstantinides, K.: Image and Video Compression Standards: Algorithms and Architectures Springer, Heidelberg (1997) 16 Chen, Z., Lin, W., Ngan, K.N.: Perceptual Video Coding: Challenges and Approaches In: ICME, pp 784–789 (2010) Using Conceptual Graphs for Text Mining in Technical Support Services Michael Bogatyrev and Alexey Kolosoff Tula State University, Lenin ave 92, 300600 Tula, Russia okkambo@mail.ru, alexey.kolosoff@gmail.com Abstract Text mining problems of natural text classification and fact extraction are important in developing information systems for Technical Support Services An approach which is based on joining acquisition of conceptual graphs and keywords search technique is presented to their solution Conceptual graphs have been created from e-mail queries sent to Technical Support Service Correct conceptual graphs acquired from email texts represent facts and situations which become patterns to search in systems resources to resolve users problems Experimental results of implementing proposed approach are presented Keywords: natural language texts classification, conceptual graphs, correctness of conceptual graphs, technical support services Introduction Text mining strategies share many techniques such as machine learning, natural language processing, text categorization, clustering, filtering, etc These techniques can be classified as ones which use texts words and others which use semantic models constructed from text Two known strategies, Latent Semantic Analysis (LSA) [1] and Formal Concept Analysis (FCA) [2] illustrate that difference LSA uses term-document matrices which describe the occurrences of terms in textual documents and has been created from documents words FCA uses conceptual models - conceptual graphs [3] and conceptual structures (formal concept lattices) which are formal models The mentioned strategies also have different mathematical nature: LSA is founded on geometry and statistics whereas FCA is founded on logic and algebra (the lattice theory) Traditionally only one approach, based on keywords or formal models, is applied in industrial text mining systems Nevertheless modern problems of textual analysis may have significant complexity and it becomes necessary to apply hybrid approaches to solve them In our work namely that complex problem is investigated As the result we decided to apply keywords technique and conceptual graphs in our Text Mining system Although each separate technique does not solve the problem, their combination produces good preliminary results S.O Kuznetsov et al (Eds.): PReMI 2011, LNCS 6744, pp 466–471, 2011 c Springer-Verlag Berlin Heidelberg 2011 Using Conceptual Graphs for Text Mining in Technical Support Services 467 Problem Statement Technical Support Services (TSS) have been intended to help users to solve specific problems with a product - electronics, goods or software Users send queries to TSS as natural language e-mail texts It is needed to resolve queries and to find an appropriate decision represented as help topics, useful URLs or e-mail reply As a rule the system’s reply is prepared manually by support team using system’s resources as it is shown on Fig.1 E-mail Support Team Web Form Forums Self-Help Resources DB (Help, FAQ, How To, etc.) Questions & Answers Database Search Engine User queries: natural language e-mail texts System’s reply: Documents or URLs Fig The structure of a Technical Support Service When the number of queries significantly grows, automation of creating TSS replies becomes very important That automation is implemented in the TSS Search Engine shown on Fig.1 There are two basic text mining problems solved by the Search Engine The first one is the problem of natural text classification The second is the problem of fact extraction These problems have some peculiarities The query text must be classified according to various resources of the TSS TSS database contains documentation, help topics, e-mails of queries and replies To find an appropriate decision it is needed to refer to all these resources The decision may exist as an example answer ready to be sent to a user or it can be constructed from separate pieces Fact extraction problem is to find two kinds of objects in the query texts: things which being analyzed text is about and situations which took users attention General Approach to Solution Considering the problems of natural text classification and fact extraction described in the previous section, we propose the following general approach to their solution: 468 M Bogatyrev and A Kolosoff Having a flow of user e-mail queries we nevertheless not apply classical machine learning technique because the style and contents of queries are very individual But it is worth to collect queries and corresponding replies in the system’s database to apply them in further analysis So, a kind of self-learning is possible in the system Things and situations described in a query are represented by words and phrases So we need to find keywords in the query text which correspond to terms described in system’s database texts Since learning technique is impossible in the system, another way of keywords extraction is needed Text filtering is standard and evidently necessary technique for e-mail texts in natural language We apply term filtering to find direct terms corresponding to the terms described in systems database texts For example, the driver word in a query in the software TSS has high probability to be the term So a text containing this word can be classified as referring to the topic ”Drivers” in the system’s database texts To implement term filtering a thesaurus as an additional resource must be created in the system Text filtering is not exhaustive technique for classification Besides terms, a query text contains many words which can also be useful for analysis The personal style of an author has certain representation in query text as a set of specific words and language grammar distortions (slang) Nevertheless our analysis of real queries shows that the following heuristic principle is valid: despite the personal style, every author uses grammatically correct phrases when describes problematic situations Therefore semantics of these grammatically correct phrases in a query text may represent useful information about situations we need to extract We apply conceptual graphs for modelling semantics of sentences or phrases of the text and use their concepts and relations for further analysis Conceptual graphs acquisition from natural language texts is the problem which has no closed solution for arbitrary texts We assume that the following circumstances cause the success of creating conceptual graphs: query texts are not long and all their sentences may be processed for acquisition in appropriate time; grammatically correct phrases in the sentences produce correct conceptual graphs possibly being sub graphs in incorrect conceptual graph of the whole sentence The following rough criterion of correctness of conceptual graph is admissible here: correct conceptual graph has no isolated concepts An isolated concept is a concept which has no connection to any relation System Implementation and Experimental Results The TSS Text Mining system works according to the following stages Using Conceptual Graphs for Text Mining in Technical Support Services 469 Text documents indexing All TSS documents have been indexed according to selected terms These terms represent topics and main notes presented in system documentation Terms are either single words or several words phrases (no more than words) Term weights are calculated via the well-known tf-idf formula [4] The TSS complex index is the only additional modification of TSS information resource realized in standard database technology (MS SQL Server) Conceptual graphs acquisition and processing Conceptual graphs are applied as an instrument of extracting keywords and key phrases according to the principle described below Search relevant documents in TSS database Keywords and key phrases corresponding to each e-mail text and extracted by conceptual graphs processing have been used as queries for full-text search in TSS indexed database Consider the last two stages in some detail 4.1 Conceptual Graphs Acquisition and Processing We use our software [5] for conceptual graphs acquisition from natural language texts The software is based on existing approaches of lexical, morphological and semantic analysis Semantic roles labeling [6] is applied as the main instrument for constructing relations in acquisition algorithm The acquisition algorithm works with our recently developed controllable grammatical templates Using these templates, it is possible to adapt acquisition algorithm as to certain language grammar (Russian or English in the current version of the system) as to some peculiarities of concrete language User interface has also tools for recognizing incorrect conceptual graphs Conceptual graphs being acquired from all sentences of a query text are applied to detect keywords and key phrases As a rule, incorrect conceptual graphs indicate that there is no useful information in processed text For example, conceptual graph acquired from the ”Thanks in advance” phrase, G1 = {[advance:”] [thank:”]}1 is incorrect since it has no relation TSS user can handle any acquired graph by using interface tools including visualization That helps finding possibly valid keywords in incorrect conceptual graphs All acquired correct conceptual graphs considered as potential source of keywords and key phrases for the next search Concepts connected with the agent relation may represent terms and have been picked as keywords Some term may consist of several words, for example Remote Agent Service The relation genitive in its graph G2 = {[remote*a:”] [service*b:”] [agent*c:”] (genitive?b?c) (attribute?c?a)} indicates that Agent Service is the single whole All graphs having simple structure with genitive and attribute relations are considered as sources of keywords and key phrases Here we use the CGIF format [3] for representing conceptual graphs 470 M Bogatyrev and A Kolosoff It is known that relations in conceptual graphs have linguistic meaning at first But some of them can directly indicate a situation That is the location relation and it is also considered as key phrases indicator For example, it is illustrated by the phrase ”stop on error” and its conceptual graph G3 = {[error*a:”][stop*b:”](location?b?a)} 4.2 Search Relevant Documents All keywords and key phrases extracted by conceptual graphs processing are then treated as queries for full-text search in TSS indexed database For each e-mail text they constitute a query vector We devoted special attention to applying LSA search strategy for such queries We also compared it with other methods which use ranking functions of Okapi BM25 [7], SQL Server iFTS [8] and ranking function of Google An experiment was conducted on the textual database with more than 7000 help topics belonging to online help systems of three different software products Employees of the products vendor company were asked to rate (from to 4) the quality of search results (including their ranking) for top 10 most popular queries retrieved from the users queries statistics A short summary is presented in the table below Table Search results ratings for 10 most popular queries Search Query LSA Okapi BM25 SQL Server iFTS Google working with grids load testing web testing Remote Agent Service name mapping template stop on error object not found UI Automation Silverlight testing flash applications web service testing Total (max 40): 4 4 3 4 37 3 4 4 29 2 4 3 32 3 4 3 4 34 Here Google refers to Google web search in the online help systems of three different software products (with URLs filtering) As one can see from the table, the LSA search gives the best result We can explain it by the following informal conclusion: Latent Semantic Analysis pretends to detect texts which are semantically similar Conceptual graphs processing produces a set of keywords which are semantically connected So, the query produced with conceptual graphs has certain portion of semantics which can resonate with semantics of TSS documents It seems that LSA, according to its mathematical nature, is namely that method which can find such peculiar semantic resonance of texts Using Conceptual Graphs for Text Mining in Technical Support Services 471 Conclusion and Future Work Hybrid approach to textual analysis in Technical Support Services is presented It is based on using conceptual graphs for extracting keywords and key phrases from query text and applying standard full-text search technique Experimental results show that conceptual graphs represent a valid tool for extracting keywords and key phrases since this tool provides semantic connection between words in key phrases Conceptual graphs technique usually produces less number of keywords and key phrases than there are in a query text that shortens the time for further search Future development of presented technology is planned on the way of creating additional information resource in the TSS system This resource will be in the form of conceptual lattice Having conceptual lattices as system’s information resource, we will apply conceptual graphs as immediate queries in search strategy according to the principles of FCA References Landauer, T., Foltz, P.W., Laham, D.: Introduction to Latent Semantic Analysis Discourse Processes 25, 259–284 (1998) Ganter, B., Wille, R.: Formal Concept Analysis Mathematical Foundations Springer, Heidelberg (1999) Sowa, J.F.: Conceptual Structures: Information Processing in Mind and Machine Addison-Wesley, London (1984) Salton, G., McGill, M.J.: Introduction to modern information retrieval McGrawHill, New York (1983) Bogatyrev, M.Y., Mitrofanova, O.A., Tuhtin, V.V.: Building Conceptual Graphs for Articles Abstracts in Digital Libraries In: Fourth Conceptual Structures Tool Interoperability Workshop (CS-TIW 2009) at 17th International Conference on Conceptual Structures (ICCS 2009), Moscow, pp 50–57 (2009) Gildea, D., Jurafsky, D.: Automatic labeling of semantic roles Computational Linguistics 28, 245–288 (2002) Robertson, S., Walker, S., Jones, S., et al.: Okapi at TREC-3 In: Proceedings of the Third Text Retrieval Conference (TREC 1994), Gaithersburg, USA (1994) Langit, L., Goff, K., Mauri, D., Malik, S.: Smart Business Intelligence Solutions with Microsoft SQL Server 2008 Microsoft Press (2009) Erratum: Evaluation of Semantic Term and Gene Similarity Measures Michal Kozielski and Aleksandra Gruca Silesian University of Technology, Akademicka 16, 44-100 Gliwice, Poland {michal.kozielski,aleksandra.gruca}@polsl.pl S.O Kuznetsov et al (Eds.): PReMI 2011, LNCS 6744, pp 406–411, 2011 © Springer-Verlag Berlin Heidelberg 2011 –––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––– DOI 10.1007/978-3-642-21786-9_76 By mistake, the following funding information was not included in the original version of the paper: Acknowledgments This paper was partially supported by the European Community through the European Social Fund _ The original online version for this chapter can be found at http://dx.doi.org/10.1007/978-3-642-21786-9_66 _ Author Index Acharya, Tinku 186 Agrawal, Rakesh Bandyopadhyay, Oishila 122 Bandyopadhyay, Sanghamitra 412 Banerjee, Minakshi 167 Banerjee, Mohua 339 Baragona, Roberto 382 Basu, Amrita 154 Ben Ahmed, Mohamed 370 Bhandari, Dinabandhu 280 Bhattacharya, Anindya 394 Bhattacharya, Bhargab B 122 Bhattacharyya, Dhruba K 110 Bhattacharyya, Malay 412 Bhowmik, Subrata 311 Bhowmik, Tapan Kumar 432, 446 Biswas, Rajarshi 86 Biswas, Sambhunath 86 Bogatyrev, Michael 466 Bolkhovityanov, Alexander 248 Bong, Chin Wei 92 Bouguila, Nizar 364 Boutemedjet, Sabri 364 Boyadzieva, Desislava 440 Brodi´c, Darko 418 Bui, The Duy 116 Doroshenko, Jaser 130 Dulkin, Lev 130 Dutta, Soma 80 Dutta Majumder, D 324 Dutta Roy, Sumantra 154 Felizardo, Rui Fenner, Trevor 3 Gaonkar, Bhakti 60 Garg, Vikram 206 Ghosh, Anupam 388 Ghosh, Ashish 173, 318 Ghosh, Shantanu 154 Ghosh, Susmita 98, 318 Ghoshal, Ranjit 446 Gluhchev, Georgi 440 Goel, Piyush 299 Gollapudi, Sreenivas Golovachev, Sergey 351 Golubev, Sergey 424 Gopal, Madan 206 Goswami, A 324 Gruca, Aleksandra 406, E1 Guha, Prithwijit 200 Gultyaeva, Tatyana A 30 Gupta, Rupesh 458 Chakraborty, Debarati 193 Chanda, Bhabatosh 122 Charrad, Malika 370 Chaudhury, Santanu 154, 206, 213, 458 Chepovskiy, Andrey 248 Chervonenkis, Alexei Ya 21 Choudhury, Lopamudra 154 Chowdhury, Manish 167 Hariharan, Divya 186 Hassan, Ehtesham 206 Hubballi, Neminath 36 Das, Sudeb 286 Das, Suprabhat 220 De, Rajat K 388, 394, 400 Desarkar, Maunendra Sankar Descombes, Xavier 142 Dey, Lipika 60 Dobrov, Boris 235 Dogra, Debi Prosad 160 Kaiser, Tim B 43 Kannan, Anitha Karan, Sankar 324 Kenthapadi, Krishnaram Khan, Md Aquil 339 Kharitonov, Evgeny 358 Kim, Sang-Woon 74 Kolosoff, Alexey 466 268 Ivahnenko, Andrey Jiang, Feng Joshi, Rahul 66 333 268 474 Author Index Komech, Sergey 142 Kozielski, Michal 406, E1 Krasotkina, Olga V 24 Kumskov, Mikhail 49 Kundu, Lopamudra 280 Kundu, Malay Kumar 167, 286, 293 Kundu, Suman 242 Lall, Brejesh 154, 213 Lam, Hong Yoong 92 Le, Thanh Ha 116 Lepskiy, Alexander 54 Loukachevitch, Natalia 235 Maiti, Arpan Kumar 293 Maiti, Saptaditya 274 Majumdar, Arun Kumar 160 Mandal, Aditi 318 Mandal, Deba P 274 Maniyar, Amit 154 Masaki, Evensen E 74 Maulik, Ujjwal 412 Michalak, Marcin 345 Mirkin, Boris Mitra, Pabitra 104, 220, 274 Mitra, Sushmita 186 Mottl, Vadim V 24 Mukherjee, Jayanta 160 Mukherjee, Suchandra 160 Mukhopadhyay, Jayanta 104, 299 Murthy, C.A 242 Nandi, Debyani 148 Nandi, Sukumar 36 Nascimento, Susana Nayak, Losiana 400 Nguyen, Duy Khuong 116 Novitskiy, Valeriy I 261 Oommen, B John 13 ă urk, Pinar 227 Oztă Pal, Rajarshi 104 Pal, Sankar K 242, 280 Pande, Nipun 200 Parekh, Nita 376 Parui, Swapan K 148, 446 Patra, Bidyut Kr 36 Perevoznikov, Aleksandr 49 Permiakov, Evgenii 49 Plastinin, Anatoliy 136 Polezhaeva, Elena 452 Popov, Alexander A 30 Prasath, Rajendra 227 Prashanth, R 154 Rigoll, Gerhard 305 Roy, Anandarup 148, 446 Roy, Moumita 98 Roy, Utpal 148 Roytberg, Mikhail 17 Sahoo, Chitta Ranjan 305 Salakhutdinov, Viktor 130 Salamat, Nadeem 180 Sanchez, A 305 Sapre, Manasi 376 Sarkar, Sudeshna 268 Sarmah, Sauravjyoti 110 Schmidt, Stefan E 43 Schomaker, Lambert 432 Shamshurin, Ivan 254 Shestov, Alexey 49 Sikora, Beata 345 Sikora, Marek 345 Singh, Arun 160 Singh, Ashish 154 Smetanin, Yury 130 Subudhi, Badri Narayan 173 Sur, Arijit 299 Sural, Shamik 160, 305 Turkov, Pavel A 24 Uma Shankar, B 193 van Oosten, Jean-Paul Venkatesh, M.S 213 Vorontsov, Konstantin Zahzah, El-hadi 180 Zhou, Lin 333 Ziou, Djemel 364 432 66 ... India Indian Statistical Institute, Kolkata, India Indian Statistical Institute, Kolkata, India Tata Consultancy Services Ltd., New Delhi, India Indian Statistical Institute, Kolkata, India University... Kharagpur, India DAIICT, Gandhinagar, India Indian Statistical Institute, Kolkata, India Indian Statistical Institute, Kolkata, India Indian Institute of Technology Kharagpur, Kharagpur, India Indian... Kong, China Higher School of Economics, Moscow, Russia Indian Statistical Institute, Kolkata, India Indian Statistical Institute, Kolkata, India Indian Statistical Institute, Kolkata, India Indian