The book Multimedia information retrieval: Theory and techniques focuses on the processing and search tools applicable to the management of new multimedia documents. These matters merge in the methodology of MIR, an organic system composed of the TR, VR, VDR and AR systems.
Multimedia Information Retrieval CHANDOS INFORMATION PROFESSIONAL SERIES Series Editor: Ruth Rikowski (Email: Rikowskigr@aol.com) Chandos’ new series of books is aimed at the busy information professional They have been specially commissioned to provide the reader with an authoritative view of current thinking They are designed to provide easy-to-read and (most importantly) practical coverage of topics that are of interest to librarians and other information professionals If you would like a full listing of current and forthcoming titles, please visit www.chandospublishing.com or email wp@woodheadpublishing.com or telephone +44(0) 1223 499140 New authors: we are always pleased to receive ideas for new titles; if you would like to write a book for Chandos, please contact Dr Glyn Jones on gjones@chandospublishing.com or telephone +44 (0) 1993 848726 Bulk orders: some organisations buy a number of copies of our books If you are interested in doing this, we would be pleased to discuss a discount Please email wp@woodheadpublishing.com or telephone +44 (0) 1223 499140 Multimedia Information Retrieval Theory and techniques ROBERTO RAIELI Oxford Cambridge New Delhi Chandos Publishing Hexagon House Avenue Station Lane Witney Oxford OX28 4BN UK Tel: +44(0) 1993 848726 Email: info@chandospublishing.com www.chandospublishing.com www.chandospublishingonline.com Chandos Publishing is an imprint of Woodhead Publishing Limited Woodhead Publishing Limited 80 High Street Sawston Cambridge CB22 3HJ UK Tel: +44(0) 1223 499140 Fax: +44(0) 1223 832819 www.woodheadpublishing.com First published in 2013 ISBN: 978-1-84334-722-4 (print) ISBN: 978-1-78063-388-6 (online) Chandos Information Professional Series ISSN: 2052-210X (print) and ISSN: 2052-2118 (online) Library of Congress Control Number: 2013941270 © R Raieli, 2013 British Library Cataloguing-in-Publication Data A catalogue record for this book is available from the British Library All rights reserved No part of this publication may be reproduced, stored in or introduced into a retrieval system, or transmitted, in any form, or by any means (electronic, mechanical, photocopying, recording or otherwise) without the prior written permission of the publisher This publication may not be lent, resold, hired out or otherwise disposed of by way of trade in any form of binding or cover other than that in which it is published without the prior consent of the publisher Any person who does any unauthorised act in relation to this publication may be liable to criminal prosecution and civil claims for damages The publisher makes no representation, express or implied, with regard to the accuracy of the information contained in this publication and cannot accept any legal responsibility or liability for any errors or omissions The material contained in this publication constitutes general guidelines only and does not represent to be advice on any particular matter No reader or purchaser should act on the basis of material contained in this publication without first taking professional advice appropriate to their particular circumstances All screenshots in this publication are the copyright of the website owner(s), unless indicated otherwise This book was originally published in Italian by Editrice Bibliografica s.r.l., Milan, Italy, with the title Nuovi metodi di gestione dei documenti multimediali Revised English edition Translated by Giles Smith Typeset by Domex e-Data Pvt Ltd., India Printed in the UK and USA List of figures and tables Figures 1.1 Set of the film 1900, by Bernado Bertolucci 21 1.2 a) Comparisons of the shapes of various pipes for visual searches of these objects b) Images of the Churchwarden pipe 23 1.3 D-R head 26 2.1 Drawing representing the famous Magritte painting 37 3.1 a) Terminological search attempts applied to a painting by Roberto Sicilia b) Content-based search founded on concrete, figurative data on the same painting by Roberto Sicilia 80 4.1 Example from Grosky: multimedia content-based indexing 95 4.2 Another example from Grosky: content-based multimedia search 97 4.3 Hierarchy of possible representative levels in a document 121 4.4 Example of ‘collaborative filtering from Amazon’s website 129 5.1 The organic MIR system 139 5.2 Example of a textual-textual search 140 5.3 Example of a textual-visual search 140 5.4 Example of a visual-textual search 141 5.5 Example of a visual-visual search 141 5.6 Example of term-based IR 143 5.7 Example of CBIR 144 5.8 Example of TR 147 ix Multimedia Information Retrieval 5.9 Example of MIR 148 5.10 a) Formal comparison between search example and b) archived model 151 5.11 Selection of a colour range as an example for searching 153 5.12 Analysis of the constitutive elements of a video 159 5.13 Video-browsing modes a) Slide show b) Storyboard 160 5.14 A scheme of representative image selection 162 5.15 A recapitulative image of the different video processing constraints in relation to their computability 163 ‘Talk to Me’ interface, didactic system of Automatic Speech Recognition 165 5.17 ‘AudioFex’ AR module of the MUVIS system 166 5.18 ‘Beat Histogram’ of different styles of music 167 5.19 ‘Similarity matrix’ of a Bach Prelude 168 6.1 Example of the Scheda F on the Album di Romana site 178 6.2 ‘Collage summary’ built starting from the descriptive metadata in a video produced by the Informedia II system 180 Model of the possible applications of MPEG-7 in information processes 183 7.1 JACOB’s start-up interface demo 196 7.2 Start page of the ECHO site 197 7.3 The MILOS system home page 198 7.4 Search phases in QuickLook a) Browsing and image model choice b) Definition of textual data c) System answer and indications of ‘relevance/non-relevance’ d) Final system response 5.16 6.3 x 199–200 7.5 AESS home page 201 7.6 Scanner realize during the VASARI project 203 7.7 Demo of a visual search by VIPER 204 7.8 The ‘PicToSeek’ search screen a) The system’s selection interface b) Upload from the Web of a search image c) Individuation of a more precise model d) New response by the system 205–6 List of figures and tables 7.9 Demo of the ‘Sphere browser’ of the MediaMill system 207 7.10 Publicity screen for the Shazam system 209 7.11 Application of AudioID via a cell phone 210 7.12 QBIC search interface a Search through colour range b Search through structural data 211 Example of a colour search of the Hermitage DC a Definition of the colour range b Search results 212 Example of a colour-formal search of the Hermitage DC a Definition of the colours and forms b Search results 213 Search interface based on colour histograms on the WebSEEK browser 214 7.16 WebSEEK module for defining the search histogram 215 7.17 Example of a search through sketches using Retrievr 216 7.18 Interface of the Virage VS LiveMedia system, developed using VideoLogger technology 217 Operating phases of the Informedia II system a Analysis of a documentary video b Relationship model of the analyzed elements 218 7.13 7.14 7.15 7.19 7.20 7.21 Phases in the demo driven by Sound Fisher a Similarity search b Filter of the search with specific varied data c Addition of music tracks to the system d Content-based rearrangement of the archive 220–1 Video Mail Retrieval system model a Browser b Search interface 222 7.22 Screen of Bing Visual Search 224 7.23 Example of Google Goggles’ functionality 225 7.24 Module for radiology content-based analysis from a system designed at the National Library of Medicine 233 Visual analysis and search screen of the GIS Web Enterprise Suite system 234 7.25 8.1 Example of content-related structural analysis of a visual document a Segmentation of an image into blocks b Grey scale calculation for each block c Complete light-dark histogram of the structure 245 xi Multimedia Information Retrieval 8.2 Automatic analysis model of a multimedia object a 3D model b Structural definition c Calculation of the representative vectors 247 Low-level characteristics in a document a Original VO b Form and skeleton c Extremities and ‘Centre of Gravity’ 250 Definitions of the ‘meanings’ of VO using low-level characteristics: models of bowling, ski slalom, golf, baseball and ski-jumping 251 8.5 Figurative example of the square-triangles 253 8.6 Example of similarity match in a musical search a Search model b Bach fugue with elements similar to the model 255 Definition and omission of a search sample a Search via model design b Modification of retrieved object c New search using the modified sample 259 Searching in the ‘photographs’ archive of PicToSeek via the ‘colour invariant’ parameter 262 Search in the ‘graphics’ archive of PicToSeek via the ‘shape invariant’ parameter 263 Demo page of the Video Content Description and Exploration tool (ViCoDE), one of the products of the Viper Group 266 8.11 Viper Group website 268 8.12 Model of fingerprint treatment according to the MPEG-7 module 271 8.13 AudioID’s promotional website 273 9.1 Noise example as a consequence of a formal search using Retrievr 279 Example of information loss in a colour search using the WebSEEK system 281 Model of an integrated content-based and ‘concept-based’ system developed during the Sculpteur project, promoted by the European Union 285 MAVIS system architecture 288 8.3 8.4 8.7 8.8 8.9 8.10 9.2 9.3 9.4 xii List of figures and tables 9.5 9.6 9.7 Demo of colour-formal query using the QBIC system interface, as applied to the Hermitage Digital Collection 290 Example of a search conducted with contentual and textual parameters, run using QBIC’s interface developed for the Sculpteur project 291 Relationship model between VR, OPAC and a Virtual Library System 296 Tables 4.1 Example of relationships between different image and user types 104 4.2 ‘Concept-based’ and content-based search models 119 5.1 Comparison between IR and MIR systems 136 5.2 The MIR system 137 5.3 Database and index creation 138 5.4 Search and retrieval process 138 5.5 Advanced functions 138 5.6 Query and search model presented by Enser 156 8.1 Steps required for the content-based treatment of materials and the creation of a database 244 8.2 Operational phases during document search and retrieval 254 9.1 Stages of implementation of the VDR project via the MILOS operating system 298–9 xiii Multimedia Information Retrieval Hjorland, B (1997) ‘Information seeking and subject representation: an activity-theoretical approach to Information Science’, (Westport: Greenwood) Hjorland, B (1992) ‘The concept of ‘subject’ in Information Science’, Journal of Documentation 48 (2): 172–200 Hjorland, B and Nissen Pedersen, K (2005) ‘A substantive theory of classification for Information Retrieval’, Journal of Documentation 61 (5): 582–597 Hjorland, B (2005) (ed), ‘Library and Information Science and the Philosophy of Science’, Journal of Documentation 61 (1 [special]) Hutchins, W J (1978) ‘The concept of “aboutness” in subject indexing’, Aslib Proceedings 30 (5): 172–181 Iconclass Research and Development Group (2012), www.iconclass.nl Ishaya, T and Eze, E (2007) ‘Context-based multimedia ontology model’, in Third International Conference on Autonomic and Autonomous Systems, ICAS07, (New York: IEEE), 2–8 Jakobson, R (1962–1988) ‘Selected writings’, (Gravenhage: Mouton) Jakobson, R and Waugh, L (1979) ‘The sound shape of language’, (Brighton, Harvester) Jansen, B J et al (2008) ‘Handbook of web log analysis’, (Hershey: Idea Group) Klusch, M (ed) (1999) ‘Intelligent information agents: agent-based information discovery and management on the Internet’, (Berlin: Springer) Knowledge Map of Information Science (2010), www.success.co.il/is/ index.html Lancaster, F W (2003) ‘Indexing and abstracting in theory and practice’, (Urbana Champaign: University of Illinois-Graduate School of LIS) Lancaster, F W (2001) ‘Intelligent technologies in library and information services’, (Medford: Information Today) Lancaster, F W (1989) ‘The perspective: natural language versus controlled language: a new examination’, in Charles Oppenheim et al (eds), Perspectives in information management, (London: Butterworths) 1–23 Lancaster, F W (1979) ‘Information Retrieval systems: characteristics, testing and evaluation’, (Wiley: New York) Lankes, R D et al (2006) ‘Participatory networks: the library as conversation’, (Chicago: ALA), www.iis.syr.edu/projects/PNOpen/ ParticiaptoryNetworks.pdf Liew, C L (2009) ‘Digital library research 1997–2007: organisational and people issues’, Journal of Documentation 65 (2): 245–266 328 Bibliography Linckels, S and Meinel, C (2011) ‘E-librarian service: user-friendly semantic search in digital libraries’, (Berlin: Springer) Lopez Yepes, J (2005) ‘Il concetto di documentazione e il suo riflesso nella formazione di professionisti e ricercatori in Spagna’, Bollettino AIB 45 (3): 277–294 Lu, K (1994) ‘Technological challenges to artists’ rights in the age of multimedia: the future role of moral rights’, Reference Services Review 22 (1): 9–19 Lukasiewicz, T (2006) ‘Fuzzy description logic programs under the answer set semantics for the semantic Web’, in Second International Conference on Rules and Rule Markup Languages for the Semantic Web, (New York: IEEE), 89–96 Mai, J E ‘The future of General Classification’, Cataloging and Classification Quarterly 37 (1–2 [special]): 3–12 McGill, M J and Salton, G 1983 ‘Introduction to modern Information Retrieval’, (New York: McGraw-Hill) Melucci, M and Baeza-Yates, R (2011) ‘Advanced topics in Information Retrieval’, (Berlin: Springer) Mescini, F (2005) ‘Le mappe topiche: come imparai a non preoccuparmi e ad amare i metadati’, Bollettino AIB 45 (1): 59–72 Messina, M (2005) ‘Un Manifesto per le biblioteche digitali’, Bollettino AIB 45 (4): 489–498 Metitieri, F (2002) ‘Il semantic Web è un sogno romantico’, Week.it 16 (36): 28 Metitieri, F and Ridi, R (2007) ‘Biblioteche in rete: istruzioni per l’uso’, (Rome: Laterza) Miccoli, S (2005) ‘Questioni di epistemologia biblioteconomica’, Bollettino AIB 45 (4): 415–438 Hsu, M.H and Chen H H (2011) ‘Efficient and effective prediction of social tags to enhance Web search’, Journal of the American Society for Information Science and Technology 62 (8): 1473–1487 Molholt, P and Petersen, T (1993) The role of the ‘Art and Architecture Thesaurus’ in communicating about visual art’, Knowledge Organization 20 (1): 30–34 Montani, P (1993) ‘Fuori campo: studi sul cinema e l’estetica’, (Urbino: Quattroventi) Morrielo, R (2008) ‘La gestione delle raccolte digitali in biblioteca’, (Milano: Bibliografica) National Library of Medicine (2011) ‘Medical Subject Headings (MeSH)’, www.nlm.nih.gov/mesh 329 Multimedia Information Retrieval Nester Kresh, D (2000) ‘Offering high quality reference service on the web: the Collaborative Digital Reference Service (CDRS)’, D-Lib Magazine (6), www.dlib.org/dlib/june00/kresh/06kresh.html OCLC/RLG Working Group on Preservation Metadata (2001) ‘Preservation metadata for digital objects: a review of the state of the art’, www.oclc.org/research/pmwg/presmeta_wp.pdf Otlet, P (1990) ‘International organisation and dissemination of knowledge: selected essays of Paul Otlet’, (Amsterdam: Elsevier) Otlet, P (1934) ‘Traité de documentation’, (Bruxelles: Mundaneum) Panofsky, E (1955) ‘Meaning in the visual arts: papers in and on Art history’, (Garden City: Doubleday) Perner, P (2012) ‘Advances in data mining: applications and theoretical aspects’, in Proceedings of the 12th Industrial Conference, ICDM 2012, (Berlin: Springer) Petrelli, D and Clough, P (2012) ‘Analysing user’s queries for crosslanguage image retrieval from digital library collections’, Electronic Library, 30 (2): 197–219 Quintarelli, A (2005) ‘Folksonomies: power to the people’, in Isko Italia Documenti Ridi, R (2007) ‘La biblioteca come ipertesto: verso l’integrazione dei servizi e dei documenti’, (Milano: Bibliografica) Ridi, R (2004) ‘La biblioteca digitale: definizioni, ingredienti e problematiche’, Bollettino AIB 44 (3): 273–344 Ridi, R et al (2004) ‘Come vogliamo chiamarli? Operatori booleani e altre tecniche di Information Retrieval negli OPAC italiani’, Bibliotime (3), www2.spbo.unibo.it/bibliotime/num-vii-3/ridiecc.htm Rijsbergen, C J K Van (1979)‚ Information Retrieval’, (London: Butterworths) Roncaglia, G (2006) ‘Folksonomies e semantiche controllate’, in Interoperabilità delle biblioteche digitali, (Rome), www.host.uniroma3 it/biblioteche/download/Roncaglia_Folksonomies.pdf Salarelli, A (2008) Biblioteca e identità: per una filosofia della biblioteconomia, (Milano: Bibliografica) Salarelli, A ‘Per un’ontologia della biblioteconomia nell’era digitale’, in Meli, R (ed), Biblioteche e informazione nell’era digitale, (Palermo: AIB Sicilia), 235–55 Salarelli, A (2004) ‘Bit-à-brac: informazione e biblioteche nell’era digitale’, (Reggio Emilia: Diabasis) Serrai, A (2001) ‘Il cimento della Bibliografia’, (Milan: Bonnard) Serrai, A (1995) ‘Guida alla Biblioteconomia’, a cura di Maria Cochetti, (Florence: Sansoni) 330 Bibliography Serrai, A (1984) ‘Dall’informazione alla Bibliografia: la professione bibliotecaria’, (Milano: Bibliografica) Serrai, A (1979) ‘Del catalogo alfabetico per soggetti: semantica del rapporto indicale’, (Rome, Bulzoni) Serrai, A (1974) ‘Indici, Logica e Linguaggio: problemi di catalogazione semantica’, (Rome: Bulzoni) Serrai, A ‘Biblioteconomia come scienza: introduzione problemi e alla metodologia’, (Florence: Olschki) Shabajee, P (2002) ‘Primary multimedia objects and ‘educational metadata: a fundamental dilemma for developers of multimedia archives’, D-Lib Magazine (6): 34–46, www.dlib.org/dlib/june02/ shabajee/06shabajee.html Shadbolt, N et al (2006) ‘The semantic Web revisited’, IEEE Intelligent Systems 21 (3): 96–101 Shiri, A (2008) ‘Metadata-enhanced visual interfaces to digital libraries’, Journal of Information Science 34 (6): 763–775 Sicilia, M A et al (2010) (eds), ‘Ontology, conceptualization and epistemology for information systems, software engineering and service science’, Proceedings of the 4th International Workshop, ONTOSE, (Berlin: Springer) Simone, R (2004) ‘Fondamenti di linguistica’, (Rome: Laterza) Soergel, D (1985) ‘Organizing information: principles of data base and retrieval systems’, (Orlando: Academic Press) Solimine, G (2007) Verso una Biblioteconomia 2.0?’ Bollettino AIB 47 (4): 433–434 Solimine, G (2004) ‘La biblioteca: scenari, culture, pratiche di servizio’, (Rome: Laterza) Solimine, G (1995) ‘Introduzione allo studio della Biblioteconomia: riflessioni e documenti’,(Manziana: Vecchiarelli) Solimine, G and Weston, P G (2007) (eds), ‘Biblioteconomia: principi e questioni’, (Rome: Carocci) Sparck Jones, K (1971) ‘Automatic keyword classification for retrieval’, (London: Butterworths) Sparck Jones, K (1970) ‘Some thoughts on classification for retrieval’, Journal of Documentation 26 (2): 89–101 Sperring, D (2008) ‘Libraries, the Internet, Web 2.0 and Library 2.0’, One-Person Library 25 (2): 5–6 Spink, A et al (2007) ‘Multitasking information behaviour: an exploratory study’, Journal of Librarianship and Information Science 39 (3): 177–186 331 Multimedia Information Retrieval Spink, A and Zimmer, M (2008) (eds), ‘Web searching: multidisciplinary perspectives’, (Dordrecht: Springer) Svenonius, E (2000) ‘The intellectual foundations of information organization’, (Cambridge: MIT) Svenonius, E (1994) ‘Access to nonbook materials: the limits of subject indexing for visual and aural languages’, Journal of the American Society for Information Science 45 (8): 600–606 Swift, D F et al (1979) ‘A sociological approach to the design of information systems’, Journal of the American Society for Information Science 30 (4): 215–223 Swift, D F et al ‘A multi-modal approach to indexing and classification’, International Classification (2): 90–94 Tammaro, A M (2007) ‘Lo sviluppo delle biblioteche digitali in Italia: i risultati di un’indagine’, Bollettino AIB 47 (3): 335–348 Tursi, A.(2005) (ed),‘Mediazioni: spazi, linguaggi e soggettività delle reti’, (Milano: Costa and Nolan) Wactlar, H D and Christel, M G (2002) ‘Digital video archives: managing through metadata’, in Building a national strategy for preservation: issues in digital media archiving, (Council on Library and Information Resources: Library of Congress), www.clir.org/pubs/ reports/pub106/video.html Waters, D J and Garrett, J (1996) ‘Preserving digital information: report of the Task Force on Archiving of Digital Information’, (Washington: Commission on Preservation and Access) Waters, D J and Kenney, A (1994)‚ The Digital Preservation Consortium: mission and goals’, (Washington: Commission on Preservation and Access) Weal, M J et al (2007) ‘Ontologies as facilitators for repurposing web documents’, International Journal of Human-Computer Studies 65 (6): 537–562 WebSemantico.org (2009), www.websemantico.org Weston, P G (2002) ‘Il catalogo elettronico: dalla biblioteca cartacea alla biblioteca digitale’, (Rome: Carocci) Williamson, N J and Beghtol, C (2003) (eds), ‘Knowledge organization and classification in international Information Retrieval’, (New York: Haworth) Pubblicato anche in Cataloging and Classification Quarterly 37 (1–2 [special]) Wilson, T D (2000) ‘Human information behaviour’, Informing Science (2): 49–55 332 Bibliography Wittgenstein, L (1953) ‘Philosophische Untersuchungen’, (Oxford: Blackwell) English ed: (1973) Philosophical investigations, (New York: Macmillan) World Wide Web Consortium (W3C) (2012), www.w3.org Xie, H (2008) ‘Users’ evaluation of digital libraries: their uses, their criteria, and their assessment’, Information Processing and Management 44 (3): 1346–1373 Zins, C (2007) ‘Knowledge map of information science’, Journal of the American Society for Information Science and Technology 58 (4): 526–535 333 Index Access to non-book materials (Elaine Svenonius), 83–4 AESS (Archivio di Etnografia e Storia Sociale) project, 201 Aleph database, 294–8 An overview of multimedia contentbased retrieval strategies (Ankush Mittal), 247 Anglo-American Cataloguing Rules (AACR), 178 Annodex technology, 187 Art and Architecture Thesaurus (AAT), 72–5 Audio browsing, 164–5 Audio Retrieval (AR), 18–19, 112–15 application, 185–8, 219, 223 audio-fingerprint, 182, 191, 208 AudioID system, 185, 208–9, 268, 270, 272 audio-visual documents See Audio-visual document automatic analysis, 167–9 document processing, 166 labelling, 208–10 management, 181–2 modalities, 268–73 music information retrieval, 62–3, 165–9, 187, 190 objectives, 269 operations, 271 processing, 272 research programmes, 208–9 SoundFisher, 219 speech recognition, 165, 219, 223 Speechbot-Audio Search, 223 SpeechSkimmer, 223 state of the art, 190–1 technical problems, 280, 282 tempo-related properties, 163–9 Video Mail Retrieval system, 219 Audio-fingerprint, 182, 191, 208 AudioID system, 185, 208–9, 268, 270, 272 Audio-thumbnailing, 164–5 Audio-visual document feature extraction, 161 frame-based extraction, 249 management, 179–81 ‘object-based’ model, 249 operations for implementing, 267–8 segmentation, 160 semantic indexing, 249 structure analysis, 161–2 video ‘abstraction,’ 162 video document browsing, 161 video-summary, 160 visual retrieval and, 158–60 Audio-visual media, 159, 226–7 Automatic Speech Recognition (ASR), 165 Batley, Sue, 101 Beat histogram, 167 Benchmarking, 44, 207, 228–9, 231 Berners-Lee, Tim, 125, 127 Biagetti, Maria Teresa, 69, 71 Bibliography, 51–2 335 Multimedia Information Retrieval Biblioteconomia come scienza (Alfredo Serrai), 33 Bisogno, Paolo, 20, 24–5, 41–2, 44, 47, 51 Briet, Suzanne, 81 British Library’s projects, 202 British Shazam system, 208 Buckland, Michael, 79–81 Casey, Michael, 187 Castellucci, Paola, 46–7 CLIOH (Cultural Digital Library Indexing our Heritage) project, 186–7 Coates, Eric, 69 Colour histogram, 280 Colour search module, 211, 214 Conniss, Lynne, 102 Conservation metadata, 294 Content Based Visual Query Project, 214–15 Content Management System (CMS), 197 Content-Based Information Retrieval (CBIR), 22–4, 82–5 audio, 163–4 automatic methods, 112, 115–16, 118, 122–3, 284–5 benefits, 105–6, 111 hybrid systems, 110, 119, 284–5 image retrieval, 117–18 intergration with semantic methods and principles, 116–24 learning methods, 256, 260 limitation, 103, 108–10 multimedia objects and, 112–14 navigation, 143 PicToSeek, 264 pragmatic approach, 145–6 purposes and objectives, 110–11 search interfaces, 290–2 searching and retrieving documents, 256–7 semantic gap, 292 336 semantic level, 119–21 stratification, 248 technical problems, 257 textual, 143–7, 286 user perspective, 118–19 visual documentation, 109 Copyright management metadata, 294 Cordeiro, Maria Inês, 64–5 Cremer, Markus, 185, 268, 270 Cross Languages Information Retrieval (CLIR), 66 Data modelling See Multimedia data modelling Data processing, multimedia, 112–13 Database biomedical, 232–3 content-based search, 94–6 image, 19, 176–7, 258 index creation, 243–7 DataBase Management System (DBMS), 267 Deb, Sagarmay, 249, 252–3 Descriptive metadata, 177–8 Digital librarianship, 38–9 Digital Libraries Initiative, 187–8 Digital library, 90 Library 2.0, 17 multimedia, 91 principles, 37–41 Web 2.0, 17 Digital reference services, 106–8 Digital technology, 15, 17 Digital video archives, 179 Dimitrova, Nevenka, 161, 186 Document See also Documentation analysis, 243–7 audio-visual See Audio-visual documents definition, 81 digital, 18 epistemology, 41–2, 44 management systems, 292–4 Index multimedia content-based, 79, 82–5 index creation in, 94 management, 90–1, 98, 175–9 metadata, 293–4 principle, 93 retrieval, 256–7 representation, 59–60, 69 surrogates, 39–40 text-only, 19 visual, 81 Documentation See also Library science and Documentation analysis, 243–7 audio, 181–2, 187 audio-visual, 179–81 definitions, 42–3 dialectic composition, 53 heuristics, 43–4 José López Yepes, 46–7 LIS and, 46–8 multimedia, 15, 42, 81–2 Paola Castellucci, 46–7 Paolo Bisogno, 41–2, 44, 47, 51 phases of search, 253–5 positioning, 51–3 principles, 41–6 systems of management, 175–9 theory, 24–5, 42–3, 81–2 visual retrieval, 265–6 Dublin Core Metadata Initiative, 181, 187 Dynamic Bayesian Networks (DBNs), 249–50 EBU P/Meta project, 181 ECHO (European CHronicles On-line), 185, 196–7 Educational metadata, 293–4 El Hadi, Widad Mustafa, 65 ELISE (Electronic Library Image Service for Europe), 202 Enser, Peter, 101–10, 117–22, 146, 148–9, 153–7, 184, 189, 201, 283–6 Epistemology, 24, 33–7, 41–2, 44, 49–50 Example Driven Image Database Querying, 215 Flank, Sharon, 229–30 Folksonomy collaborative filtering tools, 129 definition, 126 development, 128 intergration with ontologies, 126–9, 146–7 NLP and, 66, 71 reasons for, 126 semantic web, 125–6, 146–7 Frost, Olivia, 102 Gambari, Stefano, 176–7 Gargiulo, Paola, 106, 108 Gevers, Theo, 260–1 GIS (Geographical Information System), 234, 236 Google Goggles, 223, 225–6 GRIM-DBMS (GRaphical IMage DataBase Management System), 195–6 Grosky, William, 92–3 Grund, Angelika, 75–7 Guido programme, 208 Guy, Marieke, 126–8 Hare, Jonathon, 120 Herre, Jürgen, 185, 268, 270 Hjørland, Birger, 67, 71, 145 Hutchin, William, 70 Hypermedia, 15–16, 18 IBM, 157, 186, 203, 210 Ichikawa, Tadao, 112 ICONCLASS (ICONographic CLASsification System), 75–7 Iconography, 153–155 Image databases, 19, 176–7, 258 Image retrieval, 117–18, 258–60 ImageMiner, 203 337 Multimedia Information Retrieval ImageQuery, 215 IMPACT Program, 202 Indexing and abstracting aboutness model, 70 audio documents, 62 automatic procedures, 246 concept, 245–6 content-based analysis, 248 content-oriented, 69, 78–83, 94–6 criteria, 59 Elaine Svenonius, 83–5 entity-oriented, 70 images, 72–7 languages, 82–5 need-oriented, 69–70 problems, 69 purpose, 59 request-oriented, 70–1 semantic, 69–70, 248 strategies, 69–70 syntactic indexing, 248 systematic errors, 247 Indexing and abstracting in theory and practice (Frederick Lancaster), 58–63 Information retrieval (IR) See also Multimedia Information retrieval advanced proposals, 63–6 audio-visual, 18, 61–2 classification AAT, 72–4 Dewey principles of, 64 general, 64 ICONCLASS, 75–7 pragmatic method, 66–8, 71 concept-based, 63 content-based, 58, 61–3 crisis, 69–72 developments, 66–7 document searches, 22 indexing and abstracting See Indexing and abstracting international essays on, 64–85 338 interoperability, 64 limitations, 58 methodology, 58–9, 83, 90–129 multimedia See Multimedia Information Retrieval music, 62–3 NLP and, 65 principles, 58 similarity-based, 65, 96, 145, 151, 269–70 subjective language as tool, 65 text-based See Text Retrieval traditional system, 50 visual See Visual Retrieval Information society, 15–19 Informedia Project, 217–19 Intelligent Systems Lab Amsterdam (ISLA), 260–1 International Federation of Library Associations and institutions (IFLA), 185 International Standard Bibliographic Description (ISBD), 178, 295 Istituto Centrale per il Catalogo Unico e la documentazione (ICCU), 178 JACOB (Just A COntent Based search system for video databases), 196 Jörgensen, Corinne, 102 Killer app, 227 Kurth, Frank, 185 Lancaster, Frederick, 58–63 Language concept in MIR, 25–6 in indexing and abstracting, 82–5 as tool in IR, 65 Layne, Sara Shatford, 155 Learning Objects (LO), 294 Lewis, Paul, 120, 142 Librarianship, 27, 33, 38–9, 41 Index Library 2.0, 17 Library and Information Science (LIS) See also Library Science and Documentation documentation and, 46–8 hermeneutic approach, 49 MIR and, 48–53, 90–129 updated principles, 46–8 Library Science and Documentation Alberto Salarelli, 37–8, 40 Alfredo Serrai, 33–6 digitalization, 38 epistemology aesthetic approach, 20–3, 49–50 hermeneutic approach, 27–30 pragmatic and contextual approach, 24–7, 49–50 principles, 33–7, 49 processes, 35 semantic tools, 35–6 technological approach, 28 vs modern sciences, 34–5 limitations, 34–6 Linguistic-Linguistic (LL) query, 155–8 Linguistic-Visual (LV) query, 155–6 Low Level Descriptors (LLD), 270 Luo, Ying, 249 Mai, Jens-Erik, 64 MARC (MAchine Readable Cataloguing), 178 MAVIS (Multimedia Architecture for Video, Image and Sound), 287–8 MediaMill, 207 Metadata, 293–4 templates, 268–9 MetaLib system, 296 MetaSEEk, 214 Miccoli, Sebastiano, 27–8 Microsoft Bing, 223–4 MILOS (Multimedia dIgital Library for Online Search), 197–8 Mind the gap: Content Based Image Retrieval and the user interface (Colin Venters), 289 Mittal, Ankush, 247–8, 256 Molholt, Patricia, 72–4 Montani, Pietro, 20 MPEG-7 (Moving Picture Experts Group), 181, 265, 269–70 Multimedia data modelling abstract concepts, 93–4 content-based, 94–6 learning and semantics, 99 search systems, 94–6 semantic level, 249–52 temporal, 265 Multimedia document See Document; multimedia Multimedia documentation, 15, 42, 81–2 Multimedia Information Retrieval (MIR) advantages, 226–29 analyzing and indexing, 247–8 application, 183–8, 229–37, 292 audio retrieval See Audio Retrieval benefits, 136–38 bibliography and, 51–2 concept of language, 25–6 concept-based, 117–18 content-based See Content-Based Information Retrieval digital reference developments, 106–8 document management systems and, 292–4 functional requirement, 136–42 future issues, 99–100 Giovanni Solimine, 51 history, 195–223 image retrieval, 117–18, 258–60 legal and social issues, 228 library catalogue and, 292–3 library project, 294–300 339 Multimedia Information Retrieval linguistic conditioning, 26 LIS and, 48–51 mass user’s, 228–9 mathematical procedures, 112–16 methodology, 18, 90–129 modes, 253–55 multimedia data modelling, 93 multimedia object, 93, 226 ontologies in, 123–5 operational issues, 231–8, 278–83 organic integration, 116–24, 286–8 organic system, 23, 111, 135–6, 139 organic user interfaces, 288–92 positioning, 51–3 principles, 78–82 representational levels, 120–4 research programmes Italy, 195–201 rest of Europe, 201–9 US, 209–23 scientific discipline, 228 search behaviour, 290 engines, 223–6 interfaces, 289 options, 230 semantic, 98 systems See Search systems, MIR techniques, 227 term-based, 79, 139 textual, 140–1 visual, 141–2 semantic gap, 120–4, 249 semantic mapping, 177 semantic web developments, 100 shape similarity, 184 state of the art, 97–100, 188–90 systems and methods, 22 technical problems, 227, 257, 279–83 technical-organic integration, 283–5 technology, 134–5 term-based, 7, 22–3, 57–8, 72–3, 78, 109–110, 117, 134, 148, 155 340 text retrieval See Text Retrieval theory, 20, 25, 39–40, 42–5, 48–52, 64, 71, 76–7, 134–6 user’s role, 228 video retrieval See Video Retrieval visual retrieval See Visual Retrieval visual thesaurus, 157, 203, 234, 284 Multimedia Information Retrieval: international experiences and methodologies of content-based retrieval for information and documentation, 183–4 Multimedia objects CBIR and, 112–14 definition, 226 MIR and, 93, 99 spatial relationships, 113–14 video, 250–2 Multimedia technology in context (Sharon Flank), 229 Multimedia thesaurus (MMT), 287 Music Information Retrieval (MIR), 62–3, 165–9, 187, 190 NARCISSE (Network of Art Research Image Systems in Europe), 202 Natural Language Processing (NLP) applications, 65–6 folksonomy and, 66, 71 IR and, 65 nodes, 66 problems of, 66 text retrieval, 145 Net, 64–5, 69, 125, 135 Objective idealism, 71 Ontologies intergration with folksonomy, 126–9 in MIR, 123–5 as semantic support, 146–7 Index OPAC (Online Public Access Catalogue), 294–300 Otlet, Paul, 37, 47, 81 OVAL (Object-based Video Access Library), 266–7 Panofsky, Erwin, 76, 153–4 Paquet, Eric, 186 Petersen, Toni, 72–4 Pfeiffer, Silvia, 187 Photobook, 215 Photographic archives, 176, 178 PicToSeek, 204, 260, 264 PicToVision, 204 Pitch histogram, 168 Preiconography, 153–6 PRESTO project, 182 QBIC (Query By Image Content), 157, 210–14 Query, 155–8 QuickLook system, 184, 198–201 QuickTime VR, 187 Radio News Retrieval System, 208 Reference services, digital, 106–108 Retrievr, 215–16 Ribbrock, Andreas, 185 Roncaglia, Gino, 127, 129 Salarelli, Alberto, 37–8, 40 Sandom, Christine, 103–4, 120, 189, 206 Sangineto, Enver, 184 Schremmer, Claudia, 187 Search systems, MIR, 78–9 ECHO, 196–7 evaluation, 99 image types and users, 103–6 JACOB, 196 limitation, 98 MILOS, 197–8 operational applications, 260–5 QBIC, 210–14 role of imagination and creativity, 124–29 search-by-example, 113–14, 286–7, 289 search-by-humming, 190 search-by-subject, 114 semantic point of view, 102–6 traditional, 111 user expectations, 229–31 user needs, 100–108 visual, 101–4 Self-organizing Map (SOM), 191 Semantic gap, 103, 115–16, 120–4, 189, 227, 278 Semantic indexing, 69–70, 248 algorithms, 249–50 current policy, 57–63 mapping, 177, 251 modelling, 249–52 modules, 250 object based, 249 video objects, 250–52 Semantic probability, 189 Semantic tools, 35–6 Semantic Web, 66, 123, 125, 128 Sensory gap, 278 SEPIA (Safeguarding European Photographic Images) project, 178 Serrai, Alfredo, 33–6 Shape similarity, 184 Shih, Timothy, 257 Similarity-based retrieval, 65, 96, 145, 151, 269–70 Smeulders, Arnold, 260–1 SMPTE Metadata Dictionary, 181 Social tagging, 127–8 Soergel, Dagobert, 70 Solimine, Giovanni, 51 SoundFisher system, 219 Speech recognition, 165, 219, 223 Speechbot-Audio Search, 223 SpeechSkimmer, 223 341 Multimedia Information Retrieval Standard Media Exchange Framework (SMEF), 182 Start-up specialist, 128 Structured Query Language (SQL), 289 Subjective idealism, 71 Svenonius, Elaine, 83–5 Syntactic indexing, 248 System learning, 190 Tags, 126–7 Technical/management metadata, 293 Temporal document modelling, 265 Teoria della Documentazione (Paolo Bisogno), 41, 44 Term-based information retrieval, 7, 22–3, 57–8, 72–3, 78, 109–10, 117, 134, 148, 155 Text Retrieval (TR), 18, 61, 64–5, 68, 142–8 application, 237–8 classification, 144–6 definition, 142 free-text searches, 142–3, 147 heterogeneity, 144–5 integration with MIR, 286 NLP and, 145 principle, 142–3, 147 technical problems, 280–1, 282 theory, 68 use of ontologies, 146–7 The Society of Motion Picture and Television Engineers (SMPTE), 181 Three-step model theory, MIR, 76–7 Tonkin, Emma, 126–8 Touch screen, 230 TREC (Text Retrieval Conference), 231 TRECVID (TREC Video Retrieval) initiative, 228, 231 Tzanetakis, George, 166–7, 169, 187 342 VASARI (Visual Arts Systems for Archiving and Retrieval of Images), 202 Venters, Colin, 289 Video databases, 19 Video Mail Retrieval system, 219 Video Objects (VO), 250–2 Video Retrieval (VDR), 113–14 applications, 186–7 digital video archives, 179 document management, 179–81 document modelling, 266 OPAC and, 297–8 QuickLook, 198–201 research programmes, 207, 216–19 state of the art, 188–90 technical problems, 280, 282 VideoLogger, 216–17 VIPER (Visual Information Processing for Enhanced Retrieval) system, 203 Virtual Library, 294–300 Virtual Reference Desk (VRD), 176 Visual recognition, 184 Visual Retrieval (VR), 18, 101–4 AAT, 72–4 analysis and processing, 149–56 applications, 184 audio-visual documents See Audio-visual documents automatic content-based, 149 color searches, 280 cross-paradigm, 184 documentation, 265–6 effectiveness, 141–2 fine arts, 153 human mediation, 149–50 ICONCLASS, 75–7 image databases, 176–7 ISLA, 260–1 JACOB, 196 management, 175–9 metadata formats, 178 movement and, 158–63 Index OPAC and, 294–300 operations, 265–8 photographic archives, 176, 178 problems, 148–9 query, 155–8 QuickLook, 198–201 research programmes Content Based Visual Query Project, 214–15 ELISE, 202 Example Driven Image Database Querying, 215 Image Database Project, 215 ImageMiner, 203 ImageQuery, 215 NARCISSE, 202 Photobook, 215 Retrievr, 215–16 VASARI, 202 VIPER system, 203 search engines, 223–4 Visual symbolism, 83–4 Visual thesaurus, 157, 203, 234, 284 Visual-Linguistic (VL) query, 155–8 VisualSEEk, 214 Visual-Visual (VV) query, 155–7 Voice input, 230 Waal, Henry van de, 77 Wang, Ching-Sheng, 257 Web 2.0, 17, 126–8 Web Ontology Language (OWL), 125 WebSEEk, 214 Wittgenstein, Ludwig, 24–5 World Wide Web Consortium (W3C), 181 XML (eXtensible Markup Language), 181, 197 Yepes, José López, 46 Yoshitaka, Atsuo, 112 Zhang, Yanchun, 252–3 Zomax, 204, 260 343 ... attempts to frame MIR theory in relation to general and stable principles of library sciences and information search Key words: information society, multimedia documents, information handling, epistemology,... documentazione (Multimedia Information Retrieval: Methodologies and International Experiences of Content-based Retrieval for the Information and the Documentation), edited by Roberto Raieli and Perla... L’informazione multimediale dal presente al futuro: le prospettive del MultiMedia Information Retrieval (The Multimedia Information from the Present to the Future: the Perspectives of the Multimedia Information