Multiple User InterfacesCross-Platform Applications and Context-Aware Interfaces phần 5 ppt

142 ANGEL PUERTA AND JACOB EISENSTEIN <DIALOG ELEMENT ID=‘i1.1’ NAME=‘Make annotation’> <DIALOG ELEMENT ID=‘i1.2’ NAME=‘Select location’> <DIALOG ELEMENT ID=‘i1.2.1’ NAME=‘Select map point’> <FEATURES> <RELATION STATEMENT DEFINITION=‘is performed by’ REFERENCE=‘2.1.1’> <ATTRIBUTE STATEMENT DEFINITION=‘interaction technique’> onDoubleClick </ATTRIBUTE STATEMENT> </RELATION STATEMENT> </FEATURES> <DIALOG ELEMENT ID=‘i1.2.2’ NAME=‘Specify latitude’/> <DIALOG ELEMENT ID=‘i1.2.3’ NAME=‘Specify longitude’/> </DIALOG ELEMENT> <DIALOG ELEMENT ID=‘i1.3’ NAME=‘Enter note’/> <DIALOG ELEMENT ID=‘i1.4’ NAME=‘Confirm annotation’/> </DIALOG ELEMENT> </DIALOG MODEL> Note that to this point, we have linked many of the elements of the XIML components, but do not have yet a definition for how, for example, we would distribute the user tasks and selected interactors among windows in a desktop device. This is a design problem that must take into consideration various factors, such as target device and its constraints, interactors selected, and distribution strategy (e.g., many windows vs single window). The middleware unit can give support to this function via a mediator agent [Arens and Hovy 1995]. As Figure 7.12 shows, a mediator can examine the XIML specification, including the device characteristics. It can also offer a user task distribution based on an appropriate strategy for the device in question. After the process shown in this section, we have a comprehensive XIML specifica- tion for a user interface (concrete and abstract components). The specification is fully integrated and can now be rendered into the appropriate device. The middleware unit sig- nificantly simplifies the development work associated with completing the specification. 7.3.3.3. Contextual Adaptation The context in which interaction occurs has an obvious impact on what user tasks may or may not be performed at any given point in time. For example, in the scenario in Section 7.3.1, we know that the cellular phone is especially suited for finding driving directions. If the user were not driving, she could be using the PDA. The desktop work- station cannot be brought out into the field, so it is unlikely that it will be used to enter new annotations about a geographical area; rather, it will be used for viewing annotations. Conversely, the highly mobile PDA is the ideal device for entering new annotations. A middleware unit can take advantage of this knowledge and optimize the user interface for each device. The designer creates mappings between platforms (or classes of platforms) and tasks (or sets of tasks) at the abstract level. Additional mappings are then created between task elements and presentation layouts that are optimized for a given set of tasks. We can assume these mappings are transitive; as a result, the appropriate presentation model is associated with each platform, based on mappings through the task model. The procedure is depicted in Figure 7.13. In this figure, the task model is shown to be a simple XIML: A MULTIPLE USER INTERFACE REPRESENTATION FRAMEWORK FOR INDUSTRY 143 Cell phone 64 × 64 PDA 256 × 364 Mediator Desktop PC 1024 × 768 Platform model Main window Presentation model A Single window, screen-space intensive Main window Presentation model B Some windows, screen-space moderate Main window Presentation model C Many windows, screen-space optimal Figure 7.12. A mediating agent dynamically selects the appropriate presentation model for each device. collection of tasks. This is for simplicity’s sake; in reality, the task model is likely to be a highly structured graph where tasks are decomposed into subtasks at a much finer level than shown here. There are several ways in which a presentation model can be optimized for the per- formance of a specific subset of tasks. Tasks that are thought to be particularly important are represented by AIOs that are easily accessible. For example, on a PDA, clicking on a spot on the map of our MANNA application allows the user to enter a new note imme- diately. However, on the desktop workstation, clicking on a spot on the map brings up a rich set of geographical and meteorological information describing the selected region, while showing previously entered notes (see Figure 7.8). On the cellular phone, driving directions are immediately presented when any location is selected. On the other devices, an additional click is required to get the driving directions (see Figure 7.9). The ‘down 144 ANGEL PUERTA AND JACOB EISENSTEIN Cell phone 64 × 64 PDA 256 × 364 Make annotations View annotations Get to Site Desktop PC 1024 × 768 Platform model Task model Presentation model C Presentation model A Presentation model B Figure 7.13. Platform elements are mapped onto task elements which are mapped onto presentation models. arrow’ button of the cellular phone enables the user to select other options by scrolling between them. This middleware unit therefore benefits designers and developers by managing the localized contextual changes that apply across devices for a given task. The treatment by this unit of optimizations for the task structure of each device is similar to our treatment of screen-space constraints: global and structural modifications to the presentation model are often necessary, and adaptive interactor selection alone will not suffice. 7.4. DISCUSSION We conclude by offering a perspective on the ongoing development of the XIML frame- work, examining related work, and highlighting the main claims of this chapter. 7.4.1. THE XIML ROADMAP We consider that our exit criteria for the initial phases of the development of the XIML framework have been satisfied. Therefore, we plan to continue this effort. We have devised XIML: A MULTIPLE USER INTERFACE REPRESENTATION FRAMEWORK FOR INDUSTRY 145 a number of stages that we plan to follow to build and refine XIML into an industry resource. Each of the stages constitutes a development and evaluation period. The stages are as follows: 1. Definition. This phase includes the elicitation of requirements and the definition of language constructs, which we have completed. 2. Validation. Experiments are conducted on the language to assess its expressiveness and the feasibility of its use. This phase is being carried out. 3. Dissemination. The language is made available to interested parties in academia and industry for research purposes (www.ximl.org). Additional applications, tests, and lan- guage refinements are created. 4. Adoption. The language is used by industry in commercial products. 5. Standardization. The language is adopted by a standards body under a controlled evolution process. There is no single measure of success in this process. The language may prove to be very useful and successful at certain levels but not at others. In addition, the analysis of the functional and theoretical aspects of XIML is just one of several considerations that must be made in order to develop a universal language for user interfaces. It should be noted first that the meaning of the word ‘universal’ in this context is a language that has broad applicability and scope. The term should not be considered to mean a language that is used by every developer and every application. We do consider, however, that the evidence produced so far seems to indicate that further efforts are warranted. 7.4.2. RELATED WORK The work on XIML draws principally from previous work in three areas: model-based interface development [Puerta 1997; Szekely 1996; Szekely et al. 1993], user-interface management systems [Olsen 1992], and knowledge representation for domain ontolo- gies [Neches et al. 1991]. XIML shares some of the goals of these fields but is not directly comparable to them. For example, the main focus of model-based interface development systems has usually been the design and construction of the user interface. For XIML, this is just one aspect, but the goal is to have a language that can support runtime operations as well. In this point, it mirrors the aims of user-interface management systems. However, those systems have targeted different computing models and their underlying definition languages do not have the scope and expressiveness of XIML. There are also some related efforts in the area of creating XML-based user interface specification languages. UIML [Abrams et al. 1999] is a language geared towards multi- platform interface development. UIML provides a highly detailed level of representation for presentation and dialog data, but provides no support for abstract components and their relation to the concrete user interface design. Consequently, while UIML is well suited for describing a user interface design, it is not capable of describing the design rationale of a user interface. Ali et al. [2002] have recently begun to explore the integration of external task modelling languages with UIML (see Chapter 6). However, at present XIML remains 146 ANGEL PUERTA AND JACOB EISENSTEIN the only language to provide integrated support for both abstract and concrete components of the user interface. There are several existing or developing standards that overlap with one or more of XIML’s model components. XUL [Cheng 1999], Netscape’s XML-based User Interface Language, provides a thin layer of abstraction above HTML for describing the presentation components of a web page. Just as XUL overlaps with XIML’s presentation component, CC/PP [Butler 2001; W3C] and UAProf [Butler 2001] provide similar functionality to XIML’s platform component. ConcurTaskTrees provides an XML representation of a UI’s task structure [Paterno et al. 1997] (see Chapter 11). DSML [OASIS 2003], the Directory Services Markup Language, offers an adequate representation for the domain structure, at least in the case of e-commerce applications. Of course, none of these languages provides support for relational modelling between components, which is the essence of XIML’s representation framework. Since all of these languages are based in XML, it is straightforward to translate them into XIML. Consequently, we view the existence of these languages as an advantage. In the future, we hope to show how XIML can exploit existing documents in each of these languages. 7.4.3. SUMMARY OF FINDINGS In this chapter, we have reported on the following results: • XIML serves as a central repository of interaction data that captures and interrelates the abstract and concrete elements of a user interface. • We have established a roadmap for building XIML into an industry resource. The roadmap balances the requirements of a research effort with the realities of industry. • We have performed a comprehensive number of validation exercises that established the feasibility of XIML as a universal interface-specification language. • We have completed a proof-of-concept prototype that demonstrates the usefulness of XIML for multiple user interfaces. • We have successfully satisfied our exit criteria for the first phases of the XIML frame- work development and are proceeding with the established roadmap. ACKNOWLEDGEMENTS Jean Vanderdonckt made significant contributions to the development of the MANNA prototype and to the XIML validation exercises. We also thank the following individuals for their contribution to the XIML effort: Hung-Yut Chen, Eric Cheng, Ian Chiu, Fred Hong, Yicheng Huang, James Kim, Simon Lai, Anthony Liu, Tunhow Ou, Justin Tan, and Mark Tong. REFERENCES Abrams, M., Phanouriou, C., Batongbacal, A., et al. (1999) UIML: An appliance-independent XML user interface language. Computer Networks, 31, 1695–1708. XIML: A MULTIPLE USER INTERFACE REPRESENTATION FRAMEWORK FOR INDUSTRY 147 Ali, M., Perez-Quinonez, M., Abrams, M., et al. (2002) Building multi-platform user interfaces using UIML, in Computer Aided Design of User Interfaces 2002 (eds C. Kolski and J. Vander- donckt). Springer-Verlag. Arens, Y., and Hovy, E. (1995) The design of a model-based multimedia interaction manager. Artificial Intelligence Review, 9, 167–88. Bouillon, L., and Vanderdonckt, J. (2002) Retargeting Web Pages to Other Computing Platforms. Proceedings of the IEEE 9th Working Conference on Reverse Engineering (WCRE ’2002), 339–48, IEEE Computer Society Press. Butler, M. (2001) Implementing Content Negotiation Using CC/PP and WAP UAPROF. Technical Report HPL-2001-190, Hewlett Packard Laboratories. Cheng, T. (1999) XUL: Creating Localizable XML GUI. Proceedings of the Fifteenth Unicode Conference. Eisenstein, J. (2001) Modeling Preference for Abstract User Interfaces. Proceedings First Inter- national Conference on Universal Access in Human-Computer Interaction. Lawrence Erlbaum Associates. Eisenstein, J., and Puerta, A. (2000) Adaptation, in Automated User-Interface Design. Proceedings of the 5th International Conference on Intelligent User Interfaces (IUI ’00), 74–81. ACM Press. Eisenstein, J., and Rich, C. (2002) Agents and GUIs from Task Models. Proceedings of the 7th International Conference on Intelligent User Interfaces (IUI ’02), 47–54. ACM Press. Eisenstein, J., Vanderdonckt, J., and Puerta, A. (2001) Applying Model-Based Techniques to the Development of UIs for Mobile Computers. Proceedings of the 6th International Conference on Intelligent User Interfaces (IUI ’01), 69–76. ACM Press. Kawai, S., Aida, H., and Saito, T. (1996) Designing Interface Toolkit with Dynamic Selectable Modality. Proceedings of Second International Conference on Assistive Technologies (ASSETS ’96), 72–9. ACM Press. Neches, R., Fikes, R., Finin, T. et al. (1991) Enabling technology for knowledge sharing. AI Mag- azine, Winter 1991, 36–56. Olsen, D. (1992) User Interface Management Systems: Models and Algorithms. Morgan Kaufmann, San Mateo. Organization for the Advancement of Structured Information Systems (2003). http://www.oasis- open.org. Paterno, F., Mancini, C., and Meniconi, S. (1997) Concurtasktrees: A Diagrammatic Notation for Specifying Task Models. Proceedings of IFIP International Conference on Human-Computer Inter- action (Interact ’97), 362–9. Chapman and Hall. Puerta, A., (1997) A model-based interface development environment. IEEE Software,14(4) (July/August), 41–7. Puerta, A., and Eisenstein, J. (1999) Towards a general computational framework for model-based interface development systems. Knowledge-Based Systems, 12, 433–442. Szekely, P. (1996) Retrospective and challenges for model-based interface development, in Com- puter Aided Design of User Interfaces (CADUI ’96), (eds F. Bodart and J. Vanderdonckt), Sprin- ger-Verlag. 1–27. Szekely, P., Luo, P., and Neches, R. (1993) Beyond Interface Builders: Model-Based Interface Tools. Proceedings of 1993 Conference on Human Factors in Computing Systems (InterCHI ’93), 383–390. ACM Press. Szekely, P., Sukaviriya, P., Castells, P., et al. (1995) Declarative interface models for user interface construction tools: the mastermind approach. In Engineering for Human-Computer Interaction (eds L.J. Bass and C. Unger), 120–50. London: Chapman and Hall. Tam, C., Maulsby, D., and Puerta, A. (1998) U-TEL: A Tool for Eliciting User Task Models from Domain Experts. Proceedings of the 3rd International Conference on Intelligent User Interfaces (IUI ’98), 77–80. ACM Press. Vanderdonckt, J., and Berquin, P. (1999) Towards a Very Large Model-Based Approach for User Interface Development. Proceedings of First International Workshop on User Interfaces to Data Intensive Systems (UIDIS ’99), 76–85. IEEE Computer Society Press. 148 ANGEL PUERTA AND JACOB EISENSTEIN Vanderdonckt, J., and Bodart, F. (1993) Encapsulating Knowledge for Intelligent Automatic Interac- tion Objects Selection. Proceedings of 1993 Conference on Human Factors in Computing Systems (InterCHI ’93), 424–9. ACM Press. VoiceXML Forum. http://www.voicexml.org. World Wide Web Consortium. http://www.w3c.org. XIML Forum. http://www.ximl.org. 8 AUIT: Adaptable User Interface Technology, with Extended Java Server Pages John Grundy 1,2 and Wenjing Zou 2 1 Department of Electrical and Electronic Engineering and 2 Department of Computer Science, University of Auckland, New Zealand 8.1. INTRODUCTION Many web-based information systems require degrees of adaptation of the system’s user interfaces to different client devices, users and user tasks [Vanderdonckt et al. 2001; Petro- vski and Grundy 2001]. This includes providing interfaces that will run on conventional web browsers, using Hyper-Text Mark-up Language (HTML), as well as wireless PDAs, mobile phones and pagers using Wireless Mark-up Language (WML) [Marsic 2001a; Han et al. 2000; Zarikas et al. 2001]. In addition, adapting to different users and user tasks is required [Eisenstein and Puerta 2000; Grunst et al. 1996; Wing and Colomb 1996]. For example, hiding ‘Update’ and ‘Delete’ buttons if the user is a customer or if the user is Multiple User Interfaces. Edited by A. Seffah and H. Javahery  2004 John Wiley & Sons, Ltd ISBN: 0-470-85444-8 150 JOHN GRUNDY AND WENJING ZOU a staff member doing an information retrieval-only task. Building such interfaces using current web-based systems implementation technologies is difficult, time-consuming and results in hard-to-maintain solutions. Developers can use proxies that automatically convert e.g. HTML content to WML con- tent for wireless devices [Marsic 2001a; Han et al. 2000; Vanderdonckt et al. 2001]. These include web clipping services and portals with multi-device detection and adaptation fea- tures [Oracle 1999; Palm 2001; IBM 2001]. Typically these either produce poor interfaces, as the conversion is difficult for all but simple web interfaces, or require considerable per-device interface development work. Some systems take XML-described interface con- tent and transform it into different HTML or WML formats depending on the requesting device information [Marsic 2001a; Vanderdonckt et al. 2001]. The degree of adaptation supported is generally limited, however, and each interface type requires often complex, hard-to-maintain XSLT-based scripting. Intelligent and component-based user interfaces often support adaptation to different users and/or user tasks [Stephanidis 2001; Grundy and Hosking 2001]. Most existing approaches only provide thick-client interfaces (i.e. interfaces that run on the client device, not the server), and most provide no device adap- tation capabilities. Some recent proposals for multi-device user interfaces [Vanderdonckt et al. 2001; Han et al. 2000; Marsic 2001b] use generic, device-independent user interface descriptions. Most of these do not typically support user and task adaptation, however, and many are application-specific rather than general approaches. A number of approaches to model-driven web site engineering have been developed [Ceri et al. 2000; Bonifati et al. 2000; Fraternali and Paolini 2002]. Currently these do not support user task adaptation and their support for automated multi-device layout and navigation control is limited. These approaches typically fully-automate web site generation, and while valuable they replace rather than augment current development approaches. We describe the Adaptable User Interface Technology (AUIT) architecture, a new approach to building adaptable, thin-client user interface solutions that aims to provide developers with a generic screen design language that augments current JSP (or ASP) web server implementations. Developers code an interface description using a set of device- independent XML tags to describe screen elements (labels, edit fields, radio buttons, check boxes, images, etc.), interactors (buttons, menus, links, etc), and form layout (lines, tables and groups). These tags are device mark-up language independent i.e. not HTML or WML nor specific to a particular device screen size, colour support, network bandwidth etc. Tags can be annotated with information about the user or user task they are relevant to, and an action to take if not relevant (e.g. hide, disable or highlight). We have implemented AUIT using Java Server Pages, and our mark-up tags may be interspersed with dynamic Java content. At run-time these tags are transformed into HTML or WML mark-up and form composition, interactors and layout determined depending on the device, user and user task context. The following section gives a motivating example for this work, a web-based collab- orative job management system, and reviews current approaches used to build adaptable, web-based information system user interfaces. We then describe the architecture of our AUIT solution along with the key aspects of its design and implementation. We give examples of using it to build parts of the job management system’s user interfaces, AUIT: ADAPTABLE USER INTERFACE TECHNOLOGY, WITH EXTENDED JAVA SERVER PAGES 151 including examples of device, user and user task adaptations manifested by these AUIT- implemented user interfaces. We discuss our development experiences with AUIT using it to build the user interfaces for three variants of commercial web-based systems and report results of two empirical studies of AUIT. We conclude with a summary of future research directions and the contributions of this research. 8.2. CASE STUDY: A COLLABORATIVE JOB MANAGEMENT SYSTEM Many organisations want to leverage the increasingly wide-spread access of their staff (and customers) to thin-client user interfaces on desktop, laptop and mobile (PDA, phone, pager etc.) devices [Amoroso and Brancheau 2000; Varshney et al. 2000]. Consider an organisation building a job management system to co-ordinate staff work. This needs to provide a variety of functions allowing staff to create, assign, track and manage jobs within an organisation. Users of the system include employees, managers and office man- agement. Key employee tasks include login, job creation, status checking and assignment. In addition, managers and office management maintain department, position and employee data. Some of the key job management screens include creating jobs, viewing job details, viewing summaries of assigned jobs and assigning jobs to others. These interactions are outlined in the use case diagram in Figure 8.1. All of these user interfaces need to be accessed over an intranet using multi-device, thin-client interfaces i.e. web-based and mobile user interfaces. This approach makes the system platform- and location-independent and enables staff to effectively co-ordinate their work no matter where they are. Login Assign jobs to others View all jobs of department Add/Modify departments Office manager Add/Modify positions View all assigned jobs Manager Add/Modify employees Delete initiated jobs Employee View initiated job status Create jobs Figure 8.1. Use cases in the job management system. [...]... efforts have assumed the use of thick-client applications where client-side components perform adaptation to users and tasks, but not to different display devices and networks The need to support user interface adaptation across different users, user tasks, display devices, and networks (local area, high reliability and bandwidth vs wide-area, low bandwidth and reliability [Rodden et al 1998]) means... interface to a specific device, user role and user task with ease of specification and maintenance of adaptable user interfaces One consequence of specifying logical screen groups, elements and relevancies is that quite different physical screen layouts and interactions may result depending on device, user and task accessing the interface The interface the user sees is thus variable and may not be optimal i.e... specification languages and tools, such as HDM, WebML, UIML and Autoweb [Abrams et al 1998; Ceri et al 2000; Bonifati et al 2000; Fraternali and Paolini 2002; Phanouriou 2000] These all provide device-independent specification techniques and typically generate multiple user interface implementations from these, one for each device and user WebML describes thin-client user interfaces and can be translated... different combination of user, user task and device, using Java Server Pages, Active Server Pages, Servlets, PhPs, CGIs, ColdFusion and other technologies and tools [Marsic 2001a; Fields and Kolb 2000; Evans and Rogers 1997; Petrovski and Grundy 2001] This is currently the ‘standard’ approach It has the major problem of requiring a large number of interfaces to be developed and then maintained – for... different information system screens and N different user, user task and device combinations, we have to build and then maintain M*N screens We can improve on this a little by adding conditional constructs to the screens for user and to some degree user task adaptations, reducing the total number of screens to build somewhat However, for even small numbers of different users and user tasks, this approach makes... using AUIT to build two interfaces, both needing device, user and user task adaptations They built two AUIT screen implementations and up to half a dozen conventional JSP implementations AUIT: ADAPTABLE USER INTERFACE TECHNOLOGY, WITH EXTENDED JAVA SERVER PAGES 1 65 (generating HTML and WML respectively) These developers found AUIT to be straightforward to use and much more powerful and easier than the... boxes and pop-up menus have a name and value, obtained from JavaBeans when displayed and that set JavaBean properties when the form is POSTed to the web server Images have alternate text values and ranges of source files (typically, gif, jpg and wireless bit map wbm formats) Links and submit tags specify pages to go to and actions for the target JSP to perform Users, as well as user task relevance and. .. technology easier to use and more powerful for building and maintaining adaptable web-based user interfaces than other current approaches End users report they find the adaptive interfaces suitable for their application tasks, and in some instances prefer them to hard-coded, device- and user- tailored implementations REFERENCES Abrams, M., Phanouriou, C Batongbacal, A.L., Williams, S., and Shuster, J.E (1998)... Tools, 371–88 Lawrence Erlbaum Vanderdonckt, J., Limbourg, Q., Florins, M., Oger, F., and Macq, B (2001) Synchronised, modelbased design of multiple user interfaces, in Proceedings of the 2001 Workshop on Multiple User Interfaces over the Internet Varshney, U., Vetter, R.J., and Kalakota, R (2000) Mobile Commerce a New Frontier Computer, 33 (10) IEEE Press Vogal, A (1998) CORBA and Enterprise Java Beans-based... development of context-aware applications We start with a short introduction to model-based development of interactive systems and to some important ideas about task modelling Then, we consider new kinds of applications and the requirements that emerge with them A design process which can better Multiple User Interfaces Edited by A Seffah and H Javahery  2004 John Wiley & Sons, Ltd ISBN: 0-470- 854 44-8 172 . Auckland, New Zealand 8.1. INTRODUCTION Many web-based information systems require degrees of adaptation of the system’s user interfaces to different client devices, users and user tasks [Vanderdonckt. user is a customer or if the user is Multiple User Interfaces. Edited by A. Seffah and H. Javahery  2004 John Wiley & Sons, Ltd ISBN: 0-470- 854 44-8 150 JOHN GRUNDY AND WENJING ZOU a staff member. many properties the developer can specify, some mandatory and some optional. All tags have user and user task properties that list the users and user tasks to which the tag is relevant. Screen