Web Applications: XForms

Mikko Honkala

Helsinki University of Technology

XForms: Outline of the Presentation

About me
Introduction
Related technologies
XForms 1.0
Implementations
Proposed Extensions
Future work: XForms 1.1...
Conclusions

About me: Mikko Honkala

Researcher in TKK TML lab
Started in 2000

Background in content adaptation for Web applications using XML
Master's Thesis: Using XML to develop applications for WAP and Web environments
Ph.D. Thesis topic: Web User Interaction - A Declarative Approach with XForms

Invited expert in W3C XForms Working Group since 2001
1 journal and 10 conference articles
Still writing 1 article, after which the publications should be sufficient for Ph.D.

Introduction

Research Area: Web User Interaction
Web has been a phenomenal success
It started as an information repository
Simple, platform-independent hypertext content format (HTML), and simple publishing model (HTTP, URL)
It is transforming into interactive application platform

Demand for highly interactive Web applications.
Demand for mobile and multimodal Web usage.
Demand for multimedia in Web applications.

HTML was not designed for this, several new proposals exists

Interaction: DHTML, Ajax, XForms, XUL (+many research proposals)

Demand: Highly interactive Web applications

Different tasks require different levels of UI complexity

Information access - low UI complexity

Internet search (Google)

Interactive Applications - medium UI complexity

PIM: e-mail (e.g. GMail), calender (e.g. Mobical), etc.
Group Collaboration
Multimedia applications (e.g. Google Maps)

Content authoring - high UI complexity

Spreadsheet authoring
Application development
2D/3D Graphics Modeling

For instance, most current Web applications are in the Infomation Access - level

This is already changing

Demand: Mobile and Multimodal Web Usage

(c) Juha Vierinen

Demand: Mobile and Multimodal Web Usage

Device Independence

Desktop, PDA/smartphone, telephone, etc.

Input

Keyboard, stylus,mouse, microphone

Output

Desktop/laptop screen, PDA/Phone small screen, Loudspeakers, etc.

Modalities

Graphical, Speech, Gestures...

Multimodal Interaction - Combining modalities

Semantic level

In the 90's it was enough to build an UI for a certain device (desktop)
Today, a big requirement for the UI is to support different devices and modalities
In order to achieve this, semantic level must be raised

Earlier: make a green textbox, with the size of 100x400 pixels and place it in (400,300)
Today:

Content: what part of the data is inputted/outputted
Type: type of data
Styling: possible styling
Layout: how is the data related to the rest of the UI

Demand: Multimedia in Web Applications

Multimedia: Outputting different types of media simultaneously

Static media: Text, Images
Continuous media: Audio, Video
Spacial layout and styling
Temporal layout: Timing and synchronization
User interaction

TV / Movies / Radio moving to Web (e.g. podcasts, news)
Combined with interaction and static media

Requirements for a UI Description Language

Ease of authoring and maintaining
Device & modality independence

->Separation of data from presentation

Typical interactors (datatype-specific)
Typical interaction patterns

Paging (e.g. wizards), tabs
Repeating structures
Nested structures

Dependencies
Adaptation
State and synchronization
+ Reqs for layout & timing

Related Research: Requirements, etc.

N. Souchon and J. Vanderdonckt. A review of XML-compliant user interface description languages. In Proceedings of the 10th International Conference on Design, Specification, and Verification of Interactive Systems, 2003. Springer-Verlag
Jeffrey Nichols, et.al., Requirements for automatically generating multi-modal interfaces for complex appliances. In Proc. the 4th IEEE International Conference on Multimodal Interfaces, 2002.
S. Trewin, G. Zimmermann, and G. Vanderheiden. Abstract representations as a basis for usable user interfaces. Interacting with Computers, 2004
Rainer Simon, et.al, Tool-supported single authoring for device independence and multimodality. In Proceedings of the 7th international conference on Human computer interaction with mobile devices & services, 2005. ACM Press.

Related Research: Languages.

XML User Interface Language (XUL)
User Interface Markup Language (UIML)
XForms
Universal Remote Control (URC)
eXtensible Interface Markup Language (XIML)
Others, XAML, InfoPath, ...
Voice, multimodal: VoiceXML, X+V, SALT

Comparison of approaches

Web-based Interaction

Web is a channel for distributing the user interface to the client
In Web, the device accessing the application is not known
HTML Forms with CSS layout are used today for defining the user interface

Device independent

HTTP used as the communication channel

Simple request-response pattern

HTML Interaction Model

DHTML Interaction Model

AJAX Interaction Model

Problems

Pure HTML

Latency - server regenerates page
Scalability

DHTML

Still interaction problems: synchronous submissions
Heavy reliance on JavaScript: accessibility, maintainability, etc.

Async. JavaScript and XML (Ajax)

Heavy reliance on JavaScript: accessibility, maintainability, etc.

My Layered Model

XForms

W3C Recommendation 2003
Instance - content
Model - validation, constraints, calculations
User Interface - the form controls embedded in a host language
Can be seen as a Model-View-Controller (MVC) model
Additionally

Binding - binding between the instance, model and UI
Submission - send and receive the instance to/from the server

XForms Interaction Model

XForms model

Input validation and datatypes

XML Schema language

date, integer, float, string, ...
derived types: e.g. integer between 0-1000

Calculated properties

Uses XPath language
calculate - parts of data calculated from other parts
relevant - is the item shown to the user
readonly - can the user edit the item
constraint - is the item valid (extension to schema)
required - the item is required to be filled

Purchase order example: Instance


    
        
            X-Smiles desktop
            2
            50
            0
        
        
            X-Smiles PDA
            2
            100
            0
        
        
            Java debugger
            4
            75
            0
         
    ]]>

Purchase order example: Model

]]>

XForms User Interface

Requirement: Typical interactors
Dynamic UI Binding Layer
Abstract form controls

input, textarea, secret, range, select1, select, trigger
Datatype-sensitive (DEMO)

Accessibility

Separate elements for help, hint, alert, message

Purchase order example: UI


        
    Units Item
    name Price Total
    
    

            
                
                Enter the quantity of this item.
                
            
            
            
            
        
    


    Total price

        
             ]]>

	Units	Item name	Price	Total
Enter the quantity of this item.

Req: Interaction Patterns with XForms

Paging, Wizards: Switch

Switching parts of the user interface on and off
DEMO

Editing repeating constructs: Repeat

Current XForms Implementations

Different Approaches

XML Language Plugin (Formsplayer-IE, Mozilla-Firefox)
Dedicated Browser (X-Smiles)
Server-Side Transformation (Chiba, TODO)
JavaScript (TODO)

Some proof that language design was good?

Extension: Tree module

Requirement: Editing nested structures
Use cases: Help navigation, document structure, service structure, folder structure
Proposed extension: Tree module to XForms

Simple solution, that allows to navigate arbitrarily nested structures
Allows structure changes, copy-paste etc. (XForms 1.1)
Easy-to-author, rather easy to implement
DEMO: Area editor
DEMO: Slide editor

Extension: Multimodal Interaction

Multimodal application authoring is difficult with current approaches (X+V, SALT, etc.)

All modalities have to be authored separately

XForms can be used as a multimodal UI description language
It has higher semantic level than HTML, so it is possible to automatically create voice-navigation and form-filling
We have defined a novel model called XFormsMM, which includes:

XForms 1.0 with some authoring restrictions
Modality-dependent stylesheets
Interaction Manager
DEMO: SMS Message
DEMO: Car rental Message
DEMO: Email - repeating structures with speeche

XFormsMM runtime

Extension: Multimedia applications

Multimedia applications bring additional requirements
How to integrate timing and synchronization and user interaction
DEMO: Distance Education with exam

Extension: Digitally signed forms

Use cases: goverment forms, e-commerce, etc...
Buying with a credit card is not secure today in the Web!
Current Web does not offer digital signatures
XForms is the next generation XML forms language
It can be integrated with XML Signature to provide user signatures
A demo implemented: Co-signing a insurance claim

Implemented as a single XForms form + a servlet with XPath filtering

What you see is what you sign (WYSIWYS)

Future Work

XForms 1.1

SOAP integration
New Functions
Manipulating structures
Inline Rendition of Non-Text Media Types

XForms 2.0

Tree module?
Digital Signatures?
Templates?
Datatyped expressions: XPath 2.0?

Integration to other markup languages

SVG, Compound Documents, Rich UI Backplane?

Conclusions

Current trends in Web:

Web is transforming into an interactive application platform
User interfaces are breaking out of the desktop
Multimedia is becoming common

HTML model has to be upgraded
Several proposals exists: XForms, AJAX, etc.
Future will tell which will be adopted

Web Applications: XForms

Mikko Honkala

Helsinki University of Technology

XForms: Outline of the Presentation

About me: Mikko Honkala

Introduction

Demand: Highly interactive Web applications

Demand: Mobile and Multimodal Web Usage

Demand: Mobile and Multimodal Web Usage

Semantic level

Demand: Multimedia in Web Applications

Requirements for a UI Description Language

Related Research: Requirements, etc.

Related Research: Languages.

Comparison of approaches

Web-based Interaction

HTML Interaction Model

DHTML Interaction Model

AJAX Interaction Model

Problems

My Layered Model

XForms

XForms Interaction Model

XForms model

Purchase order example: Instance

Purchase order example: Model

XForms User Interface

Purchase order example: UI

Req: Interaction Patterns with XForms

Current XForms Implementations

Extension: Tree module

Extension: Multimodal Interaction

XFormsMM runtime

Extension: Multimedia applications

Extension: Digitally signed forms

What you see is what you sign (WYSIWYS)

Future Work

Conclusions

Thank you! Questions?