X-Smiles 0.6 Technical Specifications

-= X-Smiles Posse =-

© 2001 XSmiles® is licensed under an Apache based open source licence. This document is subject to various kinds of laughter, from legally trained persons. We therefore do not take any of this in account, and also state that this kind of activity will or might propably hurt our feelings. Therefore all unauthorized critique on the behalf of our innocent legal disclaimer is prohibited.

26.4.2002, Kajaani

Abstract

This document briefly covers the architecture and technical issues of the XSmiles browser. It also explains some of the terminology related with the field. We are trying to keep this document on a more general level, so that it will not suffer from minor changes in the browser code. For more specific issues you should refer to browser Javadocs found here.


Table of Contents

Chapter 1. Introduction

What is X-Smiles?

X-Smiles is a research browser. The main goal is to test out Java and XML technologies in a ubiquitous environment. As a by-product, X-Smiles can also be used as a general tool for transofrming, and viewing documents in different XML formats. There is no WYSIWYG editing possibility yet.

Even though we are a research project, we still provide some assistance trough our developer's mailing list to problems. We also encourage people to submit bugs, and whatever comments that are related with our browser.

Main Features

The browser is capable of handling XML files. It can parse, and tranform all well-formed XML documents. X-Smiles is also capable of displaying various different formats, such as SMIL, XSL-FO, and SVG. It also supports various other standards and features. A detailed description of the features can be found at our website.

This Document

The purpose of this document is to give a detailed description of the technical aspects of the X-Smiles browser and its related components. This document describes the different components, their roles and relations to other components. It also defines the environment the browser is implemented in.

Also described are some of the design limitations, requirements and suggestions concerning adding additional functionality to this browser.

This document does not attempt to describe the product to the highest possible detail at all levels. For more details, please consult the API documentation.

Understanding of this document requires that the reader has a fair knowledge of how markup-languages work, has a sufficient knowledge of UML to read the diagrams, knows the elements of object-oriented design, and has some understanding how the Internet and applications such as a browser could work. A knowledge of Java is preferred, but not necessary. If you encounter unknown acronyms, or unknown terminology, please refer to the glossary at the end of this document.

Chapter 2. Environment

Runtime Environment

The browser is JDK 1.1 compilant. At the moment, most of the features work in JDK 1.1. The SIP component requires JMF 2.1.1> installed on your computer. Also video, and sound, requires JMF.

We are also quite actively working on support for Transvirtual's Kaffe. The newest CVS version of X-Smiles should work with the newest CVS version of Kaffe, if the guys at Transvirtual have commited our newest patches. In practise, X-Smiles might not work on Kaffe from time to time.

The performance considerations that apply for Java applications on the whole apply to X-Smiles also. Right now, the minimum platform performance requirements amount to Pentium 266 MMX, and 32M of memory. It is also possible to make a minimal version of the browser, to reduce memory requirements.

X-Smiles has been reported to run on all Windows 98> environments, a few Linux distributions, Macintosh OS X, and Compaq iPaq. With Linux, there have been some problems with blackdown's jdks.

Running it

The root Java package is fi.hut.tml.xsmiles The X-Smiles files are stored in a single jar file called xsmiles.jar. To start the browser you can use a command such as: java -jar xsmiles.jar. Or if you have all required libraries and classes in the CLASSPATH: java fi.hut.tml.Xsmiles. For further detail, see the installation manual.

When setting up a development environment, be sure to include all jars in ./bin/lib directory to the classpath. Some libraries have overlapping classes, and might cause problems, when ordered wrong.

If you are using windows, and the only intension is to run the browser, the easiest solution is to use the handy installer, which comes with the release. It automagically copies the needed files to your system, and creates a shortcut on the desktop.

Building with Ant

See our website for info on how to compile the browser.

Chapter 3. Architechture

This chapter describes the technical details of the X-Smiles browser architecture. The structure of the browser is shown and the relations between the components are displayed in Figure 3.1. The browser is composed of two layers: Browser Core and Browser modules. The Browser Core ties together different parts of the browser, by providing a means of communication, within different parts. The GUI provides a means of cross-platform, device independent GUIs. This includes different device specific GUIs, and a component factory, for instantiating device specific GUI widgets for other parts of the browser. The MLFC module includes different MLFCs, which the browser's functionality mainly consists of. The XML Processing module is somewhat linked to the MLFC Management, but is mainly provided by third party libraries, such as the Xerces XML parser, and Xalan XSLT engine.

Figure 3.1. Overview of the internal modules of the X-Smiles XML browser.

The following sections describe the Browser Core, and XML Processing. Further on, in Chapter 5, we discuss the MLFCs in detail. We will also discuss the GUIs, in detail in Chapter 6

Browser Core

The browser core functionality is implemented by classes (Figure 3.2): Browser, BrowserLogic, MLFCManager, MLFCLoader, XMLParser, GUIManager, XMLBroker, and EventBroker. It is also possible to create multiple instances of the Browser, so you will result with multiple browserwindows.

Figure 3.2. The internal architecture of the X-Smiles XML browser.

Class Browser

The Browser class ties all components together. It has the classes shown in Figure 3.2 in it's vincity. The Browser implements the BrowserWindow interface, and this interface is used all classes refering to the Browser.

The main functionality includes initialization of a browserwindow, and controlling different parts of the browser.

Class MLFCManager

MLFCManager is responsible of controlling MLFCs. It keeps a list of active MLFCs, and handles creation, and destroying MLFCs. MLFCManager also contains an implmentation of MLFCListener, which handles communication between MLFCs and the rest of the browser, eliminating the need for MLFCs of having a reference to the Browser class.

Class MLFCLoader

Takes care of instantiating MLFCs.

MLFC Management

The MLFCManager manages MLFCs. It loads them into the Java Virtual Machine, gives them a graphical container to display the document, and a reference to the DOM tree of the XML file to be displayed.

An XML document has a primary ML, that is the ML specified by the main element. The MLFC handling this type of XML documents is called the Primary MLFC. If a document has a reference to another XML document, as a foreign object, then this document is rendered as a Secondary MLFC.

Class MLFCManager

Responsibilities

It finds an MLFC, and adds it to a list of active MLFCs. It also initializes, and asks MLFC to start displaying. MLFCManager is adds a MLFCListener class to the MLFC, which is responsible from communication between the MLFCs and the rest of the browser.

Hybrid documents

Hybrid XML documents are documents, which contain several XML languages, separated by a namespace.A host language usually has the default namespace in the document and it also defines the layout for the document. Typical host languages are XHTML, SMIL, and SVG. A language, which is embedded inside a host language, is called a parasite language. XML Events and XForms are examples of parasite languages.

Host and Parasite MLFCs

An MLFC can be a host, a parasite, or both. There is a strict rule: one host MLFC is created per a document. The host is identified by the documents root element and it decides the master layout for the document. The layout model can differ between different host MLFCs. For example, XHTML has a flow type of layout, while SVG uses explicit coordinates for placement.

A parasite MLFC always needs a host to live in. Parasite, such as XForms MLFC, may only define layout for its own elements. Sometimes the parasite does not have visible components (e.g., XML Events MLFC). Figure 3.3 depicts the creation of an XML DOM, where SMIL MLFC is the host and there are two parasites: XForms and XML Events. In the figure, the host elements in the DOM have white background, while the parasites have darker background.

Figure 3.3. Overview of the XML Broker functionality.

XML Broker

X-Smiles has a framework for handling hybrind XML documents. We've separated the MLFC registration and general element and attribute creation to an independent module, called XML Broker. XML Broker has three main roles:

MLFC Registering.  All MLFCs in the system register themselves to XMLBroker either by a namespace (e.g., 'http://www.w3.org/2000/12/xforms') or by the root element's unqualified name (e.g., 'smil').

Dispatching element and attribute creation.  XMLBroker checks the namespace of each element and attribute to be created, and if it matches any of the registered MLFCs, it forwards the request to the corresponding MLFC.

MLFC Instantiation.  XML Broker instantiates MLFCs on-demand, and keeps track of the MLFCs that have been instantiated for a document.

Interaction between MLFCs

The elements cannot usually live in the DOM without communicating to each other. Consider, for example, an XForms element that lives in a SMIL document. The SMIL document needs to be able to access the graphical component of the XForms element in order to place it on the screen, at the location specified by the SMIL document. Another example is an event listener that must be able to fire event handlers specified by some other language.

The host MLFC is independent of the possible parasite MLFCs. It is also possible to add new MLFCs without modifying the existing ones. Therefore, interaction between elements is solved in the framework by defining two entities: Service Provider and Service Caller as shown in Figure 3.4:

The Service Provider . This is an interface that a DOM element or attribute can implement.

The Service Caller. This is is another element or attribute that knows how to use the Service Provider's interface.

The elements then communicate directly with each other, and not via XMLBroker or browser core. Some elements may not need communication, or they may also communicate via DOM events.

Figure 3.4. Interaction between MLFCs.

Chapter 4. Roll Your Own MLFCs: Four Simple Steps

Steps

The purpose of this chapter is to provide a quick guide to creating your own MLFCs for X-Smiles. Even though it might seem like a difficult task, it is a rather simple one, if you have a little experience with the DOM API, as an MLFC is just a simple extension of DOM.

You might also want to see a simple example, such as GPSMLFC.

Step One

Extend the MLFC class, especially the methods start(), stop(), createElementNS(), and createAttributeNS().

Step Two

Do your thing. If you are creating a visual MLFC, you can get the visual component container with the getContainer() method of the MLFC. Also if you need to do some calls to the browser core, use the getMLFCListener which is the window to the browser. It contains methods such as openLocation() and setTitle().

Step Three

Add a "namesspace to implementing" class mapping for your MLFC in the config.xml, under mlfcmappings.

Step Four

Try it out!

Additional Features

Chapter 5. MLFCs

M, o, o, n. That spells, Markup Language Functional Component MLFC. They are bean-like, fairly independent components, which are used for rendering XML documents. See Chapter 4 for details on how to create your own MLFC.

SMIL

Package.  fi.hut.tml.xsmiles.mlfc.smil, org.w3c.dom.smil20

Dependencies.  Java Media Framework 2.1.1 (JMF), jmf.jar Steady State CSS, ss_css2.jar

Namespace.  http://www.w3.org/2001/SMIL20/Language

This MLFC handles the interpretation and presentation of SMIL documents.

Module hierarchy and contents.

SMILMLFC consists of several packages. They are described here. Two packages, smil.basic and smil.extension are the core logic for SMIL. Smil.viewer is a interface for GUI specific implementation. Under smil.viewer.smilmlfc there is an interface called media, which is implemented by different media, such as ImageMedia, TextMedia etc. See Figure 5.1 for more detailed description for the packages.

Figure 5.1. SMIL 2.0 package overview.

fi.hut.tml.xsmiles.mlfc.smil .  This package has SMILMLFC, which is the Viewer for SMIL. It controls the SMIL core logic and passes required information for it, such as the screen size and other system attributes.There is also TimeChildList, which is an Iterator to find all children, which have timing information.

fi.hut.tml.xsmiles.mlfc.smil.basic .  This package contains the core logic for SMIL. It has all SMIL 2.0 basic elements and functionality. These are completely GUI and browser independent classes. The elements in this package make up the SMIL-DOM.

fi.hut.tml.xsmiles.mlfc.smil.extension .  This package has all non-Basic modules for SMIL 2.0. These are also GUI and browser independent classes.

fi.hut.tml.xsmiles.mlfc.smil.viewer .  This package contains the interface for a Viewer. A Viewer is a GUI specific implementation. The classes implementing this interface will have the GUI for the SMIL player. There are several implementations, defining different kind of GUIs.

fi.hut.tml.xsmiles.mlfc.smil.viewer.awt .  This implementation of the Viewer is written using AWT. It is a standalone SMIL player using only AWT. This package is not required by the X-Smiles, but exists to show how the player can work as a standalone implementation.

fi.hut.tml.xsmiles.mlfc.smil.viewer.smilmlfc .  This implementation of the Viewer is written for X-Smiles. It uses Swing to render the media and has references to the browser to retrieve browser related information, such as the system attributes. This also passes external links to the browser.

fi.hut.tml.xsmiles.mlfc.smil.viewer.smilmlfc.media .  This package is no longer used. All media is now rendered by the X-Smiles ContentHandlers. Note that these include a native Window player, which requires jawt.dll and NativeMedia.dll in the xsmiles/bin folder. These files must be copied to the xsmiles/bin folder manually to get the native players to work.

fi.hut.tml.xsmiles.mlfc.smil.viewer.swing .  This implementation of the Viewer is written using AWT. It is a standalone SMIL player using only Swing. The X-Smiles implementation viewer.smilmlfc is inherited from these classes.

org.w3c.dom.smil20 .  This is a new package to define the interface for SMIL 2.0. It has been partly written from scratch, because W3C hasn't defined it yet.

Rendering the SMIL presentation (overview)

Rendering a SMIL presentation from the SMIL (XML) source involves the following two steps:

  1. Parsing the XML document and creating a SMIL-DOM. When the browser retrieves the XML document, it parses the source document. A DOM tree will be created. To create the DOM tree, the browser uses Xerces, which will call SMILDocument.createElementNS(). This SMIL specific method will create SMIL specific elements, instead of creating standard DOM elements. The SMIL elements are defined in the fi.hut.tml.xsmiles.mlfc.smil.basic package. The result will be a standard DOM compatible SMIL specific SMIL-DOM tree, which will implement the SMIL-DOM interface (org.w3c.dom.smil20) defined by W3C for scripting. The SMIL-DOM tree is also the presentation tree, handling all timing and logic for SMIL.

  2. Playing the SMIL-DOM.  The SMIL-DOM is ready to be played as soon as it has been completely created. The following steps must be taken to play the presentation:

    1. Call SMILDocument.initialize(). This will initialise the presentation. The SMILDocument class will call the SMIL body element's init() method, which will call its children's init() method and so on. This way the whole SMIL document will be initialised.

    2. Call SMILDocument.prefetch(). This step is optional. This can be called to prefetch all media in the presentation to local memory to speed up the playing of the presentation. Again, the SMILDocument will call SMIL body element's init() method, which will pass the call to its children.

    3. Call SMILDocument.start(). This will actually start playing the presentation. The body element's startup() will be called, which will call startup() method for its children, in a sequence (since body has the Seq semantics for timing). The children of body will call their children's startup() method in the way the timing is defined. (seq calls then in a sequence, par at the same time).

      After startup() has been called, it will create a Timer to wait until the start of simple duration. When the simple duration starts, activate() is called. It will display the media and start a new Timer to wait until the end of the simple duration.

      After the simple duration has ended, deactivate() will be called. This method will remove or freeze the media.

      When the parent time container ends, closedown() will be called for the child elements. It will remove the media (if it was only frozen) and free all timers.

    4. Call SMILDocument.stop(). This will stop the presentation, by calling closedown() for all elements. (first body, which will call its children etc.)

    5. Call SMILDocument.freeResources(). This will freeze all resources and memory used by the presentation. Again, body.destroy() will be called, which will call the children's destroy(). Every element is responsible of releasing its memory and resources.

XForms

Package.  fi.hut.tml.xsmiles.mlfc.xforms

Dependencies.  Xalan XPath,Xerced DOM implementation,Xerces Schema validations, DOM parser, javax.swing

Responsibilities.  Implements most of the XForms specification. For the table of implemented features, see the features documentation.

MLFC Overview

XForms MLFC is implemented as a parasite MLFC. That means that it always needs an host MLFC to live in. For XForms MLFC it does not matter which MLFC is the host, as long as it uses the interfaces provided by the X-Smiles Framework, such as VisualComponentService.

The MLFC constitutes of the following packages:

fi.hut.tml.xsmiles.mlfc.xforms.  This package has XFormsMLFC, which holds the state for a single page. It holds all the <model> elements for a page.

fi.hut.tml.xsmiles.mlfc.xforms.dom.  This package has the DOM implementations of all of the XForms elements. The element implementations contain most of the logic of the XForms MLFC. The common behaviour of the different XForms elements are created using inheritence. See Figure 5.2 for details.

fi.hut.tml.xsmiles.mlfc.xforms.constraint.  This package contains the implementation of the XForms dynamic constraints, readOnly, relevant, required, calculate and isValid. It has the implementation of the computation engine described in the XForms specification .

fi.hut.tml.xsmiles.mlfc.xforms.instance.  This package contains the specialized DOM implementation for the instance data. In XForms, the instance data is not live in the document DOM, but is copied to another live memory structure. In X-Smiles, a DOM implementation inherited from Xerces DOM is used.

Figure 5.2. The deployment of the XForms classes.

Instance

The instance is implemented in the package fi.hut.tml.xsmiles.mlfc.xforms.instance. In that package, there is the general interface InstanceNode that is implemented by the InstanceBaseElementImpl and both of the attribute implementations (with and w/o namespaces). The InstanceNode interface defines one important method: getInstanceItem(). That method returns a InstanceItem instance for each node in the instance.

InstanceItem.  An instance of this class is generated and linked to every instance node. This class has the following responsibilities:

  1. The form controls register and unregister themselves to this class.
  2. The calculation engine calls this class for every change in the model item properties, and it is up to this class to report changes to the form controls and save the state of the model item properties.

Binding between the UI and the instance

Every XForms element that can be bound the instance is inherited from BoundElementImpl, which provides the implementation to binding. We Xalan XPath implementation to get to the nodes corresponding to the path the author has given in the binding attributes.

It is also possible to in XForms to create nested contexts for the UI elements. For example, consider the XForms snippet in Example 5.1. The group's binding will be used for the context node for it's immediate XForms ancestor's binding. All this is accomplished in BoundElementImpl.

Example 5.1. Nested binding contexts

	<xforms:group ref="/my/items/item[1]">
		<xforms:input ref="title"/>
	</xforms:group>
	

The binding in XForms is usually static. This means, that at initialization time the BoundElementImplis called to locate the bound InstanceNode, and this binding is not changed in the future. (An exception is the Repeat processing, see below).

Repeat

Repeat is mainly implemented in the class ...dom.RepeatElementImpl. Repeat is implemented as DOM modifying action. It means, that the repeat is "rolled out", like in <xslt:for-each> processing. For examples, see the demos provided with X-Smiles.

Initialization. Initialization is performed every time the instance, where this repeat is bound to, changes. At initialization, the following successive steps are done:

  1. The nodeset, which the repeat is bound to is obtained by executing the binding XPath expression.
  2. For each nodes in the bound nodeset, the UI markup is repeated. For all repeated XForms controls, the context node is the current node in the bound nodeset.

Repeat cursor implementation. Repeat cursor is implemented the following way: every repeat (also, every instance of nested repeat) has a cursor associated with it. There is also a tree of repeats, that correspond to the nesting of the repeats. When a user sets focus to a form control, the form control checks whether it is created by a repeat. If so, it tells the repeat to set it's repeat cursor accordingly. The repeat does what it is told, and then asks it's paren t repeat to do the same.

Switch and toggle

Switch and toggle are implemented as DOM modifying actions. At initialization time, all but the initial case is removed from the DOM and saved to Switch internal memory structure. When a toggle is activated, the DOM is modified accordingly. For examples, see the demos provided with X-Smiles.

Constraints

Constraints and the calculation engine are implemented in the package defined in the section called “MLFC Overview”. The overview of the implementation is presented in Figure 5.3. It includes three distinct types of data structures:

The presentation DOM. The memory representation of the host XML document, e.g. the SMIL document that hosts the XForms elements. The implementation of this markup language is a specialized tree extended from a generic DOM implementation. It contains host DOM nodes implemented in the host MLFC and XForms DOM nodes which implement, for example, a specific form control.

The instance DOM. It is an XForms internal DOM implementation, also derived from the generic DOM, that holds the current state of the instance, namely instance data item values and their constraint states, such as 'relevant'.

Dependency graphs. They are used by the calculation engine. There are two dependency graphs per instance: The Master Dependency Graph, which is created at initialization time, and The Pertinent Dependency Subgraph, which is generated on the fly from the main graph whenever the calculation engine is run. The dependency graphs are composed of Vertices and Edges

An XForms presentation DOM node is linked two-ways to the corresponding instance DOM node. The only relevance of the presentation DOM to the calculation engine, are the instance item value updates, which are reflected to the presentation DOM and vice versa. The dependency graph's vertices are linked one-way to the instance DOM nodes. There may be many vertices linked to the same instance node, as long as they are of different type (e.g. calculate and relevant vertices), as shown in Figure 5.3.

Figure 5.3. Overview of the XForms implementation.

A computable constraint, such as calculate in a bind element, can express a computation for multiple instance data items. Each computation can refer to multiple data items. For example, a bind expression can attach a calculation of price * units to every item total.This leads to an important prequisite for building the Main Dependency Graph; the ability to obtain the referents of each XPath expression, e.g. how to find out which instance data items the calculate expression references. X-Smiles uses XPath processor in Xalan, which had to be extended to support this feature. We execute an XPath expression once in its correct context (namespace and XPath context nodes), and collect references to every node that the expression's location paths reference. We've extended some of Xalan's XPath implementations classes and use those in the package org.apache.xpath to obtain the referents.

Figure 5.4 depicts the collaboration between different objects when a value in the instance changes. In the diagram, the user changes the value in a 'selectOne' form control. The sequence is initiated when the JList component used by the 'selectOne' element dispatches the valueChanged event, which the selectOne listens to. After recording the specific form control, the base class calls setValue() for its bound InstanceItem. The instance item then calls recalculate() with a change vector containing itself. The recalculate is applied to the XForm object, which creates a new SubDependencyGraph and calls calculate() on it. SubDependencyGraph then runs the recalculation algorithm on itself, and changes the values and properties on the recalculated InstanceItems, which in turn are reflected to the user interface. After the recalculation, the revalidate() function is called on the Model object to determine whether any schema constraints have been violated or corrected.

Figure 5.4. Calculations collaboration diagram.

Schema validation

Schema validation is currently (0.51) implemented by using the Xerces 1 schema validator. Because of lack of PSVI support in Xerces 1, the whole instance is always revalidated. We're planning to move to Xerces2, which will make it more efficient because of Schema caching feature of Xerces2. Also, I hope there will be PSVI enabled DOM available in a future version of Xerces2. This is not the case with Xerces 2.0.1.

Rendering the form controls

XForms is a parasite MLFC, and trusts on the X-Smiles framework to provide inter-MLFC communication. All form control elements, that are meant to be rendered by the host language implement the interface VisualComponentService. The host language will reckognize this interface, and put the contols on screen in a place defined by the host document.

Binding to XML Events

Since XML Events is supported in X-Smiles, XForms MLFC only needs to bind to it. The binding is done at two levels:

Dispatching DOM2 events. All the form controls dispatch appropriate DOM2 events to the DOM. For instance 'click' event is posted to the DOM when the user clicks a form control with the mouse.

Implementing EventHandlerService. All the XForms actions, such as submitInstance implement the EventHandlerService. This means that they can act as handlers for any XML Event listener. There is also a special grouping handler xforms:action, which is a handler that fires all it's child handler successively when activated.

CSS Styling

The CSS styling uses the CSS implementation in X-Smiles to get the styling information for all of it's elements. Therefore it is possible to use a single stylesheet for the host and parasite document.

Posting

Serializing to XML. The class XMLSerializer serializes the instance data into XML.

Serializing URLEncoded String. The class URLSerializer serializes the instance data as URLEncoded, to be able to send it with HTTP GET.

HTTP Post with XML body. This is the default method of sending XForms data. It is done using the HTTP post method in the X-Smiles browser, and putting the serialized data into the body of the post.

HTTP GET. This is performed by creating a new URL with the serialized instance appended after the '?' - character. The browser is requested to fetch this URL.

XSL-FO

Package.  fi.hut.tml.xsmiles.mlfc.xslfo.*

Dependencies.  fixme

fi.hut.tml.xsmiles.mlfc.xslfo.*

Dependencies.  fixme

The XSL FO MLFC uses the Apache XML FOP Project (http://xml.apache.org/fop/) [FOP]. However, FOP cannot be used in the browser as such, as it provides a way to preview a XSL FO document, but allows no means of user interaction - a very basic feature in any browser. Thus, we have extended the FOP to add user interaction using:

  • external links

  • forms

Simple links are defined in the XSL FO specification (i.e., the fo:simple-link element), while forms are not within the scope of the XSL FO. Forms are introduced in the XForms specification, which is still at an early stage. Thus, we use XHTML form elements (such as input fields, buttons, etc.) with the XForms data model for rendering and input, and a subset of the XForms data model for defining the data model of the form.

Module hierarchy and contents

Apps . This package contains a slightly modified Driver and FOTreeBuilder from the original FOP. It has a bug fix for handling DOM documents and it reckognizes xform definition elements.

Form . The form package contains the form elements which are placed in the FO tree.

Render. Render package extends FOP's AWTRender to handle the displaying of XSmiles extensions.

Rendering an FO document

See [FOPArchitecture] for a more complete description of the process.

  1. Parsing the FO DOM document and creating the FO tree. There are a number of Mapping classes in fo/ that map XML elements and properties to maker() methods in the various classes. XSmiles adds the mapping to classes FormElementMapping, and FormPropertyListMapping For example, FormElementMapping has calls like: builder.addMapping(uri, "form", Form.maker()); The TreeBuilder has methods such as startElement() and endElement() to receive events and uses the mappings to call the relevant FO maker() methods when building the FO tree.

  2. Paginating and creating the Area Tree using the FO tree. FOTreeBuilder calls format() on the root FO, passing it the AreaTree reference. In turn, root calls format recursively for its children. The FOs that represent actual areas have the method layout(), which manages the layout of of children and/or generates new Areas.

  3. Rendering a page using the Area Tree. Rendering is a matter of translating the page areas into the relevant output format (AWT display). Render() is called for every Area object in the AreaTree. The render() function calls the corresponding rendering function in the AWTRenderer.

XSmiles Extension: XForm components

XForms extension affects FOP in many levels:

Detecting XForm -declaration from the DOM tree. The class mlfc.xslfo.apps.Driver2 and DocumentReader2 (taken to X-Smiles project from FOP) iterate the DOM tree. This process has been modified so that if an ?xform? element is encountered, it is recognized as Xform Data Model and sent to the mlfc.general.XformsHandler class (c.f. the section called “XForms”). This element and its children are not passed to FOTreeBulder class.

Current DOM node kept in the memory.  Our extension mechanism (XForms) have been build on top of DOM, and therefore we need the DOM element for each extension element (inside foreignObjects). Driver2 keeps the current DOM node in memory, so that Xforms extensions can access it.

New FO Tree elements.  The form UI components can only be inside the fo:instream-foreign-object. The UI components are in the xform namespace and are declared in fi.hut.tml.xsmiles.mlfc.xslfo.form.FormElementMapping.java. New FO tree elements were added to FOP for form elements. The following classes in package mlfc.xslfo.form map the DOM elements and attributes to FO Tree elements and properties:

New AreaTree elements. The form UI components can only be inside the fo:instream-foreign-object. The UI components are in the xform namespace and are declared in fi.hut.tml.xsmiles.mlfc.xslfo.form.FormElementMapping.java.

  • FormElementMapping maps the form elements

  • FormPropertyListMapping maps the form element properties

New AreaTree elements.  Mlfc.xslfo.form.FormWidgetArea Areas are created for the form widgets

Rendering.  AWTRenderer has been taken into X-Smiles as AWTRenderer2. (there were some private members which have been modified as protected.) The class .mlfc.xslfo.render.XSmilesAWTRenderer extends AWTRenderer2 and adds the form and link rendering capabilities (using CommonExtensions). The package fi.hut.tml.xsmiles.mlfc.xforms handles the form input and button clicks and maps the form widgets to corresponding xform instances (c.f. the section called “XForms”).

XSmiles Extension: Links

Since link areas are already constructed in FOP, the rendering of links is the only thing that is added in XSmiles. It is done in the class XSmilesAWTRenderer. The links use an invisible component of type .mlfc.xslfo.widget.LinkComponent to capture mouse input over the links.

XSmiles Extension: Image loading

Image handling is extended from FOP.

It contains Support for relative image URLs is added by modifying the class FopImageFactory. It calls mlfc.general.URLHandler to construct the URL

Support for simpler image loading.

Support for XML documents as image source (new class XsmilesImage)

The following classes were taken out from the FOP project (to package mlfc.xslfo.image) ("2" added to the end of the name):

FopImageFactory (new MIME types supported : application/xml and such. Basically all formats understood by X-Smiles are supported by FOPImageFactory2), FopImageConsumer, ExternalGraphic, StandardElementMapping, GifJpgImage.

XSmiles Extension: Java 1.1 support and Java font support

New renderer . We have written a new Java 1.1 compatible renderer with the AWTRenderer as basis. This renderer has the same capabilities as the AWTRenderer except that it cannot render embedded svg graphics. The renderer is in package fi.hut.tml.xsmiles.mlfc.xslfo.renderer.jdk11. The renderer, such as the Java 1.2 renderer uses the class CommonExtension to extend the basic capabilities. This renderer is not always kept up to date, therefore we keep both renderers in X-Smiles.

The Use of Java font metrics. The font metric classes have also been updated to support Java 1.1. The problem is fixed by using the Java font metrics. This fix affected the following classes, which were taken out from the FOP project (to fi.hut.xsmiles.mlfc.xslfo.render): AWTRenderer -> AWTRenderer2

The following new clases were introduced (to org.apache.fop.render.awt): AWTFontMetrics, AWTFontSetup, AWTFontInfo, FontMetricsMapper.

This classes may be removed from XSmiles when the changes are made to a future FOP release.

We've also added support for Java fonts: If a document requests a font not found in FOP, java fonts are searched instead.

XSmiles Extension: SMIL and SVG inside XSL FO document

We have extended the FOP viewer to view SMIL and SVG documents inside an FO document. The FO document refers to the secondary documents using the <fo:external-graphic> - element. The class FopImageFactory is modified to produce classes of type XsmilesImage whenever a reffered document contains XML data instead of binary image data. XsmilesImage is a "dummy" class, class, which only holds the URL reference to the secondary document.

The secondary document is realized in the rendering phase in function XSmilesAWTRenderer.renderImageArea . The function checks the image type and if it is XsmilesImage, then a secondary MLFC is created with the URL reference.

EcmaScript

Package.  fi.hut.tml.xsmiles.ecma

Depndencies.  fixme

XSmiles uses an embedded FESI or Rhino EcmaScript interpreter to evaluate EcmaScript in the documents. XSmiles currently has experimental EcmaScript support, including:

  • Substitution directly after retrieval of documen

  • Mouse events

  • Limited access to the DOM tree

  • Very limited DOM changes to SMIL documents

  • SVG changes supported

  • DOM changes to Xforms instance data.

  • Alert dialogs

SVG

Package.  fi.hut.tml.xsmiles.mlfc.svg, org.csiro.svg...

Dependencies.  fixme

SVG MLFC can render vector graphic documents. It uses the CSIRO SVG Toolkit (http://sis.cmis.csiro.au/svg/), which supports most of the SVG features. There are still several things missing or only partially implemented, since the toolkit is not yet complete. See the CSIRO web site for details.

Some slight modifications has been done in order to integrate the viewer into the browser. These modification include slight user interface adjustments. These modifications affected two classes: org.csiro.svg.viewer.Viewer and org.csiro.svg.viewer.Canvas. These are taken into the XSmiles project and named Viewer2 and Canvas2.

X3D

Package.  fi.hut.tml.xsmiles.mlfc.x3d

Dependencies.  xj3d.jar, Java 3D

Namespace.  http://www.x-smiles.org/2002/x3d

X3D MLFC can render 3d graphic documents. It uses the Xj3D Open Source VRML/X3D Toolkit (http://www.web3d.org/TaskGroups/source/xj3d.html).

X3D requires Java3D to work. This is the reason it is not included in the standard Ant build script, to avoid compiling errors if Java3D hasn't been installed. A special Ant build target exists for X3D. For binary release of X-Smiles, the users have to download an extension jar x3dmlfc.jar into their bin/extensions folder.

CommMLFC

Package.  fi.hut.tml.xsmiles.mlfc.comm

Dependencies.  JXTA jars

Namespace.  http://www.x-smiles.org/2002/comm

CommMLFC can be used for interbrowser communications. It uses the JXTA Project (http://www.jxta.org) to establish peer-to-peer communications between the browsers running in different computers.

X3D requires JXTA jars to work. This is the reason it is not included in the standard Ant build script, to avoid compiling errors if JXTA hasn't been installed. A special Ant build target exists for JXTA. For binary release of X-Smiles, the users have to download an extension jar jxtamlfc.jar into their bin/extensions folder to get it to work.

Elements. <comm> - the element to establish interbrowser communication. If the group attribute is defined, this element will immediately connect to the JXTA network, to the given group. If the group attribute is not given, then the group must be joined with a script call to comm.join("groupName", ""). The comm element will dispatch a "commEvent" event when data is received. The received data can then be read with comm.getReceivedData(), the sender name with comm.getReceivedSender(). Data can be sent to the network with comm.send("", "text") or comm.sendXML("", org.w3c.dom.Node) or comm.sendMime("", byte[] data, String mime).

Events

Package.  fi.hut.tml.xsmiles.mlfc.events

Depndencies.  fixme

Namespace.  http://www.w3.org/2001/xml-events

Dependencies.  Currently, none.

Requirements.  Every MLFC should send DOM events. See SMIL implementation.

This MLFC handles the interpretation of XML Events elements. Currently the following tags are supported: <listener>, <script>. All event and script handling should go through this MLFC. Also, if scripting is not desired, it will be easy to disable it by disabling this MLFC.

See the specification at http://www.w3.org/TR/xml-events/ for details.

GPS

Package.  fi.hut.tml.xsmiles.mlfc.gps

Depndencies.  comm.jar Some native stuff (rxtx in Linux, and Sun's DLL in windows)

Namespace.  http://www.fobar.org/2000/gps

Requirements.  You can run the GPS mlfc without a real GPS receiver hooked up to your computer with the emulation=true attribute in the gps tag.

This MLFC is used for sending GPS data to other MLFCs via XML Events. See the demostration found at http://www.xsmiles.org/demo/svg_demo/kartta.xml

Chapter 6. Device Independence

As X-Smiles is intended for a variety of different target devices, we have also focused on device independence. First of all, the browser can be interfaced through different user interfaces, which can also be added, by implementing the GUI interface.

Chapter 7. Glossary

General Terminology

CC/PP

Composite Capabilities / Preference Profiles is a W3C proposal for relaying device, and software capabilities of a device to the server side. Basically it can be used to relay other context-related information also. For building context-aware applications, this is a promising technology to use to relay information. CC/PP is not yet a recommendation, but hopefully will become one.

DOM

Document Object Model. An application programming interface (API) for HTML and XML documents, which defines the logical structure of documents and the way a document is accessed and manipulated.

DTD

Document Type Definition. XML markup declaration, that provides a grammar for a class of documents (e.g. SMIL DTD).

ECMAScript

A standard for a general purpose, cross-platform scripting language, reminiscent of JavaScript.

JFC

The Java Foundation Classes extend the original Abstract Windowing Toolkit(AWT) by adding a comprehensive set of graphical user interface class libraries. These include "Swing" GUI Components and Drag & drop from other (even non-java) app.

JMF

The JavaTM Media Framework (JMF) is the application programming interface (API), that allows developers to incorporate media data types such as audio and video into their Java applications.

HTML

Hypertext Markup Language. The most common publishing language used on the World Wide Web.

HTTP

Hypertext Transfer Protocol. A protocol for transferring files (text, graphic images, sound, video, and other multimedia files) on the World Wide Web.

ML

A markup language defines a way to describe information in a document. There are several standards, such as HTML 4.0, which define certain markup languages, but new languages can be defined for different applications and domain areas with the help of XML.

Scripting Language

In the scope of this browser, a language used to add interaction and dynamics to XML documents.

SGML

See Standard Generalized Markup Language.

Standard Generalized Markup Language

Some reasonable definition here.

See Also XML.

SMIL

(Synchronized Multimedia Integration Languag. A markup language defined with XML. SMIL gives end-users the capability to integrate different media objects into one single synchronized multimedia presentation. SMIL is specifically designed for the WWW environment and is defined by the World Wide Web Consortium

UML

Unified Modelling Language. A general-purpose notational language for specifying and visualizing complex software, especially object-oriented products It is being developed under the auspices of the Open Management Group (OMG).

URI

Unifrom Resource Identifier. URI is a way to identify points of content in the Internet (e.g text, video, audio). An URI typically contains the mechanism used to access the source, the specified computer the resource is housed in and the specific name of that content on that computer.

W3C

World Wide Web Consortium. W3C is a prestigious international organization, which produces standards of the world wide web.

XLink

XML Linking Language. The Xlink specification defines the constructs that may be inserted into XML resources to describe links between objects.

XML

Extensible Markup Language. A data format for structured document interchange on the WWW and it is defined by the World Wide Web Consortium. XML itself is not a markup language but a metalanguage for markup language design.

XPointer

XML Pointer Language. The XPointer specification defines the constructs that support addressing into the internal structures of XML documents.

XSL

Extensible Style Language. A language for expressing stylesheets in XML documents.

XSmiles Terminology

DOCTYPE

DTD and MLName is assigned with this parameter in an XML document.

Document History

Document history is a queue, which contains the URIs the users has visited during his/hers current visit to the Internet. The URIs are placed into the queue in chronological order.

Document Source

Document source is the source code of the document, that the user is currently viewing.

Xerces DOM

The third-party DOM implementation from Apache Foundation used by the browser.

Extended DOM Implementation

A method, where a markup language is implemented by adding functionality to the DOM elements themselves. This method saves memory, and makes it easier to dynamically reflect DOM tree changes, with e.g., scripts, to the screen and the user. The SMIL, XForms, and SVG are implemented in this way. This means that after creation, each DOM element knows how to display itself.

MLFC

Markup Language Functional Component. A bean-like proprietary software component for X-Smiles, which handles documents of a specific markup language. It basically is a component, which is capable of displaying a document of a certain ML in a container.

This term and its acronym was defined by the project group and is limited to the scope of this browser.

Primary MLFC

A MLFC functions as a primary MLFC, when it is the root document, which is displayed in the browser window. If a document is embedded in another document, e.g., as image, then a corresponding MLFC will be requested to function as a Secondary MLFC, to render content in the sized container assigned to it.

Secondary MLFC

If a document is embedded in another document, e.g., as image, then a corresponding MLFC will be requested to function as a Secondary MLFC, to render content in the sized container assigned to it.

See Also Primary MLFC .

Host MLFC

The MLFC created for the host language in a hybrid XML document.

Parasite MLFC

The MLFCs created for the parasite languages in a hybrid XML document.

Source MLFC

An MLFC, which is responsible for showing the source code of the XML document

Tree MLFC

An MLFC, which is responsible for showing the tree view of the document.

References

[DocBook] DocBook: The Definitive Guide.. Norman Walsh. Leonard Mueller. Copyright © 1999 O'Reilly .

[XFormsDataModel] XForms data model. W3C draft.. Micah, et. al. Dubinko. Copyright © 2000 http://www.w3.org/TR/2000/WD-xforms-datamodel-20000406/ .

Appendix A. The X-Cries CC/PP Servlet

What is X-Cries?

X-Cries is a Java Servlet, which is capable of extracting CC/PP information, and dealing pages based on this information. On the server side this is done by a site description XML file, which configures a site. This configuration file specifies which stylesheet, or file in general is sent back. The testing is achieved by setting different tests, such as "less than", "greater than", or "equals". This way we can send a page written in inuite, if the user's CC/PP profile says that her language is inuite.

The X-Cries servlet can currently be found in the the cvs. We also have it running at xdemo.tml.hut.fi. We have links to some demonstrations located on that machine on our demo page of X-Smiles.

Currently we are working with integrating the CC/PP functionality for Cocoon.

Appendix B. About This Document

This document was created with DocBook XML v. 1.44[DocBook]. The newest version of this file is located at /doc/TechSpec/specifications.xml. All related images and such should be located under the subdirectory TechSpec_files. To generate a XHTML version of this file, use the Ant: /src/build.sh docbook. To view a XSL-FO version of this document, open it in X-Smiles, the processing instruction is pointed to the correct stylesheet, and X-Smiles will tranform it automatically.

The stylesheets, are located under http://www.xsmiles.org/docbook/. The transformation is a bit slow, so be patient. The intention is to distribute transformed versions, in XHTML, XSL-FO, and PDF.