Versions & Open Source

Well, after upgrading to Firefox 3 I have realised one of the drawbacks present in the Open Source community - the XForms plugin is not released in concert with FF3. For now, this means a pause in development activity.

Of course I could get involved in the development of the plugin itself, and to this end I have been getting set up with Eclipse for C development. However accessing CVS for Firefox is a problem, as I am behind a firewall. Getting ProxyTunnel built for OS X is the next step, and this is almost complete. After that I hope to have access sorted out and be able to start some serious hacking of the code. Maybe then I can in some small way help to accelerate the release of the plugin for FF3.

Debugging status of instances in XForms

Debugging XForms at runtime has not been easy. It seems difficult to get simple and convenient access to the XForms instance data while the user is interacting with the form. I find that I want to test an XPath expression or XForms action (e.g. setvalue).

My development environment is Eclipse, with embedded Tomcat as the server. Testing the interface is through FireFox, and I have Firebug installed to monitor web server calls. Firebug does provide access to the DOM, and I guess this should mean I can look into the instance nodes. However this is less than straightforward.

In the end I realised that I could create a simple debug tool to inspect the instance data. By adding a "Debug" button to the form, I can submit an instance data back to the server. I created a simple DebugXFormsSubmission JSP. The JSP does nothing more than output the submitted data to the server's stdout. Which Eclipse captures nicely from the embedded server and displays in the Console view.

The text of the JSP is:

<%@ page language="java" contentType="text/xml; charset=ISO-8859-1" pageEncoding="ISO-8859-1" import="java.io.BufferedReader" %>

<%
  String xml = new String("");
  String thisLine;
  BufferedReader br = request.getReader();
  while ((thisLine = br.readLine()) != null) {
    xml = xml.concat(thisLine);
  }
  System.out.println("Received xml: ");
  System.out.println(xml);
  System.out.println("Done");
%>


The Debug submission in the form is:

<xf:submission id="submitDebug"
    method="post"
    replace="none"
    ref="instance('data')"
    action="/Leads/DebugXFormsSubmission.jsp"
    omit-xml-declaration="true">
</xf:submission>



The Debug button is:

<xf:submit submission="submitDebug">
  <xf:label>Debug</xf:label>
</xf:submit>



This has become a simple but invaluable means of getting quick access to the instance data. And it has no impact on the actual instance itself.

Accessing static data with ids

How to use id-based static data in repeats - using XPath expressions to show the appropriate labels instead of the id numbers or key values.

In a relational database environment, it's not uncommon to use surrogate keys - unique numbers that form the primary key on each table. These are arguably faster at a database level, because the database engine does not need to use multiple pieces of data when joining tables together.

However, surrogate keys pose a problem for a GUI designer. When managing data in forms, it's important to make the form easy to use. This generally means that we don't want to show the user the underlying keys or code values. Especially if the key value is an unintelligble number. Instead, we need to show relevant data fields that can be interpreted by the user.

For example our Client table has a foreign key to a list of Client Statuses. The foreign key column clnt-status-code-id references a table "code" with a surrogate key of code-id. Other columns on the code table contain the status description that should be displayed to the user:

code-id code-description

======= ================

1 Active

2 Inactive

3 Dead

This means that our form should ideally show a drop-down (combo-box) that shows the descriptions. The client status field needs to store the equivalent id value.

Luckily, this turns out to be pretty simple to achieve once you employ some XPath. Assuming that the static data (containing the status code values) is stored in a instance separate from the client data, the XForms select1 looks something like:

<xf:select1 ref="clnt-status-code-id">
<xf:label>Status:</xf:label>
<xf:itemset nodeset="instance('static-data')/CSTA/code">
<xf:label ref="code-short-description">
<xf:value ref="code-id">
</xf:itemset>
</xf:select1>

In the end quite simple, and this means the form maintains separation of the UI from the data, without requiring javascript to move data around.

Simplifying front-end inserts by using "origin"

Using the "origin" on xf:insert simplifies the logic for xf:repeats - no need to always have a dummy row at the end of the repeat, which should not be shown to the user by cannot ever be deleted.

I started off doing repeats the way most of the examples said to do - have a dummy row at the end, which you then use as a template for the actual repeat. Problem is that this is not an elegant solution, and results in lots of places where you end up needing to code around that dummy row. The repeat cannot (should not) show the row because it's not real data and is only there for the developer's use.

Then when you send the instance data to the server to update, you end up having to make sure that you don't try to save the dummy row into the database by mistake.

However there is a simpler way. The xf:insert tag has an attribute called "origin". This is an XPath expression that can be used to inform XForms of the location of the xml tags to be copied. Because it's just a normal XPath expression, we can use it to specify tags from other instances. This means we can separate the tags to be copied from the "real" data tags.

Using an instance for templates, separate from the instance that contains the actual data:


<xf:instance id="data">
  <data xmlns="">
    <clients>
      <client>
        <surname>Smith</surname>
        <forename>John</forename>
      </client>
      <client>
        <surname>Smith</surname>
        <forename>Jill</forename>
      </client>
    </clients>
  </data>
</xf:instance>

<xf:instance id="data-templates">
  <data xmlns="">
    <client>
      <surname />
      <forename />
    </client>
  </data>
< /xf:instance>



When used in an xf:repeat sequence, it's then simple to use the "data" instance, showing all client entities.

<xf:repeat id="client-list" nodeset="instance('data')/clients/client">

Then on the xf:trigger to insert a new Client, we can use the xf:insert as:

<xf:insert
nodeset="instance('data')/clients/client"
at="1"
position="before"
origin="instance('data-templates')/client" />

Of course in order to make this work sensibly we then need to ensure that the template and the real data entities always follow the same structure. This is where the XML Schema comes in, and that's a subject for later.

Splitting instance data by usage

When developing the proof of concept application, I started by following the documentation and using mostly a single <xf:instance> to contain all of the data that would be used by the form.

However part-way through I realised that this wasn't making life easy at the back-end when attempting to update the database. In an enterprise-level application, configurability is an important feature. This normally means that the application separates tables into two categories - static data (configuration parameters, business rules, etc), and business data (the real client records, transaction records, etc). The static data used within a form is not normally changed within the form, but rather is maintained in separate screens that are accessible only to system administrators or business analysts. Therefore, because this data is not changed it does not need to be re-sent to the server when saving the form.

Having both the static data, and the business data, and even some utilities fields (counters, etc) all in the one just complicates the back-end processing. And it turns out to be fairly straightforward to use multiple instances. In the end I found it works well to use up to four instances:

1. Static Data Instance - retrieved from the database by a JSP
2. Business Data Instance - retrieved from the database by a JSP, and submitted to the database by a second JSP
3. Template Instance - retrieved from the application server by a JSP, contains templates to be used when inserting new entities into the Business Data Instance (e.g. template containing empty fields for a new Client record)
4. Administration/Utility Instance - static instance within the main XHTML/JSP form, containing working data items (next counters, etc)
   

So an example for a Customer/Client Maintenance screen, has a <xf:model>that looks something like:


<xf:instance id="data" src="GetClientData.jsp?clnt-id=< out.print(request.getParameter("clnt-id")); %>" />

<xf:instance id="data-templates" src="GetClientDataTemplates.jsp" />

<xf:instance id="static-data" src="GetClientMaintenanceStaticData.jsp" />

<xf:instance id="util" xmlns=""
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema" >
<temp-cont-type>GEN</temp-cont-type>
<temp-cont-type-id />
<last-insert>-1</last-insert>
</xf:instance>


The key advantage of this approach is that the logic can be simplified at the back-end, both for the retrieves and for the updates. In particular, the submission now only sends the "data" instance to the server. The static data, the templates, and the admin information are retrieved but never resent back to the server.

XForms

As a veteran IT guy, I continue to toy around with various technologies as an attempt to stay up to date with what's going on. My most recent endeavours focus on the use of XForms.

XForms is an XML-based technology designed to manage client-side forms processing for browser-based applications. Defined as a W3C standard here, XForms brings rich-client functionality to the browser using a declarative approach.

The two most prominent implementations appear to be Firefox's XForms addon and FormsPlayer.

I've been building a basic application as a means of evaluating the pros and cons of the technology. A big part of what I've been learning in the process has actually been how to combine various technologies to create a working, scalable, maintainable business application. While I am clearly not creating a commerical application, it's been interesting to looks at the ways the technologies could work together within that context.

To summarise the architecture that I've ended up using so far:

• running on Tomcat
• database on mySQL
• web-based front-end running under Firefox
• using either XHTML pages or JSP depending on whether I need to manage parameters passed from other pages
• using further JSPs to retrieve data from the database, or to apply updates (no visual components involved in either case)
• POJOs at the application level, modeled on the business entities (Client, Address, Country, etc) which then encapsulate the business logic and interface with the database
• using stored procedures and dynamic SQL to access the database

Some of the most difficult parts to date have actually been working out how to use CSS to layout the XForms components on the pages. This is partially implementation-specific, and not well documented for Firefox. A future post will attempt to document some of the behaviours that I've uncovered.