MS Bugs - You mean they exist?

I have had the pleasure of playing with Microsoft Visual Studio 2008 to try a little Windows Mobile 6 development. This has been a great opportunity to learn more of C# and I have to admit that while I am learning .NET as I go, I am coming up to speed quite quickly. I decided the other night to provide some local storage for some of the values I am retrieving from my server. Microsoft provides a really light-weight version of their SQLServer which runs on the mobile platforms called SqlCe and I have been quite impressed with it so far. I read/watched some of the "How do I?" videos provided on the MSDN website and started using SqlCeResultSets. I wrote a simple loop in my code similiar to the following:

   ...
   SqlCeResultSet resultSet = _sqlController.getResultSet(ResultSetType.COUNTRIES);
   if (resultSet.HasRows)
   {
      foreach (SqlCeUpdatableRecord record in resultSet)
      {
          // do something with the result
      }
   }

However, evertime it hit the foreach statement the program would terminate. I suspected this was an infinite loop or some other issue. Well it turns out that indeed there is a bug in .NET 3.5 which causes the ResultSet to enter an infinite loop when the GetEnumerable( ) is called. An interesting article that outlines this is located Jim Wilson's Blog. Fortunately this issue only cost me a few minutes, but I was glad to see that this was a Microsoft issue and not my beginner C# programming skills at fault.

JPA Identity Interger/Long or String?

In my previous article JAXB and the nasty XmlID, I discussed how if you wanted to use @XmlID and @XmlIDREF as pointer references to XML serialized objects that the values had to be Strings. In some cases they might be fine, especially if your primary key is a compound key that you are serializing. However, the more common case for simple persisted objects is you have used some numeric identity value as the primary key. A short search will show almost all demos/tutorials and examples of JPA use either a Integer or Long wrapper. This of course is not a restriction of JPA. You can use anything you want as the primary key, but if you are going to leverage some of the @GenerateValue options, unless you are willing to define your own Alpha scheme, you will get a numeric value. Which leads me to my point: If I want to XML Serialize an object using JAXB (which is much easier than doing it by hand with a DOM Document) that contains foreign key references, surely I can do this in some way without having to convert my identities to Strings?

The simple answer is yes, but not by applying the JAXB annotations on the persisted property. Instead, you would need to create a proxy method that can convert your PK (Primary Key) into a string representation, and this method is tagged with the @XmlID annotation. In order to be used by JAXB you need at least a String property. Lets look at a simple code example.

   import javax.xml.bind.annotation.XmlID;
   import javax.xml.bind.annotation.XmlRootElement;
   import javax.xml.bind.annotation.XmlTransient;
   import javax.xml.bind.annotation.XmlAttribute;
   import javax.persistence.Entity;
   import javax.persistence.Id;
   import javax.persistence.GeneratedValue;
   import javax.persistence.Transient;

   @XmlRootElement
   @Entity
   public class PersistedObject {
       @Id
       @GeneratedValue(strategy=TABLE, generator="CUST_GEN")
       @XmlTransient // we are not going to write out the id
       private Long id = null;

       @XmlTransient
       @Transient   // this is not an entity managed attribute
       private String identityString = null;

       @XmlTransient
       public Long getId( ) { return id; }

       public void setId( Long id ) { this.id = id; }

       @XmlID
       @XmlAttribute(name="id")
       public String getIdentityString( ) {
          return ( id != null ) ? id.toString() : "0";
       }
   }

In this manner, we can denote our JPA identity with the data-type which makes sense (either a Long or Integer) yet allow for easy XML Serialization through the use of the @XmlID field on the getIdentityString() method. This is certainly not ideal, and I would've preferred to put the annotation on a method only, however JAXB requires the XmlID tag on a property.

Unfortunately for me, I only thought of this after converted my persistent beans, services and unit tests over to String Ids. Fortunately SCM tools (subversion in this case) come to the rescue and I can easily back out my changes.

Smartphone Connectivity in Visual Studio 2008

Hooking up Microsoft Visual Studio 2008 and using the network on an emulated smartphone is a rather tricky business if you are not familiar with the procedure. Fortunately, a very good blog exists to capture this information at: Akhune's Weblog. The limitation of this of course is that you can not have your mobile device AND the emulator both connected to ActiveSync at the same time.

An Update (05/23/2008): Having used the emulator a little more, I think the key point that is often lost is the usage of the "cradle" option to dock the emulator with ActiveSync. Once you get in the habit of doing this per development session, you will find it is much easier. Also, I have not had very good results of having both my Smartphone and the Emulator cradled simultaneous and recommend using one or the other. This appears to be a pretty common issue.

Flexjson limitations

The old saying goes "If it sounds too good to be true, it is too good to be true". Well this applied to Flexjson. While this tool is very capable of externalizing an object to the JSON format, there are several shortcomings which make it difficult to use at the moment. The contributors have mentioned they are working to address these issues. Currently Flexjson only can process primitives, wrappers, Strings and Dates. Objects (that are not collections) are they themselves externalized into JSON. In complex data models, you may not wish to serialize the entire downstream object. You may wish to only serialize it's ID. This (and my gripping about the implementation) is shown in one of my previous posts on JAXB and XmlID. Currently Flexjson has no clean way of supporting this. The only was to attempt this would be through the usage of include() and exclude() on the JSONSerializer. The downside of this, is that you essentially need a specific handler for each object you want to serialize since the attributes/conditions of inclusion or exclusion will change. Lets look at an example:

@Entity
public class Stamp implements Serializable {
  private Long id;
  private String description;
  private Country country;
  // ... other attributes and methods
}

@Entity
public class Country implements Serializable {
   private Long id;
   private String name;
   // ...
}

So in this example, if we wanted to serialize all of the Stamps to JSON, by default the country would be serialized for each stamp. In a typical system, we might have 200 countries and 50,000 stamps. This means that our countries are fully serilized in a redundant fashion many times over. In this situation what we really want is the country id field. We can get this in the following way:

  Writer out = // ... some writer like a PrintWriter
  Collection stamps = stampService.getAll( );
  JSONSerializer serializer = new JSONSerializer( );
  serializer = serializer.include("id","description","country.id").exclude("*");
  for( Stamp s: stamps ) { 
     out.write( serializer.serialize( s ) );
  }

While this works, If your object has many properties and object relationships, this can get a little exhaustive setting up the include and exclude parameters. It also means you either (a) need a introspective tool to read this from your beans or (b) you need to provide some handler for each bean to setup the includes and excludes properly. I personally have great faith in the Open Source community, and will look forward to leveraging the next version of Flexjson to cleaner handle this situation with a Transformer (Transformers today only handle Strings, primitives and dates). Until then I suppose I'll have to come up with some solution that is tied to my object model.

JSON Annotating of Java Beans

As I was working through a GWT Code Serializer for generating Java serialization code to serialize objects as XML or JSON, I started thinking about how nice it was to leverage JPA Annotations in documenting JSON serialization on persistent beans. Wouldn't it be nice to serialize objects to JSON as easily as we can do with JAXB? After searching the WEB for a solution, I was almost about ready to go ahead and develop an Annotation infrastructure for this very problem. That was when I came across Flexjson, which does exactly this. In fact, unlike JAXB's marshalling terminology, Flexjson actually calls it serialization. As of yet, I have not tried Flexjson, but from looking at the examples shown in the link above, it seems to do exactly what one would expect.

The Engineers Approach to Baby Crying

I have had the fortunate gift of becoming a father recently to a beautiful little girl "Angela". However, as a first time parent, trying to deal with a crying baby was next to impossible. That was until we found out what all the sounds meant and what worked and didn't work. Now she is three weeks old and almost sleeping through the night. The kep principle is to feel confident and calm that you can take care of whatever the needs of the baby are. That being said, a little decision matrix can go a long way in assisting this. For that reason, I have put together this nice "binary flow chart" which will guide you through to happier parenting!

Olympus and Poor Usability - Lesson 1

Today I was helping my father install a version of Olympus Master 2.0 on his Vista laptop. We were struck back when it asked him to enter the Country. The list seemed to include every country in the world, however they were in no particular order. In his case we has looking for "Canada", and hitting the letter "C" brought up "Cuba". Suffice it to say, we gave up trying to find Canada since finding it in two hundred odd items in no particular order was impossible and frustrating. We selected the first item "United States". Which makes me wonder. Does all software that prompts for a country put "United States" at the top? Is it because the software engineers feel the average American is too stupid to find their country in an alphabetical list? It is these little short cut techniques in software that really destroys to usability of a product.

JAXB and the nasty @XmlID

I wonder why the developers of JAXB decided to make the @XmlID annotations support Strings only? You would think that a String or any primitive type would've been acceptable. The Javadoc of XmlID states:

The usage is subject to the following constraints:

• At most one field or property in a class can be annotated with @XmlID.
• The JavaBean property's type must be java.lang.String.
• The only other mapping annotations that can be used with @XmlID are:@XmlElement and @XmlAttribute

The other property annotations like XmlAttribute and XmlElement support primitives and wrappers. This of course means if you are using JAXB to XML Serialize a JPA Entity, your primary ID key needs to be a string instead of a Integer/Long value. JPA Persistence will still treat this as an integer in your datastore if you have the @GeneratedValue annotation set, so at least from this perspective your data model does not need to change. The advantage of using the @XmlID annotation is it allows you to use the @XmlIDREF tag in other Entities (meaning the entire entity is not XML Serialized only it's @XmlID value. Here is a simple example:

   @Entity @XmlRootElement
   public class Company implements Serializable {
      @Id @Column(name="company_id") @GeneratedValue
      @XmlAttribute @XmlID
      private String id;

      // ... setters, getters and other methods.
   }

   @Entity @XmlRootElement
   public class Employee implements Serializable {
      @Id @GeneratedValue
      @XmlAttribute @XmlID
      private String id;
   
      @JoinColumn(name="COMPANY_REF", referencedColumnName = "company_id")
      @ManyToOne(optional=false)
      @XmlIDREF
      private Company worksfor;

      // ... setters and other methods
   }

Now if we marshall an Employee with an id of "50" who works for a company with an id of "20", the resulting XML would look something like the following:

   <employee id="50">
      <worksfor>20</worksfor>
   <employee>

Eclipse, Toplink, JPA and a Lost Evening

So I decided to put together a simple web-application using the dynamic web project in Eclipse. However, my recent workspace had become rather "corrupted" and so I had created a new one named "eclipse 3.3" (under my common \dev\workspace area). To my dismay, I simply could not get my web-application which was working in my old workspace to work. The crazy thing about this, is I know my persistence.xml and project was setup correct. The Tomcat log was producing the following:

INFO: The configured persistenceUnitName is: MileageTracker
[TopLink Config]: 2008.05.01 10:23:15.703--ServerSession(2165595)--Thread(Thread[main,5,main])--The alias name for the entity class [class org.javad.mileage.model.Vehicle] is being defaulted to: Vehicle.
javax.persistence.PersistenceException: No Persistence provider for EntityManager named MileageTracker: The following providers:
oracle.toplink.essentials.PersistenceProvider
oracle.toplink.essentials.ejb.cmp3.EntityManagerFactoryProvider
Returned null to createEntityManagerFactory.

at javax.persistence.Persistence.createEntityManagerFactory(Persistence.java:154)
at javax.persistence.Persistence.createEntityManagerFactory(Persistence.java:83)
... 

Now what is curious about this, is Toplink was found, and it determined some information about my EntityBean. I spent a lot of time trying to discover what was wrong here, checking my other projects from the previous workspace etc. It turns out the problem was an Eclipse bug in which you have workspaces with spaces (" ") it will fail to find the Entity Manager Factory. The reference bug ID is 210280 Sufficit to say, this cost me a few hours of productivity.....

GWT and JPA with Servlets

I am doing some investigation of GWT and in writing an application, I wanted to integrate the servlet end of my application with a JPA services oriented architecture (either through the servlet itself, or a standalone JPA service). I have written a few JPA applications and there are several things I like about JPA:

• There is a nice model(bean)/service view of the world.
• Defining your persistence behavior on your JavaBeans "feels" right.
• Takes care of all the ugly connection/pooling stuff for you.
• You do not need to be a service "guru" (personally I prefer the business layer and presentation layer) to get a working application with persistent storage.

While there are certainly disadvantages with JPA, I wanted to leverage my JPA knowledge, tools and services with a GWT application using RPC-Servlets. The nice thing about using RPC-Servlets is you can code your servlet in a very DSL like manner. (I was going to look into REST and Restlets but decided GWT was enough to learn for now). Getting JPA and RPC-Servlets proved rather difficult at first, and I almost abandoned the approach in favor of a pure servlet "service" approach delivering JSON or XML objects and then using HTTPRequests to get the responses from GWT.

My first attempt was to try and stick my servlet(s) that referenced my JPA services outside of the GWT client packages and then try and run these within the hosted tomcat within GWT. This proved problematic since GWT/tomcat bundles an earlier version of the xerces library which is not compatible with the persistence.xml xsi schema. (I also wasn't very convinced this would work anways since I am not sure if the imbedded tomcat runs with a 1.4-compliant JDK or not (thus preventing JPA annotations)). So I needed another approach. I had read that GWT hosted browser could run without the Tomcat imbedded, hence allowing you to work and debug within the hosted browser, but use a standard OOTB tomcat service. This appealed to me alot, since I can easily setup a dynamic web project in eclipse to host my servlet/service and I can work in the natural hosted browser of GWT. After a little tweaking I got this to work in eclipse and so far, while not as clean as a pure GWT hosted-mode environment I am able to do everything I need to do.

Application Description

I am going to write an application which allows me to store user profiles (website profiles, not Windows profiles) in a database with encrypted passwords. If you are like me, after signing up for a few sites, you can never remember you user ids and passwords. The application will be called ProfileManager and it will be written in GWT as a web-application, using JPA to access the database through a ProfileService.

GWT Application Setup

I created an eclipse project for my GWT application using the projectCreator and applicationCreator scripts provided by GWT with the -eclipse flag. This provides you with the basic scaffolding necessary for the creation of a GWT application. For the purposes of this blog, the package path of my module is org.javad.profile.gwt.ProfileManager. Executing the ProfileManager.launch script launches the GWT toolkit development shell and my application. I am not going to go into more details here as these procedures are well documented in text and websites.

Step 1: Create the Service Interface

My service interface under the package org.javad.profile.gwt.client.rpc. The interface has to extend the google interface RemoteService. For simplicity this interface defines only a single method getAll( ) that looks like the following:

package org.javad.profile.gwt.client.rpc;

import java.util.List;
import com.google.gwt.user.client.rpc.RemoteService;


public interface RPCProfileService extends RemoteService {
   public List getAll( );
}

As more functionality is added, I will add the signatures to the service interface. Along with the interface, I need an Asynchronous interface which is defined in the same package:

package org.javad.profile.gwt.client.rpc;
import com.google.gwt.user.client.rpc.AsyncCallback;

public interface RPCProfileServiceAsync {
   public void getAll( AsyncCallback callback );
}

Step 2: Create your Serializable Bean

Since JPA uses annotations, we unfortunately have to translate the JPA POJO to a GWT DAO. One product you might want to look at is HiberObjects. For this project, I created a simplified model of my JPA POJO called "ProfileBean" under org.javad.profile.gwt.client.model. Since this overview does not use the ProfileBean directly, I am not going to say any more on it, other than you'd need it for a full GWT Application.

Step 3: Create an Externalized JAR

In order to implement the service interface (and retrieve ProfileBean objects) you need to bundle these into an external JAR to associate to your web project. Within eclipse you can do this with the File->Export ...->JAR File functionality. This JAR forms the externalized view of our service which we'll need to implement within the web project.

Step 4: Modify the .launch Script

When you create your project, the GWT toolkit created a ProfileManager.launch script which you can use to launch your application. The problem is, this launch script will launch the Toolkit Development shell with an imbedded Tomcat, will attempt to connect on port 8888 (default for GWT toolkit) and will not include a web-application root name, which if you are deploying with web project will be needed.

The GWTShell command can take some arguments which we'll use to adjust this. Editing the ProfileManager.launch shell, you need to change the value of the stringAttribute " org.eclipse.jdt.launching.PROGRAM_ARGUMENTS"

1. First, we need to tell the shell not to launch an imbedded Tomcat. This is done by specifiying the -noserver option.
2. The port needs to be specified. In my case, my Tomcat is running on port 8080 which can be defined by specifying the -port 8080 option.
3. Finally, we need to change the application which is launched by the application script by prepending the web-application name in front of the module's HTML file.

An example of this line from my ProfileManager.launch script looks like the following:

<stringAttribute key="org.eclipse.jdt.launching.PROGRAM_ARGUMENTS" value="-noserver -port 8080 -out www Profiles/org.javad.profile.gwt.ProfileManager/ProfileManager.html"/>

Now executing the script will open the Toolkit Shell an attempt to execute the GWT application with the correct port number and web-application name. A tomcat instance will not be started by the Toolkit Shell. The navigator window of course will not be able to connect to your application (since we have not hooked it up yet) so your window should look something like this:

Step 5: Make a call to the Service

Before we go on to the servlet project, it'll be helpful to know that we have the connection to the service working within GWT when the servlet is ready. To test this, I modified the the onModuleLoad( ) method of the application to simply dispatch a call to the RPC service. Obviously if you are going to use a DAO/Controller pattern this would be abstracted, but this is simply a test to know you are on the right path.

   ...
   RPCProfileServiceAsync service = (RPCProfileServiceAsync)GWT.create(RPCProfileService.class);
   ServiceDefTarget endpoint = (ServiceDefTarget) service;
   String moduleRelativeURL = GWT.getModuleBaseURL() + "servlet/ProfileServlet";
   endpoint.setServiceEntryPoint(moduleRelativeURL);
   AsyncCallback callback = new AsyncCallback() {
      public void onSuccess(Object result) {
         System.out.println("in callback");
      }
      public void onFailure(Throwable caught) {
         caught.printStackTrace();
      }
   };
   service.getAll(callback);

If we were to launch the application now, it would fail since there would be no response from the service. However when we complete the next section, we should see to "in callback" message shown in the console of the GWTShell.

Servlet Project For the servlet project, I am using Eclipse Europa with the WTP 2.0. This includes Dali, which allows you to easy define you JPA POJOs using the built in editor. Step 1: Create the Project Create the project within Eclipse using the "File->New->Project..." and selecting the dynamic web project under the Web project types. Enter a project name (this will be the default web-application name) so for mine I chose "Profiles". For the Project Facets step, make sure you choose the "Java Persistence" facet. This will allow you to manage your JPA objects.

Step 2: Add the GWT RPC library When we exported the RPC library from our GWT project (see Step 3 above) we created the contract that the servlet needs obey. We now need to import this JAR into the web project as a references library. After adding it as a referenced library, we also need to ensure that it is copied to the server deployment location. This is done by selecting the Properties of our project and selecting the necessary JARs under the J2EE Module Dependency option. I also included the TopLink JPA, MySQL (connector library) and gwt-servlet.jar as can be seen from the image below:

Step 3: Create your RPC Servlet

To implement our servlet, I created a class GWTProfileServlet which extends the RemoteServiceServlet and implements our RPCProfileService. This is located in the org.javad.profile.servlet package under the src variant of my web project.

package org.javad.profile.servlet;

import java.util.ArrayList;
import java.util.Collection;
import java.util.List;
import org.javad.profile.gwt.client.rpc.RPCProfileService;
import org.javad.profile.model.Profile;
import org.javad.profile.service.ProfileService;
import com.google.gwt.user.server.rpc.RemoteServiceServlet;


public class GWTProfileServlet extends RemoteServiceServlet implements RPCProfileService {
   @Override
   public List getAll() {
      ProfileService service = ProfileService.getInstance();
      Collection profiles = service.getAll();
      System.out.println("debug: got all profiles" + profiles);
      return new ArrayList();
   }
}

In the example above, I make a call to the ProfileService which is a JPA Enabled service for managing profile objects from the datastore. You could also add your JPA persistence code directly to the servlet here. I am also printing out a debug message just to show that I am getting the output in the servlet.

Step 4: Create the servlet mapping

Since we want the GWTProfileServlet to be mapped under the GWT Application, in the web.xml for the web project, you need to define the servlet-mapping that includes the GWT Module name. So for our application, the servlet path was appended to the module path, making a url-pattern name as follows:

<servlet>
<description></description>
<display-name>RPCProfileServlet</display-name>
<servlet-name>RPCProfileServlet</servlet-name>
<servlet-class>org.javad.profile.servlet.GWTProfileServlet</servlet-class></servlet>
<servlet-mapping>
   <servlet-name>RPCProfileServlet</servlet-name>
   <url-pattern>/org.javad.profile.gwt.ProfileManager/servlet/ProfileServlet</url-pattern>
</servlet-mapping>

Step 5: Copy over the base GWT Application

In order for the web application to properly server the GWT Application to the imbedded browser we need to provide some files to the Tomcat.

First compile the GWT Application that you wrote in the first section above using the ProfileManager-compile.cmd through the External Tools in Eclipse.

Create a folder under the www-root named "org.javad.profile.gwt.ProfileManager". In this folder copy the following files from the www-root folder from your GWT Project:

   ProfileManager.html
   org.javad.profile.gwt.ProfileManager.nocache.js
   hosted.html
   gwt.js

Now you should be able to start your application server, and launch the GWT Toolkit browser and debug the application (both the servlet and the GWT application) using the eclipse debugger.