Showing posts with label JPA. Show all posts
Showing posts with label JPA. Show all posts

Thursday, November 1, 2012

Diving into J2EE6 with JTA

I have been silent for a long time on the blog (I have been busy - never fear) but simply have not spent the time here to document my activities.

I was in the process of doing a major overhaul on my server-side JAX-RS WEB Services and their accompanying persistence services to bring things up to a little more standard approach. In particular my goals were to:

  1. Utilize more dependency injection (from JSR-299) to avoid having to separately manage my services in a factory.
  2. Utilize JPA using the JTA transactions and a JDBC datasource (I was previously using Resource_Local transaction modes
  3. Inject the EntityManager (or factory - see below)
  4. Leverage in-container testing using Arquillian for the services (to verify the transactional integrity of the services)
  5. Clear out remnants and methods from when the services were used by a Swing Application (which was replaced by Stamp Web 1.0 in 2008)
  6. Leverage MOXy with JAXB for JSON serialization from JAX-RS WEB Services in place o the Jersey implementation with Jackson and mapped notation (which had required me to create custom JABXContext objects, register properties in my custom mapping file etc.)
  7. Provide a proper JAX-RS implementation using Response Builders. Previously I was returning a JSON object with a success flag instead of using valid HTTP Status objects like NOT_FOUND (404)
  8. Provide complete testing of all JAX-RS WEB Service methods in particular using both JSON and XML supported content encodings and verifying all response code for non-conditions.

As you can imagine this has turned out not to be a small effort and I have had to learn and re-learn a lot of areas of J2EE I had convienently left behind in my memory space.

The following is probably the most important points to remember when designing an application like this:

  1. Start small. I didn't simply start modifying my codebase. Instead I started a new project and brought over the highest level concepts (in my case my PersistableObject, AbstractPersistenceService and a few other classes) and built it up from there - testing as I went. This allowed me to verify the new JTA based approach and also let me figure out a few concerns. For example, previously I had to manage all commit/rollback code. Initially I was thinking to mark methods as Transactional and write an interceptor (which led me down a deep hole of Interceptors) only to realize this was not needed in a JTA context.
  2. Early on I read a blog article that stated that you shouldn't use @PersistenceContext (injecting an EntityManager) because it wasn't thread safe and should instead use @PersistenceUnit (injecting an EntityManagerFactory). This seemed like a good idea and worked... until I tried to save 150 stamps (which each created a EntityManager at a lower level of the code). It appears when using JTA and Container Managed transactions (CMT) creating an EntityManager from the factory will establish a different connection in the pool (I couldn't verify this other than I was seeing max-connection errors and deadlocks in my arquillian tests - but only when this test was used). After some reading, I saw that using @PersistenceContext in CMT it becomes the responsibility of the container to close and manage the EntityManagers. Doing so (in place of using the factory to get a new one) instantly rid me of the connection errors and I see a balance of connection threads - Note the test that hit this is one that never happens in my WEB application (yet) but I am glad to have hit this now - rather than after deploying on my production server.
  3. Arquillian took a little effort to get completely working right, but now that I have it working it seems to work nicely for testing my services "in context". I am currently testing my WEB Services using the jersey-client module and running them after my application is deployed to glassfish. I do want to do some further experimentation to see if I can use Arquillian for these (this article looks promising: http://www.samaxes.com/2012/05/javaee-testing-introduction-arquillian-shrinkwrap/)

Along with my services, I am actually going to rewrite my client side application from a GWT/GXT 2 based solution to SmartClient JS. I originally was going to convert to GXT 3, but I have become pretty disillusioned with the roadmap from Sencha, and therefore are planning to go with a JS based client application. In order to pace this, I need to keep the existing GXT 2 + WEB Services running (which is why I am creating a new Web application project) and will be converting/creating client portions on a priority basis. As well, from the services perspective I am attempting to capture a new level of test code coverage (before it was about 45% now I am aiming for 70%+). To accomplish this, I am not writign any WEB Services beyond my first one to ferret out the testing/WADL approaches to prevent me from working on the client code until the server is complete and solid.

Sunday, January 17, 2010

Unable to deploy JPA to Tomcat (Linux server)

I had to reimage my linux server due to a catastrophic disk failure. In doing so, I had to reinstall tomcat(s) (I use two... one for running my stamp app and the stamp test app (selenium testing) and one for running hudson builds). When my ant script for my stamp application deployed to the "production" tomcat, the start target would fail. I looked in the tomcat logs and saw this:

Exception Description: Predeployment of PersistenceUnit [stamp-test] failed.
Internal Exception: java.lang.RuntimeException: Exception [EclipseLink-7018] (Eclipse Persistence Services - 1.1.2.v20090612-r4475): org.eclipse.persistence.exceptions.ValidationException
Exception Description: File error.
Internal Exception: java.io.FileNotFoundException: /etc/rc.d/init.d/tempToDelete.xml (Permission denied)
at org.eclipse.persistence.exceptions.PersistenceUnitLoadingException.exceptionSearchingForPersistenceResources(PersistenceUnitLoadingException.java:121) 
at org.eclipse.persistence.jpa.PersistenceProvider.createEntityManagerFactory(PersistenceProvider.java:133)
at org.eclipse.persistence.jpa.PersistenceProvider.createEntityManagerFactory(PersistenceProvider.java:67)


This threw my for a loop, until the path where the file was being read/written to tipped me off. I have startup scripts that use the java service daemon (jsvc) which are located in /etc/rc.d/init.d. Sure enough, this was the problem. Essentially eclipselink was trying to create a temporary file while attempting to clone an object when TABLE_PER_CLASS inheritance is being used. The fix turned out to be quite simple. I simply had to perform a change directory prior to calling the jsvc to the $CATALINA_HOME(or wherever your tomcat lives) prior to invoking the jsvc.

Wednesday, February 18, 2009

JPQL and Upper case queries in LIKE

I had a situation where I needed to write a query which would perform upper case comparisons in a LIKE statement. It took me a while to figure this out, mainly because I didn't realize the right-side of the LIKE statement does not support any of the functions. So given a query like this:

  SELECT name FROM Countries WHERE name LIKE '%someValue%';

to write this in a valid JPQL format (assuming we have a persistent bean Country with an attribute name, you would write it out as follows:

  String searchExp = "find me";
  Query qs = entity.createQuery("SELECT c.name FROM Country c WHERE UPPER(c.name) LIKE '%:val%'");
  qs.setParameter("val",searchExp.toUpperCase());

The key here is to convert the right-side in code and then you can use the function UPPER( ) for the left-side of the expression.

This was tested with MySQL and HSQLDB with EclipseLink. There was an article I found here which states that with MySQL 5.1 onward you don't need to use the LOWER function, but I have not verified this yet (nor found the collaborating evidence on MySQL's website). Running against my test system in HSQLDB (in memory) it did not find the results in my unit tests.

Tuesday, August 26, 2008

JPA and Optional Associations

I was writing some unit tests to test a few of my queries and noticed they were failing to return results (even though the resultant rows were clearly in the database). Looking into my changes, the only differences was that I recently added the EclipseLink eclipselink.join-fetch for two one-to-many relationships. From looking at the resultant SQL, it became clear what the problem was. By adding these join-fetch statements, my query became more efficient (since I didn't have subsequent row-by-row lazy fetches later), but it also became invalid in some circumstances. In particular, the fetch join added some AND statements to the WHERE clause whereby the foreign key id was equal to the primary table id. However if the many-side relationship is empty, then this statement will not return any rows. I think the way around this would be to box the AND statement in a compound OR statement with an exists condition. Currently to my knowledge the JPA implementations do not support this, and I am going to research whether or not this is achievable by manually modifying the join-fetch statement.

Sunday, July 27, 2008

Automatically recording Create/Update Timestamps with JPA

Now that I am using Bugzilla to track various bugs and enhancements for my projects, I finally got around to addressing this issue of "Provide the ability to record the create/update timestamp." for my stamp objects. This seems simple enough, and most database applications record this information, however by default this is not something that is automated by the JPA frameworks (unlike the Primary Key with the @Id annotation). Let us first examine the ways we could accomplish this:
  1. Provide a database trigger to try and insert the timestamps automatically.
  2. Manually set them (or have each persistent service set them) before performing a persist() operation with an EntityManager.
  3. Use aspects to dynamically insert the timestamp.
  4. Provide an EntityListener which automatically sets the creation/modification timestamp at persist time
  5. .

Obviously the first solution is very database specific and is not really tied to the JPA code. The second solution is likely to be error prone and easily missed. The final solution is the best way to handle his. Daniel Pfeifer provides an excellent walkthrough of this technique in his blog here. I have a few comments on this. First off, the concept of an entity listener, does not follow the normal "implement this interface" convention. An entity listener is any POJO class which contains one or more methods in the format:
public void someMethod( Object obj )


It should be noted that there are javax.persistence annotations for each of the JPA lifecycle states. In the case of create timestamps, annotating a method with @PrePersist will allow it to be used to provide the creation timestamp. For the modify timestamp, annotating a method with @PreUpdate will call this method on persisting an entity which was previously persisted. The entity listener can be registered either in the orm.xml file (as described by Daniel Pfeifer) or one the Entity class itself using the annotation @EntityListeners( class ... ). Personally I prefer this technique as it allows me to programatically tie a class with its behavior (such as storing timestamps on create/update) without having an external configuration. Using an external configuration in several modules, and unit tests quickly becomes muttled in that you forget to update the file under test etc. It also increases the developer's awareness of this association, and to be candid, it is unlikely that I really want to swap out the persistence timestamp handling at package time with a different solution. Since the annotation is applied to an abstract @MappedSuperClass annotated class, any implementing classes will automatically inherit this behavior. I might change my stance on this approach in the future, but at least for this this seems to be the right way to approach this.

Sunday, July 13, 2008

EclipseLink Startup Lag

I took a few minutes to look into the startup timing differences between EclipseLink and Toplink Essentials. It looks like with EclipseLink they have changed the way the login information is handled for the decryption of the password. Previously in TopLink Essentials, the password was decrypted through the SecurableObjectHolder, which forced the JCEEncryptor to be initialized only upon creating the new instance. In EclipseLink, rather than initializing on first use, they are pre-initializing the JCEEncryptor through the SecurableObjectHolder and then decrypting the password. Another difference between the EclipeLink and TopLink Essentials encryptor, is with EclipseLink they are creating a separate cypher for encryption and decryption. Having looked at the code, this makes sense from a performance scaling perspective. Having separate encryptors/decryptors means that you do not need to reinitialize them for each encryption or decryption. Of course, on startup that would mean the instantiation of not one cypher (for decrypting the password) but two, which does account for about a 4000 ms* difference in the profile runs. This interesting enough is also the difference between the application startup running either EclipseLink and TopLink Essentials.

* Since I am profiling in Netbeans with All Classes the performance numbers themselves are quite poor. What is more interesting is the ~ 18% performance degradation on startup this causes.

Saturday, July 12, 2008

EclipseLink - Initial Impressions

As was suggested by Doug Clarke, I took a look at EclipseLink. I was actually caught a little off guard on the whole subject, as with the birth of my daughter I had pretty much gone under a rock for the past few months. EclipseLink is yet another JPA provider, but there are some interesting aspects to it:
  • It was chosen by Sun Microsystems to serve as the reference implementation for Java Persistence 2.0.
  • The JPA development community has essentially switched from TopLink JPA right over to EclipseLink.
  • EclipseLink is bringing further capabilities with support for MOXy (JAXB), SDO and OSGI to name a few.

There are several good articles on the tool itself from the EclipseLink home page.

I downloaded the most recent release from the Eclipse website, and set about setting up my applications to use it. Generally speaking, I had tried to make only API calls to the javax.persistence APIs, thus I had very little package dependency changes. Since EclipseLink is based off TopLink virtually all the classes from TopLink appear in EclipseLink under a different naming convention. Basically oracle.toplink.essentials became org.eclipse.persistence. Since I have a nice level of unit testing around my services, I was able to quickly identify the places that were failing. In particular, I had some Query Hints that I was applying if the provider was the TopLink provider. I had to replicate this functionality for EclipseLink code paths (fortunately it was only in a few places). I think the biggest impact was on the Upgrade Tooling in which the SQLSchemaUpdateUtility had package dependencies on TopLink Essentials. Instead I roled this into a StatementExecutor interface/implementation and used reflection to call the APIs. Similarly, I did something similar for EclipseLink if I detected it as the persistence provider.

One observation I have noticed is that EclipseLink takes significantly longer to "initialize" than TopLink Essentials. This is most noticeable with the Unit Tests and HSQLDB, where the execution time is approximately ~ 5.0 seconds different. I brought up my JavaSE application in Netbeans 6.1 using the Profiler, and there is a significant time lag building and initializing the EntityManagerFactory. I am not convinced yet that there isn't a setting I am missing which is causing this lag. For example, launching my JavaSE application (which will query to see if there are any upgrades needed, query for all collections and countries, display the Swing UI etc.), almost 56% of the total startup time was spent in the deploy() method of the EntityManagerSetupImpl.

EclipseLink provides a nice compact javax.persistence_1.0.0.jar file which represents all of the J2EE javax.persistence classes required to compile. This is great for application development, and means you can provide your JPA provider at runtime (along with the persistence.xml configuration). Of course, this assumes you are using non-compile dependencies for things like QueryHints etc.

The documentation for EclipseLink seems a little disorganized, but overall there is a lot of information available through the EclipseLink Wiki User Guide.

I am looking forward to using some of the query features (such as fetches for foreign key objects) and it will take some time to really take advantage of the broader features being offered.

Saturday, July 5, 2008

HSQLDB and Toplink : Uniqueness Constraints

After some effort, I was able to figure out a way to place the UNIQUE constraint from the @Column definition on an JPA entity bean, and have it handled properly with the Toplink JPA. I was actually surprised that even with build 40 or Toplink Essentials v2.1 this was still an issue. The problem was, if you defined a value of unique=true in your JPA column annotation, Toplink Essentials would insert the UNIQUE keyword in the create table routine. This would break on HSQLDB, which does not support this keyword during column descriptor creation. The challenge was to fool Toplink into handling the unique attributes like contraints which it would add after the table was created with an ALTER TABLE sequence.

The route I have chosen for now to solve this for my unit testing needs was to provide replacement Toplink-Essentials class files ahead of the Toplink-Essentials JAR file used by my application. Therefore my applications all run with the approved Toplink distributable, however my unit tests run with the instrumented files. There were two small changes I had to make:

(1) Modified oracle.toplink.essentials.tools.schemaframework.FieldDefinition to not write out the unique field keyword if the database was HSQL. From Line 168 of the appendDBString method:
   if(isUnique() && !session.getPlatform().isHSQL()) {
      session.getPlatform().printFieldUnique(writer, shouldPrintFieldIdentityClause);
   }


(2) Modified oracle.toplink.essentials.tools.schemaframework.DefaultTableGenerator to add the unique constraints if the database is HSQL on line 253 of initTableSchema()
 if( dbField.isUnique() && databasePlatform.isHSQL()) {
   tblDef.addUniqueKeyConstraint(dbTbl.getName(),dbField.getName());
 }

Currently I have not posted any information to the Glassfish project with these updates. I am not certain this is the ideal way to achieve this, but from my unit testing perspective I appear to be off to the races. If you are interested in these changes, please contact me and I'll look into working to get them submitted for Glassfish.

Thursday, July 3, 2008

JPA Unit Testing

Up until now, I have been testing my stamp services using a test-schema that lives on my MySQL server. This server is remote (well in my basement connected to a 100Mb ethernet). While this has worked well for me, as the number of unit tests have increased in my code, the time of the tests is also increasing. Eskatos's wrote a great little blog article which introduced HSQLDB to my vernacular (see Unit test JPA Entities with in-memory database). Of course in his scenario, hibernate was used in place of Toplink. However I was intrigued by the idea of using a in-memory only database for unit tests. So I set about to get my unit tests to run. This turned out to be tricky to get working using Toplink. The first issue I ran into was that the tables refused to be created on startup. It turns out, this is due to an issue with the toplink.target.database property missing from the persistence.xml file. This was outlined in a useful blog TopLink JPA and HSQLDB Quirk.

Even after making these changes however, I still could not get toplink to properly create the tables. It turns out, I had several entity beans which had name fields defined as unique=true. This caused the UNIQUE keyword to be written in the CREATE TABLE statements by Toplink which appears to be an invalid syntax for the HSQLDB database. After removing this JPA constraint from the affected objects I was able to successfully create the tables and run my tests. I also had some minor refactoring to do in some SQL utilities to leverage the persistenceUnit configuration, but I was very impressed with the speed.

Overall, my test suite went from executing in approximately eighteen seconds down to just four. While eighteen seconds may not seem like a long time, it was sufficiently long to disrupt my work efficiency. I decided to retain my MySQL peristence unit (for occasional "live" DB testing), and have now configured two test targets in Eclipse which take a org.javad.jpa.serviceName environment variable to switch between the hsqldb and toplink-test persistence units.

The final activity I will have left to do, is to determine a way to reinsert the unique statements in my entity beans without have the SQL generated for HSQLDB. There are a few threads out there, so I should be able to come up with something.

Finally, here is my persistence unit configuration for the HSQLDB database:
<persistence-unit name="hsqldb" transaction-type="RESOURCE_LOCAL">
  <provider>oracle.toplink.essentials.PersistenceProvider</provider>
  <class>org.javad.stamp.model.Album</class>
  <class>org.javad.stamp.model.CatalogueNumberReference</class>
  <class>org.javad.stamp.model.Category</class>
  <class>org.javad.stamp.model.Country</class>
  <class>org.javad.stamp.model.Stamp</class>
  <class>org.javad.stamp.model.StampCollection</class>
  <class>org.javad.model.ClassVersion</class>
  <class>org.javad.services.TestEntityWithIdentity</class>
  <properties>
    <property name="toplink.jdbc.user" value="sa"/>
    <property name="toplink.jdbc.password" value=""/>
    <!-- <property name="toplink.logging.level" value="FINEST"/> -->
    <property name="toplink.jdbc.url" value="jdbc:hsqldb:mem:."/>
    <property name="toplink.jdbc.driver" value="org.hsqldb.jdbcDriver"/>
    <property name="toplink.ddl-generation" value="create-tables"/>
    <property name="toplink.target-database" value="HSQL"/>
  </properties>
</persistence-unit>

Sunday, June 8, 2008

MySQL with Servlets - Poor uptime

Now that I have my mobile application working on the smartphone I have been quite pleased with the client. Then I started getting strange timeouts and no matter what I was trying to do I was unable to execute queries against the database. From looking in the Tomcat logs I discovered an interesting exception:

Last packet sent to the server was 3 ms ago.
        at com.mysql.jdbc.MysqlIO.reuseAndReadPacket(MysqlIO.java:2579)
        at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:2867)
        at com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:1616)
        at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:1708)
        at com.mysql.jdbc.Connection.execSQL(Connection.java:3255)
        at com.mysql.jdbc.PreparedStatement.executeInternal(PreparedStatement.java:1293)
        at com.mysql.jdbc.PreparedStatement.executeQuery(PreparedStatement.java:1428)



This led me to research whether this could be a MySQL issue. It turns out, that the connection pool will close all connections after eight hours of inactivity. Their solution of using the "autoReconnect" property apparently will not work under most circumstances. This is covered in section 26.4.5.3.4 on the MySQL reference manual.

The solution? Well it has been suggested the writing a small daemon thread which wakes up every hour and executes some small query should be sufficient to keep the connections open. I have not implemented this yet, but this seems reasonable. In my case I'll probably tie it to one of my servlets in their init() methods.

Friday, May 23, 2008

JPA Identity Interger/Long or String?

In my previous article JAXB and the nasty XmlID, I discussed how if you wanted to use @XmlID and @XmlIDREF as pointer references to XML serialized objects that the values had to be Strings. In some cases they might be fine, especially if your primary key is a compound key that you are serializing. However, the more common case for simple persisted objects is you have used some numeric identity value as the primary key. A short search will show almost all demos/tutorials and examples of JPA use either a Integer or Long wrapper. This of course is not a restriction of JPA. You can use anything you want as the primary key, but if you are going to leverage some of the @GenerateValue options, unless you are willing to define your own Alpha scheme, you will get a numeric value. Which leads me to my point: If I want to XML Serialize an object using JAXB (which is much easier than doing it by hand with a DOM Document) that contains foreign key references, surely I can do this in some way without having to convert my identities to Strings?

The simple answer is yes, but not by applying the JAXB annotations on the persisted property. Instead, you would need to create a proxy method that can convert your PK (Primary Key) into a string representation, and this method is tagged with the @XmlID annotation. In order to be used by JAXB you need at least a String property. Lets look at a simple code example.
   import javax.xml.bind.annotation.XmlID;
   import javax.xml.bind.annotation.XmlRootElement;
   import javax.xml.bind.annotation.XmlTransient;
   import javax.xml.bind.annotation.XmlAttribute;
   import javax.persistence.Entity;
   import javax.persistence.Id;
   import javax.persistence.GeneratedValue;
   import javax.persistence.Transient;

   @XmlRootElement
   @Entity
   public class PersistedObject {
       @Id
       @GeneratedValue(strategy=TABLE, generator="CUST_GEN")
       @XmlTransient // we are not going to write out the id
       private Long id = null;

       @XmlTransient
       @Transient   // this is not an entity managed attribute
       private String identityString = null;

       @XmlTransient
       public Long getId( ) { return id; }

       public void setId( Long id ) { this.id = id; }

       @XmlID
       @XmlAttribute(name="id")
       public String getIdentityString( ) {
          return ( id != null ) ? id.toString() : "0";
       }
   }


In this manner, we can denote our JPA identity with the data-type which makes sense (either a Long or Integer) yet allow for easy XML Serialization through the use of the @XmlID field on the getIdentityString() method. This is certainly not ideal, and I would've preferred to put the annotation on a method only, however JAXB requires the XmlID tag on a property.

Unfortunately for me, I only thought of this after converted my persistent beans, services and unit tests over to String Ids. Fortunately SCM tools (subversion in this case) come to the rescue and I can easily back out my changes.

Tuesday, May 13, 2008

Flexjson limitations

The old saying goes "If it sounds too good to be true, it is too good to be true". Well this applied to Flexjson. While this tool is very capable of externalizing an object to the JSON format, there are several shortcomings which make it difficult to use at the moment. The contributors have mentioned they are working to address these issues. Currently Flexjson only can process primitives, wrappers, Strings and Dates. Objects (that are not collections) are they themselves externalized into JSON. In complex data models, you may not wish to serialize the entire downstream object. You may wish to only serialize it's ID. This (and my gripping about the implementation) is shown in one of my previous posts on JAXB and XmlID. Currently Flexjson has no clean way of supporting this. The only was to attempt this would be through the usage of include() and exclude() on the JSONSerializer. The downside of this, is that you essentially need a specific handler for each object you want to serialize since the attributes/conditions of inclusion or exclusion will change. Lets look at an example:
@Entity
public class Stamp implements Serializable {
  private Long id;
  private String description;
  private Country country;
  // ... other attributes and methods
}

@Entity
public class Country implements Serializable {
   private Long id;
   private String name;
   // ...
}
So in this example, if we wanted to serialize all of the Stamps to JSON, by default the country would be serialized for each stamp. In a typical system, we might have 200 countries and 50,000 stamps. This means that our countries are fully serilized in a redundant fashion many times over. In this situation what we really want is the country id field. We can get this in the following way:
  Writer out = // ... some writer like a PrintWriter
  Collection stamps = stampService.getAll( );
  JSONSerializer serializer = new JSONSerializer( );
  serializer = serializer.include("id","description","country.id").exclude("*");
  for( Stamp s: stamps ) { 
     out.write( serializer.serialize( s ) );
  }

While this works, If your object has many properties and object relationships, this can get a little exhaustive setting up the include and exclude parameters. It also means you either (a) need a introspective tool to read this from your beans or (b) you need to provide some handler for each bean to setup the includes and excludes properly. I personally have great faith in the Open Source community, and will look forward to leveraging the next version of Flexjson to cleaner handle this situation with a Transformer (Transformers today only handle Strings, primitives and dates). Until then I suppose I'll have to come up with some solution that is tied to my object model.

Friday, May 2, 2008

JAXB and the nasty @XmlID

I wonder why the developers of JAXB decided to make the @XmlID annotations support Strings only? You would think that a String or any primitive type would've been acceptable. The Javadoc of XmlID states:
The usage is subject to the following constraints:
  • At most one field or property in a class can be annotated with @XmlID.
  • The JavaBean property's type must be java.lang.String.
  • The only other mapping annotations that can be used with @XmlID are:@XmlElement and @XmlAttribute
The other property annotations like XmlAttribute and XmlElement support primitives and wrappers. This of course means if you are using JAXB to XML Serialize a JPA Entity, your primary ID key needs to be a string instead of a Integer/Long value. JPA Persistence will still treat this as an integer in your datastore if you have the @GeneratedValue annotation set, so at least from this perspective your data model does not need to change. The advantage of using the @XmlID annotation is it allows you to use the @XmlIDREF tag in other Entities (meaning the entire entity is not XML Serialized only it's @XmlID value. Here is a simple example:
   @Entity @XmlRootElement
   public class Company implements Serializable {
      @Id @Column(name="company_id") @GeneratedValue
      @XmlAttribute @XmlID
      private String id;

      // ... setters, getters and other methods.
   }

   @Entity @XmlRootElement
   public class Employee implements Serializable {
      @Id @GeneratedValue
      @XmlAttribute @XmlID
      private String id;
   
      @JoinColumn(name="COMPANY_REF", referencedColumnName = "company_id")
      @ManyToOne(optional=false)
      @XmlIDREF
      private Company worksfor;

      // ... setters and other methods
   }

Now if we marshall an Employee with an id of "50" who works for a company with an id of "20", the resulting XML would look something like the following:

   <employee id="50">
      <worksfor>20</worksfor>
   <employee>

Thursday, May 1, 2008

Eclipse, Toplink, JPA and a Lost Evening

So I decided to put together a simple web-application using the dynamic web project in Eclipse. However, my recent workspace had become rather "corrupted" and so I had created a new one named "eclipse 3.3" (under my common \dev\workspace area). To my dismay, I simply could not get my web-application which was working in my old workspace to work. The crazy thing about this, is I know my persistence.xml and project was setup correct. The Tomcat log was producing the following:
INFO: The configured persistenceUnitName is: MileageTracker
[TopLink Config]: 2008.05.01 10:23:15.703--ServerSession(2165595)--Thread(Thread[main,5,main])--The alias name for the entity class [class org.javad.mileage.model.Vehicle] is being defaulted to: Vehicle.
javax.persistence.PersistenceException: No Persistence provider for EntityManager named MileageTracker: The following providers:
oracle.toplink.essentials.PersistenceProvider
oracle.toplink.essentials.ejb.cmp3.EntityManagerFactoryProvider
Returned null to createEntityManagerFactory.

at javax.persistence.Persistence.createEntityManagerFactory(Persistence.java:154)
at javax.persistence.Persistence.createEntityManagerFactory(Persistence.java:83)
... 
Now what is curious about this, is Toplink was found, and it determined some information about my EntityBean. I spent a lot of time trying to discover what was wrong here, checking my other projects from the previous workspace etc. It turns out the problem was an Eclipse bug in which you have workspaces with spaces (" ") it will fail to find the Entity Manager Factory. The reference bug ID is 210280 Sufficit to say, this cost me a few hours of productivity.....

Tuesday, April 29, 2008

GWT and JPA with Servlets

I am doing some investigation of GWT and in writing an application, I wanted to integrate the servlet end of my application with a JPA services oriented architecture (either through the servlet itself, or a standalone JPA service). I have written a few JPA applications and there are several things I like about JPA:

  • There is a nice model(bean)/service view of the world.
  • Defining your persistence behavior on your JavaBeans "feels" right.
  • Takes care of all the ugly connection/pooling stuff for you.
  • You do not need to be a service "guru" (personally I prefer the business layer and presentation layer) to get a working application with persistent storage.

While there are certainly disadvantages with JPA, I wanted to leverage my JPA knowledge, tools and services with a GWT application using RPC-Servlets. The nice thing about using RPC-Servlets is you can code your servlet in a very DSL like manner. (I was going to look into REST and Restlets but decided GWT was enough to learn for now). Getting JPA and RPC-Servlets proved rather difficult at first, and I almost abandoned the approach in favor of a pure servlet "service" approach delivering JSON or XML objects and then using HTTPRequests to get the responses from GWT.

My first attempt was to try and stick my servlet(s) that referenced my JPA services outside of the GWT client packages and then try and run these within the hosted tomcat within GWT. This proved problematic since GWT/tomcat bundles an earlier version of the xerces library which is not compatible with the persistence.xml xsi schema. (I also wasn't very convinced this would work anways since I am not sure if the imbedded tomcat runs with a 1.4-compliant JDK or not (thus preventing JPA annotations)). So I needed another approach. I had read that GWT hosted browser could run without the Tomcat imbedded, hence allowing you to work and debug within the hosted browser, but use a standard OOTB tomcat service. This appealed to me alot, since I can easily setup a dynamic web project in eclipse to host my servlet/service and I can work in the natural hosted browser of GWT. After a little tweaking I got this to work in eclipse and so far, while not as clean as a pure GWT hosted-mode environment I am able to do everything I need to do.

Application Description

I am going to write an application which allows me to store user profiles (website profiles, not Windows profiles) in a database with encrypted passwords. If you are like me, after signing up for a few sites, you can never remember you user ids and passwords. The application will be called ProfileManager and it will be written in GWT as a web-application, using JPA to access the database through a ProfileService.

GWT Application Setup

I created an eclipse project for my GWT application using the projectCreator and applicationCreator scripts provided by GWT with the -eclipse flag. This provides you with the basic scaffolding necessary for the creation of a GWT application. For the purposes of this blog, the package path of my module is org.javad.profile.gwt.ProfileManager. Executing the ProfileManager.launch script launches the GWT toolkit development shell and my application. I am not going to go into more details here as these procedures are well documented in text and websites.

Step 1: Create the Service Interface

My service interface under the package org.javad.profile.gwt.client.rpc. The interface has to extend the google interface RemoteService. For simplicity this interface defines only a single method getAll( ) that looks like the following:


package org.javad.profile.gwt.client.rpc;

import java.util.List;
import com.google.gwt.user.client.rpc.RemoteService;


public interface RPCProfileService extends RemoteService {
   public List getAll( );
}
As more functionality is added, I will add the signatures to the service interface. Along with the interface, I need an Asynchronous interface which is defined in the same package:

package org.javad.profile.gwt.client.rpc;
import com.google.gwt.user.client.rpc.AsyncCallback;

public interface RPCProfileServiceAsync {
   public void getAll( AsyncCallback callback );
}

Step 2: Create your Serializable Bean

Since JPA uses annotations, we unfortunately have to translate the JPA POJO to a GWT DAO. One product you might want to look at is HiberObjects. For this project, I created a simplified model of my JPA POJO called "ProfileBean" under org.javad.profile.gwt.client.model. Since this overview does not use the ProfileBean directly, I am not going to say any more on it, other than you'd need it for a full GWT Application.

Step 3: Create an Externalized JAR

In order to implement the service interface (and retrieve ProfileBean objects) you need to bundle these into an external JAR to associate to your web project. Within eclipse you can do this with the File->Export ...->JAR File functionality. This JAR forms the externalized view of our service which we'll need to implement within the web project.

Step 4: Modify the .launch Script

When you create your project, the GWT toolkit created a ProfileManager.launch script which you can use to launch your application. The problem is, this launch script will launch the Toolkit Development shell with an imbedded Tomcat, will attempt to connect on port 8888 (default for GWT toolkit) and will not include a web-application root name, which if you are deploying with web project will be needed.

The GWTShell command can take some arguments which we'll use to adjust this. Editing the ProfileManager.launch shell, you need to change the value of the stringAttribute " org.eclipse.jdt.launching.PROGRAM_ARGUMENTS"

  1. First, we need to tell the shell not to launch an imbedded Tomcat. This is done by specifiying the -noserver option.
  2. The port needs to be specified. In my case, my Tomcat is running on port 8080 which can be defined by specifying the -port 8080 option.
  3. Finally, we need to change the application which is launched by the application script by prepending the web-application name in front of the module's HTML file.

An example of this line from my ProfileManager.launch script looks like the following:

<stringAttribute key="org.eclipse.jdt.launching.PROGRAM_ARGUMENTS" value="-noserver -port 8080 -out www Profiles/org.javad.profile.gwt.ProfileManager/ProfileManager.html"/>

Now executing the script will open the Toolkit Shell an attempt to execute the GWT application with the correct port number and web-application name. A tomcat instance will not be started by the Toolkit Shell. The navigator window of course will not be able to connect to your application (since we have not hooked it up yet) so your window should look something like this:

Step 5: Make a call to the Service

Before we go on to the servlet project, it'll be helpful to know that we have the connection to the service working within GWT when the servlet is ready. To test this, I modified the the onModuleLoad( ) method of the application to simply dispatch a call to the RPC service. Obviously if you are going to use a DAO/Controller pattern this would be abstracted, but this is simply a test to know you are on the right path.


   ...
   RPCProfileServiceAsync service = (RPCProfileServiceAsync)GWT.create(RPCProfileService.class);
   ServiceDefTarget endpoint = (ServiceDefTarget) service;
   String moduleRelativeURL = GWT.getModuleBaseURL() + "servlet/ProfileServlet";
   endpoint.setServiceEntryPoint(moduleRelativeURL);
   AsyncCallback callback = new AsyncCallback() {
      public void onSuccess(Object result) {
         System.out.println("in callback");
      }
      public void onFailure(Throwable caught) {
         caught.printStackTrace();
      }
   };
   service.getAll(callback);

If we were to launch the application now, it would fail since there would be no response from the service. However when we complete the next section, we should see to "in callback" message shown in the console of the GWTShell.

Servlet Project For the servlet project, I am using Eclipse Europa with the WTP 2.0. This includes Dali, which allows you to easy define you JPA POJOs using the built in editor. Step 1: Create the Project Create the project within Eclipse using the "File->New->Project..." and selecting the dynamic web project under the Web project types. Enter a project name (this will be the default web-application name) so for mine I chose "Profiles". For the Project Facets step, make sure you choose the "Java Persistence" facet. This will allow you to manage your JPA objects.

Step 2: Add the GWT RPC library When we exported the RPC library from our GWT project (see Step 3 above) we created the contract that the servlet needs obey. We now need to import this JAR into the web project as a references library. After adding it as a referenced library, we also need to ensure that it is copied to the server deployment location. This is done by selecting the Properties of our project and selecting the necessary JARs under the J2EE Module Dependency option. I also included the TopLink JPA, MySQL (connector library) and gwt-servlet.jar as can be seen from the image below:

Step 3: Create your RPC Servlet

To implement our servlet, I created a class GWTProfileServlet which extends the RemoteServiceServlet and implements our RPCProfileService. This is located in the org.javad.profile.servlet package under the src variant of my web project.


package org.javad.profile.servlet;

import java.util.ArrayList;
import java.util.Collection;
import java.util.List;
import org.javad.profile.gwt.client.rpc.RPCProfileService;
import org.javad.profile.model.Profile;
import org.javad.profile.service.ProfileService;
import com.google.gwt.user.server.rpc.RemoteServiceServlet;


public class GWTProfileServlet extends RemoteServiceServlet implements RPCProfileService {
   @Override
   public List getAll() {
      ProfileService service = ProfileService.getInstance();
      Collection profiles = service.getAll();
      System.out.println("debug: got all profiles" + profiles);
      return new ArrayList();
   }
}

In the example above, I make a call to the ProfileService which is a JPA Enabled service for managing profile objects from the datastore. You could also add your JPA persistence code directly to the servlet here. I am also printing out a debug message just to show that I am getting the output in the servlet.

Step 4: Create the servlet mapping

Since we want the GWTProfileServlet to be mapped under the GWT Application, in the web.xml for the web project, you need to define the servlet-mapping that includes the GWT Module name. So for our application, the servlet path was appended to the module path, making a url-pattern name as follows:


<servlet>
<description></description>
<display-name>RPCProfileServlet</display-name>
<servlet-name>RPCProfileServlet</servlet-name>
<servlet-class>org.javad.profile.servlet.GWTProfileServlet</servlet-class></servlet>
<servlet-mapping>
   <servlet-name>RPCProfileServlet</servlet-name>
   <url-pattern>/org.javad.profile.gwt.ProfileManager/servlet/ProfileServlet</url-pattern>
</servlet-mapping>

Step 5: Copy over the base GWT Application

In order for the web application to properly server the GWT Application to the imbedded browser we need to provide some files to the Tomcat.

First compile the GWT Application that you wrote in the first section above using the ProfileManager-compile.cmd through the External Tools in Eclipse.

Create a folder under the www-root named "org.javad.profile.gwt.ProfileManager". In this folder copy the following files from the www-root folder from your GWT Project:

   ProfileManager.html
   org.javad.profile.gwt.ProfileManager.nocache.js
   hosted.html
   gwt.js
Now you should be able to start your application server, and launch the GWT Toolkit browser and debug the application (both the servlet and the GWT application) using the eclipse debugger.

Tuesday, March 4, 2008

QueryHint and TopLink Essentials

Since I am using TopLink Essentials (I tend to try and keep this updated - currently using version 2.1 build 22), I was finding a few query issues where I was getting some stale objects for certain queries from the cache. This seems possible, since I am performing a lot of queries in background tasks (executed as threads) using ThreadLocal entity managers, however all the entity managers should be using a shared EntityManagerFactory. I decided to send Query Hints with the query to force a refresh on called queries. Since I have provider information stored in the ServerFactory class, I added a new method createQuery( EntityManager em, String query ) to not only create the query, but also set the QueryHint and FlushMode. It turned out, I was setting the FlushMode on almost every query as it was, so this at least gives me some more control over this. The EntityManager is used to create the query, and also for getting the persistence provider. To force a refresh, I am using: query.setHint("toplink.refresh", "true"); when the provider is TopLink Essentials. Currently, for other providers I am not doing anything special. A great blog on this is posted by Wonseok Kim on Java.net: http://weblogs.java.net/blog/guruwons/archive/2006/09/understanding_t.html