Wednesday, February 18, 2009

JPQL and Upper case queries in LIKE

I had a situation where I needed to write a query which would perform upper case comparisons in a LIKE statement. It took me a while to figure this out, mainly because I didn't realize the right-side of the LIKE statement does not support any of the functions. So given a query like this:

  SELECT name FROM Countries WHERE name LIKE '%someValue%';

to write this in a valid JPQL format (assuming we have a persistent bean Country with an attribute name, you would write it out as follows:

  String searchExp = "find me";
  Query qs = entity.createQuery("SELECT c.name FROM Country c WHERE UPPER(c.name) LIKE '%:val%'");
  qs.setParameter("val",searchExp.toUpperCase());

The key here is to convert the right-side in code and then you can use the function UPPER( ) for the left-side of the expression.

This was tested with MySQL and HSQLDB with EclipseLink. There was an article I found here which states that with MySQL 5.1 onward you don't need to use the LOWER function, but I have not verified this yet (nor found the collaborating evidence on MySQL's website). Running against my test system in HSQLDB (in memory) it did not find the results in my unit tests.

Saturday, December 6, 2008

GXT Tree and Loading ... Alternative Design (Validated)

In my previous blog I had proposed a design which I could use to dynamically load the tree based on several different modeled objects. I was successful in implementing this. One of the tricks was to ensure the stores were loaded prior to building the tree, and to do this, I wrote a handler which would load each of the stores and listen for the completion of the loads and then build the tree from the model structures.

YSlow and Cache-Headers

If you run YSLow on one of your web applications, you can obtain some really useful information about improving the performance of your application. One common performance improvement is to set expiration headers on static resources (such as CSS, JS and image files). Setting this header, will help preserve the file in the browser cache, thus preventing reload of the image on subsequent page refreshes. Web servers like Apache make this easy (with a configuration file change). However, if you are using Tomcat both as a Web server and application container, the solution here is relative easy, but requires some coding. To facilitate this in Tomcat, you need to create a ServletFilter to add the Cache-Control header to the response, and then register your filtered types in your web.xml. A blog on this was posted by Byron Tymvios on jGuru here.

Tuesday, December 2, 2008

GXT Tree and Loading ... Alternative Design

This is a followup blog to my previous blog entry on GXT Tree and Loading from Restful URLs. While this approach worked, I had concerns over the number of AJAX requests as the tree is fully "expanded". As well, I was considering another possible issue:
  • The BaseModel objects representing the Country, Album objects are used elsewhere in the client (not just within my tree).
In this sense, the tree model is really a "view" into the real data. Having the tree be dynamically loaded prevents me from really being able to easily sync up changes made to the real data used elsewhere. I had already recognized that I likely needed to provide a singleton store pattern for obtaining countries and albums. The only problem was I was using this for lists and selection drop-downs which are modeled as ListStores (which co-incidentally are incompatible with TreeStores). However the fact that I have data being manipulated in two places just seems rather dangerous to me. For this reason, I think the process I intend to use here is still to use a DataProxy for the TreeStore, but instead have it's load method pull values from the corresponding singleton CountryStore and AlbumStore, and then provide some form of a Binder such that I can bind my TreeStore to the CountryStore (thus when a Country is created, deleted or modified, the TreeStore will be notified of the change in the CountryStore and take appropriate action). This also means I might be able to use a lighter-weight "representation" of the Country in the Tree (for example, just the "name" (for display) and "id" (as the foreign key to the CountryStore). In this case, my tree would really boil down to a BaseTreeModel object with three attributes ("name", "id", "type"). I'd want the type so that I can:

  1. Know the singleton store to access (if I need to full model)
  2. Toggle/display to proper icon in the tree
  3. Handle the leaf/childless cases (such as for Country)
One challenge in this case is I would need to ensure the AlbumStore, CountryStore and StampCollectionStore's are loaded prior to initializing/loading the tree store. As well, I would need to make sure the JSON output from the Album and StampCollection Restful Web Services includes an array of the ids of their children. (It actually is doing this today - but I bring this up as a point that others might consider). For example, an Album JSON String might look like the following:

{"total":1,"albums":[{"name":"Australia","id":154,"countries":[1104,733,3851,704]}]}
In the above example, the album for Australia contains references to the countries 1104, 733 etc. If the StampTreeDataProxy came across a parent BaseModel which was an Album reference, I could obtain the concrete Album from the AlbumStore to get the Country references and thus create an output list for the Callback.onSuccess() method.

While this above approach will take me more time to implement, I think it will provide one major performance improvement. The tree building time is strictly driven by tree AJAX calls to obtain the concrete Country, Album and Stamp Collection data models, as opposed to "N" calls for each node in the tree. It would also give a consolidated view of the tree and allow management of the models to be controlled by their respective stores in a more uniform fashion.

Finally, using a ReferenceModel approach, I actually do not even need the DataProxy. I'd have to test this out, but it looks like I can simply use the TreeBinder to bind a Tree to a TreeModel, and the TreeModel can be built programatically once the singleton data stores are created.

Monday, December 1, 2008

GXT Tree and Loading with REST

Anyone who has used GXT can tell you that it is very powerful, but examples (and the javadoc) are still few and hard to find. The examples that come with the SDK are great (see Ext GWT 1.2 Samples), but if you need to something beyond these you typically need to figure it out on your own. I hope to be able to write a series of blog articles on some of the designs I have figured out for the conversion of my Stamp application from Swing to GXT. (Some of my readers may remember I was experimenting with EXT-JS (more on this in a future blog)).
Getting a tree to work in GXT is not too hard. In fact the demos do a pretty job of illustrating this. However, my tree was a little more complex than the trees shown. First some features of my tree:
  1. Has three separate types of objects (Stamp Collections, Albums and Countries)
  2. Stamp Collections and Albums may contain children (Albums and Countries respectfully)
  3. A Restful WebService is used to return the content of a particular item. (for example to get the albums of a stamp collection with id of "5" the URI might look like http://hostname/web-app/resources/collections/albums/5).

My original approach was that I wanted to make a request on the selected tree node when an expansion occured. However when I used filtering, this caused the invocation of the filter to actually load all the nodes (which was very slow on the first filter). You can turn this off, however that defeated the point of the filter. It also didn't seem like the right solution since at least in the examples the trees were being loaded via an RPC call. So the problem really boiled down to making the tree make an AJAX call to the Restful Web Service upon requesting the expansion of an unloaded node. First I should mention that trees in GXT actually act on BaseModel objects. One point of interest here is that BaseModel object are not TreeModel's (in this way it does work differently than Swing-based trees).
The approach I took was to create a customized DataProxy and pass this proxy to the store on creation. Thus as nodes are loaded/expanded the data proxy will be used to obtain the node's children. My proxy essentially overloaded the load( ) method and for convienence added another method getRequestBuilder( ). I would send requests using the output of the request builder to the Restful Web Service and process the results with a JSONReader and output the load result in the callback's onSuccess( ) method. So lets look at some code:
public class StampTreeDataProxy implements DataProxy<BaseModel,List<BaseModel>> {

   public StampTreeDataProxy( ) {
      super();
   }

   /**  
    * Generate the request builder for the object. If it is the root node (ie. null)
    * then request the stamp collections.  Else, get the appropriate Albums or Countries.
    * For any other type of object, since we do not know how to build a Restful Web
    * URI, simply return a null value.
    * 
    * @param item   The current (parent) item.
    */
   protected RequestBuilder getRequestBuilder( BaseModel item ) {
      RequestBuilder builder = null;
      if( item == null ) {
 builder = HttpUtils.getRestfulRequest(StampModelTypeHelper.getResourceName(StampCollection.class));
      } else if( item instanceof NamedBaseTreeModel ){
         String pathInfo = ((item instanceof Album) ?
                StampModelTypeHelper.getResourceName(Country.class) :    
                StampModelTypeHelper.getResourceName(Album.class)) +"/" + ((NamedBaseTreeModel)item).getId();
         builder = HttpUtils.getRestfulRequest(
                StampModelTypeHelper.getResourceName(item.getClass()), pathInfo, HttpMethod.GET );
      }
      return builder;
   }
  
   public void load(DataReader<BaseModel, List<BaseModel>> reader, 
       final BaseModel parent, final AsyncCallback<List<BaseModel>> callback) {
      
      RequestBuilder builder = getRequestBuilder( parent );
      // If the builder is null, then we do not have a Restful Builder we can handle.
      if( builder == null ) {
         callback.onSuccess(new ArrayList<BaseModel>());
         return;
      }
      builder.setCallback(new RequestCallback() {
      
         public void onError(Request request, Throwable exception) {
            GWT.log("error",exception);
         }

      @SuppressWarnings("unchecked")
      public void onResponseReceived(Request request, Response response) {
         if( response.getStatusCode() == Response.SC_OK ) {
            if (HttpUtils.isJsonMimeType(response.getHeader(HttpUtils.HEADER_CONTENT_TYPE))) {
               JSONValue json = JSONParser.parse(response.getText());
               Class _c = StampCollection.class;
               if( parent instanceof StampCollection || parent instanceof Album) {
                  _c = ( parent instanceof StampCollection ) ? Album.class : Country.class;
               }

               // Modified JsonReader which can read structures of ModelType definitions.
               // For this, I believe a regular JsonReader would work, however
               // it would create instances of BaseModel objects instead of
               // my specific types (which have nice accessor methods).
               StructuredJsonReader<BaseListLoadConfig> reader = new
               StructuredJsonReader<BaseListLoadConfig>(new StampModelTypeHelper(), _c );
               ListLoadResult lr = reader.read(new BaseListLoadConfig(), json.toString());
               callback.onSuccess(lr.getData());
            }
            } else {
               GWT.log("a non-status code of OK" + response.getStatusCode(), null);
            }
         }
      });
      try {
         builder.send();
      } catch (RequestException e) {
         e.printStackTrace();
      }
   }
}

Using this DataProxy, I can create an instance and pass it to the store I am creating for the tree:
public class BrowseStore extends TreeStore<BaseModel> {
  
   public BrowseStore( ) {
      super(new BaseTreeLoader<BaseModel>( new StampTreeDataProxy()));
   }
 
   // Simplified method to create the tree content and load it 
   public void load( ) {
      removeAll();
      getLoader().load();
   }
  
}

In this desgin, when load() is called (either on the loader of the store or the store itself), the DataProxy will be called for the root element. Since I am ignoring the reader (argument to the load( ) method) I create the new StructuredJsonReader. As stated above, the reason for this is three fold:
  1. It handles structured ModelTypes (ie. a ModelType with a field representing another ModelType.
  2. The ability to delegate the creation of the BaseModel to another class (in this case the StampModelTypeHelper which will create the appropriate instance of the modeled object (eg. Country).
  3. Finally uses the class provided to create the propert model type definition.

There is one negative of this solution. Currently it is posting a single request per node in the tree (eg. given an Album get all the countries). I plan on redesigning this to be a little more efficient, however given the mixture of BaseModel types, in order to get an efficient structure downloaded, I may need to forgoe the clean model structure in preference for a more efficient algorithm.

Friday, October 3, 2008

ExtJS Action.submit response

When an ExtJS form is submitted, the successful completion of the asynchronous call will call the function mapped to the success config value. This function will take two parameters form and action. If you are returning JSON from the call you an access this directly from the action parameter.

var _form = // ... get your form (eg. formPanel.form) _form.submit({ scope.this, waitMsg:'Doing someting',url: someurl: method: somemethod, success: function(form, action ) { Ext.Msg.alert('Success?', action.result.success ); Ext.Msg.alert('Data returned.', action.result.data.key1 ); } });

The return value should look something like the following:

{"success":true,"data":{"key1":"key 1 result value"}}

Collections and JAX-RS

As I had previously reported, both RestEasy and Jersey both suffered from the inability to return a collection of JAXB marshalled objects. I was thinking on this a little, and while I could use the JAXBCollection fix in Jersey, it seemed a little bit of a hack until it goes into the 1.0 release. Instead (for now), I have written a few wrapper classes which they themselves are XmlRootElements thus supporting marshalling with JAXB. This actually worked and is a reasonable workaround (since I really only need to return collections for four to five persistent objects. These wrapper objects simply have a collection/list of the persistent objects with the XmlElement set to have the appropriate name of the child elements for processing:

package org.javad.stamp.model.collections; import java.util.ArrayList; import java.util.List; import javax.xml.bind.annotation.XmlElement; import javax.xml.bind.annotation.XmlRootElement; import javax.xml.bind.annotation.XmlTransient; import org.javad.stamp.model.Country; @XmlRootElement(name="countryList") public class CountryList { @XmlTransient public List countries = new ArrayList(); public CountryList( ) { } public void setCountries( List c ) { countries = c; } @XmlElement(name="country") public List getCountries() { return countries; } public void addCountry( Country c ) { countries.add(c); } }

Deploying RestEasy in Tomcat 6.0

Since I was successful deploying Jersey to Tomcat 6.0, I decided to check out RestEasy from JBoss. One issue I ran into with Jersey was it's inability to XML serialize a collection of objects. This is bug ID is 18, and it is fixed for the 1.0 release (however a beta of this does not appear to be available yet), but I decided to see if RestEasy (a non-reference implementation) has the same issue.

UPDATE I have discovered from some testing that neither XML nor JSON is currently supported as a return type for a Collection/List of items. An issue has been filed for RestEasy for JSON (RESTEASY-134) with an RC1 delivery, however I did not see one for XML.

Here are the steps I used to get this working:

  1. Download RestEasy from the JBoss website: http://www.jboss.org/resteasy/
  2. Follow the steps 2 to 4 of my previous blog Deploying Jersey in Tomcat 6.0
  3. Download and create a Eclipse library for JavaAssist. Include this in your WEB project (or provide the javaassist.jar for Tomcat)
  4. Create a new RestEasy library in eclipse which contains the content of the rest easy installation lib location. You can probably skip a few of the jars if you do not need all the functionality (such as jyaml.jar and possibly mail.jar)
  5. Modify your web.xml for your project to include the following (this comes right from the sample web.xml in the rest easy install):

    <?xml version="1.0" encoding="UTF-8"?> <web-app xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns="http://java.sun.com/xml/ns/javaee" xmlns:web="http://java.sun.com/xml/ns/javaee/web-app_2_5.xsd" xsi:schemaLocation="http://java.sun.com/xml/ns/javaee http://java.sun.com/xml/ns/javaee/web-app_2_5.xsd" id="WebApp_ID" version="2.5"> <display-name>TestWeb</display-name> <welcome-file-list> <welcome-file>index.html</welcome-file> </welcome-file-list> <context-param> <param-name>resteasy.scan</param-name> <param-value>true</param-value> </context-param> <!-- set this if you map the Resteasy servlet to something other than /* <context-param> <param-name>resteasy.servlet.mapping.prefix</param-name> <param-value>/resteasy</param-value> </context-param> --> <!-- if you are using Spring, Seam or EJB as your component model, remove the ResourceMethodSecurityInterceptor --> <context-param> <param-name>resteasy.resource.method-interceptors</param-name> <param-value> org.jboss.resteasy.core.ResourceMethodSecurityInterceptor </param-value> </context-param> <listener> <listener-class>org.jboss.resteasy.plugins.server.servlet.ResteasyBootstrap</listener-class> </listener> <servlet> <servlet-name>Resteasy</servlet-name> <servlet-class>org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher</servlet-class> </servlet> <servlet-mapping> <servlet-name>Resteasy</servlet-name> <url-pattern>/*</url-pattern> </servlet-mapping> </web-app>
  6. Start your tomcat in Eclipse and you should be good to go!

Thursday, October 2, 2008

Deploying Jersey in Tomcat 6.0

Jersey is a reference implementation for the JAX-RS (JSR-311) for building RESTful Web services. It is nearing approval with the JSR committees. While there are many wikis and articles in using Jersey with Netbeans (downloading the Netbeans 6.1 EE package includes everything you need for JSR-311), there was very little information on using Tomcat 6.0 with Jersey. After piecing together several blogs, I was able to get a simple resource working in Tomcat using JSR-311.

The following are the steps I used to get this working:

  1. Download and unjar the Jersey distribution (I used 0.8) from https://jersey.dev.java.net/ to a location on your system (lets call it JERSEY_HOME for this article).
  2. Download and install the Java-WS distribution (I used 2.1) from https://jax-ws.dev.java.net/. (lets call it JAXWS_HOME for this article).
  3. Rama Pulavarthi wrote a blog (in fact the key for me) in configuring tomcat to reference the Jersey distribution jars. To summarize, in TOMCAT_HOME/conf/catalina.properties modify the shared.loader property entry to point to your JAXWS_HOME/lib/*.jar. Mine looks like this:

    shared.loader=C:/dev/jaxws-ri/lib/*.jar
  4. Create a new Dynamic Web Project in Eclipse using Tomcat 6.0 as the application server.
  5. Create a new library in which you add the following JARs:
    1. JERSEY_HOME/lib/asm-3.1.jar
    2. JERSEY_HOME/lib/jersey.jar
    3. JERSEY_HOME/lib/jsr311-api.jar
  6. Next modify the web.xml to include the adaptor for jersey. Most of the Blogs refer to a different class than what appears in 0.8 I am not certain which is the right class, only the following works for me (and the documented one is not available throught the downloads!):

    <servlet> <servlet-name>ServletAdaptor</servlet-name> <servlet-class>com.sun.jersey.impl.container.servlet.ServletAdaptor</servlet-class> <load-on-startup>1</load-on-startup> </servlet> <servlet-mapping> <servlet-name>ServletAdaptor</servlet-name> <url-pattern>/resources/*</url-pattern> </servlet-mapping> <session-config> <session-timeout>30</session-timeout> </session-config>
  7. If you start Tomcat, you will see the following error:

    com.sun.jersey.api.container.ContainerException: The ResourceConfig instance does not contain any root resource classes.

    This error is due to not providing any root RESTful resources.

  8. Create a new class and include the following:

    import javax.ws.rs.GET; import javax.ws.rs.ProduceMime; import javax.ws.rs.Path; @Path("/test") public class TestResource { @GET @ProduceMime("text/html") public String getMessage( ) { return "hello"; } }
  9. Start and test the application with the following url:

    http://localhost:8080/appName/resources/test

    and you should see a hello in your web-browser.

Making your servlet application URLs more Restful

If you are a developer who likes to work with Java you may have come to know REST and Restful Web Services. The real advantage of REST is its ability to get rid of some of the "webspeak" and make your URLs a little more platform independent. As a client developer it is much nicer to make a request to http://somehost/Application/stamp/5533 to retrieve stamp 5533 than the traditional http://somehost/Application/servlet/StampServlet?id=5533. Theoretically you could rewrite your application services layer without modifying the client. If you are not familiar with REST and Restful Web Services, a good dissertation on this can be found here by Roger L. Costello.

So all of this is nice, but how can we apply this to a servlet-based Web Application? There are projects out there such as the Java Restlet API, and while I think this is good, it does mean essentially having a non-servlet compatible solution (since the Restlet takes the place of a servlet). Instead, you can take advantage of some of the REST themes by using servlet-based Web Applications following these steps:

  • We want to write a Rest-like servlet for accessing Stamps. We have a servlet of the class StampServlet which is mapped in the web.xml of our servlet container as stamp. This would look like the following:
    <servlet> <description>Servlet for processing Stamp Restful requests</description> <servlet-name>stamp</servlet-name> <servlet-class>org.javad.stamp.servlet.StampServlet</servlet-class> </servlet> <servlet-mapping> <servlet-name>stamp</servlet-name> <url-pattern>/stamp/*</url-pattern> </servlet-mapping>

    The key here is the /* in the url-pattern of the servlet-mapping. This will mean any URI that is received that starts with "stamp" will send it to the StampServlet, but we can use any of the information on the URI chain to help direct the servlet to process the request in a Restful way. For example, if we are retrieving the details of a stamp our URL would look like:
    http://hostname/StampApp/stamp/563

    Using the GET method, where the ID of the stamp in this case is 563. If this was a modify function, the URL would look similar only a method of PUT would be used.
  • REST uses many of the less-popular methods of HTTP. In particular the PUT and DELETE methods. Fortunately, the HttpServlet class does implement these with the doPut(HttpServletRequest,HttpServletResponse) and doDelete(HttpServletRequest,HttpServletResponse) methods. If you are not comfortable using these methods (for example the HttpServlet Javadoc does mention PUT for use in placing files on the server like FTP), we can make our URLs be a little less Restful, but still Restlike by using POST method and inserting a keyword after the ID of the object in question. I should note, that for some applications like GWT, using protocols other than POST or GET is difficult. In GWT we can attach a header value (X-HTTP-Method-Override) to indicate we'd like to use DELETE, but it is still sent to the POST method receiver initially. In this case, using a POST method URL along with the method override header we can still have a Restful URL that would look like the following:
    <form action="http://hostname/StampApp/stamp/563" method="POST"> </form>

    More practical would be the use in an AJAX application using the XMLHttpRequest in Javascript (since we can override the X-HTTP-Method-Override header:
    var req = new XMLHttpRequest(); req.open( "POST", "http://hostname/StampApp/stamp/563" ); req.setRequestHeader("X-HTTP-Method-Override","DELETE"); req.send();
  • URL Construction is great, but how do I make use of this in my servlet doPOST( )? Since we mapped the servlet with a /*, anything to the right of the stamp in the URI is treated as part of the PathInfo. Therefore within your doPOST() method, you can request the path-info and take the appropriate action. Similar to the following:
    protected void doPost(HttpServletRequest request, HttpServletResponse response) { String pathInfo = request.getPathInfo(); Method method = request.getHeader("X-HTTP-Method-Override"); if( pathInfo.isEmpty() ) { createNewStamp( ); // purely a POST } else if( method != null && pathInfo != null) { long id = Integer.parseInt( pathInfo.split("/")[0]); if ("DELETE".equalsIgnoreCase( method ) ) { doDelete(request,response); // or call deleteStamp(id); } else if ( "PUT".equalsIgnoreCase( method )) { doPut( request, response ); // or call modifyStamp(id); } else { // if there are further elements in the pathInfo call the appropriate code... } } else { log("POST method called for stamp details without Method Override header. Use GET method to retrieve stamp details or specify a Method Override header."); } }

While this is not perfect, it is certainly easier to program a client to a URI like stamp/563 than the traditional way of stamp?id=563&action=DELETE. In the above example, it is likely that the code for servicing an action like Delete is in a specified method, so instead of trying to call the doDelete( ) simply a call to deleteStamp(id) would probably be more appropriate. Using this technique allows you to support both methods of processing your objects in a consistent way, while being flexible in support for the client technology. While it does muck up your doPost() methods a little bit this is a minor tradeoff for more readable URIs.

I should also mention that additional values in the pathInfo are used in Rest to indicate additional actions to take place against that object. For example:

http://hostname/StampApp/stamp/563/catalogues

With a method of GET would refer to a request to retrieve the catalogues for the stamp with the ID 563. In this situation having a controller (such as that provided by the Restful application) would definitely help in forwarding these to the correct service. Depending on the complexity of your application, you may be able to simply do this internally within your servlet such as in the else case mentioned above, however for more than a few actions this can be complex. Especially if catalogues (using the example above) can exist outside of the context of a stamp. This would mean you'd have to provide a servlet or some application that can process catalogues and find a way to tie the stamp servlet with the catalogue servlet. The worst case I can think of in my application would be trying to get all of the stamps that are in a country, filtered by an album in a stamp collection. This might look like:
http://hostname/StampApp/collection/56/album/25/country/76/stamps

As you can see, this is not quite as readable. Since I can also get stamps by album or simply by collection I might be more prone to simple request the stamps for collection 56 and then take on the album/country ids as queryString data:

http://hostname/StampApp/collection/56/stamps?album=25&country=76

Not pure Rest, but Rest-like. If I truly did want to retain the Restful URI, I would likely write a controller which returned stamps, and if the pathInfo of the GET request for the collection servlet contained stamps in it I would directly call it passing the pathInfo and then allow the controller to decide how to filter the URI (in my case I have a StampFilter object which accepts the three objects and calls the appropriate JPA Query based on the filter setup so this would be quite easy for me to do).