Every Byte Counts

Well, it has been a while since I've posted anything. I've been really, really busy trying to figure out how to save every last byte that I can over the network for mobile applications. When all you have is an EDGE network, you really can't spare anything.

Here are some of my findings:

• Don't use SOAP. If you're going to use a standardized communication method, prefer REST to SOAP. Why? Because even if SOAP offers a lot of information in its message envelope that is very useful for implementing integrity, security, transactions and not to mention making it easy to orchestrate via an enterprise bus, it also happens to be much larger. Expressing a resource in REST will save you a lot of bytes.
• Another reason, is that REST isn't bound to XML like SOAP is. JSON is so much shorter than XML. Furthermore, most Java ME platforms have zero support for XML. To be fair, Java ME has zero support for JSON too. Support for both is readily available using 3rd party libraries however, again, the JSON library is much smaller than the XML one. Using JSON, not only do we get something that is more compact than XML, we don't lose any of the flexibility. You can express any XML construct in JSON, and just like XML, you can extend your JSON documents in the future with elements that can be ignored and parsed over by older clients.
• Gzip encoding is your friend. Mostly all HTTP servers support it, just add the "accept-encoding" header to your request. Needles to say, Java ME doesn't support gzip but once again 3rd party libraries are readily available
• Keep alive connections. HTTP 1.1 supports keep alive connections out of the box. What that means is that you can send one http request, read the response and instead of closing the socket connection, you can re-use it to send and receive additional requests and responses. This is also unsupported from Java ME, and no, to my knowledge, there are no 3rd party libraries available that support this. Fortunately, HTTP is a really simple protocol, you will need to forgo the use of the HttpConnection class and use a raw socket via the SocketConnection class. You just need to send well formed HTTP 1.1 requests to the server and you should be able to read responses just fine. This saves you the setup and tear down times associated with a full TCP socket connection every time you need to send a request. Please note however that you will need to support, at the very least, the "chunked" encoding since HTTP 1.1 compliant servers are free to use this encoding whenever they so choose.

However, most disappointingly, probably one of the best tool you can use to save bytes over the network is not available. I am talking about HTTP pipelining. HTTP pipelining as specified in version 1.1 of the protocol allows you to send many requests to the server on the same socket before you start reading any of the responses. Unfortunately, my server of choice, Glassfish, has no support for it. If Glassfish has no support, most certainly no Java EE container does either. The HTTP parser found in Glassfish is none other than Tomcat's Catalina engine. Since Catalina is used verbatim in almost every Java EE container out there, then that's it.

Most unfortunately, the Grizzly project team members have decided that HTTP pipelining is not a mandatory or important feature of HTTP 1.1 since the browsers don't enable it by default. Grizzly is the HTTP server used inside Glassfish and fronts the Catalina engine in Glassfish instead of Tomcat's own Coyote server. I disagree with this conclusion. I think this W3C document states that pipelining is indeed mandatory.

I would love to hear from someone with Jetty experience here if this feature is supported or not. I use many Java EE 5 features that would make the transition to Jetty impossible but the Apache Geronimo project does feature a certified Java EE 5 server built around Jetty.

Finally, if anyone would like to suggest more ways to save bytes in a portable manner, please feel free to comment.

Glassfish JDBC Realm Authentication

Judging by the numbers of posters on the Glassfish and NetBeans users list that are having problems with this, I thought I would explain how I got Glassfish to play well with JDBC for authentication.

1. The first gotcha is the database driver. Most developers get their copy of Glassfish bundled with NetBeans. Please note that even if the NetBeans IDE comes with drivers for Java DB, MySQL, PostgreSQL and the ODBC bridge, that is not the case for the bundled copy of Glassfish that ships with the IDE.

You will need to install the proper JDBC driver for your database in $GLASSFISH_HOME/domains/yourDomainNameHere/lib or in $GLASSFISH_HOME/lib if you want the drivers to be available from all domains. Additionally, you can also use the database drivers that come with NetBeans and just copy them over to Glassfish instead of hunting them down. The JDBC drivers installed with NetBeans 6 are found in "$NB_HOME/ide8/modules/ext".

2. If you do not have a database pool setup. You will need to define a database resource from Glassfish. You can do this from the admin console by going to "resources" -> "JDBC" -> "Connection Pool". Setup a pool using "javax.sql.DataSource". Complete the wizard and make sure you can "ping" the database.

3. Setup a new JDBC resource for the pool you selected in step 2. JDBC resources can be defined in "resources" -> "JDBC" -> "JDBC Resources". Select whatever JNDI name you want for the resource. Assign the JDBC pool you selected in step 2 above to the new resource.

4. You need to define a new realm for authentication. You can do this in "Configuration" -> "Security" -> "Realms". Assign a name for this realm, let's say "dbtest" for this example and select "com.sun.enterprise.security.auth.realm.jdbc.JDBCRealm" as the class name.

We now need to setup all the properties here for the new realm:

• JNDI. Set the name you selected in step 3 above.
• User Table. The table name in your database that has users
• User Name Column. The column that represents the user name in the user table
• User Name Column. The column that represents the column in the user table
• Group Table. The group table.
• Group Name Column. The column that represents the column in the group table.
• Digest Algorithm. Glassfish will transform the clear text password using this algorithm before comparing it to the value in the database.
  

Gotcha #2 here is that database user/password fields are completely useless in this field, just leave them blank.

Gotcha #3 is that the group name column absolutely needs to be a string. So, for example, if you use numeric foreign keys in your database, you will need to create a view that maps user names to user groups by string. You then use this database view for the group table/group name column fields.

Gotcha #4 is that digest algorithm field is actually required even if this doesn't look to be the case from the web page. Furthermore, the "none" algorithm found in Tomcat for example doesn't exist here, you need to digest your passwords, you can specify any Java supported algorithm here, i.e., MD5 or SHA-256.

Finally, to test it out, just use the NetBeans tutorial here but change the realm name in the web.xml file of the application to the one you specified in step #4 above.