Codepimps

Dennis Ritchie R.I.P.

2011-10-13T20:24:00.003-04:00

Some perspective by Rob Pike on Ritchie's passing.

Managing Jar Hell

2011-07-03T18:48:00.004-04:00

When I moved from Netbeans to Eclipse, I couldn't understand why the Eclipse library support was so bad. By bad, I mean it doesn't support relative paths in its user libraries feature. You can see the bug report here. It was reported in July of 2004. Why is it that?

Well, it turns out, it's because the people at Eclipse know that you can handle this just fine now, probably not in 2004 though, using a dependency manager and it will do a much better job of it. Ultimately, the problem with the Netbeans approach, and to be fair, we'll see that Netbeans does support the alternatives suggested here, is that it requires you to check in library modules, the jar files, into source control or some kind of shared resources.

Enter Ivy... and moving on to Maven. Ultimately, these two systems work with Maven central do automatically grab all your jars for you. Not only, but they will also grab javadoc and source jars for you at the same time.

If you're struggling with this, give Ivy and Maven a look. I started using Ivy a little while ago and it worked like advertised but hit some limitations with the AppEngine SDK and native libraries (OpenGL and OpenCL bindings).

Ultimately, I moved our builds to Maven. Not only does Maven manage jar dependencies, including native libraries and big and complex SDKs like AppEngine and GWT, it also allows you to generate your IDE projects.

Having the maven project file be the authoritative source means we can generate projects for Eclipse or IntelliJ and Netbeans can read maven pom files directly. On many tasks, we can move away from Eclipse but still get access to all its plugins when we need it.

There are other up and coming systems like Buildr which uses Ruby instead of XML and Gradle that uses Groovy but for now, my build and dependencies are driven by Maven. Upgrading jars to never versions is changing one digit in an XML file. New team members just need to have a maven aware IDE and everything just starts. No setup time, no nothing.

One More

2011-05-11T23:52:00.000-04:00

Also top pick is the new svg renderer library for Android. This is the same library that Androinify uses.

Google I/O 2011 Top Picks

2011-05-11T22:39:00.000-04:00

Yes, the Android keynote was very interesting and all. Here are my picks for this year's "best of show":

ForPlay. Liked the Angry Birds in Chrome demo? I did too but for technical reasons. I immediately wanted to know what they used to port the game. They obviously have a Java version of the game and the physics are powered by Box2d which has a Java port. Angry Birds for the web is built around GWT and this new library. I'm a big fan of GWT and their new release schedule since version 2.1 is really making it easier to use the library for real work. It will be interesting to see what kind of uptake this has with casual game developers.
Android tools. Hopefully, soon, we will finally get relief here. The current emulator isn't good and really hampers progress. The current visual designers are also horrible.

Some Hope for the Android Emulator

2011-03-28T09:27:00.001-04:00

Google has publicly stated that they're working on making the Android emulator faster and that they feel our pain.

I'm all for statements to such effects and all, but nothing says we're doing it like some cold hard commits:

https://review.source.android.com/#dashboard,1005670

This adds hardware (GPU) acceleration to the emulator.

NoSQL != No SQL

2011-03-11T21:34:00.005-05:00

Time and time again, I'm reminded that the NoSQL abbreviation is probably the worst coined term ever.

NoSQL doesn't mean "no SQL", as in there is no SQL here. It means "Not Only SQL". Granted, the two biggest heavyweights in the NoSQL space, Apache's HBase and Cassandra, don't support SQL. However, it doesn't mean you can't use a domain specific query language on them.

Cassandra has CQL which means exactly what you think, i.e., Cassandra Query Language. This is very close in syntax to SQL but adapted to work with Cassandra.

Now, I encourage users of both systems to discover the power of Pig. Pig works works with both HBase and Cassandra. Personally, I find Pig much easier to use than SQL. The key to its simplicity is that it has a concept that represents the current row. If you've ever had to deal with non trivial SQL before, you know all too well the evils of sub-selects that could be easily avoided if you just had a way to refer to the current item when you needed to.

Android Emulator Slow?

2011-01-26T13:05:00.002-05:00

I have an 8 core Core i7 processor and the emulator was really slow for me. I'm not talking it's slow to start and then it runs fine. I'm saying it takes a long time to start and it's always slow.

The trick it seems, at least for me, is to disable the loading animation, in Eclipse, you can so do in: Debug Configurations > Your Debug Configuration Name > "Target" tab

Dock-able Phones

2011-01-06T09:13:00.001-05:00

They're coming:

http://www.engadget.com/2011/01/06/motorola-atrix-4g-hd-multimedia-dock-and-laptop-dock-hands-on/

and a video:

Programming Windows Phone for Free

2010-11-04T17:21:00.004-04:00

I usually don't bother with books anymore to learn a new platform, don't need it, they just slow me down. However, I would be lying if I didn't have a special place in my heart for the great Charles Petzold.

Back in 1992, I very much needed books to learn a new language or concept and "Programming Windows 3.1" was my introduction to the world of UI programming and event loops.

Not only was it clear and concise, a "problem" that has since been "fixed" with later editions of the book, it was actually quite entertaining. No small feat considering it took over a hundred lines of C code back in the day just get a window to show up.

Now the Great is back with a book on programming for Windows Phone. I haven't read it but Microsoft is comping the book in support of its new platform and is available to all for free. Microsoft is hoping it will introduce a new generation of developers to Windows Phone like the original did for Windows, I can only hope it will introduce a new generation of developers to Petzold.

You can download the book from here.

The Challenge of Using Multiple Languages

2010-08-21T18:58:00.002-04:00

After years of using Netbeans, I finally gave up. We have started using Eclipse. Why? In one word: plugins.

To be more exhaustive, I have not been very comfortable with the acquisition of Sun by Oracle, and that was even before they announced that awful lawsuit against Google. I would have been more comfortable with the acquisition if Oracle hadn't just closed on buying BEA one year before Sun.

Oracle's Java EE application server offering, OC4J, was, and is, simply awful. To their credit, Oracle didn't ignore the problem, so they went out and bought BEA for $1 billion dollars to acquire WebLogic. They also made it clear, that although they will support both application servers for now for their enterprise customers, WebLogic is their "strategic" server. Meaning that OC4J is on the endangered species list.

If it weren't for the BEA acquisition, Glassfish would have taken center stage in Oracle's portfolio. However, even if both servers share common components, e.g., both use the Metro web services' stack which is developed as part of the Glassfish group, Oracle didn't buy Sun for Glassfish, they bought it for Java. Makes sense considering they spent $1 billion to buy WebLogic.

Then we had the Netbeans vs Oracle's JDeveloper scenario. Unfortunately, if that was the only variable, Netbeans' victory is absolute. JDeveloper is not very good, and I'm being polite here. However, JDeveloper is not just an IDE, it's an IDE designed to get you to use as many Oracle middleware products as possible. The other variable is that Oracle, like everybody else, supports Eclipse. Meaning that Oracle is now spending money to support 3 IDEs, I suspect that will not last indefinitely.

All this to say, even if Glassfish is still the best and Oracle is continuing to invest in both Netbeans and Glassfish. We still decided to move ahead and proceed with our migration from these products. We also decided to take it by steps and steps and slowly. We have now successfully migrated to Eclipse but continue to use Glassfish.

So why Eclipse? Well, our primary toolkit is Google's web toolkit(GWT). We use GWT everywhere and profitably too. However, Google's IDE of choice is Eclipse and the plugin for GWT, and AppEngine too, made by and supported by Google is for Eclipse. So it was hard to always be behind on Netbeans when it came to GWT. Not only, but we are targeting AppEngine to replace Glassfish. If that wasn't enough, we also have a very real need to develop for Android and we love Scala. Again, all of these have world class plugins for Eclipse but not for Netbeans. So we just grew tired of being second class citizens all the time.

So how about Eclipse? I'll tell you one thing, I was nervous having had really awful experiences with it before. It has a very steep learning curve. However, once you wrangle it, it definitely pays off and with dividends too. It's definitely a match for Netbeans and more. I can't think of an IDE that has better refactoring and source navigation features. Heck, there's even a Silverlight plugin for Eclipse.

So now we have one IDE for pretty much all platforms and languages: Java SE, Java EE, Scala, Android, AppEngine and C++... or do we? Well it turns out it's not that simple.

Recently, on a project, I had to code in 4 different languages at the same time: C++, Objective C, Java and C#. Java on the server. C++ for some glue code. C# for a Microsoft Office extension and Objective C for an iPhone application... and I ended up using 4 development environments.

I had to use Visual Studio for the Office plugin. I used Eclipse for the Java server stuff. I used Netbeans for the C++ part and finally, I used Xcode for Objective C.

It turns out coding in multiple languages, even on the same day, isn't that hard, however, using multiple IDEs in the same day is a different story. You see, I always keep the default key bindings of an IDE so that I learn them as I go. It also means that if you're a veteran of a particular IDE, Netbeans in my case, it's very, very hard to switch and not continuously mess up which accelerator you're suppose to use.

So there you have it, the real challenge of using multiple languages: the IDE key bindings. So if you're going to be in this situation and you aren't dumb like me, I suggest you import key bindings in all the IDE's that you can.

The Significance of a USB host

2010-06-11T09:36:00.003-04:00

For the last 2 years, I've been saying to anyone willing to listen that the future of computing is a incredibly powerful phone that you cradle when you get home or at work, etc. This cradle has a mouse, a keyboard, a large screen and a good sound system.

Android enthusiasts seem to have taken another step to making this vision a reality. Notice in the video that he launches X-Window with a desktop environment from his phone that isn't the Android shell.

That being said, the Nexus One, as powerful as it may be, still isn't the all powerful phone I've been describing for the last 2 years, that phone still doesn't exists.

HTML5 Update

2010-06-03T22:49:00.002-04:00

Yes, we know about some of the good stuff in html5 already but a lot of new specs have been added to the html5 family in the last year that will try to make the following available to web applications, i.e.,

TTS
Speech Recognition
Recording
Microphone Access
Camera Access
WebGL
Positional Audio, still very early, don't know if will be based on OpenAL, OpenSL ES or something else
File & Device API
and more...

Android can be Dumb Too

2010-06-03T14:43:00.002-04:00

Here's something you would expect if you were coding on the BlackBerry platform but this is most definitely on Android. Android doesn't implement roaming itself.

What? Yes, it's true. If you enable roaming in global system settings, it doesn't actually stick unless each and every single application you use honors the flag. Furthermore, if you don't implement it, Google states that it's the application developer's problem if your users complain about very large data bills.

So to implement it in every application, you need to check ConnectivityManager to see if you're allowed to use the network, e.g.,

final ConnectivityManager cm = (ConnectivityManager) getSystemService(Context.CONNECTIVITY_SERVICE);

// TODO check the connection to see if it's allowed

You also need to check for the CONNECTIVITY_ACTION intent in case this changes later on. Honestly, this should just be blocked by the system. They're afraid someone won't be able to get to emergency services. It should just support an emergency level flag for network connections instead of getting all developers to implement this in their apps.

iPhone dev on Windows

2010-06-03T13:40:00.000-04:00

As of VirtualBox 3.2.2 it is now possible to do iPhone dev on your Windows box. VirtualBox 3.2.0 introduced support for OS X as a guest OS, however there was a bug that broke iPhone/iPod Touch usb support on Windows hosts, this has been fixed in the 3.2.2 release.

So without further ado, here are the (amazingly simple) instruction for installing OS X 10.6.3 as a guest OS in VirtualBox on Windows.

1) Create a new virtual machine, the guest OS is Mac OS and for the version I used Mac OSX Server, not server 64-bit (I haven't tried 64-bit yet), disable EFI.
2) Download EmpireEfi and choose the BootCD.iso as your CD image.
3) Edit the VM's xml file and add the following to the section:

<ExtraDataItem name="VBoxInternal2/SmcDeviceKey" value="ourhardworkbythesewordsguardedpleasedontsteal(c)AppleComputerInc"/>

4) Boot from the CD and follow the EmpireEfi instructions. You can install OS X using a retail DVD.
5) Unlike VirtualBox 3.2.0, the built-in EFI bios will not boot OS X, so keep using the EmpireEfi boot cd.

The Beauty of Google Chrome

2010-04-07T11:48:00.005-04:00

Google Chrome is really an amazing application. A lot has been said about its performance and how Opera 10.5 is slightly faster in terms of rendering and JavaScript performance. However, Opera doesn't implement any of the sandboxing techniques that Chrome does, if it did, it would suffer a significant performance penalty. This is a look at Google Chrome on Windows.

What exactly does make Chrome so special? It's the fact that it uses recommended practices for Windows security. It's funny reading some of the archived documentation on MSDN. The documentation uses examples on how a future release of Internet Explorer or Outlook , carefully disclaiming that current releases do not leverage these security features, could use such feature to provide additional protection to users.

Needless to say, Internet Explorer (IE) and Outlook still do not implement these recommendations but Chrome does. It implements and leverages the following technologies from Windows:

1. Multi-process architecture

Yes, IE 8 has this also but their implementation is very different from the one found in Chrome. In both browsers, each tab represents a distinct process. On Windows Vista and higher, both mark the process as a low integrity level, more on this later. This is pretty much where the similarities' end.

In IE, the browser leverages the fact that each thread in Windows can have a message pump and have user object likes windows, buttons, etc. That means that each child process is managing the user interface of the tab directly. It also means that this child process, its threads and its resources live in the same security context that you do. The only difference however is that most risks are mitigated by marking the process as a low integrity one. This means however that squatter and shatter attacks can still be performed on other windows, just as long as those processes are also low integrity ones. All this to say that child processes render content to your screen.

In Chrome, child processes are also renderers but they do not actually manage anything on screen. They're renderers and these processes do not have a message pump that is coming from Windows' user interface layer. Instead, their message pump is a custom one that is listening on a named pipe in Windows (a socketpair() on OSX and Linux). They listen for requests, render data and send back the information to the broker process which is actually the chrome of Chrome.

In other words, they render everything to an off screen surface that never interacts with your screen. Even if it did, we'll see later that it wouldn't matter anyway.

That means when you click on something, a message from the broker process, Chrome's chrome, is sent to one of the renderers, once it's done processing the message, the renderer communicates back the result to the broker for on screen rendering. We can see how such communication would be expensive and how it would slow down Chrome. Making Chrome's performance, and Windows' implementation of named pipes, even more impressive to witness.

It's also important to note that the lifetime of a renderer process in Chrome can also be very short. If you navigate to something like "bank.com" to "robbers.com" using the same tab, Chrome has killed the renderer that was handling "bank.com" and spawned a new one to handle "robbers.com" even if you did not switch tabs. Not only does this make it impossible to leak any information from the first renderer to the new one, it's also a very effective way of doing garbage collection. All resources, including leaks, that were used by the now defunct renderer are reclaimed automatically by the kernel.

2. Restricted Tokens

Windows NT has token based security. Most objects, i.e., files, sockets, pipes, etc., in Windows are securable, if not in the beginning, the capabilities have been iteratively added in subsequent versions.

Chrome sets up a token for its child processes that translate to "deny access to everything token". This means child processes cannot acquire any resources that were not implicitly inherited from the broker. Even the named pipe that is used for communication with the broker cannot be opened directly by the child, it must be inherited.

3. Use of Windows job objects

Windows 2000 introduced support for jobs. Jobs in Windows have 2 distinct responsibilities:

They include the ability to act on a group of processes atomically
They allow additional security restrictions to be placed on this group of processes

The second one is of interest here. I'm not saying Chrome doesn't use the first, I'm saying it's of no importance to our stroll on its implementation of security.

Windows doesn't allow security to be set on various API it inherited from OS/2 and 16-bit Windows. This includes manipulating any User or GDI objects , this is what allows for things like shatter attacks, capturing keystrokes and all kinds of bad things. It's also what makes the desktop experience possible, so windows, copy & paste, etc.

It does so because introducing new interfaces that would allow for fine grained security and deprecating the old ones would pretty much deprecate every Windows application out there, including Microsoft's. It also wouldn't make for the same end user experience. A lot of the mechanics that users' take for granted today on Windows, OSX, etc., just wouldn't be possible or very easy to implement.

Microsoft, in the earlier days of Windows, also decided not to do this for another reason: performance. We can imagine that requiring to check the security permission of every single window object when broadcasting a message would be very expensive for computer's in service 10-15 years ago when a simple logon session routinely has hundreds of windows. Don't forget, there are a lot of hidden windows on your desktop.

However, instead of adding new functions with additional parameters for security, you can set the security and continue to use the same function. This is what Microsoft did with the job object. Specifying the job object allows to set other limits on these child processes that normally would have free reign, allowing you to forbid the following:

Access to the clipboard
Creation of new processes
Access to other windows
Access to the window station object's global atom's table
Access to virtual desktop objects, ability to switch between them
Logoff, login
Creating new User objects
Change system parameters
Prevent hook functions, like trapping every key message, from being set

So once the job restrictions are set, normal function calls that would otherwise go unchallenged by the operating system are now restricted.

4. Use of Windows virtual desktop objects

The initial release of Windows NT, 3.1, had support for virtual desktops. However, this is circa 1989 for release in 1991, virtual desktops on Windows NT were not implemented to allow usage that is more in line with our thinking of virtual desktops today. For example, there is no way to switch a window from one desktop to another.

Instead, virtual desktops in NT were designed to be a security mechanism. They are the boundary of communication for User objects. This means if you install a global keyboard hook, you can also do so on your own desktop. If you broadcast a message to all windows, you're sending a message to every window on your desktop.

They were introduced so that login would be secure, that is, when you log on to Windows NT, there are 3 desktops created. The default one, the logon one and the screen saver one.

The logon one is only active when you lock your workstation and you need to unlock it. This allows for secure login since processes in the default desktop cannot create a keyboard hook into the logon desktop.

The screen saver is used to run the screen saver in its own sandbox. This means that a screensaver that is potentially malware would not be able to access your default desktop.

Of course, you still have the ability to create new windows on any desktop unless the proper steps are taken to secure them and this is indeed what Chrome does. Since its child processes don't have a user interface you can actually see, the child processes are nonetheless in a new desktop.

All these child processes also run in a job object that forbids them from accessing any of the virtual desktop functions. This means these processes can still interact with other low integrity processes but the only other processes these renderers can see are the one's found on Chrome's dedicated desktop for renderers. This is a step further than just marking the children as low integrity. The only possible squatter or shatter attacks that would be possible would be on other processes found in this desktop, problem is however for the would be attacker, is that these renderers don't actually have a user interface either.

5. Integrity level

The integrity level mechanism introduced in Vista allows for processes to be imbued with a level of trust. If a process wants to interact with a process of higher integrity, a UAC prompt is required to escalate the process. Also, low integrity processes do not have the ability to read any system objects or modify them

Chrome and Other Browser's Today

None of these mechanism are being leveraged by Firefox, Opera or Safari. Internet Explorer uses jobs and multi-processes but the sandbox is limited in scope compared to the one found in Chrome. Just food for thought on what browser you should be running to navigate the hostile web.

A final note, Microsoft has announced that it does not plan to support Internet Explorer 9 on anything less than Windows Vista SP2 stating only that "a modern browser requires a modern operating system". We're quick to point out that Chrome does indeed run on Windows XP. However, it is interesting to understand Windows XP's impact on Chrome.

It impacts Chrome in a few fundamental ways, the first one, is that, integrity levels were introduced in Vista, so it's not possible to mark child processes as low in Windows XP. This means these child processes have more privileges. However, due to other efforts realized to execute the renderer's under a proper sandbox, the scope is somewhat limited. The problem lies more in the ability to change system objects like registry keys. Registry key access is also governed by tokens but most users run as administrators under Windows XP.

More importantly however are the following:

Windows XP does not require NTFS. It obviously supports it but installation on FAT32 is also supported. This means that the restricted token assigned to the child processes under such a file system will not forbid a compromised renderer from opening new files. Funnily enough, an attacker would need to deduce the path to any file he wishes to open however since listing of files would be prevented by the restricted token. Unfortunately, it's more than easy enough to deduce the path to important Windows' system files.
Sockets are not securable objects. That's right, before the introduction of Vista, sockets are not governed by any token or any kind of security at all. That means a compromised renderer could create any number of new network connections. This could be used to download malicious code and then inject them in the files that were opened in the step above.

Ultimately, even on Windows XP, the sandbox setup by Chrome is nonetheless very impressive. Users should still be wary of that particular version of Windows compared to what is offered by Vista and above.

Chrome and Virtual Desktop Manager

2010-03-28T21:05:00.000-04:00

Now that I have my new laptop, I decided it was time to find a better virtual desktop manager. I had been using the creatively named Virtual Desktop Manager with mostly good experiences, but a few quirks. One of the more annoying ones is artifacts that occur sometimes, but are persistent, when switching from a desktop with Chrome to one without. After a few days of uptime, I would find ghostly outlines of Chrome windows on all my desktops, and sometimes even orphaned status messages of the variety you get in the bottom left of the browser window when you mouse over a link or are loading a page. Pretty annoying! The second quirk is a blank and non-responsive "Write (no subject)" window courtesy of Thunderbird when you compose an e-mail and switch desktops before sending it.

Anyway, all that has been resolved with the use of Dexpot, I heartily recommend it.

Two Finger Click and Scroll

2010-03-28T12:33:00.001-04:00

One of the (many) things that Apple get right on their Macbooks is the touch pad, it is large and supports nice multi-touch gestures.

My new laptop, an Asus G73, features the nice large touch pad and supports multi-touch gestures, like pinch-zoom, but it misses the best two gestures - two-finger scroll and two-finger tap to right-click. Fortunately, there is a remedy to this deficiency: Two-Finger-Scroll. So now I get the Macbook quality touch pad experience with twice the cores, way better graphics and more than enough money left over for a fast ssd!

Java3D offscreen rendering and changing colours

2010-03-15T15:44:00.000-04:00

I just ran into an irritating quirk (perhaps bug?) in Java3D. I'm using the offscreen rendering feature of Canvas3D to render a simple 3D scene to a PNG file. The requirement is simply to produce a PNG given certain input parameters, notably colour.

My first attempt was very straight-forward; create my scene, with the correct colours and render it to the PNG. Repeat as required. The last part, repeat as required, is problematic. Each time you create an offscreen Canvas3D you create an OpenGL PBuffer. My graphics card as 32 pbuffers so the 33rd time you render the scene, the program crashes.

A second attempt is obviously required. The requirement being to use only one Canvas3D object and thus only one scene graph and to change the colour of the scene before re-rendering it. First, some background, my objects are Boxes and to set their colour you have to set an appropriately configured Appearance. For example:

final Material material = new Material();
final Appearance appearance = new Appearance();
final Box cube = new Box(1, 1, 1, appearance);

material.setDiffuseColor(someColor);
appearance.setMaterial(material);

This simply creates a cube using the Box utility class and sets a color on it. Java3D will optimize the scene graph and part of that optimization assumes that the scene doesn't change, unless you explicitly state that it can. Changing the colour would be one of those things you have to state. There are plenty of forum topics about changing the colour of a Box and they usually suggest changing the Appearance object, like so:

final Box cube = new Box(1, 1, 1, Box.ENABLE_APPEARANCE_MODIFY, someAppearance);

[...] // Some time later
box.setAppearance(aDifferentAppearance);

The ENABLE_APPEARANCE_MODIFY flag simply tells Java3D to assume that the Appearance object associated with the Box can change. This should allow you to change the appearance at will. Except in offscreen rendering mode, this doesn't work. Not only does it not work, but any initial colour you set is ignored and you get a white Box. After much messing about, I found that ENABLE_APPEARANCE_MODIFY was the cause. The solution to this is to change the colour on the material directly and not simply set a new Apperance.

final Material material = new Material();
final Appearance appearance = new Appearance();
final Box cube = new Box(1, 1, 1, appearance);

material.setCapability(Material.ALLOW_COMPONENT_WRITE);
material.setDiffuseColor(someColor);
appearance.setMaterial(material);
[...] // Some time later
material.setDiffuseColor(aDifferentColor);

And there you have it, Boxes that change colour in an offscreen rendered scene.

Red Rings Again and for the Last Time

2010-03-03T13:59:00.003-05:00

Once again, my Xbox 360 has been hit with the 3 Red Rings of Death error. Once again, I called Microsoft only to be told that it was out of warranty this time around and it would cost me $150 to repair it. After explaining my history, this was my 4th console after all, they graciously offered to repair it for free. The catch is that I had to get it to a UPS store myself which isn't anywhere near me.

Well, I got the console back again this last Monday and got 4 Red Rings this time. Called support again, they were willing to repair it but when they mentioned I would have to get it back to the UPS store, I decided enough is enough. The customer service representative explained that my preferred courier was indeed supported, i.e., the one that is just a few blocks away and the same one I received the latest damaged console from, but that the Xbox support system was randomly generating labels to even out the various couriers Microsoft uses. I politely asked for a label from another courier and was told there was nothing they could do.

So this will be the last time I have issues with Red Rings. The games, accessories, etc., are heading to EB Games and the console is headed to the garbage bin. Fool me once Microsoft... but I've had enough.

The Sanity of Google Chrome Frame

2010-01-29T11:39:00.002-05:00

OK, so having had to support CSS in Internet Explorer 6, 7 and 8. This isn't so insane after all. Turns out, it's much less effort to fix IE with a free add-on, that doesn't even require administrator privileges, than it is to fix CSS to work in IE.

I now recommend using Google Chrome Frame over actually making stuff that works correctly in IE. This just shows you the sorry state the IE is in. Microsoft is working on fixing IE with 9 and beyond but it just won't help all those people who don't really know what a browser is. However, asking for a plugin, and one from Google no less, will reach all audiences.

So the people at Google weren't insane, they just felt it was less work to fix IE than to break sites to comply with IE's view of the web. What I didn't understand at the time however, is that you can't just show a page when your user reach your site telling them to install another browser because too many people don't know what a browser is really. Also, in a corporate environment, they probably can't install one either even if they do.

Moving on to Scala... Hopefully

2009-11-19T11:55:00.003-05:00

Ah, Scala, I became infatuated with this language some time ago when I saw the following video presentation from fosdem 2009.

This language is such an evolutionary step over what was there before that's it really makes you think what's coming up in the future. By that, I mean what else will come from this field which was once thought to be done, where original research was basically considered to be all over.

So, why hopefully? Well, the problem with Scala, much like any new language, is tooling. The good news however is that Scala produces Java Bytecode instead of starting from scratch.

Meaning that's it's still a pain to use your favorite IDE, Netbeans in my case, with this language. However, leveraging the existing Java platform means that it doesn't suffer from things that would normally suffer any nascent language:

Uses the JVM, so performance is what you would expect. It's fast and works well.
Tons and tons of libraries because it can access any Java library out there.
Continuous integration servers and other tools that you would want to complete your development life cycle are all readily available since it's Java based
Mature application servers. Code produced from the compiler is Java Bytecode, hence any Java EE server, like Glassfish, will just work.

Unfortunately, for now, until the Netbeans support matures, I won't be using this. However, I would recommend developers still have a look at that video. It's hard to look at this video without being in total awe of what's going on.

Kicks & Giggles with Windows 7

2009-10-22T16:06:00.002-04:00

256 core system with 1TB of RAM. Windows 7 supports up to 2TB of RAM. You'll need Windows Server 2008 R2 (Windows 7 Server) if you need more however.

Netbeans and GWTTestCase

2009-10-22T15:44:00.003-04:00

In the "yet another problem I couldn't find an answer on Google for" unofficial series.

The problem lies in the fact that the GWT compiler operates on Java source code and not Java bytecode. Unfortunately, a standard WAR project on Netbeans will exclude the Java sources from a WAR file.

The first step is to go edit your project and go to "Build" and in "Exclude From WAR File:" (their spelling here not mine) field remove the value there which by default is "**/*.java,**/*.form".

The second thing you need to do is go to the run section and specify heap options in the "VM options" field. This is needed because tests run outside the application server you're using.

Finally, just hit "Test Project" and see your GWTTestCase's go. However, I would be remiss if I didn't mention that the vast majority of your GWT test code should not use GWTTestCase. Simply put, if your views are properly segregated, the majority of your GWT client code will run as regular Java code, so just use regular JUnit test cases for those.

Capturing UIWebView Touches

2009-10-03T20:24:00.001-04:00

I’ve searched everywhere on how to do this and everywhere the same result. Either use a transparent overlay that you put over the web view to capture touches but you then lose the ability to click on HTML anchors, pinch and move around or subclass UIWebView and disable user interactivity.

First, a word of caution, the technique I describe here is not for the faint of heart nor is it for the inexperienced. If you mail me for help, most people do instead of leaving comments for some reason, I will silently discard your email with a smile on my face.

The first problem is that UIWebView internally uses many undocumented views. The iPhone has many very useful, undocumented views. Unfortunately, using them means an automatic rejection from Apple’s AppStore. The UIWebView is a composite of HScroller, UIWebDocument, UIImageView and other more minor views. Out of all of those, only UIImageView is documented.

Furthermore, the UIWebView documentation states that you should not subclass this view. Considering how completely useless doing that is, it’s actually sound advice since it will not hurt your AppStore submission approval process. UIWebView delegates all its functionality to internal components, overriding any method found in there doesn’t buy you anything, just lost time.

The first thing you need to know is that UIWebView is a very narrow view of the underlying WebKit engine. Apple probably did this to make its system secure but also keep people from doing too many things with it too. If you’re on Android, just smile and be happy you don’t have to do business with these people.

The second thing you need to know is that it’s OK to reference undocumented views as long as you use documented API’s to get them and you store such references in a documented class with the obvious choice being UIView.

OK, so let’s describe what we’re going to do before we show some code. The steps are the following:

Define a new protocol that extends the UIWebViewDelegate protocol, we’ll call this UIWebViewDelegateEx for now. Define a new method called “tappedView”.
Define a view, not a view controller, that is completely transparent. This view holds the reference to the UIWebView you want to capture information from. This view implements the UIWebViewDelegate protocol in full and holds a reference to your custom UIWebViewDelegateEx object.
We define a couple of flags in this view, basically didMove and didShouldStartLoadViewRequestGetCalled.
We define a timer in this class.
The overlay should be above web view. Completely transparent. It doesn’t have to be over all of it but at least the section you want to get events from.
We find the view that we need that was touched by the user by using hit testing. Again, only documented API’s are used to get this view.
We define a timer that we start when we detect a tap. If this timer fires before the UIWebView delegate method “shouldStartLoadWithRequest” is called, we consider this a tap, otherwise, we consider that you activated an HTML child object of some kind, most likely an anchor.

So, at the end we have something that looks like this:

#import <UIKit/UIKit.h>

@protocol UIWebViewDelegateEx<NSObject, UIWebViewDelegate>

/**
* Called when the view was touched by the user and wasn’t an anchor.
*/
- (void)tappedView;

@end

/**
* Intercept any touch events by displaying a transparent
* overlay on top of a web view.
*/
@interface WebOverlayView : UIView<UIWebViewDelegate> {
        /**
         * The view that we are monitoring, i.e., the view that we will
         * possibly steal events from
         */
        UIWebView *webViewComposite;
        NSObject<UIWebViewDelegateEx> *delegate;

@private
        BOOL didMove;
        BOOL didShouldStartLoadViewRequestGetCalled;
        NSTimer *timer;
}

@property(nonatomic, retain) IBOutlet UIWebView *webViewComposite;
@property(nonatomic, retain) IBOutlet NSObject<UIWebViewDelegate> *delegate;
@property(nonatomic, retain) NSTimer *timer;

@end

From here, we have to filter what we want, the strategy is basically is that if the user touches the view and the UIWebViewDelegate method “shouldStartLoadWithRequest” does not get called, we can assume that the user touched the view without triggering an HTML object like an anchor, this is where the timer comes in. If after the elapsed time, “shouldStartLoadWithRequest” has not been called, we call our custom “tappedView” message.

#import "WebOverlayView.h"

@implementation WebOverlayView

@synthesize webViewComposite;
@synthesize delegate;
@synthesize timer;

#pragma mark -
#pragma mark NSObject

- (void)dealloc {
    [webViewComposite release];
    [delegate release];
    [timer invalidate];
    [super dealloc];
}

#pragma mark -
#pragma mark WebOverlayView

- (UIView *)findViewToHitForTouches:(NSSet *)touches
        withEvent:(UIEvent *)event
        inView:(UIView *)v {
    UITouch *touch = [touches anyObject];
    CGPoint pt = [touch locationInView:v];

return [v hitTest:pt withEvent:event];
}

- (UIView *)findViewToHitForTouches:(NSSet *)touches withEvent:(UIEvent *)event {
return [self findViewToHitForTouches:touches withEvent:event inView:webViewComposite];
}

- (void)timerFired:(NSTimer *)t {
    timer = nil;
    if (didShouldStartLoadViewRequestGetCalled == NO)
        [delegate tappedContent];
}

#pragma mark -
#pragma mark UIView

- (void)touchesBegan:(NSSet *)touches withEvent:(UIEvent *)event {
[[self findViewToHitForTouches:touches withEvent:event] touchesBegan:touches withEvent:event];
didMove = NO;
}

- (void)touchesMoved:(NSSet *)touches withEvent:(UIEvent *)event {
[[self findViewToHitForTouches:touches withEvent:event] touchesMoved:touches withEvent:event];
didMove = YES;
}

- (void)touchesEnded:(NSSet *)touches withEvent:(UIEvent *)event {
    [[self findViewToHitForTouches:touches withEvent:event] touchesEnded:touches withEvent:event];
    if (didMove == NO) {
       [timer invalidate];
       didShouldStartLoadViewRequestGetCalled = NO;
       timer = [NSTimer scheduledTimerWithTimeInterval:0.75
                target:self
                selector:@selector(timerFired:)
                userInfo:nil repeats:NO];
    }
}

- (void)touchesCancelled:(NSSet *)touches withEvent:(UIEvent *)event {
[[self findViewToHitForTouches:touches withEvent:event] touchesCancelled:touches withEvent:event];
didMove = YES;
}

#pragma mark -
#pragma mark UIWebViewDelegate

- (void)webViewDidStartLoad:(UIWebView *)webView {
[delegate webViewDidStartLoad:webView];
}

- (BOOL)webView:(UIWebView *)webView shouldStartLoadWithRequest:(NSURLRequest *)request
        navigationType:(UIWebViewNavigationType)navigationType {
    didShouldStartLoadViewRequestGetCalled = YES;
    return [delegate webView:webView shouldStartLoadWithRequest:request navigationType:navigationType];
}

- (void)webViewDidFinishLoad:(UIWebView *)webView {
[delegate webViewDidFinishLoad:webView];
}

- (void)webView:(UIWebView *)webView didFailLoadWithError:(NSError *)error {
[delegate webView:webView didFailLoadWithError:error];
}

@end

If this code was useful to you, drop me a line in the comments (not e-mail). Obviously, you need to apply the technique to your own application but at least it shows it’s doable.

The Insanity of Google Chrome Frame

2009-09-23T13:05:00.002-04:00

So Google is fixing Internet Explorer for Microsoft. Great, fine. You know what really pisses me off however? Is that Google decided this was a priority when you have people like Ray Ryan at Google I/O 2009 telling people we don't have infinite resources to get GWT 2/UiBuilder out the door.

Or the same speech about various AppEngine features. Like, for example, how about a JPA implementation that is actually worth a damn in AppEngine? Like instead of giving us a big speech about de-normalizing data on AppEngine, how about you use the current JPA annotations to do it for us, you have enough information in those annotations to do it for us.

Or a browser on Android that is actually good, you know with multi-touch and pinching. And don't give me that crap about Apple patterns. Apple is at war with Google, Google is just too dumb to realize it yet. Also, people, don't call the Android browser Chrome Lite, it's not fast enough to be deserving of the name Chrome just because it's WebKit based. The architecture of these 2 browsers are very, very different.

Ending Series on C++0x Feature Focus

2009-08-24T12:18:00.001-04:00

Considering that the "x" in C++0x now appears to be an hexadecimal number, I'm no longer interested in continuing my series on this topic.

WebKit: Bridging the Great Mobile Divide

2009-06-28T11:45:00.003-04:00

I hate Objective C and I hate Apple's Xcode and I think I hate Apple's AppStore policies even more but the people do love their iPhones. Furthermore, if you've had to deal with development on many different smartphones on the same project, you know how painful this is. You code one version for the iPhone, then you port it to BlackBerry, then Windows Mobile, then Nokia N60, and if you're lucky, Android.

So, if you want to develop for iPhone you either have two choices: develop a web application or go native. Unfortunately, even if Mobile Safari is a fantastic browser, it doesn't yet implement all of the HTML 5 features. Assuming iPhoneOS 3 here, Safari supports the following:

The new video/audio tags. Albeit using only mp4 containers, h264 video and AAC audio. Furthermore, it does so by opening the video player application, in effect exiting the browser
It supports app cache. With the proper manifest, your application's assets will be stored locally. Opening the door for the application to run without connectivity
Support for HTML 5 databases. First is the assets, then it's the data. With both, your application can potentially run without connectivity.
Canvas. It supports canvas for more complex rendering.
Workers. Thread workers so that long running JavaScript code doesn't need to lock up the user interface

The only big in-draft HTML 5 (the file desktop API isn't in the draft yet) feature it doesn't support is geolocation. That being said, what if you need access to the accelerometers? Or the iTunes music library? Sorry, you're back to native and coding in Xcode and Objective-C.

Well, a funny thing happened to me the other day while watching this Google I/O session. I was expecting an introduction to the C/C++ kit and that's it. Turns out that was pretty boring. What wasn't boring however was the idea of using a web view and injecting custom JavaScript element in the document's DOM.

That concept introduces a new, more sane, alternative to coding in Objective C and Xcode and supporting many different platforms. Basically, you write an HTML 5 application using GWT. So you're writing with Java 5 syntax in the IDE (and debugger) of your choice for the majority of the application. You only write a small shell containing web views in your iPhone application.

So how about porting to other phones? Well consider the following phones that have WebKit in some variant or another:

BlackBerry. Ships with a WebKit based browser since 4.6. That means the Storm/Bold, won't cover most of your users but eventually it will. Please note however that JavaScript is disabled on BlackBerry phones by default.
Android. Ships with a WebKit based browser.
Nokia S60. WebKit is now the default browser engine in S60 phones. Write a shell using C++ and Qt.
Windows Mobile. WebKit is nowhere to be found. Again, not a problem since the Qt port to this platform has WebKit, write a shell using C++ and Qt.
Pre. Ships with WebKit. Not only, but Palm only wants web based, HTML 5, apps to be written for its phone using WebKit. They even introduce some new DOM elements to access other services in their implementation.
Finally, iPhone, write a shell in Objective C embedding WebKit.

The strategy is the same, use a native application so that you're not jailed by the browser but keep most of your code portable by embedding a web view using WebKit. Furthermore, thanks to GWT's deep JavaScript integration and GIN, you could write interfaces that get bound to the proper native/platform specific implementations automatically at runtime.

So now we actually have a platform for sharing a lot of code between the major smartphones out there. And what if you need those iPhone like visual effects? Well, it's absolutely possible with JavaScript. Consider gwt-fx, it has all the standard iPhone visual effects and more and when you're stepping out of what the standard iPhone controls do out of the box, is actually easier and less verbose to use in terms of code. For your enjoyment, gwt-fx has an effect playground available here.

Ah, NSObject reference counts, how will I miss thou? Well, not at all I must say.

Windows 7 Exploit Source Released

2009-06-22T19:14:00.004-04:00

In an effort to get Microsoft to change their mind on UAC's default behavior in Windows 7, the source code for the UAC injection technique has been released. Considering this system has been covered here many times, I won't rehash anything here but you can find the source and information about the tool here.

Microsoft Research For the Win?

2009-06-06T08:40:00.002-04:00

Experience in reviewing Microsoft counts. Paul Thurrott is one of the most credible Microsoft technology reporter in the business.

In his latest podcast with Leo Laporte, episode #110, he rants very humorously at Project Natal. All I can say is ouch and thanks for putting this technology in perspective.

Microsoft Research For the Win

2009-06-02T08:18:00.001-04:00

Just a light post to say how impressed I am with "Project Natal".

EnGadget has the details here.

Move Construtors

2009-05-30T21:36:00.001-04:00

For simplicity sake, we assume the name space 'std' is used here. Given the following:

vector<vector<string> > v;

We now have an expandable array of string arrays. Now I don't want to poke inside the possible different implementations the string class could have here. We assume the worst, that string is implemented using a single C array of characters and leave it at that.

However, also for the sake of this discussion, we assume that the vector template class capacity is always at least bigger than its size. So we can assert the following:

assert(v.size() <= v.capacity());

Where the capacity holds numerous empty slots at the end of the array for future allocations. So, if we have the following:

vector<string> v1, v2;

v.reserve(32);
v.push_back(v1);
v.push_back(v2);

We are can safely state that the insertion of v2 inside of v did not cause v1 to be re-created, copied and destroyed.

The problem comes from the following:

v.push_back(v1);
// v.push_back(v2..v(n - 1) where n == capacity == size
assert(v.capacity() == v.size());

The next time we insert into this vector, the capacity will be expanded, most implementations will double the size to avoid any extra copying that might be caused by being over-conservative.

This means we have the following algorithm going on inside v the next time we insert:

1. Save the capacity to a local variable and double it, we call this n
2. Allocate a new array of vector of size n.
3. Copy construct a copy of every element found in the original array
4. Destroy every element found in the original array
5. Delete the original array
6. Capacity is now equal to n
7. The new array replaces the old one
8. The new element is inserted at the position "size" and size is incremented by 1

The problem here are steps #3, #4 and #5. Vector simply can't move the pointers found in the array because ISO C++98 dictates that vector (and all standard C++ containers) be safe for storing objects, i.e., data types that have a virtual pointer table. In fact, in this case, vector can't even hold pointers to its elements because the standard also dictates that vector, like all other standard based containers, is value based for maximum flexibility.

So the problem is that all these vectors found inside 'v' are copied element by element, string by string and then destroyed.

Now, considering that capacity is usually expanded by 2 every time this limit is reached, this shouldn't be too bad right? Well, unfortunately no, without any compiler optimizations, returning the value from a function or method is sufficient to trigger a copy. So simply put:

v = do_something(v);

where do_something is defined to be:

vector<vector<string> > do_something(vector<vector<string> > v) {
return v;
}

is sufficient to trigger 2 copies, one by the value argument, and one by returning the value. Suffice it to say this is not very efficient especially considering how widespread the standard containers have now become in the world of C++.

The new ISO C++0x standard introduces a new feature to address this shortcoming, a move constructor. In the previous example, our vector template class defined a copy constructor using the following syntax in order to perform copies:

template<typename T>
vector<T> (const vector<T>& other) {
// perform the copy
}

However, we can now complement this constructor with a move constructor following this new syntax:

template<typename T>
vector<T> (vector<T>&& other) {
// OK, let's steal some memory, so for example
m_rawArray = other.m_rawArray;
other.m_rawArray = 0;
m_size = other.m_size;
other.m_size = 0;
}

Here, the move constructor only gets called when a copy is performed and the previous copy is also known to be at the end of its life. So that means it's OK for us to "steal" memory that belonged to our forebearer.

This is a simple feature that will address the main performance concern of the standard C++ based containers. Even if end users are unfamiliar with this new feature, they will still reap the benefits of it the next time they compile since most standard C++ library implementations (and the Boost libraries) will be enhanced to use move constructors. By using a move constructor here, vector can now avoid all the expensive copy operations it needed to perform in the previously showcased algorithm.

C++0x Feature Focus

2009-05-25T09:18:00.002-04:00

Starting a new series of blog posts related to language changes found in the upcoming C++0X standard. These will focus on core language changes and not on any libraries. You will need gcc from the head branch or the Visual Studio 2010 beta 1 to compile any of the upcoming samples. Even then, some of the samples will not build even with these compilers since some features have yet to be implemented.

I want to highlight the following new features:

Move Contructors
Perfect Forwarding
Variadic Templates
Closures

Closures, a.k.a lambdas, are already implemented today in the libraries but the ISO committee decided it was time to put this in the core to have a unique syntax across all systems.

A Peek at the Compatibility Future

2009-04-24T18:31:00.004-04:00

Users enjoy the broad compatibility Windows has. Even 20 year old 16-bit DOS based applications still run on Windows 7. However, the flip side of this is that it has made Windows very hard to maintain for Microsoft and very hard for developers to use.

That is, targeting multiple version of Windows is very difficult. Bugs present in one version and not another meant different run-time behavior for the application. Add to that the complexity that earlier versions of Windows were also fundamentally different depending on which version of Internet Explorer was installed, even the kernel was touched, meant you add a significant number of operating systems to test and debug. Ultimately, the best solution for portable applications for Windows was to use some kind of layer between your application and the operating system, most notably Java. Sun, or whoever, does the heavy lifting and you target one platform.

If you're familiar with Microsoft however, there's one very interesting technology that you just know is the future. The future of compatibility that is. That technology is called MED-V. Yes, it's a stupid name but the technology is far from stupid. MED-V is based on Virtual PC but instead of running a different operating system with a different desktop it runs your applications side by side.

What is stupid however, other than the name of course, is that this technology is currently only for enterprise customers who have volume licensing. This is a very sad state of affair considering that this would allow for a much more streamlined user experience for people who only want the latest and greatest and much greater compatibility for those who do not. What I mean by that is that ever since Vista shipped, Windows is a considerably more modular operating system with the capability to turn various features on or off. The potential still isn't realized today but Windows 7 goes a step further by allowing various applications to be "turned off".

Now, with MED-V, you could go that extra step, on your brand spanking new machine, you could, say, no 32-bit application, so that entire part of the OS goes missing. No resources no nothing. It also means that even if you're running these compatibility modules, it doesn't mean they're running all the time or at the same time. Furthermore, since these are full virtual environments, they're actually a copy of the target OS that the legacy application was designed for. What's also very interesting is even if these are virtual environment, they're running on the bare metal which means you still have the full power of your actual machine unlike actually using a virtual box.

Well everyone who knows about MED-V knows it's the future, well, at least the technology if not the name. This technology would allow for Windows to grow at a faster pace and cut off deprecated APIs and SDKs without breaking compatibility with older applications.

Which brings me to today, Microsoft has unveiled Windows XP Mode (XPM) for Windows 7. This is the same technology that powers MED-V but what it means is that every copy of Windows 7 will ship with a fully licensed copy of Windows XP SP3 for compatibility mode. So if you're a large enterprise with poorly written line of business (LOB) applications, all of them really, you'll be able to depend on this Windows 7 module to run all those applications. If you just happen to not care about this and just consider it bloat, that's fine too, just turn it off. In fact, if you don't really care about any of this, XPM doesn't actually run unless you use something that needs it.

Reviews of pre-release versions of Windows 7 praise how lean it is compared to its predecessor. That you agree with that or not doesn't really matter, thanks to XPM and eventually the full MED-V, you'll be able to turn off huge, gigantic portions of the operating system and developers and users alike will be able to enjoy new versions of Windows that are free from binary compatibility with older applications but nonetheless offer full compatibility which will mean a much leaner, more secure operating system for all.

Google Updater

2009-04-13T10:54:00.003-04:00

Google has been very busy this past week making new release after new release. The release however of Google Updater as open source is definitely very interesting.

One of the most annoying aspect of the modern Windows XP/Vista desktop is keeping your applications up to date. That is, you have an auto-update for Windows and other Microsoft software. You have one from Adobe for your Adobe applications, one from Mozilla for Firefox and the same software but yet another instance for Thunderbird. You have one for Vuze, you have one for any number of applications. Keeping all your software up to date today just requires too much work.

It was hoped that with Windows Vista, Windows Update would be opened to 3rd parties beyond the driver space but that hasn't happened.

The availability of a high quality open source application updater for Windows doesn't guanrantee it will become the de-facto standard nor that all companies that end up using it will share the same instance/server on the back end but it may pressure Microsoft to finally open up Windows Update to 3rd parties.

Anyway, I see this as good news and just hope that we will finally see some progress in this area.

Let there be Light! (or Java)

2009-04-08T10:10:00.002-04:00

Google has just added Java support to their AppEngine framework. Obviously, the main selling point here for using this system besides free resources for up to 5 million page views a month is the database.

AppEngine uses the Big Table database which is a fully distributed, replicated database and you just get it for free.

Now, if Google would just support NetBeans out of the box already.

Secure IPC in OS X - Part 2

2009-04-06T21:09:00.002-04:00

I wrote the first of what was intended to be a set of Secure IPC in OS X posts. Instead, Part 2, will be The End.

The idea is fundamentally flawed. The easiest way to see this is to consider that what I was trying to achieve was an escalation to root privileges for a particular operation without any user interaction. This is a security hole with a few easily exploitable vectors:

First, the dynamic validation of code signatures does not work as you might expect - it is only shorthand for validating the files on disk.

Second, I was assuming that the IPC system is secure, as pjulien points out here, any process can inject arbitrary code into any other process of the same privilege level. This means that your process can, without needing to modify any files on disk, be made to execute code that was not written by you.

Third, assuming dynamic validation worked like I wanted it to, then the first two problems would be solved. The third problem is that this option is exposed to a user via a GUI. The GUI of any application can be programatically controlled (for instance, by a unit testing tool).

I'm sure there are other attack vectors that I haven't thought of. The point is, when escalating privilege, you need to implement some kind of barrier to entry that asks a user for permission, otherwise it can circumvented programatically.

It turns out this type of mistake is not uncommon. Microsoft recently committed a similar error with their more friendly UAC system for Windows 7. This article explains how the UAC whitelist in a Windows 7 Beta can be trivially subverted by the second problem (code injection).

Reading and Writing to Another Process

2009-03-11T14:33:00.003-04:00

I had a conversation with a friend the other day about this topic, he wasn't aware that this was indeed possible on OSX. As a side note, if your operating system supports debugging, and more specifically, attaching and detaching a debugger to an already running process then the operating system has to have some support for this. Just as a reference, here's a rundown of the functions you can use to read or write bytes to other processes on Linux, Windows and Mac OS X.

On Linux, you need to look no further than open(), read(), write() and close(). You just find the PID you are looking for under the special "/proc" file system, in there, change to the directory that corresponds to your PID. Inside, there is a file named "mem". If you have access, you can just open this file and read and write to it.

On Windows, given a PID, get a process handle using the OpenProcess() function. From there, you can reserve space in the other process by using VirtualAllocEx(). You can use WriteProcessMemory() and ReadProcessMemory() to write and read to this other process.

On Mac OS X, you first obtain a Mach task using task_for_pid(), once you have that, you only need to use vm_read() and vm_write() to read and write to the other process.

All of the above assume that the operating system is clamping your access to other processes based on your credentials otherwise you get something like this.

Secure IPC in OS X - Part 1

2009-03-10T11:05:00.011-04:00

This is the first in a series of posts about a specific implementation of a secure IPC solution for OS X.

First, what do I mean by secure IPC? IPC is the communication between two or more processes on a single computer. There are many methods for doing IPC, some are applicable to most operating systems and others can be quite specific. On OS X, the most obvious choices for IPC are:

In each of these cases, lets assume there is a server process running with elevated privileges that handles requests from one or more client processes. In a typical IPC setup, the server waits for instructions from the client to perform a task. None of the three methods have the ability to establish the sender's identity "built-in." Thus, a malicious process could communicate with the server and get it to do undesirable things. Identifying the sender will be covered later in the series.

You might be thinking there are well-documented industry standard ways to secure client-server communication, in use in important applications like online banking. These methods employ public-key cryptography to ensure the privacy of the communication as well as the identities of the communicating parties. This hinges upon a malicious user not having access to the private key, in other words it is security based on what-you-know. On a single computer, what-you-know security fails because the programmer has to assume that no secrets will remain secret for very long if they are installed on a user's machine (for instance, an embedded private key can be extracted from the binary). The alternative is security based on who-you-are.

For my purposes, the data being transferred between the processes did not need to be encrypted, only the identity of the communicating parties needed to be confirmed. Both OS X and Windows have the ability to establish the identity of an application using code-signing. Thus, the goal of secure IPC, for my purposes, can be thought of as, retrieving and validating the code-signature of the processes participating in IPC.

Part 2 will deal with determining the identity of these processes at run time.

Using code-signing in OS X - 10.6 update

2009-03-10T09:25:00.003-04:00

I wrote a few days ago about the private code-signing API in OS X, and expressed some doubt as to whether applications using it will get burned when 10.6 is released.

There is some good news. I have tried the latest developer seed build, 10A286, and the codesign tool, and my code, works!

theothermike.printf("hello\n") && lessons(XmlSerializer)

2009-03-05T14:43:00.010-05:00

This is my first article here, so I figure I'd do a little introduction first. I'm a professional software developer/geek, with expertise mainly in Perl and *nix environments. I've been toying around with Java and C# in the last couple of years, and more recently, I've started using C#/.NET at my day-job in a specific project.... which brings me to my first post.

While working on making a SOAP request and using the XML serialization facilities in .NET, I spent the better part of a day tracking down why all of a sudden my code had started throwing runtime exceptions when changing a class member from being an ArrayList to a generic List<SomeClass>

Thinking at first that the serialization services didn't actually work with generic collections, I had found several examples that proved otherwise..

Now keep in mind that I'm fairly new to Visual Studio (this project is in 2005), and IDEs in general as I had mainly been using Emacs in a terminal window before. So, when I was presented with the exception dialog, I was extremely annoyed at how many 'levels' I had to dig into the dialog to actually get to the exception information I needed. For some reason, it doesn't word wrap either, and even stretching the dialog across my 2 monitor setup doesn't reveal everything in the exception.

Almost ready to give up using generic collections, I started modifying the code to use ArrayList and arrays[], but cleaner than it had been before.

Now for some reason, the GUI was STILL throwing an exception... but at least it 'bubbled' up to a different exception dialog that would actually let me see the text of the entire exception*.

Way down in the stack, I saw the error message causing my problems wasn't related to the fact that I was using generic collections, but rather due to the fact that the class used in the generic collection (ie. SomeClass) didn't have a default constructor anymore since I had created another constructor to take parameters for convenience. This does make perfect sense after I realized my mistake, but it didn't even occur to me as I'm still fairly new to C# (I had assumed the default constructor would still be there even though I didn't define it)

Adding a default constructor of course took care of the problem, and I reverted back to using List instead of ArrayList.

Applying some tips from pjulien, I also made the class 'sealed' and exposed the List using IList instead ..... much nicer:)

... live and learn

* of course you can see the entire exception in the 'output' window of VS, but this I learned after the fact

Using code-signing in OS X

2009-03-05T14:37:00.008-05:00

OS X 10.5 includes code-signing support. Unlike Windows, code-signing in 10.5 doesn't really affect the user experience, but that will almost certainly change in future versions. The only code-signing aware features in OS X are the firewall and parental control systems. Both of these systems use the code-signature to identify an application independently of its location on disk and of its version, therefore, firewall preferences can be saved accross upgrades.

OS X 10.5 allows you to sign your applications and verify their on disk and runtime signatures. Here's the catch; Apple only support doing this via the codesign utility.

But what if I want to make use of code-signing in my application, for instance, checking the validity of a process I'm about to do IPC with? If you are like me, you don't like the idea of executing codesign directly and picking up the return code, you'd prefer an API. Well one exists and its really nice. Here is the code to verify the runtime signature of a process:

SecCodeRef code = 0;
SecCodeCreateWithPID(pid, kSecCSDefaultFlags, &code);
SecCodeCheckValidity(code, kSecCSDefaultFlags, 0);

Obviously, you'll have to check return codes and such, but 3 lines for code-signature validation is pretty nice. But all it checks is that the code hasn't been tampered with, no identity checks here. Adding your own extra requirements (like an identity check) is also easy (see man csreq for details on requirement strings):

SecRequirementRef requirement = 0;
SecCodeRef code = 0;
SecRequirementCreateWithString("certificate root H=\"abcdef12345\"", kSecCSDefaultFlags, &requirement);
SecCodeCreateWithPID(pid, kSecCSDefaultFlags, &code);
SecCodeCheckValidity(code, kSecCSDefaultFlags, requirement);

For brevity, I lied about being able to pass a char* to create the requirement (it should be a CFString) and I didn't check my return codes. Replace "abcdef12345" with your public key's SHA1 and you can verify that the code is signed by you. Pretty nice. Much better than the 100 or so lines of CryptUI code I had to write to do the same thing. As always though, there is a downside, the API is private, and codesign doesn't work on the preview builds of Snow Leopard, which makes me think they may have changed the something. Bummer.

Still, if you're interested download libsecurity_codesigning from the Darwin source code page and enjoy.

New Contributor

2009-03-05T11:15:00.006-05:00

I'm going to starting contributing to this blog. I will mostly post about things I find interesting as I learn my way around OS X and Objective-C, from the perspective of a person coming from Windows/Linux and C++.

By way of introduction, a bug report. NSFileManager reads past the end of allocated memory. The details can be found at rdar://6636446, Apple have fixed the issue for some future release (10.5.7 or 10.6?).

I came across this by running my application with Guard Malloc. For the Linux developer, Guard Malloc is basically DUMA for Mac. Like Linux, it is easily used via the Mac (pseudo-)equivalent of LD_PRELOAD, DYLD_INSERT_LIBRARIES, or if you can stand using XCode long enough, it can be enabled at the bottom of the "Build" menu.

Windows 7 Shadow Copies

2009-03-04T19:10:00.002-05:00

Since Windows Server 2003, Windows has had a feature called shadow copies. Basically, this allows you to get back to previous versions of existing or deleted files. The feature was later enhanced in Vista to make the "system restore" feature. Namely, the system takes snapshots of important system files your computer needs to boot by creating "restore points". At any point in the future you can revert to such a restore point. This can be because your system stop rebooting or you just noticed that game demo included SecuROM and is still installed even if the game demo no longer is.

The feature works has advertised however, there was no way in Vista to know how much disk space this feature was using or could use. Windows 7 includes new configuration options that control these settings.

JavaFX

2009-03-04T08:49:00.005-05:00

Now that the 1.0 release has been superseded by the 1.1 release, I decided it was time to look at JavaFX and see if there was anything there.

The first thing that hits you when looking at the samples is how smooth everything is on screen. That's good because Sun really needs to impress with these samples if it has any hope of getting developers to use them.

The second thing that hits you is the claim about there is no JavaFX mobile, there is only JavaFX. Code you write in JavaFX will work over "all the screens of your life". It's certainly possible to write a JavaFX application that would work over a phone, BD player, TV, browser and your desktop. Unfortunately, you wouldn't want too, the effort required to get that going wouldn't be worth it. The problem doesn't arise when you look at what's there but what isn't. Most basic controls like a text box or a list are imported from Swing verbatim or are simply missing. In other words, you have to use Java/Swing to add the missing functionality in your own application.

Now, if you're planning on deploying to all of the above platforms. All these Java stacks have swing with the exception of Java ME/CLDC. The CLDC environment happens to be the one that sits on those 2 billion phones that Sun keeps talking about and that's an issue. It's possible but highly unlikely that your application will not require any user interaction beyond flashy effects. The real problem with the CLDC however, is that it doesn't have AWT either. So beyond writing a complete replacement for these controls in CLDC, which is what the LWUIT project is doing, there isn't much you can do here. Is it possible that Sun will finally start to push Java ME/CDC for smart phones? I don't know, time will tell. However, the BlackBerry is still the most popular business oriented smart phone out there and it's based on CDLC. I say "based", because that virtual machine has some serious issues that I feel invalidates it as a Java virtual machine in the first place. Meaning getting JavaFX to run on a BlackBerry is still a big question mark.

The other big news is that Sun has reported they now have done 100 million installs of JavaFX on the desktop. I was really impressed by this at first because I thought this was people going to javafx.com, downloading the bits and installing them. What else could it be right? There wasn't a new Java 6 update so they couldn't have been piggy backing on the regular Java runtime from Sun to deploy JavaFX right? Wrong. Java 6 update 12 is still the latest release, but if you go to java.com right now and inspect the current installer, you can see that the installer binary has been modified to include JavaFX. 100 million desktop installs is still very impressive, Silverlight just recently passed this milestone and it has been out for far longer. However, let's face it, JavaFX is getting deployed because of Java Update, not because end users are excited about it and getting the bits themselves.

JavaFX on the desktop is still interesting. Thanks to the massive work that was done in Java 6 update 10, it can leverage the completely overhauled applets and web start systems in that release. Which means you can deploy your FX code as applets or web start applications or a combination of both. For example, most of the samples on javafx.com run in the browser, however installing on the desktop just requires the user to press "Alt" and drag the applet outside the browser to have the application live on your desktop. What's also good about this process is that the install is per user, so there are no UAC escalation prompts so anyone can install these applications that they be an administrator or not.

More good stuff:

Sun decided that it wasn't going to re-invent a completely new set of tools for content designers. Instead, it wrote plugins for Adobe's creative suite.
Media codecs. Finally, video comes to the Java platform. JMF (Java Media Framework) is dead and buried, it was last updated in 2003. JavaFX features On2 codecs and plugs into DirectShow on Windows and QuickTime on OSX.
JavaFX isn't a virtual machine, it runs on top of an existing JVM. That's good news indeed because in conjunction with Java Web Start, if your application is started by someone that still doesn't have JavaFX, just Java, the application will still load just fine. The JavaFX jar is just another jar which means web start will just grab it just like any other jar that is part of your application. Furthermore, if your users don't have Java, you can still use Sun's JavaScript script for Java deployment to get it to end users.
Updatable runtime even for phones. This is a departure from how Sun has been doing business on phones. Having the runtime updatable over the air is a really a good thing. Even if you bought a first generation iPhone, you haven't been left out in the cold, your system has been upgradable to every release that has been put out there. Sure, it's not over the air, you need a computer to install it but this is still a good thing.

Now some bad stuff:

Applets still take too long to start. Java 6 update 10 really helps but it still isn't good enough. It needs to be instantaneous if Sun has any hopes of replacing Flash with this technology. When you get to a web page, your banner ads have to be showing immediately, they can't be showing the Java spinning logo.
The Java system tray icon and bubble advertisement. Are you kidding me here? Really, really annoying.
No Android support outside the traditional model. That means Android handset developers need to license it from Sun and it doesn't come over the air. It follows the traditional model of what ships with the phone, dies with the phone. Of course, Android unfortunately hasn't exactly caught on fire, might be a non-issue here.
Not all JavaFX versions have been released. Namely Linux.
No visual editor. I've said many times over now that NetBeans' excellent matisse editor needs to move to the next level, beyond just Swing/AWT support. Please note that visual editors are available from 3rd parties.

All in all, I hope Sun keeps working really hard on this. The 1.1 release came mere months after the 1.0 release and I think it needs to keep doing that. Release very often so that it can patch holes and missing functionality bit by bit, forget the huge releases here, if it does that, it will be in a position where JavaFX 1.1 is competing against Silverlight 3. JavaFX's scope, i.e., the total number of classes, already doesn't compare favorably to Flex or Silverlight, let it gain ground bit by bit instead of waiting till the platform is abandoned before making another release.

Next up, Silverlight...

Windows 7 DST Heads Up isn't New

2009-03-04T08:48:00.001-05:00

Turns out this feature is in Vista also.

Windows 7 DST Heads Up

2009-03-03T21:05:00.002-05:00

Windows 7 Editions

2009-02-03T16:40:00.004-05:00

One thing that Linux and MacOS X users enjoy over their Vista counterparts is that they don't have to worry about choosing an edition of their operating system or upgrading to the same operating system but to a different edition, e.g., from Home Premium to Ultimate works but Home Premium to Business doesn't.

The Windows SuperSite today has an article about the editions that are being removed. Also, each edition is a super-set of the previous one, so upgrading is much simpler since there aren't any distinct branches anymore in the editions.

That being said, I still think they are still too many versions. There should be only 3, Starter, Home and Professional with the starter edition being targeted at netbooks only.

Most Recently Used Files From the Start Menu

2009-01-25T10:11:00.008-05:00

Instead of a global menu item that shows a pot pourri of recent documents for all applications, the Windows 7 menu is smarter on how it shows you recent documents.

Recent documents are displayed and bound next to any compliant application. Pictures are worth a thousand words here:

Windows 7 Hidden Features

2009-01-24T12:33:00.003-05:00

Starting a small series of posts about new features in Windows 7 that don't have a lot of flash, hence are unlikely to be covered in the mainstream, but are still neat and interesting.

Finally, Windows can burn ISO images directly from its shell. It's about time Microsoft did this considering the APIs have been there since Windows XP. I'm excited about this since most vendors usually bundle some crapware in order to provide this feature to their customers. Hopefully, this will mean a cleaner hard drive on purchase.

Now, if Microsoft would just get around to finally supporting virtual desktops. The APIs have been in there since at least Windows NT 4.

Windows 7 has the ability to mount Virtual PC VDC disk images. You can see the mounted disks like any other drive in the "Computer" view. This would have come very handy a while ago, I have a collection of virtual machines from Windows 95 up to Vista. With Windows NT 4 and earlier, it was really hard to download updates and/or service packs since:

a) Out of the box, they either don't come with any browser or come with Internet Explorer version 1. If you try to use Internet Explorer version 1 on mainly any site, including Mozilla, it doesn't work, the browser errors out.
b) You can't use a shared folder because you can't install the guest additions until you've updated to the latest service packs/updates but you have no means to get them on there.

This feature would have made it easier to just put the latest updates on these deprecated operating systems and get to the point where I could just install the guest additions. Ultimately, I had to put all the updates on an FTP site and use the text based ftp command that comes with NT versions 4 and earlier. I was still looking for a solution for Windows 95 and this is what finally made it possible to update. My 95 disks are original Windows 95 floppy disks, so I didn't have the FTP command on this install either.

Windows 7 Codec Support

2009-01-24T12:16:00.003-05:00

In addition to the codecs supported by previous versions, Windows 7 supports an impressive list of new codecs and media containers out of the box.

New containers:

MP4
MOV
3GP
AVCHD
ADTS
M4A
WTV (new Windows Media based format for PVR based recordings)

New codecs:

H.264
MPEG4-SP
DivX
Xvid
MJPEG
DV
AAC-LC (regular AAC)
LPCM
AAC-HE

You can get all these codecs and more with the K-Lite Codec Pack, however, this is the 32-bit version. The 64-bit version of K-Lite doesn't offer many codecs at all.

Now, if Microsoft can just add support for the Matroska file format, we'll be all set.

The Nokia N96

2009-01-19T16:05:00.003-05:00

Just got my brand new toy. The Nokia N96, I honestly don't see the mass market appeal for such a device but I don't think I've felt this young in quite some time. This device is a whole lot of fun.

When I say fun, I don't mean this phone has a touch screen or a full keyboard, more fun like it supports live TV and has a PVR, a pretty damn good web browser, tons of codecs for all your media, good podcasting software, etc, etc. An unlocked version of this thing will set you back $800 USD. I have no idea what the target market for this device could possible be but I like it.

That being said, the N97, if not for the price tag, would be a much more serious product. Touch screen, full screen keyboard... and yes TV and PVR among other things.

Anyway, the N96 is a very good product, everything on this phone just works. The Java support is top notch with support for SVG based interfaces even which makes for some really interesting user interfaces. Again, however, budget conscious users might want to stay away here.

A video review is available here.

Microsoft SkyBox

2009-01-19T14:19:00.005-05:00

The net is abuzz today about the rumored new user interface for Windows Mobile based devices. You can see some more information here.

However, that's not the major pain point when developing for Windows Mobile. Yes, the user interface is the major pain point for users when dealing with Windows Mobile but not for developers.

The real issue for developers is deployment. That's right, when developing for Windows Mobile, you have basically 3 options:

Java ME
.Net Compact
Native

Native is a pain due to the many different CPUs you'll encounter when dealing with Windows Mobile devices. Furthermore, many third party libraries just haven't been certified/tested on this platform. Here, I'm thinking mostly about Boost. Additionally, you'll either have the choice of using MFC or using the C Windows API. Just fun all around. Deploying a cabinet file for these devices if you can straigthen out all these issues is straight forward but getting there is a long road.

Java ME, when present, is usually the CLDC variant. Meaning it's a pain, it's light, missing many important classes, has pre-Java 5 syntax which means you can't use the majority of 3rd party libraries on it, open source or otherwise, and well, usually, not even loaded on the phone to begin with. Furthermore, if you don't have it on your phone, there isn't a convenient way to get it or upgrade it. What ships with the phone is what dies with the phone. Again, deploying a JAD file isn't terribly difficult, that is, if the phone has a Java VM to begin with and that Java VM is of decent quality. As it turns out, most Java VMs that ship on Windows Mobile aren't worth the bytes they fill up.

The last choice is .Net Compact. This one isn't too bad except for the fact that most phones don't ship a recent version of .Net Compact. Usually, you'll find .Net Compact 2 with no service packs, which is a total mess, or .Net Compact 1 ranging from service pack 3 to no service pack at all. The minimum that is actually usable is .Net Compact 2 SP2. So the problem you have now is upgrading .Net compact. The only official supported way of doing this is to download an MSI package to Vista/XP and the next time your phone connects to your PC, your desktop will upgrade your phone. Not very useful if you want to deliver applications on the show room floor.

So keep going Microsoft, keep going, and maybe someday you'll look back on this and say, what were we thinking?

The Last Cylon

2009-01-17T19:24:00.003-05:00

Seems I was wrong, Ellen is the final Cylon. The actress tells the LA Times she's been sitting on the secret for over two years.

Battlestar Galactica

2009-01-16T23:01:00.004-05:00

The season premiere just ended and I don't want to spoil anything for anyone. However, talk about living up to the hype. The show has been on hiatus for a year now courtesy of the writer's strike.

I always liked sci-fi but not sci-fi shows, e.g., Star Trek and Star Wars to name a few, just placeholders, something to fill the void but not particularly enjoyable. This show, however, goes beyond its genre.

The premiere was beyond anything I could have hoped for. I believe this show will live on as a pillar of American television. A milestone, a new standard of what quality should be and what it needs to be to keep audiences engaged.

Now, spoiler/theory alert. Do not keep reading if you don't want to know what I think.

There is no way in hell that Helen is the last Cylon. However, that doesn't mean she isn't a Cylon of course. We now know, thanks to Saul Tigh, that a Cylon can grow older. I believe Helen is an aged #6 model.

Who knows what Starbuck is at this point, or Dee, or anyone for that matter. It just seems to point to that everyone is a Cylon or at least a descendant of one.

Windows 7 Beta

2009-01-11T08:26:00.006-05:00

Still in the early stages of use but I have to say, an apology is needed here. I've been laughing at all the glorious reviews Windows 7 has been getting. Considering Windows 7 is just some spit and polish of Windows Vista, obviously if these people liked Windows 7, then they must have liked Windows Vista.

Well, having used 7 for a couple of days now, I have to admit, that this is some serious spit they used here.

What can I say other than "wow". It just shows that in this release, having nailed all the architectural underpinnings in Vista, Microsoft was free to concentrate on the user interface, be it the shell or the bundled applications. All I can say is that this is one serious calculator program.

So far, I only have 2 bad things to say about the system:

The return of the "My" prefix, so "my music", etc. Shoot the person in the head who thought bringing this back was a good idea already.
UAC is toned down. I always felt UAC was a good thing. Vista got a lot of heat for pointing out bad programs to users. Seriously, how would Linux users have reacted to a program that needed constant write access to /user/bin? The levels are good since it should stop some people from disabling it completely. However, I think the highest level should be maintained as the default. I think if Windows Explorer wouldn't show you a dialog that an operation is going to need escalation, hence 2 escalation dialog warnings, and would just put the escalation shield icon directly in the menu item, this would solve the problem most users had with UAC to begin with.

Anyway, all this to say, I apologize for laughing at all these glorious reviews of Windows 7. Windows 7 is really quite something.

However, not that I want to finish on something negative, I can't help myself to take a shot at Windows Mobile. How is it possible that the same company that is producing something so good like Windows 7 is also producing, at the same time, something that is so mediocre, namely Windows Mobile?

Every Byte Counts

2008-12-19T21:12:00.004-05:00

Well, it has been a while since I've posted anything. I've been really, really busy trying to figure out how to save every last byte that I can over the network for mobile applications. When all you have is an EDGE network, you really can't spare anything.

Here are some of my findings:

Don't use SOAP. If you're going to use a standardized communication method, prefer REST to SOAP. Why? Because even if SOAP offers a lot of information in its message envelope that is very useful for implementing integrity, security, transactions and not to mention making it easy to orchestrate via an enterprise bus, it also happens to be much larger. Expressing a resource in REST will save you a lot of bytes.
Another reason, is that REST isn't bound to XML like SOAP is. JSON is so much shorter than XML. Furthermore, most Java ME platforms have zero support for XML. To be fair, Java ME has zero support for JSON too. Support for both is readily available using 3rd party libraries however, again, the JSON library is much smaller than the XML one. Using JSON, not only do we get something that is more compact than XML, we don't lose any of the flexibility. You can express any XML construct in JSON, and just like XML, you can extend your JSON documents in the future with elements that can be ignored and parsed over by older clients.
Gzip encoding is your friend. Mostly all HTTP servers support it, just add the "accept-encoding" header to your request. Needles to say, Java ME doesn't support gzip but once again 3rd party libraries are readily available
Keep alive connections. HTTP 1.1 supports keep alive connections out of the box. What that means is that you can send one http request, read the response and instead of closing the socket connection, you can re-use it to send and receive additional requests and responses. This is also unsupported from Java ME, and no, to my knowledge, there are no 3rd party libraries available that support this. Fortunately, HTTP is a really simple protocol, you will need to forgo the use of the HttpConnection class and use a raw socket via the SocketConnection class. You just need to send well formed HTTP 1.1 requests to the server and you should be able to read responses just fine. This saves you the setup and tear down times associated with a full TCP socket connection every time you need to send a request. Please note however that you will need to support, at the very least, the "chunked" encoding since HTTP 1.1 compliant servers are free to use this encoding whenever they so choose.

However, most disappointingly, probably one of the best tool you can use to save bytes over the network is not available. I am talking about HTTP pipelining. HTTP pipelining as specified in version 1.1 of the protocol allows you to send many requests to the server on the same socket before you start reading any of the responses. Unfortunately, my server of choice, Glassfish, has no support for it. If Glassfish has no support, most certainly no Java EE container does either. The HTTP parser found in Glassfish is none other than Tomcat's Catalina engine. Since Catalina is used verbatim in almost every Java EE container out there, then that's it.

Most unfortunately, the Grizzly project team members have decided that HTTP pipelining is not a mandatory or important feature of HTTP 1.1 since the browsers don't enable it by default. Grizzly is the HTTP server used inside Glassfish and fronts the Catalina engine in Glassfish instead of Tomcat's own Coyote server. I disagree with this conclusion. I think this W3C document states that pipelining is indeed mandatory.

I would love to hear from someone with Jetty experience here if this feature is supported or not. I use many Java EE 5 features that would make the transition to Jetty impossible but the Apache Geronimo project does feature a certified Java EE 5 server built around Jetty.

Finally, if anyone would like to suggest more ways to save bytes in a portable manner, please feel free to comment.

No More Heroes 2

2008-10-10T14:14:00.003-04:00

I'm so happy that the first game did well enough to warrant a sequel. If you don't own a Wii, there's never been a better excuse to get one.

Trailer for No More Heroes: Desperate Struggle.

First Android Device Fails to Impress

2008-09-24T12:09:00.004-04:00

T-Mobile in cooperation with Google yesterday announced the G1. I watched the video of the entire press conference. I think Google is not adapting to this new market instead relying on doing business the same way it does on the web. This may have proved successful for their web business but I don't think it will work out very well for the mobile market. Things that just don't work include:

No desktop application for syncing. Google is positioning everything to sync from the cloud, gmail, google calendar etc. Yeah, this is just going to be such a joy for organizing your music and other media files on your device.
No iTunes support. The largest music retailer in the world will not function with Android.
Sim locked to T-Mobile. 2 year contract with a retail price tag of $175. OK, so the device costs a lot more than $175 hence the contract, so why can't we just buy the full version? So much for this open thing.
The device is bulky, favoring a full pull out keyboard instead of a touch keyboard similar to the iPhone. I think this is more of a problem with HTC than the Android platform itself. However, now that I think about it, I haven't seen a touch keyboard on this platform period. The emulators don't have it.
Limited Bluetooth support and APIs. Google has pledge that future versions of the software will expand on these services.
Google Android Market is a good thing but are these devices actually upgradable? Can we upgrade from Android 1.0 to future Android releases?
For the relatively small price difference, that 8 GB flash drive found in the iPhone is looking very attractive. Honestly, dump the SD cards already.
Tooling support is still sub par. No visual designer is available but the latest Eclipse plugin has visual preview so this may be in the works. However, DroidDraw is available and is pretty good.

Not all is bad however, the strength of the platform is still Java SE. Android is a subset of the full Java SE 5. You can see the number of packages for Android here and for Java SE 5 here. It may be a subset of Java SE 5 but it's still huge compared to any other mobile platform out there. The other thing is that the Android compiler works on Java byte code. So any library written for the Java SE 5 platform, and yes, there are quite a few of them out there, can be potentially used on this platform.

Recipient Address Rejected with PostFix and JavaMail

2008-09-05T10:24:00.003-04:00

Really not sure anymore what I wanted to talk about this exactly. I had this problem and the only thing I can recommend if you do have this issue is that the bug is usually in the order of the parameters you pass to JavaMail and not in your PostFix configuration proper.

JavaMail has way too many parameters that are just strings, with methods that take a lot of strings, it becomes way too easy to mix up one string parameter with another.

Folders in a Database

2008-09-05T10:08:00.002-04:00

If you're using folders and storing them in a database, the conventional approach is to use recursion for things like folder validation, moves and membership.

Using recursion with a database like this is usually pretty bad performance wise, consider using a token instead. A token could be, for example, a folder id string field that keeps the path of the folder using the identifiers of all its parents.

So, if you had folders "a/b/c", the folder id string for the "c" object could be "/100/101/" representing the path to this folder.

In application code, you should obviously never manipulate this string directly, the string should be completely private to your inner most classes that represent your data model.

Using this technique however, it becomes very simple to answer some very basic questions like:

Can I drop this folder into this other folder? Simple, just check if the string starts with the same path or is equal. So "c" can be dropped into "a" or "b", but "a" cannot be dropped in "a" or "b" because the path for "c" contains "a".
How many items are contained in this folder? Just check how many items have a matching string.
Makes it easy to find siblings, all the items found in the same folder have the same folder id/path.

If you do keep using recursion, make sure your code is secure at least, don't allow for an infinite number of recursions.

Push vs. Poll

2008-09-04T10:09:00.007-04:00

It seems pretty obvious at first but this is no simple matter. For years, I have been in the poll camp. The idea was that if you had long running processes that would poll for updates every 20 to 60 minutes, the cost of setting up, performing requests and tearing down a connection would be less than the cost of maintaining a connection on the server.

Well, there are several problems with this scenario. The first one being that under such conditions, the total number of bytes needed to perform the requests on the server usually ends up being a higher number than the total bytes needed to produce responses to indicate that no new data is available since the last poll. The second is that it is much harder to limit incoming requests to the capacity of your servers. Namely, given n number of client processes out in the field, entropy sets in and will cause your clients to poll at different times. However, as the number of client processes gets bigger and bigger, the number of simultaneous polls that hit the server at the same time grows at the same rate. Using push, we can feed updates to clients at a rate that the server can handle. Since the server is responsible for contacting the clients, it can do so at its own pace completely sidestepping the “Thundering Herd” problem. Third is that not all bytes are of equal cost. In this ever heterogeneous computing landscape, e.g., the cost of a byte on a high speed DSL line isn’t the same than the cost of a byte on a cellular edge network. In the scenario above, sending requests that have empty responses should be a no cost operation on a domestic high speed link but this doesn’t hold true for a user on the cellular network. Fourth, given an update that needs to be pushed, that single piece of information is completely independent of any updates that came before it and will come after it, meaning that it is easy to distribute this work to a farm. For example, we can imagine that for a poll, we would need to query the database for each and every poll. For a push, we would push the result of running a single query to multiple clients. The fourth argument is much more domain specific since your data might not allow for this at all but it’s still perfectly possible on some data sets. Fifth, web applications are not kiosks, they can't be polling only once every hour. However, even for kiosks, push is still superior since it allows for instant monitoring of every client in the field.

The term coined for server side push is comet, or reverse Ajax, and you use this to get to your clients, be it a web application or not, to achieve a server side push. However, the problem is that too often the solutions ends up looking something like this:

but this is not the most optimal solution. First, it does not get rid of the extra request traffic that we are trying to avoid. This is not really server side push but more of a constant poll where the response timeout is extremely high. With comet, the idea is to get to a point where you're shaping your traffic like this:

Not only does this get rid of our unwanted traffic, it truly does provide a scalable way for the server to publish change sets to clients without having to exert itself.

Now the bad news, achieving comet isn't just about behaving in a certain way at the network level. Reaching this level of sophistication means that the server is maintaining work queues of the data that it needs to send to clients. This isn't too bad but remember that not every client will be synchronized to the latest updates when sending out new updates. Your clients will still need to get to the synchronized state using regular requests before they can get updates via comet.

Finally, I would like to mention that there are several tools and kits and what not to add comet support to your application, you don't need to do it yourself. However, I have found that Grizzly, the http web server that ships by default in Glassfish, to be an excellent choice for all my comet needs. Grizzly is really an impressive piece of code, using non-blocking I/O everywhere with a minimal amount of threads to achieve maximum throughput. You'll still need to handle things like dropped connections and other exceptional conditions but the hard part of getting a scalable comet implementation going in the first place will be done at least. Furthermore the plan for Grizzly 2 calls for enhancing the server with the new asynchronous I/O facilities being introduced in Java SE 7, which should make this already impressive beast scale to new heights.

NetBeans Exception Reporting

2008-08-17T09:28:00.003-04:00

I've talked before about how crash reporting hasn't yet entered the mainstream of available services from open source hosting sites. GNU Savannah, Sourceforge, Google Code, none of them provide crash reporting to their customers. If your program crashes in the field, you either need your user to report a bug manually or you need to implement or reuse a crash reporting system, but more importantly, you also need to host your own crash reporting service.

I've talked before on how this is really a shame especially considering that Google produces breakpad, certainly good enough for the C and C++ community to start improving the quality of their software thanks to good crash reporting and some projects definitely are, albeit, again, using their own servers.

To this day, Microsoft is the only company that is hosting an error reporting service that it makes available to all developers in the form of Windows Error Reporting. Kudos to Microsoft for this effort.

I've been very enthusiastic about my jump from C++ to the Java platform, or to a much lesser extent the .Net (Compact) platform. The reasons for this aren't related to language features, C++ armed with smart pointers and modern libraries and idioms can easily compete with the Java 5 language and C#.

No, the reason is how much richness is kept in the compiled binary format representation of these platforms. Of course, the massive advantage of Java over, well, pretty much everything else, in the number of high quality libraries that are available certainly helps too. The point being if you've had to deal with post-mortem debugging before, the rich and high quality stack traces you can get from a Java exception versus the information you get from a minidump file is night and day. You can't get this from a C or C++ program even if you do all your symbol generation correctly because a lot of good and necessary information is optimized away. While with the other two, the non optimized version is still available on disk since it's compiled just in time.

Taking a step back here for a second to talk about ahead of time versus just in time compilation. I'm not too familiar with .Net outside of the compact edition but the performance of Java SE 6 is simply amazing, I've done enough high level C++ to know that Java SE 6 beats it hands down performance wise. Forgetting the ton of adaptive optimizations that the HotSpot virtual machine can do here for a moment, one of the major reason for this is the heap. Heap implementations in C++ are still based on C's malloc which isn't exactly good at dealing with tons of little polymorphic object allocations. C's malloc is still geared toward the C program, you allocate a bunch of memory at start up and you use that as a pool until your program ends. Not exactly friendly to a high level C++ program nor a multi-threaded one.

Getting back on track here, the lack of error reporting services unfortunately extend to the Java world. Even the tree major Java hubs out there, in addition to the tree named at the start of this post, namely java.net, Apache and Codehaus do not provide crash reporting services. I guess Apache is off the hook since they provide mostly libraries and server software, but still, much could be done in the area of crash reporting.

All of this brings me to how NetBeans handles unhandled exceptions that occur in the wild. NetBeans pops up this little dialog and asks you to send in your report. Once you do send it, it's almost magical, reports are queued, analyzed, associated to either a new report or an existing report and finally the generic reports are associated to an issue in IssueZilla and it does it all automatically. If the issue is fixed, it even tells you in which, upcoming or not, version it has been fixed.

This is how it should be done. I don't know what license this system is under, but how about getting this into the greater community? Let's get this system moved from being NetBeans specific to all projects hosted at java.net for example. And what's really nice about this is that it doesn't have to be limited to Java SE/EE, this can work just fine with Java ME too. Now, wouldn't that be something?

BlackBerry + WiFi

2008-07-25T19:20:00.005-04:00

Taking a short break from the planned article list to rant about my new least favorite feature of BlackBerry OS.

If you have a BlackBerry that also happens to support WI-FI, you might be surprised to find out that your favorite networked application doesn't favor WI-FI over APN when you're in range.

In fact, these applications will never favor WI-FI over your regular data network link. This is because the BlackBerry operating system does not abstract this for the application programmer. In other words, unless the application programmer knows about this and explicitly codes support for WI-FI in his application, the application will never use WI-FI.

So how do you get WI-FI support in there? You have to append the magical string ";interface=wifi" to any URL that you want to open. So, assuming standard Java ME/CLDC network programming here, you would do something like:

String uri = "http://www.spectrumdt.com;interface=wifi";
HttpConnection c = (HttpConnection) Connector.open(uri);

Otherwise, your application will inform you that "Could not open tunnel - Max time out" or go over your regular data link.

And that's about it. Oh, and by the way, if you don't have WI-FI or you're out of range, the OS won't do you any favors and fall back to your regular data link, so you have to make sure to try out the regular data link in case of errors too.

Finally, I would like to say that this feature is mostly impossible to find in the BlackBerry documentation. I found it in a beta version of an upcoming manual. The only reason I noticed this is that my development BlackBerry doesn't have a sim card, the browser worked, but my application didn't. Also of interest, almost all non first party applications I tested couldn't use WI-FI either. I'm guessing quite a few people don't know about this.

GWT + j_security_check

2008-07-13T11:27:00.001-04:00

I went out looking for a solution to this recently and found very little information. A lot of questions unanswered led me to believe I was not the only one who wanted to do this. I present here a small sample on how to accomplish this.

Before presenting the sample, let's go over a few things first on why you would want to do this and more importantly why you still need an Ajax style security token for all your GWT services even after this is implemented.

The reasons you would want to do this? One is automatic single sign-on (SSO) if supported by your container, e.g., Glassfish supports SSO out of the box for its web, EJB and web services tiers. If you use j_security_check, you're in; you're into all these services. Furthermore, if you use a J2EE authorization agent like OpenSSO inside Glassfish, you're into any services or sites provided by that federation. So even if you’re fronting your application with GWT, your back end might be composed of all these elements and more.

This also allows you to use any standard security annotations in your EJBs or, if you're orchestrating like me, in OpenESB. You can do this because you now have a Java security principal setup inside the container.

This also allows you to leverage the clustering features, if any, of your application server. Glassfish, again, contains great out of the box support for clustering. Session replication and expiration is managed for you and it just works. One call can proceed on one server while the other can proceed on another while the caller remains completely oblivious to where its work is actually being executed.

What you can’t do however is assume that you should no longer have an object representing your logged in user in your GWT client code. GWT RPC calls are still out in the open with this technique, you should pass in at the very least your session id value as a parameter to the GWT remote call so that the RCP service can do validations.

You still need SSL. Use SSL to make sure your password is transmitted securely over the wire. Hashing on the client side is completely useless. First, because MD5 is usually about the only thing available in JavaScript and that is easily broken. Second, because you should be counting on SSL to do its job correctly. Third, once on the server, you should hash your password with something that is secure, i.e., not MD5 and throw away the clear text representation of the password.

I use NetBeans 6.1 and release candidate 1 of GWT 1.5 for this example. The idea is to use the RequestBuilder class to touch a resource that has been marked off limits by the container. We do this because the j_security_check, in Java EE 5 at least, cannot be invoked directly in a portable manner. When the server sees the access to the protected resource by an unknown, it redirects you to the login page. When this occurs, the method “onResponseReceived” is called. We ignore the redirect and submit our own form. The form just needs to have at least 3 named elements present:

The form name, j_security_check
The user name field, j_username
The password field, j_password

If you don’t touch a protected resource first and simply submit your form immediately, you will find that you still don’t have a security principal since no login actually occurred in the container.

You would apply the same rules for defining a protected area than you would for a normal non-GWT application. You can see on how to do this in Glassfish and Tomcat by following the tutorial here.

Here is some sample code that fulfills a login on Java application server. I originally had more here in terms of error handling but it seemed it got to be too large for presentation purposes.

package org.codepimps.client;

import com.google.gwt.core.client.EntryPoint;
import com.google.gwt.core.client.GWT;
import com.google.gwt.http.client.Request;
import com.google.gwt.http.client.RequestBuilder;
import com.google.gwt.http.client.RequestCallback;
import com.google.gwt.http.client.RequestException;
import com.google.gwt.http.client.Response;
import com.google.gwt.user.client.Window;
import com.google.gwt.user.client.ui.Button;
import com.google.gwt.user.client.ui.ClickListener;
import com.google.gwt.user.client.ui.FormHandler;
import com.google.gwt.user.client.ui.FormPanel;
import com.google.gwt.user.client.ui.FormSubmitCompleteEvent;
import com.google.gwt.user.client.ui.FormSubmitEvent;
import com.google.gwt.user.client.ui.HorizontalPanel;
import com.google.gwt.user.client.ui.Label;
import com.google.gwt.user.client.ui.PasswordTextBox;
import com.google.gwt.user.client.ui.RootPanel;
import com.google.gwt.user.client.ui.TextBox;
import com.google.gwt.user.client.ui.VerticalPanel;
import com.google.gwt.user.client.ui.Widget;

public class MainEntryPoint implements EntryPoint {
public void onModuleLoad() {
    final FormPanel formPanel = new FormPanel();
    VerticalPanel vpanel = new VerticalPanel();
    HorizontalPanel hpanel = new HorizontalPanel();
    TextBox userName;
    TextBox password;

    formPanel.setAction("j_security_check");
    formPanel.setMethod(FormPanel.METHOD_POST);
    hpanel.add(new Label("User name:"));
    hpanel.add(userName = new TextBox());
    userName.setName("j_username");
    vpanel.add(hpanel);

    hpanel = new HorizontalPanel();
    hpanel.add(new Label("Password:"));
    hpanel.add(password = new PasswordTextBox());
    password.setName("j_password");
    vpanel.add(hpanel);

    vpanel.add(new Button("Login", new ClickListener() {
      public void onClick(Widget arg0) {
        String url = GWT.getHostPageBaseURL() + "protected/protectedText.txt";
        RequestBuilder rb = new RequestBuilder(RequestBuilder.GET, url);

        rb.setCallback(new RequestCallback() {
          public void onResponseReceived(Request request, Response response) {
            formPanel.submit();
          }

          public void onError(Request request, Throwable caught) {
            throw new UnsupportedOperationException("Not supported yet.");
          }
        });

        try {
          rb.send();
        } catch (RequestException ex) {
          // TODO present the error in a text area or something
        }
      }
    }));

    formPanel.addFormHandler(new FormHandler() {
      public void onSubmit(FormSubmitEvent event) {
      }

      public void onSubmitComplete(FormSubmitCompleteEvent event) {
        // TODO actually make sure the login succeeded. Present error in a text area
        // for the user to see
        Window.Location.assign(GWT.getHostPageBaseURL() + "/protected/protectedText.txt");
      }
    });

    formPanel.add(vpanel);
    RootPanel.get().add(formPanel);
}
}

Java2html

Upcoming

2008-07-09T10:45:00.000-04:00

The blog has been very silent but I have quite a few articles in the queue coming up. Upcoming on this blog are:

j_security_check + GWT
NetBeans exception reporting
push vs poll (or how much do you love Grizzly?)
Folders in a database without using recursion
Postfix + Recipient address rejected from Glassfish/JavaMail

I'm so busy I don't know when I'll have time to write any of these but hopefully they will be up before long.

Just How Bad is Windows CE?

2008-06-11T09:34:00.001-04:00

Lately, I've been developing concurrently on the iPhone and Windows Mobile. The .Net Compact framework is actually pretty good compared to the iPhone SDK but the user interface of Windows Mobile is just so very, very awful. I can't think of anyone who has tried absolutely anything else and like this. I can't even think of anyone liking this period. Considering how good a job Microsoft is doing with its general and server operating systems, how is it possible that they are even willing to release this product in the first place? Considering the quality and usability of the desktop and server systems that Microsoft puts out, Windows Mobile isn't even worthy of calling itself Windows. Sure the underlying kernel and technologies are pretty solid but it doesn't change the fact that the user interface is just crammed for such a small screen. I was using this the other day just to make a call and kept thinking to myself how many people I know wouldn't even be able to answer or make a call with this thing.

The real problem with Windows Mobile is that it isn't trying to be a really good phone, it's trying to be a pocket PC which not so coincidentally was its original brand. It just doesn't have the screen real estate to pull that off.

At WWDC 2008, Apple announced 3G iPhones that are actually cheaper than the current models. This is bad news for every smart phone vendor out there. I see really two segments now, the people who just want a basic phone, and the people who want an iPhone. The iPhone is really just that disruptive of a technology. I can't explain in words just how horrible it is to start using another smart phone again. Furthermore, it's the only phone that I have encountered that makes mobile web browsing viable.

So what is everybody else doing? Well, you have things like Android and the Samsung Instinct which are clearly heavily influenced by the iPhone but neither are yet available. The iPhone 3G, with its 7 Mbps Internet access, goes on sale early July of this year.

All is not lost for other vendors however. Let's face it, even with its new focus on the enterprise with the iPhone, Apple is really not very good at dealing with businesses and developers at this scale. Those users won't put up with half the things the regular Apple fan boy puts up with. If you're in the iPhone developer program right now, you probably don't like Apple very much. The things they make you do to actually deploy an application to real hardware is pretty obscene. At this point, I think it's Apple's market to lose. We'll see if Apple will learn a bit of humility here and open up this platform instead of imposing their will on everything closely related to the iPhone. This is really where the door is open for something like Android to come in and really give iPhone a run for its money.

We'll only know for sure over time but Google, now more than ever, needs to get real hardware out into the wild and at the very least refresh the Android public SDK. The Android SDK drop is still the one from last March and, thanks to controlled leaks, doesn't look or feel anything like the Android of today.

So just how bad is Windows CE? Well, it's so bad I don't even think it's a contender anymore. Let's face it, convergence is still ever creeping in. In a few years, your primary computer will probably be your phone that you cradle to get a full sized screen and keyboard. This is already happening, just have a look at the Celio REDFLY. I would think this makes Microsoft pretty nervous.

The BlackBerry Curve

2008-05-06T10:50:00.000-04:00

I have spent quite a bit of time dissecting this phone lately. The BlackBerry is certainly quite popular with some estimating its market share at 41% of all smart phones. On a recent trip to Pennsylvania, I saw many, many phones on planes and at the airports and I can certainly confirm that ratio. I counted close to 400 phones, half of which were some model of BlackBerry.

I have to admit, so far, this is the most impressive Java ME/CLDC capable phone found I have tried. Not only are many optional CLDC packages available but many more BlackBerry Java packages are present. That being said, Research in Motion (RIM), implements custom GUI packages that are specific to its platform. What that means is that if you code an application that follows the look and feel of a BlackBerry application, it will not function on any other phone.

I have to say however that this device is too expensive. At a suggested retail price upward of $400, it only features 64 MB of memory and an empty micro-SD card slot. A similarly priced iPhone comes with 128 MB of SDRAM and between 8 and 16 GB of flash memory. The iPhone screen supports touch and it's twice the vertical size of the Curve's screen.

The iPhone comes with Wi-Fi support too. Which means you can enjoy much faster networking on any hot spot and potentially save on those outrageously expensive over the air rates for bandwidth too. In Canada, MB per dollar access is ridiculously expensive. Rogers, for example, charges $10 per MB outside a plan. If you have a plan, prices range from $25 for 4MB per month to $100 for 100MB per month. Just disgusting if you ask me. Getting back on track here, you have expensive bandwidth and no memory to implement any kind of good caching on the phone. The only strategy left is to implement watered down services. Consider the browser on the BlackBerry for example, it has no JavaScript support and it mostly ignores any image properties found on a page in order to keep bandwidth consumption at a minimum. Sure, you can access the web but it doesn't mean it's any good.

RIM also develops probably one of the worst integrated development environments (IDE) you've ever seen. The JIDE is an all Java IDE that you use to code for the BlackBerry device. JIDE redefines what the word "ugly" means. Fortunately, RIM has started to develop a plugin for Eclipse and a plugin for Visual Studio. It's also possible to use NetBeans if you have enough energy to invest into it.

Looking at the BlackBerry, I have to wonder if the company suffers too much from the "not invented here" mentality and how much of that affects the price tag of its products. The company develops an IDE, customizes the licensed library and CLDC virtual machine from Sun, has its own executable format, implements its own content designers tools, etc, etc, all of which are very expensive. Talking to various colleagues these last few months, I am convinced that RIM must refresh its hardware in order to stay competitive. By refresh, I not only mean the power and visual appeal of their devices but the overall quality of the BlackBerry development platform or risk irrelevance.

JavaOne is this Week

2008-05-04T15:09:00.000-04:00

The Java front news has been dry for the last little while since everyone is holding off on announcing anything, instead waiting for the conference in the hopes of getting more press.

I'm really eager to see what will be made available this week. That being said, I'm hoping for major announcements on JavaFX from Sun. At the very least, I'm hoping to see some strong tooling support for content creation on that platform for graphic designers. Also, I'm expecting a new Java ME/CDC operating environment for phones based on the assets acquired from SavaJe and the PhoneME Advanced project. This new environment will feature JavaFX across the board.

Now, I still think Android is better than Java ME/CDC, but the CDC with all optional profiles is equivalent to Java SE 1.4 which is still pretty enticing. Furthermore, Java ME + JavaFX might go a long way to catching up with Android. By the way, PhoneME Advanced implements all the optional CDC profiles, so yes, it's equivalent to Java SE 1.4.

Mac OS X 7 Years Later

2008-05-01T08:14:00.000-04:00

Continuing my work in the mobile space, I recently had to purchase a Mac. By recently, I mean the day that Apple made their iPhone SDK public. I like the iPhone, it's the device that has raised the bar, the device that has forced all other mobile manufacturers to rethink their own product lines. We should see some real advancements this year from mobile hardware, that it be in terms of screen quality, power and features from competing companies thanks to the iPhone.

If the iPhone and iPod Touch are neat little devices, I have to admit I'm not much of a fan of Mac OS X however. I remember being so excited when I got my copy of OSX 10.0.0. That excitement was promptly crushed when I saw it under perform on my lowly PowerPC G3. After that, I pretty much left the world of Macs behind until the iPhone SDK was publicly made available.

The first I notice is how little has actually changed since 2001. OSX today is pretty much the same OSX that I used in 2001. The performance is much better of course but pretty much the same nonetheless.

I don't like the inherent security policies that are setup after install time. For example, to install an application, you usually have to start an installation wizard to guide you through the process. At one point during the installation, you're usually presented with a privilege escalation dialog. Entering your credentials will allow the installation to proceed. Unfortunately, the same cannot be said about uninstalling. First of all, there is no global registry of installed applications, so there is no uninstall wizard. You just go to the applications folder and drag an application to the trash to uninstall it. That's fine, my point is that you don't actually need to escalate to drop your applications into the trash. That's just plain wrong. Furthermore, it's only fine if this is where the application installed all of its files in the first place. Some applications will also install software in more traditional UNIX locations. If you happen to install such a beast, you'll be stuck with those extra utilities or have to rely on using a custom uninstaller before doing a regular uninstall. These custom uninstallers are usually Bourne shell scripts.

Speaking of escalation. Apple may poke fun at Microsoft about UAC in their commercials, but let me tell you they have nothing to brag about. First of all, they have an escalation prompt too, just like UAC. However, a filtered administrator, instead of a single click on 'Continue', must enter his password every time or optionally cache his credentials for a limited amount of time. Both options are terribly insecure. First, because if a malicious piece of software on both these systems makes a dialog box that mimics the appearance of this dialog, you get two very different results. On Vista, you have given out a mouse click (by clicking on 'Continue'), while on OSX you have given out your escalation credentials. Yeah Apple, keep making jokes. Furthermore, caching of credentials? Really? So everything I execute for the next little bit is running escalated? Really? At this point, I think OSX is only secure because it's less of a target than Windows, that's it, nothing else. It really has nothing to brag about. You can say all you want about the underlying UNIX security model found in OSX but if the user interface handles security this way, it's all for nothing.

I officially don't like Objective C. You can talk about the virtues of your favorite programming language all you want or not at all but it doesn't change the real reason some languages are popular and some aren't, in a word taste. Considering mostly all languages these days are Turing complete and object oriented, what a language looks like on screen is pretty much the only differentiator. When people like something, they have a reaction to it and they like it. Same thing when they don't like it. It's the same really for programming languages, when you look at a piece of code on screen, something in you reacts and decides if you like it or not, it really is that simple. When the dust settled, most people decided they didn't like how Objective C felt and used something else. Well, I'm now part of that group too except I can't use something else on the iPhone. I don't completely dislike it, e.g., I like the big class library and I like being able to retrieve the constructor of a class with nothing else but a string that represents the class name, I do, I really do... but that's about it really. The overly verbose method calls, the unified diff challenged syntax, i.e., '+' for static methods and '-' for instance methods is just plain evil when you're reviewing patches.

Using memory pools manually is also really just annoying. What is this? C or Objective C? I'm still not clear on how to correctly use these things across threads on the iPhone. I'm aware that the full OSX supports garbage collection now.

What I really despise however, what turned me off to Mac OS Classic and still does it for me today is the workbook style windows found in applications. If you've never used Mac OS, early versions of Visual Basic and Delphi suffered from this problem on Windows too. You have a single application but instead of being contained into a single master window, you have many little windows floating around everywhere. If you have multiple applications opened at the same time, you can see the windows of other applications within the holes of your current application. Most annoying is that there doesn't seem to be a standardized shortcut for quickly moving focus around between the windows of a single application. This makes use of Xcode really annoying. If I just happened to miss this keyboard shortcut, please, for the love of any deity you worship, please leave me a comment and tell me what it is.

Of course, it's not all bad. One very good feature of this OS is the startup and shutdown times. They both put Windows and Linux to shame.

The UNIX command line is a very nice touch too. I have to admit, as much as I like Vista, I don't think I could use it, or any other version of Windows for that matter, if it weren't for Cygwin.

Google's App Engine

2008-04-16T08:20:00.000-04:00

Seriously, what happened? Python? I don't mean to offend anyone but why come up with this nice, huge, not to mention free, application server if all you're going to support is Python?

Now, I'm sure they are tons of people who like Python, I just happen to not be one of them. I don't like languages that are brace challenged. What's really annoying is that Google has been pushing its Web Toolkit(GWT) this last year, a toolkit that lets you write Java code for the client that gets compiled down to JavaScript so that is can execute on the client. Why is this not supported?

That being said, since GWT compiles down to static HTML and JavaScript files, it's still possible to use the client side, compiled product from Google's App Engine. The server end of the application would have to be written in Python. Furthermore, the RPC mechanism provided by GWT would need to be dismissed in favor of REST or JSON for communications.

All this to say that the top four requested features on the Google Code site for this project are support for other languages. I can understand Google not supporting Ruby, PHP and Perl. Google doesn't have any significant history with any of these languages. Java on the other hand? I just don't get it.

NetBeans 6.1: The Best just got Better

2008-03-19T08:34:00.000-04:00

When it comes to IDEs, I've been around. I've used Visual C++ since version 1.5. Tried other environments over the years, but Visual Studio was by far the best no matter what other people claimed.

I started using Eclipse since that's what everyone around me was using for some Java work and I hated it. I'll reserve my thoughts on that for a future blog post but it was never a comfortable fit for me. Needless to say, I've converted my entourage from Eclipse to NetBeans now.

Version 5.5.1 of NetBeans was pretty good but the editor kept, well, I'm not quite sure on how to describe this, but it kept being wrong... or feeling wrong at least. Not sure if that makes any sense but that's the best I've got to offer.

NetBeans 6 was simply fantastic. It was just great. Probably one of the best IDEs I've ever used. That being said, it had a few shortcomings that were really annoying. On huge files, code completion was somewhat slow. Not that slow, but code completion is one of those features where you must literally work at the speed of thought of the developer. Otherwise, you break the flow of the developer. Startup time was annoying but manageable.

NetBeans 6.1 has really been great for me, so great that I've ditched 6 for the 6.1 beta. They are several reasons for this. First, it seems to me like this thing has had a lot more debugging time than a fully patched 6.0.1 release has. So it just works even if it's just a beta. That being said, a small caveat here is that I shun JavaScript work in favor of GWT. So I don't know how the rewrite has impacted that editor.

However, it's the little things that make the difference. For example, my favorite new feature by far is the default location of the Glassfish domain. Silly? Maybe, but I use Vista across the board. It has always been considered impolite under Windows to write to "Program Files" after installation has completed. Starting with Vista however, that folder has now been locked down. Meaning that with NetBeans 6, the default developer domain lived under "Program Files\Glassfish\domains\domain0". Under Vista, you had to either unregister your Glassfish server from the IDE and re-register it again with a more polite location, or you had to go into the Glassfish folder in "Program Files" and grant yourself write privileges to the domain's folder in order to support all scenarios. NetBeans 6.1 under Vista, and hopefully under any other Windows version too, creates the default domain inside your home directory. I think I almost cried when I saw that.

That being said, as much as I love this IDE now, ending a 14 year old run on Visual Studio/C++, I still have my pet peeves. In no particular order:

it's still slow to startup even it's a lot faster than it used to be.
I hate the fact that they are no project grouping feature similar to the solution concept in Visual Studio. NetBeans does actually have a project group feature but I just hate it. I hate it because it doesn't live on the file system, hence I can't commit it to source control. This means sharing project groups between team members is really annoying. I hate it because it clutters up the UI. The idea of a master project I find is also broken. Simply because you don't know right off the bat which of the project in a hierarchy is the master. I'm hoping that full support for Maven in NetBeans 7 will take care of this.
I use this IDE for pretty much everything now except for Windows Mobile and Android development. Which means I use it for C++ too. I don't use C++ that much anymore but it does come in handy. It works really well under Windows with cygwin, I even have Microsoft's Platform SDK hooked into this thing. Again, works pretty well. That being said, g++ is the only supported compiler under Windows. Considering the Visual C++ compilers ship with the Platform SDK now, I would like to be able to just use those instead of g++.
I would like to be able to run multiple instances of the IDE at the same time. The scenario here is when I want to debug both the client and server end of an application at the same time.
Better GWT support. Matisse is great but with all these new languages and toolkits like JRuby, Groovy and GWT coming into the Java ecosystem now, isn't it about time this thing started to learn new tricks? Look at SharpDevelop for instance. It has a visual form designer for all the languages it supports.
Message queues in the services tab. I really like the services tab. I really like the fact that databases are showing up there too now with this release. The Java DB menu item looked out of place in the tools menu in 6. Support for stand alone instances of JMS brokers would be a welcomed addition. I use Glassfish a lot, but I also use ActiveMQ outside of any Java EE container a lot too. Why? Because of the additional interoperability features it provides. Very handy to have around. And yes, before anyone comments on this, I know about the STOMP adapter from CodeHaus could probably be installed on OpenMQ.
Support for Groovy, Groovy and Groovy.
Lastly, I would like to see a Geronimo server plugin. Yeah, I know this a weird one considering I'm all Glassfish. However, I have one project where Geronimo was the best compromise I could reach. Maybe this is selfish but in my defense, I would probably have started coding this if it hadn't been for this thread here.

All in all, I'm pretty happy. I won't be switching back to Visual Studio anytime soon. The only real problem I have with NetBeans is that since everything is in the open and announced upfront, I find I'm always eager to get my hands on the next release, never truly satisfied with the current one. I would imagine however that's a good problem for NetBeans to have.

TopLink Lives

2008-03-17T20:40:00.000-04:00

It was my hope that the union of BEA and Oracle would lead to the enventual demise of TopLink in favor of KODO(Apache OpenJPA). Obviously, Oracle being and enterprise player, any phase out plan would be over many, many years. For example, I think Oracle will move their customers to the vastly superior WebLogic over their own Java EE offering but will still probably continue to offer and develop OC4J for many years to come. However, when it comes to a Java EE server, they are many parts. Each part could conceivably be marketed and sold individually.

Of course, things were never that simple. The core of TopLink, TopLink Essentials, is the JPA version 1 reference implementation, is open source and ships with Sun's Glassfish server which also happens to be the Java EE 5 reference implementation.

Futhermore, Oracle donated not just Essentials but the full TopLink product a while back to the Eclipse foundation under, curiously enough, under a different, Eclipse compatible, license.

Today comes the announcement that TopLink Essentials is indeed being phased out but TopLink is moving up in the world. The full TopLink, the one donated to Eclipse, is now EclipseLink. EclipseLink has been chosen as the JPA version 2 reference implementation and will ship with the next version of Glassfish, also know as the Java EE 6 reference implementation.

All in all, this is good news nonetheless. The double donations, under different licenses no less, being consolidated in one code base is certainly less confusing. The Eclipse license, from EclipseLink, is also more permissive than the CDDL/GPL from TopLink Essentials. In fact, so permissive that the Apache Software Foundation considers it just fine too. For example, the Java compiler from Apache Harmony is actually the Eclipse Java compiler.

Now, if we can just get comprehensible error messsages from future releases, all should be good. That being said, I've been using TopLink Essentials more and more, and after a while, you get into this weird Zen state and actually start to recognize and understand the error messages. When you do, it actually works pretty well. That being said, I still think those error messages are nuts.

I'm Tired of Closing Files

2008-02-26T13:18:00.000-05:00

Not long ago, color commentator Bruce Eckel shocked the Java world with his article entitled: "Java: Evolutionary Dead End".

The press went wild stating that Java was doomed since one of its biggest supporters was now shunning it. If you actually read the article, this doesn't call for the end of Java, just slowing down or completely stopping the adoption of new features to the language akin to what C and C++ are today. C and C++ are governed by slow ISO committees that make a new language specification every ten years or so. Instead, he argues, hit the reset button and make a new language on top of the JVM. Certainly, the JVM is no longer a stranger to non-Java languages, both Groovy and JRuby seem to be carving a niche for themselves.

Furthermore, the Java virtual machine, at least the one from Sun, is constantly improving both in terms of speed and memory consumption. The version 6 virtual machine of HotSpot from Sun is nothing short of spectacular. New language, same great infrastructure, sounds perfectly reasonable doesn't it?

Now, why would he choose such a tittle? I'm familiar enough with this author, I probably still have the 1st edition of Thinking in C++ somewhere, to know that it was probably on purpose. He knew the attention he would get with this kind of title than with one that more accurately portrayed the contents of his article. Nonetheless, it made a lot of people think about the core issue, should Java be frozen or not?

I've been thinking about that lately and I've come to the conclusion that I'm tired of closing files or any other type of resources for that matter. I think it's reasonable to freeze Java (the language) but only after it has indeed reached adulthood. You can talk about Java 7's closures, asynchronous I/O and super packages all you want but it doesn't change the fact that I'm still sick of closing files.

With a garbage collector present, it is no longer clear when the destructor/finalizer of a class will execute. If I were to write a program that operates on files, I would have to manually close any file object created or risk running out of file descriptors depending on the operating system I am running on. The same can be said for any object that holds kernel level resources. Coming from C++, this is something I no longer needed to care about, why should I care about it in Java?

C# does something interesting here, any object that implements the IDisposable interface can be used inside a "using" statement. This means that at the end of the "using" block, the dispose method is automatically called. Something like this:


using System;
using System.IO;

class Program {
 static void Main(string[] args) {
      using (Stream stream = File.Open("foo", FileMode.Open)) {
          // Operate on stream to foo
      }
 }
}

The problem I have with that is that it's still bound to one variable per "using" block. So, to copy a file to another, I need at least two "using" blocks.

In C++, the resource initialization is acquisition idiom is increasingly popular, made even more popular by the use of smart pointers, so assuming the use of shared_ptr here:

if (shared_ptr<x> p = someManager.get(id)) {
 // operate on p
} else {
 // p is also not in scope here
}
// p is no longer visible

All this to say, that the equivalent construct is missing in Java. Not only, but I don't like the C# solution as it's possible to go further.

If you have multiple "using" (or "if") blocks, why can't we just make a list? In other words, if I can acquire this list of resources, then execute the block otherwise don't? For example:

// Fake Java code sample
using (FileChannel r = new FileInputStream(path).getChannel(),
 WritableByteChannel w =
       Channels.newChannel(response.getOutputStream())) {

 r.transferTo(0, length, w);
} finally {
// handle failure
}

Instead, I have to do something like:

// might throw
FileChannel fc = new FileInputStream(file).getChannel();
WritableByteChannel w =
   Channels.newChannel(response.getOutputStream());

if (w != null)
    fc.transferTo(0, file.length(), w);

// null or not, w is still in scope
// Don't really have to close, but waiting for GC might be fatal later on
fc.close();

I write a lot of Java code now and I'm really enjoying it, I wouldn't go back to C++ at this point. C++ is only the last resort language now, the language that makes things work if it doesn't anywhere else. So, when I hear things about freezing Java, I think about things like this. I wonder how much time Mr. Eckel spends writing code versus teaching it. As an educator, I can understand the importance of a non-moving target, but in the field, I need relief for some common idioms. Saying let's do a new language won't make all the tooling that's available today for Java suddenly appear on that language. For example, I really like Groovy but I don't use it. The IDE support in NetBeans just isn't good enough for me to work with it.

BDJ Revisited

2008-02-26T09:25:00.000-05:00

My last blog entry talked about Blu-Ray Java a.k.a. BDJ. Hanging out in the groups however, not all is well in the land of BDJ.

The first problem is fragmentation. BDJ is really young and the fact that fragmentation is already a problem is something I find completely unacceptable. Fragmentation means that either the reliability or availability of the code that ships with the device varies enough that you have to change your application code to work around bugs or missing features. In layman's term, it means you need to ship different SKUs and that costs money. Money to package, money to validate and money to distribute. Java ME fragmentation has really gotten out of control these last few years. The Transformers movie tie-in game is one of the biggest Java ME project ever undertaken. In all, some 25,000 SKUs shipped in order to support all the phones out there. Yes, that's 25,000, twenty five thousand SKUs. I can only imagine what the cost of getting that game to run on all these handsets was. No horror story of this magnitude in the land of BDJ yet, however the "Chicken Little" movie application apparently had 30 different variants in order to run.

Not all is lost however. Unlike Java ME (CLDC or CDC variants) on phones. The Java ME/CDC stack that is shipped with BDJ is upgradeable. If your device is networked, it can phone home and check for upgrades. If your player isn't networked, upgrades can still ship with a movie disk. I'm not sure what the experience is for the end user considering most users won't even understand what this is upgrading in the first place but let's hope it's very transparent.

For the moment anyway, this needs to be dealt with. Content authors are overwhelmingly choosing HDMV for their content over BDJ. Only 30 of the 600 or so Blu-Ray movies that have shipped, i.e., roughly 5%, use BDJ.

It could be just familiarity with HDMV since these authors have been using it for quite some time on regular non-HD disks, or could it be the complete lack of tools? This being the second problem. I've blogged before about JavaFX. JavaFX is many things, but what it really is a codeword to right all the past wrongs of Java on the client. So, even if we haven't seen any JavaFX authoring tools for designers, Sun has confirmed that it's working on components for Adobe products to output JavaFX scripts. Now that could be something that is enticing for content authors. The HDMV tools don't even come close to the Adobe design tools.

Finally, my last gripe is why did anyone even bother with Java ME in the first place for these devices? For crying out loud, these devices can decode and output 1080p video. I think they can handle the full Java SE platform. Considering that Sun has announced their intention to eventually phase out Java ME over the next decade to converge with Java SE, did they really have to add another few hundred million devices as part of that phase out plan?

HD-DVD is Dead. Where's the SDK for BDJ again?

2008-02-19T08:31:00.000-05:00

So, it finally hit the fan. Toshiba has ended its HD-DVD business. This obviously impacts consumers since most people have wisely decided to stay away from this market until one format won out against the other.

What consumers don't know about these mortal enemies is that this war was being fought on two fronts. The other front being software, mainly codecs and platforms. Unbeknownst to the regular DVD Joe and Sally, all Blu-Ray players are shipped with Java ME.

Considering the horror that is Java ME on phones, you'll be happy to know that the Java ME that ships on Blu-Ray devices is the CDC profile. The CDC profile has considerably more features than the CLDC one that usually ships on phones but still falls short of .Net Compact and Java SE. Unfortunately, the CDC still has many optional packages too which means that fragmentation card is still in the deck. I'm not sure anymore if .Net Compact was shipping on HD-DVD players, information on the subject is very scarce but my belief is that it was. I know for a fact that .Net Compact ships on the X-box 360.

Officially, the Java that ships on Blu-Ray is called BDJ and they are basically two profiles, one that offers networking and the other one does not. So, for example, the Java that ships with the PS3 is BDJ with networking. Well, all these features are basically meant to offer a richer interactive experience on BR disks. However, with the rise of casual gaming and the presence of USB ports on most of these devices, I wouldn't be too surprised if some games started coming down the pipe using this technology. The problem so far has been about getting an SDK. In other words, BDJ has not been widely available since the movie studios aren't sure if they want developers writing code for these devices. The fear is that these SDKs will be used for cracking the copy protection of disks.

Well, I don't know if this coincidental or not, but we are finally starting to see BDJ related bits being made publicly available. You can see a introduction to BDJ from its main architect here. The HD cookbook project is also available from java.net.

So what does this all mean for consumers and developers? Well, if you bought an HD-DVD drive, welcome to the world of betamax. If you bought a Blu-Ray player, congratulations, you'll be able to get content for many years to come. For the Java developer, it means yet another way to leverage your existing skill set. With potentially hundreds of millions of Blu-Ray players that will be sold in the next few years, you have yet another avenue to market applications. For the .Net Compact developer, you still have a small, but sturdy niche of Windows CE based smart phones that you can still code for but that's about it.

No More Heroes Again

2008-02-17T08:46:00.001-05:00

I just finished the game. This is what fun is all about. In any case, I want a sequel. So go buy it. If you don't own a Wii, there's never been a better excuse.

In any case, if you have played the game and are wondering what Jeane was talking about at the end, it's available from YouTube. No wonder it was fast forwarded in the game, that "M" rating would have turned to an "AO" one real easy. If you haven't played it yet, don't watch it, it spoils pretty much the ending.

The magic of the tablet

2008-01-28T20:33:00.000-05:00

This blog post is special, it was all written by hand. That is, it was written using a Tablet PC running under Vista.

You know, people all around me are always complaining about how bad my penmanship is, but this thing is pretty good at figuring out the words that I'm writing.

So kudos to Microsoft how figuring out my bad writing. That being said, there's one slight problem. You know those toolkits that have custom widgets that do their own drawing? For example, Qt, Java swing and Mozilla based products to name a few, well, they don't work.

The problem is that the handwriting controls are tied to the native window class. So, if you have a window that has a native class of "text box", then Windows knows to pop up the handwriting widget button. When doing custom painting with widgets, the native Win32 class for all windows is, well, "window". So Windows can't tell that it's supposed to put up the handwriting pop up.

Too bad, Sun is really trying to position Java for rich client Internet apps. Is this important? Maybe, maybe not, but this is definitely something that the .Net Framework can do. Sun now has a lot of people working in Redmond to make Java run on Windows better. Well, this is one item that they can put on their todo list.

No More Heroes

2008-01-23T08:38:00.000-05:00

This blog is usually about technology but every once in a while, a game pisses me off and I write bad things about it here, i.e., Resident Evil: The Umbrella Chronicles.

Even more rarely, I get to experience a real gem and it does take quite a lot for me to actually write something about it. After all, a lot of good games came out in 2007 and I didn't bother to write anything about the majority, if not all, of them.

No More Heroes, just released exclusively for the Wii, is such a gem. Do yourself a favor gamers, pick up a copy. You can see a video review here.

Use Eggs to Defeat the Final Boss in RE4 Wii Edition

2008-01-21T08:36:00.000-05:00

Oracle Buys BEA

2008-01-20T09:24:00.000-05:00

If you haven't heard by now, Oracle has just acquired BEA for a lot of money. Here's what I'm hoping will come out of the deal, or at least some predictions:

TopLink dies a horrible death replaced by KODO.
TopLink Essentials dies a horrible death replaced by KODO core (Apache OpenJPA).
Oracle OC4J still doesn't do Java EE 5, it's still stuck on Java EE 1.4. It has almost no market share and BEA WebLogic has something like a 33% market share of application servers. Not sure what will happen here...
JRockit will hopefully die a not so horrible death. JRockit is a full Java SE implementation but it's actually licensed from Sun with some custom patches applied that apparently improve performance. Enough already, if those custom enhancements are really worthwhile, just contribute them to the OpenJDK project already and move on.
IDEs? Don't know what will happen there, both companies have their own Java IDE which have both failed to gain any significant momentum. To me, 0 + 0 is still 0. Here, please note that I am talking about market share and not the quality of these products

KODO was acquired by BEA which donated the core bits to Apache which formed the OpenJPA project. OpenJPA is now used by WebSphere, Geronimo and WebLogic. Having OpenJPA standard in Glassfish would be nice but even if Oracle decides to move forward with KODO, it still doesn't mean Sun would decide to dump the TopLink Essentials code that Oracle contributed to Glassfish. Well, here's hoping, see my previous blog post on TopLink Essentials on why I think this matters.

Frustrated Android

2008-01-08T10:55:00.000-05:00

Well, I've been coding some mobile demo applications up recently. Not much really, just a message of the day application on a server that offers SOAP and Restful services.

The idea is to showcase a mobile client for each of the major toolkits out there i.e., Java ME (CDC and CLDC), .Net Compact and Android.

Well, it has been a lot tougher than I thought, every one of the platforms had hurdles except .Net Compact. In Java ME, both editions didn't have anything to parse XML. No big deal, had to pick up a 3rd party open source library but it's a good thing the server had Rest and not just a SOAP API. Microsoft .Net Compact was just a charm to work with. Really no problems. In all, it took 1 hour for the .Net client, 4 hours for the CDC client and about 6 hours for the CLDC client.

Enter Android, I was really eager to start work on this client. Java 5 syntax. Most of the Java 5 platform classes. What could possibly go wrong?

Well, the first problem is the IDE. NetBeans isn't supported. You can say what you want about the NetBeans vs Eclipse debate but NetBeans 6 is a killer application. Going back to Eclipse hurts.

The second problem is the complexity of the user interface building blocks. What I mean by that is that you can't just have a combo box and fill in some data for example, you have to setup an adapter that wraps around your data and plug that into the combo box. I'm all for flexibility but too often this is overkill. Not enough common adapters are provided in the platform. I think this is a real issue. Look at Qt 4, it suffered from the same problems, great flexibility in its models, but to do even the simplest of task meant huge amounts of coding. Trolltech introduced common model classes in subsequent releases of Qt 4.x to address this. I really think Google needs to do the same here too, just too much of a pain to fill in widgets when you don't need flexibility.

Where do I report bugs? Because I encountered a lot. No public issue tracking system is available.

The documentation has issues too, most of the time I just had to guest what a return value of an adapter method had to do. Also, I can't find anything recommendations on how to use threads on this platform or should they all just be shunned in favor of Android services?

All in all, it was a very painful process. I wanted this to be my favorite platform and I ended up cursing at it. At this point, I've spent two days on the Android client and I'm done for now. I'll pick this up again when Google refreshes their SDK.

Now that the dust has settled, I find it a bit unsettling that .Net Compact 3.5 was by far the best mobile platform. Competition with .Net has really made Java SE and EE superior products, but where are the Java ME improvements? What's really annoying too is that my .Net client can run as a rich client and Silverlight application without modification, just different build settings. I know, I tried, it works.

Java vs Java

2007-12-21T08:18:00.000-05:00

More specifically the Java language versus the Java platform. I have been playing around with some non-Ajax technologies for rich web clients. I think creating Ajax sites by hand is nuts. Thankfully, we have GWT, RAP and Visual Studio 2008 seems to have Ajax support directly in its palette. At the same time, it's clear that some very powerful forces are converging on Ajax. Namely Flash, Silverlight and JavaFX.

These 3 technologies do something that Ajax can't really do, or at least not very well. They bridge the gap between developers and designers. For example, if you are working in Visual Studio on your client application using Silverlight, the designer can open your Visual Studio project and edit the XAML form from Expression Blend. That's right, the Microsoft designer tools use Visual Studio projects as their project files.

I don't want to spend too much time on this, but you can script Silverlight in any language that the DLR supports. You can code Silverlight in any language that the CLR supports. You can code in Silverlight using C#, IronPython or JavaScript for example. Silverlight is really just a .Net runtime that runs as a browser plugin that depends on WPF to do all its shiny effects. These explanations are bit crude but I want to get to JavaFX sometime this year.

JavaFX is the Flash, Silverlight competitor from Sun. JavaFX is also a completely new scripting language i.e., it's not XML based and I have to ask why? Did Sun forget again that Java is a platform and not just a language? Why can't JavaFX be just like XAML? Forget the new language and just allow us to code in any language supported by the Java platform. I've seen some JavaFX scripting code and I don't care for it. I do not want this. I just want the presentation part. I want the code behind the presentation to be Groovy or any language of my choice that runs on the JVM.

I've talked before about how Matisse, the NetBeans GUI builder, only outputs Java code. The NetBeans JSP page editor, I'm not sure if that is also Matisse or not, suffers from the same problem. Any tooling made available for JavaFX will most likely have the same flaw.

At this point, I just wish I could say I don't care about JavaFX, I really wish I could. Unfortunately for me and fortunately for JavaFX, Sun has committed to bring JavaFX to Java ME, even the puny CLDC profile. With over a billion phones out there running the CLDC profile, it's sort of hard to ignore this. Java ME is a hard place to develop, you have the CLDC, CLDC/MIDP and CDC profiles. All profiles have different versions with tons of packages being optional and not to mention vendor specific packages. Anything JavaFX can do to help in that space would be appreciated.

In the meantime, JavaFX is far from version 1 and while Sun toils away at its new scripting language, you have to ask where are the designer tools? The tools that will actually compete with Microsoft Expression and Adobe Flex.

Compilers no Longer a Threat?

2007-12-16T09:01:00.000-05:00

This all started innocently. I've been using the excellent version 6 of NetBeans as my primary IDE for a while now. Have been doing so since the beta releases. Since Java is now my primary platform. I use it for Java, Groovy, JRuby and even C++. I also have the Microsoft Platform SDK hooked in so I can pretty much do all my Win32 C++ work from there also.

The NetBeans project claims NetBeans is the only IDE you need. Well, not quite, if you're looking into .Net for example, NetBeans is a dud. Not that you couldn't write plugins to support it but that seems unlikely. I was really interested in looking into .Net, well .Net Compact more precisely.

I have no commercial interests in .Net, I like my solutions, all jokes aside, to be portable. The compact edition of the framework however is something I find truly irresistible compared to the Java ME headache.

If you want to test drive .Net, you're in luck, you can just download a copy of Visual Studio Express from here and incur away. I've been having quite a bit of fun with Visual C# Express 2008. If you want to try .Net Compact however, well the express editions don't do compact. What to do? Both Visual Studio 2005 and 2008 offer a trial version available from here and here respectively and this is what I used initially.

Both these products have restricted licenses and I started looking for an IDE project that would work directly on the top of the .Net SDK. I wasn't expecting much really but I was simply shocked when I found out about SharpDevelop.

A very, very good clone of the Visual Studio IDE. It doesn't support C++ but it supports Boo, Visual Basic, C# and has a new IronPython plugin. This IDE is really good. It could even teach NetBeans a few tricks or two. For example, the visual form designer in SharpDevelop works with all the supported languages, not just C#. Compared this to Project Matisse who only works with Java, and according to this blog entry at least, getting it to work on other languages will be difficult.

What does all of this have to do with this blog entry? Well, when I was initially surveying the SharpDevelop web site, the requirements for the product caught my eye. Namely, SharpDevelop doesn't require the .Net SDK, only the .Net runtime.

OK, since SharpDevelop works out of the box with Boo on Windows, then we can conjecture that SharpDevelop ships a copy of Boo. Correct.

OK, since SharpDevelop works out of the box with C# and Visual Basic on Windows, then we can conjecture that SharpDevelop ships a copy of the C# and VB compilers from Mono. Incorrect.

Process Explorer put this to bed very quickly. SharpDevelop was using "csc.exe" for compiling. I never installed the .Net SDK on this machine, where did it come from? Well, it turns out that, just like Java, the compiler is available from .Net as a service. Unlike Java however, the compiler executables for C# and VB actually ship with the .Net runtime. If you look in "C:\Windows\Microsoft.Net", you'll find that you have a folder in there for each version of the .Net runtime you have installed. You'll also find compilers for C# and VB inside those folders. This means that on a fully patched Windows Vista system running .Net 3.5, a user has 6 compilers installed, a C# compiler version 1, 2 and 3 and VB compiler version 7, 8 and 9.

For an old UNIX war horse like me, it's tough to see all these compilers. A long time ago, it was common practice to make sure all compilers were disabled on UNIX servers. So, if you had an attacker using say FreeBSD version 2 attacking a server running IRIX or HPUX, the attacker could not upload a C source of a password cracker to the system, compile it and run it. It was uncommon that an attacker would go to the trouble of setting up a cross compiler just to attack a server.

The popularity of Linux and/or Perl "fixed" that problem. Both systems now just run Linux on Intel and at one point, most UNIX systems started shipping Perl out of the box around version 4 of the interpreter. Disabling the compiler isn't as important has it used to be. Using the noexec bit on the file system is obviously more important than ever.

We can make the same assertion about Windows, someone can easily compile from their system and attack the target system. Still, I feel the only reason why all these compilers aren't being used for more mischief is because they are much more easier alternatives. Now, is this just me? Am I just an old dinosaur in the age of .Net? Am I the only one that finds the presence of these compilers on ordinary non-developer boxes unsettling?

TopLink Error Messages

2007-12-10T12:30:00.000-05:00

Whoever is responsible for the error messages found in Oracle's TopLink product is one sick individual.

TopLink Essential is a Java EE 5 persistence provider. It was contributed by Oracle to Glassfish and serves as the reference implementation for JPA. If you're a Glassfish user, do yourself a favor and setup another JPA provider.

Following these instructions, getting Apache OpenJPA to run inside Glassfish takes about 2 minutes. You can also use Hibernate if you so wish by following these instructions.

Glassfish JDBC Realm Authentication

2007-12-10T11:43:00.001-05:00

Judging by the numbers of posters on the Glassfish and NetBeans users list that are having problems with this, I thought I would explain how I got Glassfish to play well with JDBC for authentication.

1. The first gotcha is the database driver. Most developers get their copy of Glassfish bundled with NetBeans. Please note that even if the NetBeans IDE comes with drivers for Java DB, MySQL, PostgreSQL and the ODBC bridge, that is not the case for the bundled copy of Glassfish that ships with the IDE.

You will need to install the proper JDBC driver for your database in $GLASSFISH_HOME/domains/yourDomainNameHere/lib or in $GLASSFISH_HOME/lib if you want the drivers to be available from all domains. Additionally, you can also use the database drivers that come with NetBeans and just copy them over to Glassfish instead of hunting them down. The JDBC drivers installed with NetBeans 6 are found in "$NB_HOME/ide8/modules/ext".

2. If you do not have a database pool setup. You will need to define a database resource from Glassfish. You can do this from the admin console by going to "resources" -> "JDBC" -> "Connection Pool". Setup a pool using "javax.sql.DataSource". Complete the wizard and make sure you can "ping" the database.

3. Setup a new JDBC resource for the pool you selected in step 2. JDBC resources can be defined in "resources" -> "JDBC" -> "JDBC Resources". Select whatever JNDI name you want for the resource. Assign the JDBC pool you selected in step 2 above to the new resource.

4. You need to define a new realm for authentication. You can do this in "Configuration" -> "Security" -> "Realms". Assign a name for this realm, let's say "dbtest" for this example and select "com.sun.enterprise.security.auth.realm.jdbc.JDBCRealm" as the class name.

We now need to setup all the properties here for the new realm:

JNDI. Set the name you selected in step 3 above.
User Table. The table name in your database that has users
User Name Column. The column that represents the user name in the user table
User Name Column. The column that represents the column in the user table
Group Table. The group table.
Group Name Column. The column that represents the column in the group table.
Digest Algorithm. Glassfish will transform the clear text password using this algorithm before comparing it to the value in the database.

Gotcha #2 here is that database user/password fields are completely useless in this field, just leave them blank.

Gotcha #3 is that the group name column absolutely needs to be a string. So, for example, if you use numeric foreign keys in your database, you will need to create a view that maps user names to user groups by string. You then use this database view for the group table/group name column fields.

Gotcha #4 is that digest algorithm field is actually required even if this doesn't look to be the case from the web page. Furthermore, the "none" algorithm found in Tomcat for example doesn't exist here, you need to digest your passwords, you can specify any Java supported algorithm here, i.e., MD5 or SHA-256.

Finally, to test it out, just use the NetBeans tutorial here but change the realm name in the web.xml file of the application to the one you specified in step #4 above.

Krita

2007-12-10T11:25:00.000-05:00

Just a quick blurb on Krita. It has been a while since I checked in on this project. I was involved in Krita a while back. At that point, the project, Krayon at the time, had been abandoned. I started work on Krita because I wanted to learn the Qt, KDE and the KOffice libraries. From this perspective, the project was a success for me since I learned all these libraries while working on Krita.

Real life got in the way of my involvement as I went to work in digital signage for a few years. That being said, before I left, I was able to kick start a new round of developers on Krita by answering questions on the actual code base.

I don't want to make it sound like I did much but trying out Krita today, I'm happy to see that it has continued and thrived. Congrats to all the Krita developers.

Dynamic Languages for a New World

2007-12-07T12:49:00.000-05:00

Long ago, almost a lifetime now, I was a Perl developer. Perl 5 had just come out and I was a happy camper. Then, I started coding some very large programs, and I wasn't happy anymore. Moved away from Perl to C++ and pretty much stayed there for the last decade.

With my recent incursions in Java world and Java EE, I felt there was a lot of good technologies I could use in there but one part still bothered me, namely JSP and AJAX. JSP seems to be a capable technology but JSP and Ajax seems to be a marriage made in hell.

The prospect of writing HTML, CSS and JavaScript by hand, even with the help of toolkits like jMaki, is something that has always scared me so I kept on looking. I finally came across Google's web toolkit. I was happy again, full Java on the server and compile down Java to JavaScript on the client.

Now, I already knew about IronPython and JRuby, I decided to look around for a Perl for .Net and Java just to see what was available. Well, needless, to say I didn't find much except a discontinued Perl on .Net product from ActiveState and abandoned Perl on Java project. Signs of Perl's waning popularity I suppose.

I did come across a number of interpreters however, and I was quite surprised by the ample choice that was available, mainly:

IronPython. C# implementation of the Python language. Licensed under the MS Public License. Impressive but I never liked Python much, I'm too much of a braces ('{', '}') man myself.
IronRuby. C# implementation of the Ruby language. Also licensed under the MS Public License.
JRuby. Java implementation of the Ruby language. Triple licensed, available under the LGPL.
Jython. Java implementation of the Python language. Licensed under the Python Foundation License.

Of all these interpreters, Jython is the oldest and is the only one who is not employed by either Sun or Microsoft to work on his interpreter. Sun and Microsoft announced their support of JRuby and IronPython at around the same time, and Microsoft added support for IronRuby this year when it became clear that Ruby on Rails was going to make Ruby considerably more popular than Python.

Now, I tried all these languages and none of them felt right. All of them felt odd because of the importance of maintaining compatibility to their C counterpart. Is it a Ruby string or a Java string? A Python string or .Net one? To both questions, it depends on the context of what that module is importing.

I was a little bit discouraged. I felt Java everywhere except mobiles and C# on mobiles was just too monogamous. I like to spices things up but at the same time I didn't feel like moving away from GWT was a good move.

Well, it turns out that there are open source scripting languages being built natively on Java and .Net at Codehaus.org. Groovy for Java and Boo for .Net.

Both support support static typing instead of just duck typing. Now, I don't know much about Boo, I'm not looking to complement C# since I only use that on mobiles. I will say this however, Boo is supported by the very good SharpDevelop IDE, including visual form design and everything.

Groovy on the other hand has been a really good experience especially considering I use NetBeans and the Groovy plugin for that IDE is still incomplete. I get a first class citizen of the Java platform. At first, due to the fact that I wanted to use GWT, I was planning to limit my use of Groovy to testing code, i.e. write my JUnit stuff in Groovy. After all, I couldn't write Groovy code for GWT right? That things compiles Java down to JavaScript, it won't know what to do with my Groovy code right?

Enter Grails. Now, I can't for the life of me think of why anyone would use JRuby on Rails, i.e. using Rails from JRuby instead of Ruby, but I can see why a lot of Java developers would migrate from JSP/jMaki to Grails.

Grails follows the Rails philosophy of RAD web development but also features a GWT plugin. Unlike Rails, production deploys of Grails are done to a standard Java EE container using a WAR file.

Now, I can only imagine doing all my web services and back end stuff in Java, doing the web front end using Groovy and GWT, just very exciting times. Now, I'm still in the process of learning and using Grails and still fiddling to see how good GWT support is in Grails but exciting times nonetheless.

Ninja Gaiden Dragon Sword

2007-12-04T20:47:00.000-05:00

Went to the shopping mall today, I'm always excited when I go to mall, or at least this mall. This mall has a Toys "R" Us store with a Nintendo DS demo station. I'm always excited to see which demo I'll download today.

Well, today, I was right to be excited. I blogged before about the success of the DS would start to attract more established franchises, the result was only passable with Call of Duty 4, but this new Ninja Gaiden doesn't take any prisoners.

Wow! What kind I say, what an incredible game this is going to be. Incredible visuals, by far the best on the DS yet. Incredible audio. Incredible controls. Incredible action. Incredible everything. The only bad thing I have to say about the game is that I can't buy it, turns out it will only be released in February of 2008.

Java ME vs Android Take 2

2007-12-04T20:15:00.000-05:00

I have just completed my first ever .Net program. This program, as it turns out, is built using the .Net Compact framework.

All in all, I believe .Net has been very beneficial for Java. .Net lit a fire under Sun's behind and this helped propel Java, both SE and EE, to new heights in order to stay competitive. Unfortunately, seeing how much better .Net Compact is, it appears this statement doesn't hold true for Java ME.

The first problem is the lingo. Java ME comes in basically 3 flavors: CLDC, CLDC/MIDP and CDC. CLDC is puny and doesn't offer much, it currently runs and ships on over a billion phone. CLDC/MIDP is a version of the CLDC with additional packages to make the experience a little less dreadful. Finally, we have the CDC, the CDC is a modern Java ME that supports a feature set that is considerably richer and on par with .Net Compact and Android.

Now, if it would end there, one could easily ask what the point of Android was? Unfortunately, it doesn't end there, that would be too simple. Each of the above profile has different versions and all of them are still being actively developed. If that's not enough fragmentation for you, Sun doesn't actually provide Java for mobiles. Instead it licenses the code to carriers and these carriers port the profile to their operating system. Carriers can also ship other ME implementations, e.g., the one from IBM. Just to spice things up, most, if not all of these carriers add their own packages on top to compensate for missing features. It also means that without a standard distribution, unlike .Net Compact, updating the version of Java on a mobile is practically impossible because carriers don't usually bother publishing updates. If a phone is shipped with an old version of the MIDP, that's it, deal with it.

That being said, I missed the real killer benefit of Android the first time I blogged about it. All these profiles from Sun still use the Java 1.4 language. So you can forget about using generics or anything of the sort. I could not find anything about updating the Java dialect to 5 or 6 for Java ME. Android uses the Java 5 dialect. The other good thing about Android is that there is, at least for now, only one SDK, not one per mobile.

All is not lost for Java ME however. Sun is still busy toiling away at its own Java mobile environment called JavaFX. Sun is still hoping to converge Java ME to the full Java SE in the future, good deal.

The best hope for Java ME in the short term is that Sun has now GPL'ed Java ME and is maintaining ports of the distribution for Linux and Windows CE. The project doesn't seem that active but it will at least give a standard build of ME that developers can distribute to their customers directly. Unfortunately, this project also has no information about updating the Java dialect to 5 or 6.

I am now a big supported of the Java platform, i.e. Groovy, JRuby, Java SE 6 and Java EE 5. Until Java ME cleans up its house or until Android finds a market, I'll stick to .Net Compact thank you very much. I would not consider .Net for any type of project outside of mobiles due to, all jokes aside, the lack of portability, but I can't think of anything better for mobiles right now.

Latest Apple anti Vista campaign is retarded

2007-12-03T08:40:00.000-05:00

The latest advertisement from Apple promoting the benefits of a Mac versus PC states: "Ask not what Vista can do for you but what can you buy for Vista?". Saying that even a simple printer will not work and that you will need to buy a new one.

Well, not only will your printer work with Vista. You have to ask yourself what kind of backwards compatibility does OSX offer versus Vista.

When OSX showed up, its compatibility was simply disastrous. You had to run MacOS classic inside an emulator, a very slow and painful process. Not only did all the old software not work, but Apple excluded a huge number of models from the the OSX compatibility list since all but the absolute newest PowerPC models had enough horsepower to run the new OS.

But that was just the migration from MacOS classic to X right? Nope, on every point release (or is it non-free service pack?), Apple keeps on dropping older models with extreme prejudice.

Finally, what about all those people who bought a PowerPC based Mac instead of an Intel based on? Surely Apple will keep on releasing software for them right? Yeah, don't hold your breath.

The truth is, Apple may make fun of Microsoft for its compatibility woes with Vista but underneath it all, Apple customers are terribly jealous of the compatibility offered by Microsoft to its customers.

Red Rings of Light Again

2007-11-17T12:53:00.000-05:00

I got my xbox console, or should I say another used console, two days after sending it in via the good folks over at Purolator. New serial number and new manufacturing date. I also got a letter thanking me for my patience, a one month xbox live gold card and a letter explaining that is a "certified Microsoft refurbished console".

Unfortunately, I also got a 3 red rings of light within the very first half hour the damn thing was connected. I'm starting to wonder if Microsoft really understands what the problem with these consoles really are, or if the defect can't really be fixed at all without major hardware replacement.

In any case, I'll be sending back the replacement again, maybe I'll get another xbox gold card.

Google Android

2007-11-17T09:18:00.000-05:00

Google recently released a preliminary SDK for their mobile platform called Android for developers. Off the top of my head, looking at their technical documentation, I can't think of anything that is in Android that also isn't in Java ME (or .Net Compact for that matter). That being said, Java ME is an SDK only while Android is a full platform and it comes bundled with standard applications and user interface. Sun has already announced that they too are working on their own platform for mobiles called JavaFX.

Unless I'm mistaken here, the SDK for JavaFX will be Java ME and it too will come with its standard user interface and set of applications. I don't know what the motivation for developing JavaFX was, but I'm guessing Sun is trying to create a synergy similar to the one Microsoft enjoys between Windows Mobile and .Net Micro.

Now that Android has entered the picture, it's sort of hard to not question its existence. Let's face it, Java ME is already on millions of handset worldwide which makes Android seem irrelevant. On the other hand, Google has made Java on handsets look good and exciting again, I can't remember when I felt like that about Java ME.

Now that I'm writting this, Android does codify something I feel is really missing in Java and not just the micro edition, a standard XML syntax for user interfaces. Microsoft is really doing great work with its XAML technology. Not only does this technology have the potential to bridge the gap between web and desktop applications but also the gap between designers and developers. Android doesn't fill that gap obviously since even Google's other toolkits don't support this XML syntax, this is exciting nonetheless. I like GUI editors that generate XML, not when they generate code directly in a class somewhere. This has been, and remains Sun's stategy for Java. Fortunately, even if it's not included directly in Java, I can still use XAML in Java using eFace.

One thing that I find solely missing in Android however is crash reporting. Google is obviously aware this is important, see their breakpad project. I'm not committed to quality in my software, it goes beyond that, I'm committed to perfection. If Google wants to provide a platform, well I think crash reporting is one of those services that developers expect to be part of any platform nowadays. That being said, this also happens to be missing from standard Java but again, I'm not talking about providing software to capture and report crashes only, this also exists for Java. No, I'm talking that platform holders should provide servers where developers can register their binaries like Microsoft allows you to do. Open source developers have tons of resources available to them in the form of Sourceforce, GNU Savannah and Google Code to name a few, but it appears that crash servers aren't readily available from any of these sites.

Finally, I would be remiss if I were to not mention the #1 reason why I like Android. The inclusion of the SQLite database. Meaning that, if you have an Android phone, you have SQLite in there. Words cannot describe how passinate I am about this database. I have used this database extensively in the past and will certainly continue to do so probably till the day I die. I have tried using Derby and SQL Server Compact but if you're looking for embedded, small and complete they just don't come close to SQLite.

Capcom Thinks Wii Owners are Dumb

2007-11-16T12:58:00.000-05:00

I had never played Resident Evil before 4. The genre just didn't interest me. The only reason I bought 4 was because of the fantastic, near perfect reviews.

That being said, my experience with Resident Evil 4 on the Gamecube wasn't very positive. I couldn't aim correctly so I never even made it to the first type writer.

Well, a few months ago, I picked up the game again but for the Wii this time. Now, I could aim and now I understood what those rave reviews were talking about. What a fantastic game!

Now, eager to play some more Resident Evil, I picked up the new installment of the franchise for the Wii... Resident Evil: The Umbrella Chronicles. Well, if you're looking for the next great RE game, don't look here.

To put it mildly, this game seems to have been dumbed down for the Wii users. It has more of an arcade style to it. You can play the game pretty much without the nunchuck.

I love my Wii. I think it's the only true next generation console. Unfortunately, most 3rd party publishers are content with dumping bad games on it and then complaining that Nintendo has too much of a market share on their consoles.

Well, I have news for these publishers, Nintendo makes really good games hence why people are willing to give money to buy them. In any case, if you own a Wii, avoid The Umbrella Chronicles, just play the 4th game again.

Red Rings of Light

2007-11-07T12:16:00.000-05:00

Once again, my xbox console has succumbed to the dreaded 3 red rings of light problem. This is now the 3rd time this year. Obviously, I am a little peeved even with this now being covered by the xbox warranty.

That being said, this is not another rant on this problem. This has been covered to death by so many people, I don't see how one more would help (or hurt).

This post is more about praising the xbox support team. When I called yesterday, once again, I talked to somebody that was incredibly nice, incredibly helpful in getting this taken care of. Microsoft also threw in a one month xbox gold card for my troubles. I can only imagine how many people this person fields in just a day just for this specific problem. Even assuming all customers are nice and polite, the repetitiveness of this task must be daunting at best.

Yes, it's annoying, but I still think Microsoft is a company that has learned from its mistakes, continues to learn everyday and keeps on improving. As much as I hated Microsoft in the nineties, preaching the Linux gospel and what not, I feel that no one, not even Microsoft is beyond redemption.

More DS good news

2007-11-07T12:00:00.000-05:00

Following up on my post from yesterday, saw this article from next-gen.biz on a Ubisoft investor meeting. In the article, the CFO states that:

"CFO Alain Martinez added that DS games, with their relatively low development costs, achieve profitability at around 100,000 units sold, while a next-gen game for PS3 or Xbox 360 needs to sell around 1.3 million."

This should encourage many 3rd party publishers to bring their top franchises to the DS. However, I'm still worried about what the overall quality of these games will be and if any of the control schemes attempted will be physically hurtful like Metroid Prime Hunters.

My advice, make sure you do what Advance Wars: Dual Strike does. This superb title offers touch screen based controls but also offers a "classic" control scheme that is identical to the Game Boy games of the franchise. This is fortunate since the touch screen control scheme of that game is unusable. Offering a classic control scheme salvaged this gem instead of dooming it.

That being said, offering two control scheme is not always that easy to implement. Zelda: Phantom Hourglass is nothing short of fantastic. One of the reason is the new control scheme based completely on "touch". If given a choice, I suspect many users would have opted for a more classic control scheme and would have missed out on the shear genius that are the touch controls in this game.

Call of Duty 4

2007-11-06T14:07:00.000-05:00

OK, I think it's a given by now that developing for the Nintendo DS is hard. That being said, game publishers can no longer ignore the massive success of the handheld now having shipped more than 50 million units.

The good news is that titles like Call of Duty 4: Modern Warfare are coming to the console. The bad news is that tiles like Call of Duty 4: Modern Warfare are coming to the console.

On the surface, the availability of a title like CoD4 on the DS is really good news. Having played Call of Duty 2 and 3, I can attest that they are really great, if not very long, games. The problem is fitting that franchise on the DS. The title only comes with one control scheme, this control scheme was pretty much the one used in Metroid Prime Hunters and it didn't work in that title and it doesn't work here either. In fact, the harshest criticism I had about Metroid DS was how hard it was on the hands to play that title, I gave up on it only after a few plays.

The problem with CoD 4 on the DS is that it's clear that the traditional control scheme of the franchise, found on the XBox 360 version of the game for example, is a much better fit. Unfortunately, Activision decided that one control scheme was enough and didn't include an option for it.

I'm all for innovating but more often than not it seems that the price for innovation on the DS is bad controls. Call of Duty 4 on the DS is nonetheless a game with high production values, even delivering speech for all the text in the game, however, it's also clear that the control scheme just doesn't work. Finally, the most frustrating thing about the experience is the DS has all the buttons and controls necessary to actually enable the traditional control scheme.

Tomcat vs. Geronimo

2007-10-30T11:44:00.000-04:00

I talked in an earlier post about Java "word salad" and I still stand by it. It's only after playing with the various products that I finally understand the differences between Tomcat, Geronimo and Glassfish for example.

In plain English, Tomcat is a web and application server that implements a java servlet container.

Geronimo and Glassfish are the implementation of the Java EE platform by Apache and Sun respectively. Like Tomcat, Java EE servers need to provide the servlet feature. This can be accomplished by including Tomcat proper and this is indeed what Geronimo does for example. However, a Java EE server also needs to implement tons of other stuff too: transactions, EJB, JDBC, message queues, persistence, clustering, management beans and an internal SQL database to name a few.

Each set of features provided implements interfaces specified in the various java community specification documents. This means you can swap various features for others, e.g., Glassfish implements the JMS API using Sun's Java System Message Queue product, a.k.a OpenMQ, but Geronimo uses ActiveMQ. You could for example put ActiveMQ in GlassFish to gain additional features found in that product. That being said, you could also install ActiveMQ or OpenMQ standalone on additional servers to either complement your Java EE application servers or just for use as a standalone message queue.

Not all components found in these open source Java EE platforms are distinct however. The internal database found in both products is Apache Derby for example.

The Java Ecosystem

2007-10-30T10:41:00.000-04:00

I promised myself to look at the .Net 3.5 Beta by now but I've been overwhelmed by the number of technologies found in the Java ecosystem that I haven't yet found the time. It doesn't help that I only work 2 to 3 hours a day either.

But seriously, I feel like a newborn experiencing all these interesting technologies:

1. Google Web Toolkit. Write a Java application and compile it down to JavaScript to get an Ajax site. Really interesting and really fund to code. I can definitely think of a large application or two that I wrote on the desktop that would have been on the web had this been around circa 2002.
2. Project Phobos. So, if you're planning on writing the web front end in java, why not write the server in JavaScript? OK, haven't played with this one yet but interesting nonetheless and I certainly will find something to do with this if it works well.

The number of tool kits that Sun has made available under its open source initiative is staggering. Need federated single sign on? No problem, just use OpenSSO. It also supports OpenDS as an LDAP back end. If you consider that OpenSSO is extensible and can be made to support InfoCard. Even more attractive it becomes when you consider you can use it from any language thanks to its support of web services via SOAP and Rest.

Moving Blog

2007-10-24T10:46:00.001-04:00

Apparently some of my Unix friends still hate Microsoft so this blog is moving over to blogspot. What I find funny is how these guys still hate Microsoft for abusing their market position... i.e., Microsoft the evil corporation.

Personally, I find the Microsoft of today to be significantly different than the one that launched Windows 95. The focus today is really on creating incredibly high quality software and innovation, e.g., UAC and the Office 2007 ribbons to name a few.

The other thing that I find funny is that http://codepimps.blogspot.com is actually taken. The codepimps.org domain will now point to http://codepimpsdotorg.blogspot.com/.

Fresh Pot of Java or The Java Platform, Ten Years Later

2007-10-24T10:33:00.002-04:00

I haven't looked at Java in a long while, almost ten years. I remember walking away very unimpressed with the language and runtime back then. The promise of "write once, run everywhere" was laughable at most. The only thing I really liked was applets but ultimately dismissed them due to the need of distributing the JRE to clients. Not exactly sure when, but I also played around with JSP when it came out. Ultimately, I felt Java was slow, big, not portable, that the language was just too clean and lacked some very important features and that crafting libraries to compensate for those missing features was just too painful.

I also felt the supporters of the technology downplayed real issues (and still do?) when building applications in order to market the language. Correct use of threading and memory management to name a few are still no walk in the park no matter what the environment supports or not. I also worried about deployment issues related to applications and the runtime itself.

In any case, I started looking at Java again recently. Not sure why, maybe it's the idea of the virtual machine that I always liked, just plain boredom or maybe just a new outlook on life in general.

With the latest version of Java, i.e., 6, I can honestly say it's nice, real nice. I still don't know how portable it really is but I don't particularly care anymore. Let's face it, with most applications delivered over the Web now, the application/JVM would run on a server/OS that would be fully validated anyway.

Things I really like:

1. The huge class libraries
2. Support for generics. I can't tell you how I hated having containers full of "Object" classes and having to rely on runtime exceptions to determine correctness. This is the job of the compiler to tell me and it's about time it did so.
3. Tools. Eclipse and NetBeans to name a few. The NetBeans 6 Beta is really nice.
4. Application servers. Only tried Glassfish and Tomcat, neither really have everything I want but interestingly enough, if combined they would most likely be ideal.
5. Management of application servers. Haven't tried to do anything with them really yet scalability wise but I have written quite a few highly scalable servers in my day and configuration and management was always lacking. Why? Time simply, budgets had to be spent on other areas. The administrative application included with Glassfish is good and does its job really well.
6. Web deployments
7. Java DB, A.K.A. Apache Derby. If you're dealing with simple brochure website or something like that, it might be worthwhile to just use this DB and not bother with anything else if your developing in Java, it's included in the SDK after all.
8. Java.nio was something I solely missed previously. Let's face it, sometimes simple blocking I/O just won't do even with thread support. In my line of work, standard blocking I/O is too often the wrong answer.
9. Support for scripting. JRuby, JPython and Sun also has its own scripting language whose name escapes me right now.

Also, I found that speed and application interactivity is fine on modern hardware and that assertions are finally supported.

In all, I invested two full days on Java and its related technologies and came away impressed. Using Eclipse, I wrote a sample console application. Using NetBeans, I wrote a sample web application. The application was unimpressive but the web application I wrote was much more interesting.

This web application was a less (much less) ambitious version of the DinnerNow .Net sample application. Ajax support, Google maps, mobile backend, web marketplace driven by Apache Derby in network configuration of all things and a simple workflow using BPEL. I didn't bother with all features, i.e. command line client, management console and secure authentication didn't make the cut. However, I did enough to get a feel for Java and found it pleasant to work with which is a lot more than I can say about it ten years ago.

That being said, not everything is positive. I've been around this industry for a while. It's easy for me to pick up new toolkits and languages. I really feel the Java documentation is hard for newcomers to get into easily. The name of the different Java technologies and tools is one such barrier. A much more complete map of Java related technologies is needed and more importantly, it needs to offer guidance on how these technologies relate to one another.

Don't believe me? Have a look at the Apache Software Foundation site. Obviously, a priceless resource for any aspiring Java developer. Look at the software directory on the left side of the main page... You will see a list of very colorful names like Cayenne, Jakarta, Lucenne and Maven among others with no clear indication of what these really do for you or where to start. To learn about each project, you have to dwell into each of their individual websites and somehow track all the projects yourself, great. Needless to say, Apache is not Sun but the company does suffer from the same techno word salad in its own documentation of the Java platform.

It's still too early to tell if I will use using Java in a future project but I'm seriously considering it. I plan on delving into .Net shortly in the future since I haven't bothered looking since 1.1 beta something. This evaluation will obviously impact this one on Java.