Thursday, November 4, 2010

Programming Windows Phone for Free

I usually don't bother with books anymore to learn a new platform, don't need it, they just slow me down. However, I would be lying if I didn't have a special place in my heart for the great Charles Petzold.

Back in 1992, I very much needed books to learn a new language or concept and "Programming Windows 3.1" was my introduction to the world of UI programming and event loops.

Not only was it clear and concise, a "problem" that has since been "fixed" with later editions of the book, it was actually quite entertaining. No small feat considering it took over a hundred lines of C code back in the day just get a window to show up.

Now the Great is back with a book on programming for Windows Phone. I haven't read it but Microsoft is comping the book in support of its new platform and is available to all for free. Microsoft is hoping it will introduce a new generation of developers to Windows Phone like the original did for Windows, I can only hope it will introduce a new generation of developers to Petzold.

You can download the book from here.

Saturday, August 21, 2010

The Challenge of Using Multiple Languages

After years of using Netbeans, I finally gave up. We have started using Eclipse. Why? In one word: plugins.

To be more exhaustive, I have not been very comfortable with the acquisition of Sun by Oracle, and that was even before they announced that awful lawsuit against Google. I would have been more comfortable with the acquisition if Oracle hadn't just closed on buying BEA one year before Sun.

Oracle's Java EE application server offering, OC4J, was, and is, simply awful. To their credit, Oracle didn't ignore the problem, so they went out and bought BEA for $1 billion dollars to acquire WebLogic. They also made it clear, that although they will support both application servers for now for their enterprise customers, WebLogic is their "strategic" server. Meaning that OC4J is on the endangered species list.

If it weren't for the BEA acquisition, Glassfish would have taken center stage in Oracle's portfolio. However, even if both servers share common components, e.g., both use the Metro web services' stack which is developed as part of the Glassfish group, Oracle didn't buy Sun for Glassfish, they bought it for Java. Makes sense considering they spent $1 billion to buy WebLogic.

Then we had the Netbeans vs Oracle's JDeveloper scenario. Unfortunately, if that was the only variable, Netbeans' victory is absolute. JDeveloper is not very good, and I'm being polite here. However, JDeveloper is not just an IDE, it's an IDE designed to get you to use as many Oracle middleware products as possible. The other variable is that Oracle, like everybody else, supports Eclipse. Meaning that Oracle is now spending money to support 3 IDEs, I suspect that will not last indefinitely.

All this to say, even if Glassfish is still the best and Oracle is continuing to invest in both Netbeans and Glassfish. We still decided to move ahead and proceed with our migration from these products. We also decided to take it by steps and steps and slowly. We have now successfully migrated to Eclipse but continue to use Glassfish.

So why Eclipse? Well, our primary toolkit is Google's web toolkit(GWT). We use GWT everywhere and profitably too. However, Google's IDE of choice is Eclipse and the plugin for GWT, and AppEngine too, made by and supported by Google is for Eclipse. So it was hard to always be behind on Netbeans when it came to GWT. Not only, but we are targeting AppEngine to replace Glassfish. If that wasn't enough, we also have a very real need to develop for Android and we love Scala. Again, all of these have world class plugins for Eclipse but not for Netbeans. So we just grew tired of being second class citizens all the time.

So how about Eclipse? I'll tell you one thing, I was nervous having had really awful experiences with it before. It has a very steep learning curve. However, once you wrangle it, it definitely pays off and with dividends too. It's definitely a match for Netbeans and more. I can't think of an IDE that has better refactoring and source navigation features. Heck, there's even a Silverlight plugin for Eclipse.

So now we have one IDE for pretty much all platforms and languages: Java SE, Java EE, Scala, Android, AppEngine and C++... or do we? Well it turns out it's not that simple.

Recently, on a project, I had to code in 4 different languages at the same time: C++, Objective C, Java and C#. Java on the server. C++ for some glue code. C# for a Microsoft Office extension and Objective C for an iPhone application... and I ended up using 4 development environments.

I had to use Visual Studio for the Office plugin. I used Eclipse for the Java server stuff. I used Netbeans for the C++ part and finally, I used Xcode for Objective C.

It turns out coding in multiple languages, even on the same day, isn't that hard, however, using multiple IDEs in the same day is a different story. You see, I always keep the default key bindings of an IDE so that I learn them as I go. It also means that if you're a veteran of a particular IDE, Netbeans in my case, it's very, very hard to switch and not continuously mess up which accelerator you're suppose to use.

So there you have it, the real challenge of using multiple languages: the IDE key bindings. So if you're going to be in this situation and you aren't dumb like me, I suggest you import key bindings in all the IDE's that you can.

Friday, June 11, 2010

The Significance of a USB host

For the last 2 years, I've been saying to anyone willing to listen that the future of computing is a incredibly powerful phone that you cradle when you get home or at work, etc. This cradle has a mouse, a keyboard, a large screen and a good sound system.

Android enthusiasts seem to have taken another step to making this vision a reality. Notice in the video that he launches X-Window with a desktop environment from his phone that isn't the Android shell.

That being said, the Nexus One, as powerful as it may be, still isn't the all powerful phone I've been describing for the last 2 years, that phone still doesn't exists.

Thursday, June 3, 2010

HTML5 Update

Yes, we know about some of the good stuff in html5 already but a lot of new specs have been added to the html5 family in the last year that will try to make the following available to web applications, i.e.,
  • TTS
  • Speech Recognition
  • Recording
  • Microphone Access
  • Camera Access
  • WebGL
  • Positional Audio, still very early, don't know if will be based on OpenAL, OpenSL ES or something else
  • File & Device API
  • and more...

Android can be Dumb Too

Here's something you would expect if you were coding on the BlackBerry platform but this is most definitely on Android. Android doesn't implement roaming itself.

What? Yes, it's true. If you enable roaming in global system settings, it doesn't actually stick unless each and every single application you use honors the flag. Furthermore, if you don't implement it, Google states that it's the application developer's problem if your users complain about very large data bills.

So to implement it in every application, you need to check ConnectivityManager to see if you're allowed to use the network, e.g.,

final ConnectivityManager cm = (ConnectivityManager) getSystemService(Context.CONNECTIVITY_SERVICE);

// TODO check the connection to see if it's allowed

You also need to check for the CONNECTIVITY_ACTION intent in case this changes later on. Honestly, this should just be blocked by the system. They're afraid someone won't be able to get to emergency services. It should just support an emergency level flag for network connections instead of getting all developers to implement this in their apps.

iPhone dev on Windows

As of VirtualBox 3.2.2 it is now possible to do iPhone dev on your Windows box. VirtualBox 3.2.0 introduced support for OS X as a guest OS, however there was a bug that broke iPhone/iPod Touch usb support on Windows hosts, this has been fixed in the 3.2.2 release.

So without further ado, here are the (amazingly simple) instruction for installing OS X 10.6.3 as a guest OS in VirtualBox on Windows.

1) Create a new virtual machine, the guest OS is Mac OS and for the version I used Mac OSX Server, not server 64-bit (I haven't tried 64-bit yet), disable EFI.
2) Download EmpireEfi and choose the BootCD.iso as your CD image.
3) Edit the VM's xml file and add the following to the section:

<ExtraDataItem name="VBoxInternal2/SmcDeviceKey" value="ourhardworkbythesewordsguardedpleasedontsteal(c)AppleComputerInc"/>

4) Boot from the CD and follow the EmpireEfi instructions. You can install OS X using a retail DVD.
5) Unlike VirtualBox 3.2.0, the built-in EFI bios will not boot OS X, so keep using the EmpireEfi boot cd.

Wednesday, April 7, 2010

The Beauty of Google Chrome

Google Chrome is really an amazing application. A lot has been said about its performance and how Opera 10.5 is slightly faster in terms of rendering and JavaScript performance. However, Opera doesn't implement any of the sandboxing techniques that Chrome does, if it did, it would suffer a significant performance penalty. This is a look at Google Chrome on Windows.

What exactly does make Chrome so special? It's the fact that it uses recommended practices for Windows security. It's funny reading some of the archived documentation on MSDN. The documentation uses examples on how a future release of Internet Explorer or Outlook , carefully disclaiming that current releases do not leverage these security features, could use such feature to provide additional protection to users.

Needless to say, Internet Explorer (IE) and Outlook still do not implement these recommendations but Chrome does. It implements and leverages the following technologies from Windows:

1. Multi-process architecture

Yes, IE 8 has this also but their implementation is very different from the one found in Chrome. In both browsers, each tab represents a distinct process. On Windows Vista and higher, both mark the process as a low integrity level, more on this later. This is pretty much where the similarities' end.

In IE, the browser leverages the fact that each thread in Windows can have a message pump and have user object likes windows, buttons, etc. That means that each child process is managing the user interface of the tab directly. It also means that this child process, its threads and its resources live in the same security context that you do. The only difference however is that most risks are mitigated by marking the process as a low integrity one. This means however that squatter and shatter attacks can still be performed on other windows, just as long as those processes are also low integrity ones. All this to say that child processes render content to your screen.

In Chrome, child processes are also renderers but they do not actually manage anything on screen. They're renderers and these processes do not have a message pump that is coming from Windows' user interface layer. Instead, their message pump is a custom one that is listening on a named pipe in Windows (a socketpair() on OSX and Linux). They listen for requests, render data and send back the information to the broker process which is actually the chrome of Chrome.

In other words, they render everything to an off screen surface that never interacts with your screen. Even if it did, we'll see later that it wouldn't matter anyway.

That means when you click on something, a message from the broker process, Chrome's chrome, is sent to one of the renderers, once it's done processing the message, the renderer communicates back the result to the broker for on screen rendering. We can see how such communication would be expensive and how it would slow down Chrome. Making Chrome's performance, and Windows' implementation of named pipes, even more impressive to witness.

It's also important to note that the lifetime of a renderer process in Chrome can also be very short. If you navigate to something like "" to "" using the same tab, Chrome has killed the renderer that was handling "" and spawned a new one to handle "" even if you did not switch tabs. Not only does this make it impossible to leak any information from the first renderer to the new one, it's also a very effective way of doing garbage collection. All resources, including leaks, that were used by the now defunct renderer are reclaimed automatically by the kernel.

2. Restricted Tokens

Windows NT has token based security. Most objects, i.e., files, sockets, pipes, etc., in Windows are securable, if not in the beginning, the capabilities have been iteratively added in subsequent versions.

Chrome sets up a token for its child processes that translate to "deny access to everything token". This means child processes cannot acquire any resources that were not implicitly inherited from the broker. Even the named pipe that is used for communication with the broker cannot be opened directly by the child, it must be inherited.

3. Use of Windows job objects

Windows 2000 introduced support for jobs. Jobs in Windows have 2 distinct responsibilities:

  • They include the ability to act on a group of processes atomically
  • They allow additional security restrictions to be placed on this group of processes

The second one is of interest here. I'm not saying Chrome doesn't use the first, I'm saying it's of no importance to our stroll on its implementation of security.

Windows doesn't allow security to be set on various API it inherited from OS/2 and 16-bit Windows. This includes manipulating any User or GDI objects , this is what allows for things like shatter attacks, capturing keystrokes and all kinds of bad things. It's also what makes the desktop experience possible, so windows, copy & paste, etc.

It does so because introducing new interfaces that would allow for fine grained security and deprecating the old ones would pretty much deprecate every Windows application out there, including Microsoft's. It also wouldn't make for the same end user experience. A lot of the mechanics that users' take for granted today on Windows, OSX, etc., just wouldn't be possible or very easy to implement.

Microsoft, in the earlier days of Windows, also decided not to do this for another reason: performance. We can imagine that requiring to check the security permission of every single window object when broadcasting a message would be very expensive for computer's in service 10-15 years ago when a simple logon session routinely has hundreds of windows. Don't forget, there are a lot of hidden windows on your desktop.

However, instead of adding new functions with additional parameters for security, you can set the security and continue to use the same function. This is what Microsoft did with the job object. Specifying the job object allows to set other limits on these child processes that normally would have free reign, allowing you to forbid the following:

  • Access to the clipboard
  • Creation of new processes
  • Access to other windows
  • Access to the window station object's global atom's table
  • Access to virtual desktop objects, ability to switch between them
  • Logoff, login
  • Creating new User objects
  • Change system parameters
  • Prevent hook functions, like trapping every key message, from being set

So once the job restrictions are set, normal function calls that would otherwise go unchallenged by the operating system are now restricted.

4. Use of Windows virtual desktop objects

The initial release of Windows NT, 3.1, had support for virtual desktops. However, this is circa 1989 for release in 1991, virtual desktops on Windows NT were not implemented to allow usage that is more in line with our thinking of virtual desktops today. For example, there is no way to switch a window from one desktop to another.

Instead, virtual desktops in NT were designed to be a security mechanism. They are the boundary of communication for User objects. This means if you install a global keyboard hook, you can also do so on your own desktop. If you broadcast a message to all windows, you're sending a message to every window on your desktop.

They were introduced so that login would be secure, that is, when you log on to Windows NT, there are 3 desktops created. The default one, the logon one and the screen saver one.

The logon one is only active when you lock your workstation and you need to unlock it. This allows for secure login since processes in the default desktop cannot create a keyboard hook into the logon desktop.

The screen saver is used to run the screen saver in its own sandbox. This means that a screensaver that is potentially malware would not be able to access your default desktop.

Of course, you still have the ability to create new windows on any desktop unless the proper steps are taken to secure them and this is indeed what Chrome does. Since its child processes don't have a user interface you can actually see, the child processes are nonetheless in a new desktop.

All these child processes also run in a job object that forbids them from accessing any of the virtual desktop functions. This means these processes can still interact with other low integrity processes but the only other processes these renderers can see are the one's found on Chrome's dedicated desktop for renderers. This is a step further than just marking the children as low integrity. The only possible squatter or shatter attacks that would be possible would be on other processes found in this desktop, problem is however for the would be attacker, is that these renderers don't actually have a user interface either.

5. Integrity level

The integrity level mechanism introduced in Vista allows for processes to be imbued with a level of trust. If a process wants to interact with a process of higher integrity, a UAC prompt is required to escalate the process. Also, low integrity processes do not have the ability to read any system objects or modify them

Chrome and Other Browser's Today

None of these mechanism are being leveraged by Firefox, Opera or Safari. Internet Explorer uses jobs and multi-processes but the sandbox is limited in scope compared to the one found in Chrome. Just food for thought on what browser you should be running to navigate the hostile web.

A final note, Microsoft has announced that it does not plan to support Internet Explorer 9 on anything less than Windows Vista SP2 stating only that "a modern browser requires a modern operating system". We're quick to point out that Chrome does indeed run on Windows XP. However, it is interesting to understand Windows XP's impact on Chrome.

It impacts Chrome in a few fundamental ways, the first one, is that, integrity levels were introduced in Vista, so it's not possible to mark child processes as low in Windows XP. This means these child processes have more privileges. However, due to other efforts realized to execute the renderer's under a proper sandbox, the scope is somewhat limited. The problem lies more in the ability to change system objects like registry keys. Registry key access is also governed by tokens but most users run as administrators under Windows XP.

More importantly however are the following:

  1. Windows XP does not require NTFS. It obviously supports it but installation on FAT32 is also supported. This means that the restricted token assigned to the child processes under such a file system will not forbid a compromised renderer from opening new files. Funnily enough, an attacker would need to deduce the path to any file he wishes to open however since listing of files would be prevented by the restricted token. Unfortunately, it's more than easy enough to deduce the path to important Windows' system files.
  2. Sockets are not securable objects. That's right, before the introduction of Vista, sockets are not governed by any token or any kind of security at all. That means a compromised renderer could create any number of new network connections. This could be used to download malicious code and then inject them in the files that were opened in the step above.

Ultimately, even on Windows XP, the sandbox setup by Chrome is nonetheless very impressive. Users should still be wary of that particular version of Windows compared to what is offered by Vista and above.

Sunday, March 28, 2010

Chrome and Virtual Desktop Manager

Now that I have my new laptop, I decided it was time to find a better virtual desktop manager. I had been using the creatively named Virtual Desktop Manager with mostly good experiences, but a few quirks. One of the more annoying ones is artifacts that occur sometimes, but are persistent, when switching from a desktop with Chrome to one without. After a few days of uptime, I would find ghostly outlines of Chrome windows on all my desktops, and sometimes even orphaned status messages of the variety you get in the bottom left of the browser window when you mouse over a link or are loading a page. Pretty annoying! The second quirk is a blank and non-responsive "Write (no subject)" window courtesy of Thunderbird when you compose an e-mail and switch desktops before sending it.

Anyway, all that has been resolved with the use of Dexpot, I heartily recommend it.

Two Finger Click and Scroll

One of the (many) things that Apple get right on their Macbooks is the touch pad, it is large and supports nice multi-touch gestures.

My new laptop, an Asus G73, features the nice large touch pad and supports multi-touch gestures, like pinch-zoom, but it misses the best two gestures - two-finger scroll and two-finger tap to right-click. Fortunately, there is a remedy to this deficiency: Two-Finger-Scroll. So now I get the Macbook quality touch pad experience with twice the cores, way better graphics and more than enough money left over for a fast ssd!

Monday, March 15, 2010

Java3D offscreen rendering and changing colours

I just ran into an irritating quirk (perhaps bug?) in Java3D. I'm using the offscreen rendering feature of Canvas3D to render a simple 3D scene to a PNG file. The requirement is simply to produce a PNG given certain input parameters, notably colour.

My first attempt was very straight-forward; create my scene, with the correct colours and render it to the PNG. Repeat as required. The last part, repeat as required, is problematic. Each time you create an offscreen Canvas3D you create an OpenGL PBuffer. My graphics card as 32 pbuffers so the 33rd time you render the scene, the program crashes.

A second attempt is obviously required. The requirement being to use only one Canvas3D object and thus only one scene graph and to change the colour of the scene before re-rendering it. First, some background, my objects are Boxes and to set their colour you have to set an appropriately configured Appearance. For example:
final Material material = new Material();
final Appearance appearance = new Appearance();
final Box cube = new Box(1, 1, 1, appearance);

This simply creates a cube using the Box utility class and sets a color on it. Java3D will optimize the scene graph and part of that optimization assumes that the scene doesn't change, unless you explicitly state that it can. Changing the colour would be one of those things you have to state. There are plenty of forum topics about changing the colour of a Box and they usually suggest changing the Appearance object, like so:
final Box cube = new Box(1, 1, 1, Box.ENABLE_APPEARANCE_MODIFY, someAppearance);

[...] // Some time later
The ENABLE_APPEARANCE_MODIFY flag simply tells Java3D to assume that the Appearance object associated with the Box can change. This should allow you to change the appearance at will. Except in offscreen rendering mode, this doesn't work. Not only does it not work, but any initial colour you set is ignored and you get a white Box. After much messing about, I found that ENABLE_APPEARANCE_MODIFY was the cause. The solution to this is to change the colour on the material directly and not simply set a new Apperance.
final Material material = new Material();
final Appearance appearance = new Appearance();
final Box cube = new Box(1, 1, 1, appearance);

[...] // Some time later
And there you have it, Boxes that change colour in an offscreen rendered scene.

Wednesday, March 3, 2010

Red Rings Again and for the Last Time

Once again, my Xbox 360 has been hit with the 3 Red Rings of Death error. Once again, I called Microsoft only to be told that it was out of warranty this time around and it would cost me $150 to repair it. After explaining my history, this was my 4th console after all, they graciously offered to repair it for free. The catch is that I had to get it to a UPS store myself which isn't anywhere near me.

Well, I got the console back again this last Monday and got 4 Red Rings this time. Called support again, they were willing to repair it but when they mentioned I would have to get it back to the UPS store, I decided enough is enough. The customer service representative explained that my preferred courier was indeed supported, i.e., the one that is just a few blocks away and the same one I received the latest damaged console from, but that the Xbox support system was randomly generating labels to even out the various couriers Microsoft uses. I politely asked for a label from another courier and was told there was nothing they could do.

So this will be the last time I have issues with Red Rings. The games, accessories, etc., are heading to EB Games and the console is headed to the garbage bin. Fool me once Microsoft... but I've had enough.

Friday, January 29, 2010

The Sanity of Google Chrome Frame

OK, so having had to support CSS in Internet Explorer 6, 7 and 8. This isn't so insane after all. Turns out, it's much less effort to fix IE with a free add-on, that doesn't even require administrator privileges, than it is to fix CSS to work in IE.

I now recommend using Google Chrome Frame over actually making stuff that works correctly in IE. This just shows you the sorry state the IE is in. Microsoft is working on fixing IE with 9 and beyond but it just won't help all those people who don't really know what a browser is. However, asking for a plugin, and one from Google no less, will reach all audiences.

So the people at Google weren't insane, they just felt it was less work to fix IE than to break sites to comply with IE's view of the web. What I didn't understand at the time however, is that you can't just show a page when your user reach your site telling them to install another browser because too many people don't know what a browser is really. Also, in a corporate environment, they probably can't install one either even if they do.