While thin client architectures have become prevalent in many domains, there are still lots of use cases that call for desktop applications written in Java. Outer Colony is certainly one of them. Choosing the right deployment mechanism is very important in these cases, and after experimenting with Oracle’s out of the box solutions, I was a bit surprised at how limited their capabilities were, as each proved inadequate for my purposes. In the end, I decided to use a third party solution called install4j, a multi-platform installer builder, that simply addressed all of my distribution requirements.
In this entry, I’ll walk through my attempts to use Java Web Start, then writing my own launchpad and auto-updater, using Oracle’s self contained packaging and Inno Setup, and finally why I decided to use install4j instead of either of my other semi-functional solutions.
Outer Colony’s Requirements
Before getting into my experimental solutions, it’s important to understand the general deployment requirements for Outer Colony. The following points were absolutely necessary for any potential solution:
- The setup process needs to handle setting up Java for the end user. Even when internally distributing Outer Colony to close friends who are fairly tech-savvy, they couldn’t really handle setting up Java themselves. This should be obvious, but if you’ve spent the last few years as a backend Java developer, it’s easy to forget that end users won’t have any idea what to do when you tell them to “install Java 8, first”. Even if you give them a link to the download page and specific instructions in an email, half of them will do it wrong. As such, the installer needs to handle every aspect of setting up the JRE in a way that’s completely automated from the user’s perspective.
- I need to be able to specify the exact JRE to use. Outer Colony needs a 64 bit JRE for its higher memory modes, and a late Java 8 version is essential. If a user happens to have Java 7 already installed on their machine, that won’t suffice.
- Like many games, Outer Colony will have a very active maintenance lifecycle. Patches for bug fixes, new content, and all sorts of adjustments will probably occur frequently, so there needs to be some kind of auto-update functionality built into the system. When a new patch is released, Outer Colony needs to seamlessly update itself in a way that’s typically experienced by computer game users.
Falling short in any one of these three categories is a deal breaker. With these points understood, I started to explore my options.
Java Web Start
Java Web Start is a technology directly from Oracle that’s used to distribute Java applications via the internet. At first glance, it seemed like the perfect choice for Outer Colony. Initially, I was confused as to why its use wasn’t more widespread, and this did give me pause. General googling revealed that Java Web Start didn’t seem like it was being used by all that many modern applications, and that’s often a bit of a spooky sign for something that’s a part of the Java platform itself.
Still, it’ll automatically set up the JRE for the client, it provides a really straightforward mechanism for auto-updates, and was pretty easy to configure. With a few hours, I had configured my Apache server properly, set up the necessary XML, and was deploying Outer Colony via Java Web Start. It took another day or two to figure out all the details of jar signing, but once I’d overcome those issues, it seemed like this was going to work.
Then I hit a seemingly insurmountable barrier. Outer Colony needs a 64 bit JVM to run in its higher memory modes, and I couldn’t find any way to force Java Web Start to install a 64 bit JVM. There was no way I could see to specify which JRE should be used to run Outer Colony when it’s launched from Java Web Start. Even worse, JWS decides which JRE to tell the user to install based on the user’s browser. This is a problem, because almost every browser is a 32 bit application. As such, 32 bit JREs were being installed, which limited OC to 4GB of RAM, which would limit users to relatively small worlds.
This was unacceptable. I searched and searched and searched, but couldn’t find a working solution. The -D64 flag seemed to do nothing, and Oracle’s documentation couldn’t answer my questions. Even StackOverflow didn’t contain any good answers.
The second problem with Java Web Start is that, while it almost worked for me, the user experience is decidedly clunky. It feels very 1998 and is highly alarmist with respect to security popups and warnings. End users just don’t see things that look like Java Web Start anymore, and a friend I was showing it to at the time said he might not proceed with it, because it “looks like a virus”. While this sort of thing is fine in an enterprise environment, user trust is essential in my case, and I don’t want people to turn away from Outer Colony because its installer looks sketchy. This and the 64 bit issues made me give up on JWS and try something else.
Self Contained Packaging, Inno Setup Installer, Custom Auto-Updater
My next attempt was to use Oracle’s self contained packaging to bundle Outer Colony’s jar with the right JRE, install the whole thing using Inno Setup, and write my own updater to grab new Outer Colony jar files from the web and configure clients’ setups to run the new version.
After some thought and a bit of design work, I decided that Outer Colony would consist of two jars, one serving as a launchpad / auto updater, and the second comprising the application itself.
I hit Oracle’s documentation and started experimenting. I updated my build script (an ant script) to include an fx:deploy task, and within a few hours, I was building my jars, creating a folder structure that looked like it contained the necessary JRE, and felt like I was on my way to success. I started work with Inno Setup, packaged up all the assets that were arrayed by my ant script, and built an installer to jam the whole thing, appropriate JRE and all, onto a client’s machine via a pretty conventional-looking installer.
I wrote all the code for my launchpad jar, which included some basic web functionality to read an XML file from my website, compare the version there to the one in the client, download the new version as necessary, overwrite it on the user’s disk, and start everything up properly. I then went to test the whole thing out, running the installer on a fresh machine, and everything was going perfectly. Inno Setup put all the files where they should be, and the Outer Colony launcher started everything up using its own JRE. I tested the auto-updater out, it brought down an updated jar, attempted to “java -jar OuterColony.jar”, and wham! A failure.
After digging around for a while, I realized that java.exe wasn’t contained within my bundled JRE that I made using the fx:deploy task in my ant script. I thought that I must have something wrong in my script, but I just couldn’t figure it out. After hours of detective work, I found the answer in Oracle’s documentation, addressing my exact situation (I bolded parts for emphasis):
"Also by default a self-contained application does not contain full copy of Java Runtime. We only include set of mandatory components. Part of the reason why this approach was taken is that we want to reduce the package size. However, there are situations where your application may depend on these optional components and in that case you will need a way to add them to the private runtime. For example https connections will not work if jre/lib/ext/sunjce_provider.jar is missing. Currently this can be achieved by providing a custom config script that is executed after application image is populated. Like in the example above with the icon, you need to enable verbose output to find the name of the script file and then drop it to the location where packaging tools will find it. Note that scripting language is platform specific too. Currently we only support shell for Mac/Linux and Windows Script on windows."
I don’t want to convey the wrong idea here, as I love both Java and Oracle, and I think Oracle has done a stellar job since they absorbed Sun. However, why Oracle would decide that package size is a valid reason to do this, I simply can’t imagine. In the year 2016, do a few extra megabytes matter that much? In my view, the cost in added complexity for developers by hacking out seemingly random parts of the JRE vastly outweighs any benefit this approach might confer.
Java.exe is excluded by fx:deploy, sunjce_provider.jar is excluded, and frankly, a whole bunch of who-knows-what may or may not also be excluded. I couldn’t find any definitive definition of which parts of the JRE do make it in, and which ones are considered “mandatory”. For me, the whole thing is “mandatory”, because I don’t want to spend 100 hours figuring out which parts of my application no longer work, because fx:deploy doesn’t really bundle a full JRE.
Ironically, the very example provided by Oracle of https connections using sunjce_provider.jar not working is case in point of why hacking parts of the JRE out of the bundle is not such a great idea. A developer will test software in the IDE, find it all functional, test it in the build, and find it to mysteriously not work. Without this special piece of knowledge that this specific part of the JRE isn’t bundled by fx:deploy, it’s extremely time consuming to try to piece together what is going on.
Once I figured out the issue, I set out to solve the problems by writing a custom script to copy the missing parts of the JRE into my build folder. I wound up using Inno Setup’s .iss scripts to manually copy over some files that I identified as missing from fx:deploy and necessary for Outer Colony, built a new installer, and tested it out. The installer set up Outer Colony, complete with its own fortified semi-JRE, I ran the auto-updater, it brought down an updated version of the application, restarted it properly, and everything seemed to work.
I handed the installer off to my brother to test on his machine, and, of course, it didn’t work. It “did something”, but when he tried to launch Outer Colony via its desktop icon, “nothing happened”. After guiding him through launching Outer Colony via the command line, he was able to send me the stack trace of a ClassDefNotFoundError, which I determined was the result of another missing file from fx:deploy’s incomplete JRE.
At this point, I’d had about enough. The whole solution was starting to reek of brittleness and convolution. I’m relying on an ant script centered on a poorly understood and barely functional task that I can’t debug, followed by a funky .iss script that’s manually copying a bunch of files into a build folder, a custom-built auto updater jar that makes the whole architecture of Outer Colony significantly more complex than it would otherwise be, and a freeware installer tool that, while nice in its own right, does have a somewhat outmoded and primitive feel to it.
One’s natural engineering sensibilities are offended by this kind of byzantine setup, and even after days of effort, it still wasn’t working. It was time to move on.
It’s my hope that this blog post can help developers with similar requirements skip all the headaches described above and proceed directly to this point. I had first run into install4j at a previous software engineering job back in 2009, and while it was only peripheral to my role, my employer was using it to effectively distribute their desktop Java application to Windows, Linux, and Mac users.
I headed to EJ Technologies’ website, downloaded an install4j trial, and within a couple hours I had gone from being a new user to having built an installer for Outer Colony, containing exactly the right JRE, and doing everything I needed it to do. install4j even contains built in support for auto updates, which I had working in no time. The documentation is exceptional, the tutorials are fantastic, and the whole thing is as simple and intuitive as it can be. Best of all, the end result is very professional looking and streamlined. It will instill confidence in clients. After all, the first part of their interaction with your software is its installation, so it’s probably wise to start things off on the right foot.
At the end of this adventure, I have another experience that demonstrates the old adage that it’s cheaper to buy software than to build it. In this case, it’s cheaper and better. Software engineer hours are expensive, and for a non-trivial Java application, you’ll burn scores of them if you try to build a custom deployment and auto-update solution. At the end of that development, whatever you’ve built will almost certainly be inferior to what install4j can give you with a bare minimum of expense, both in terms of time and effort.
Early testers of Outer Colony are even commenting, without any prompting, about how much they like Outer Colony’s installer and auto-updater. I’ve never heard of an end user doing that, but here’s the quote, right from our forums:
2) The auto-update process. Seamless. Did you code all that yourself or are you using an add-on? I’ve used a program that automatically creates an installer from a Unity game before, but I don’t think it supported auto-updating like that. Yours is really sleek and user friendly!
This was posted right here by one of our closed beta testers. An actual end user, remarking on how much they like an auto-updater! Unprecedented! I wish I could take credit for having built an installer and auto-update platform of this quality myself, but it’s all install4j’s doing. This should tell you every thing you need to know about install4j!