As a special bonus, I’ve also thrown in coverage analysis using EMMA. And before people ask, yes I have tried to use Clover with PDE Build, but have had little luck due to Clover’s dependence on a specialized Java complier. If anyone has gotten this to work, I’d love to hear about it.

And finally, I had to change this example to use a test plug-in instead of a test fragment. For some reason when running Eclipse 3.4 (maybe related to p2?), the ETF is not finding tests in fragments. If anyone has any ideas here as well, please let me know.

You can download the new example here.

NOTE: I’ve discovered that the EMMA instrumentation task fails if you’re running the script inside of Eclipse and you have spaces in your workspace path. Avoid spaces in your path and you should be good to go.

Update – October 6, 2009: The sample projects have now been updated to work with Eclipse 3.5.1

• It’s not easy to run all the tests in a set of plug-ins. The test launcher allows you to run all the tests in a single project, but RCP and OSGi developers are usually working with a large set of test plug-ins. Sure it’s possible to create test suites, but keeping suites up-to-date is a real pain.
• It’s not easy to use test fragments. To find out why you’d want to use test fragments instead of test plug-ins, check out my previous post on Testing Plug-ins with Fragments. The problem is that even the standard suite-based solution does not work with fragments. There are workarounds, but they’re not very pretty.
• It’s not easy to run all of your tests during an automated build using the Eclipse Testing Framework. This is related to the first point above, and again you can use suites to handle this. 

You can then run the test suite both inside of the Eclipse IDE and using the Eclipse Testing Framework. I should note that this works only for JUnit 3.x tests. JUnit 4 describes suites using annotations which makes it (as far as I can tell) impossible to dynamically generate a suite at runtime. If anyone has a solution to this, I’d love to hear it. 

As always, comments and fixes are much appreciated.

A common request I’ve received, though, is to provide a similar sample build that includes the running of JUnits using the Eclipse Test Framework. Well here it is! Just download and import the sample projects, and follow the directions in the included readme.txt file.

Note: If you want to run this sample build using Eclipse 3.2, you’ll need to work around a PDE Build defect. For instructions on how to do this, check out the “Problems parsing feature file in a product build” section on the PDE Build wiki page.

What follows is the list of plug-ins being obfuscated in my Market Contours application. A total of 809 classes are obfuscated in one way or another (some have exclusions that protect class and method names). The obfuscation was done with Zelix KlassMaster and line numbers were not scambled (scrambling line numbers can increase file sizes). So here are the numbers:

Plug-in							Before	After	Diff	% Diff

com.marketcontours.cache_1.0.0.200707122146.jar		5,103	5,679	576	11.29%
com.marketcontours.client_1.0.0.200707122146.jar	98,565	98,751	186	0.19%
com.marketcontours.context_1.0.0.200707122146.jar	102,159	93,646	-8,513	-8.33%
com.marketcontours.core_1.0.0.200707122146.jar		22,559	23,949	1,390	6.16%
com.marketcontours.data.text_1.0.0.200707122146.jar	18,404	20,097	1,693	9.20%
com.marketcontours.data.web_1.0.0.200707122146.jar	13,323	14,401	1,078	8.09%
com.marketcontours.data_1.0.0.200707122146.jar		26,926	28,042	1,116	4.14%
com.marketcontours.evaluation_1.0.0.200707122146.jar	176,009	180,630	4,621	2.63%
com.marketcontours.future_1.0.0.200707122146.jar	18,391	17,849	-542	-2.95%
com.marketcontours.resource_1.0.0.200707122146.jar	132,675	138,066	5,391	4.06%
com.marketcontours.result_1.0.0.200707122146.jar	28,213	27,802	-411	-1.46%
com.marketcontours.runtime_1.0.0.200707122146.jar	56,840	52,486	-4,354	-7.66%
com.marketcontours.stock_1.0.0.200707122146.jar		12,468	12,069	-399	-3.20%
com.marketcontours.ui.builder_1.0.0.200707122146.jar	98,212	95,134	-3,078	-3.13%
com.marketcontours.ui.chart_1.0.0.200707122146.jar	12,835	14,392	1,557	12.13%
com.marketcontours.ui.data.text_1.0.0.200707122146.jar	8,594	8,526	-68	-0.79%
com.marketcontours.ui.data.web_1.0.0.200707122146.jar	2,375	2,399	24	1.01%
com.marketcontours.ui.data_1.0.0.200707122146.jar	33,472	31,890	-1,582	-4.73%
com.marketcontours.ui.result_1.0.0.200707122146.jar	63,844	64,452	608	0.95%
com.marketcontours.ui.runtime_1.0.0.200707122146.jar	7,537	8,285	748	9.92%
com.marketcontours.ui.series_1.0.0.200707122146.jar	67,262	62,512	-4,750	-7.06%
com.marketcontours.ui.tradeshape_1.0.0.200707122146.jar	20,757	19,691	-1,066	-5.14%
com.marketcontours.ui_1.0.0.200707122146.jar		196,728	200,764	4,036	2.05%

Total change: 	-1,739	-0.14%

The results show that this obfuscation resulted in little if any space savings, with some jars actually increasing in size. One reason for this is that there is a good deal of inter-plugin API which causes many package names to be excluded from obfuscation. This type of API is pretty typical of RCP applications, so it’s likely that they’re generally not going to shrink much during obfuscation. And then when you add the Eclipse plug-ins and a JRE to your distribution, the small size decrease will appear even smaller.

So in short, there are good reasons for obfuscating an RCP application, but decreasing the size of your distribution isn’t one of them.

Selecting an obfuscator

The first challenge is selecting the right tool for the job. The most important feature of an RCP application obfuscator (assuming it does a good job at obfuscating!) is that it can obfuscate multiple jars at the same time. Some tools expect you to obfuscate one jar at a time, and unless your RCP application contains only one jar, these tools will not work.

In my case, I chose a tool called Zelix Klassmaster. This obfuscator handles multiple jars, and also integrates nicely into a build process managed by Ant scripts. The process I describe is based on Klassmaster. I’ve also heard that the open-source obfuscator ProGuard can handle multiple jars at the same time, but I haven’t tried this tool myself. If anyone manages to get a similar process working with ProGuard, I’d appreciate it if you’d let me know.

The big picture

The big picture for an obfuscation process includes all of the jars that are distributed with your application. These jars can then be divided into two categories: those that will be obfuscated and those that will not.

• Obfuscated – Jars that make up your own proprietary plugins
• Non-Obfuscated – All third-party plugins and libraries, including the RCP plugins. This category may also include your own plugins that cannot be obfuscated for one reason or another.

Zelix Klassmaster operates on these two sets of jars, loading them all into memory at the same time and then obfuscating only those in the first category.

The build process

An obfuscation process begins after an RCP product has been exported either by the Product Export Wizard or by an automated build process. The exported application is then obfuscated using an Ant script and a related Zelix script. The Ant script controls the process, and it has three functions:

1. Create lists of jars and other parameters which are then substituted into the Zelix script.
2. Execute the obfuscate Ant task provided by the Zelix obfuscator.
3. Replace the original plugin jars with the new obfuscated jars.

In the obfuscate Ant task, Zelix loads all of the plugin jars into memory and then obfuscates those in the list to be obfuscated. These jars are placed into a new folder (in my case a folder called “obfuscated”). These jars are then merged back into the plugin directory of the application.

Of course, if your plugins are not exported as jars, then this process will be a little more involved. Another good argument for migrating to the new packaging structure.

Excluding package names – choosing a naming convention

Exclusions are one of the trickiest part of obfuscating any application, and in particular an RCP application. A good general rule of thumb is that any class referenced in an extension or extension point should be excluded. This doesn’t mean that the code in the class won’t be obfuscated, but that both the package and class names associated with the class need to be excluded. I’ll deal with class names in a bit, but first I’d like to address the issue of package exclusion.

The important thing about package exclusion is that when you exclude a package name that is deep in your package hierarchy, you are required to exclude all other packages above it. This can make it difficult to exclude packages deep in your hierarchy without abandoning package name obfuscation entirely.

The solution I’ve come up with is to make a clear distinction between obfuscated and non-obfuscated packages in the package naming conventions I use. It turns out that this convention fits nicely with another naming convention used in the Eclipse code itself that separates code making up the public API from internal code. Basically, package names that can be obfuscated are always considered to be “internal” packages and will never contain classes referenced by extensions or extension points. Public packages are excluded from package name obfuscation and may contain excluded classes.

To make this convention as useful as possible (to allow as many packages to be obfuscated as possible), this distinction between public and internal needs to be made very high in the package structure. My convention differs from the Eclipse convention in that I make this branch even before the plugin name portion of the package structure. Here is an example of the convention I use.

• Public package – com.marketcontours.plugin-name.subpackage
• Obfuscated package – com.marketcontours.internal.plugin-name.subpackage

Of course, only public packages can be included in the “Exported Packages” list for a plugin. If you specify a package that will be obfuscated, the plugin will not be able resolve the package name at runtime. This is just one more reason to align the concepts of public/internal packages and non-obfuscated/obfuscated packages. And finally, by maximizing the amount of classes in internal packages (which is a good thing to do for many reasons), we can keep the amount of unobfuscated package names to a minimum.

Excluding class names – a new properties file

The second challenge is to exclude the names of classes and interfaces referenced by extensions and extension points. I’ve found that the easiest way to do this is to track the exclusions at the plugin level with a new properties file called obfuscation.properties. Whenever I need to add a new exclusion, I put it into this property file, and then at build time these exclusions are combined by Zelix into a master exclusion list. The exclusion format I use is particular to Zelix, and is meant to exclude both the package and class names of the classes in question. To see the details of what the exclusions look like, check out the sample properties file included in the download below.

Remember that you need to include any class referenced in either your plugin.xml or manifest.mf files. This includes the Plugin (or Activator) class, if you use one.

Also, for now, each obfuscation.properties file must be added by hand to the the Zelix script. So every time you create a new plugin, you’ll need to edit the script. One area for improvement would be to have the Ant script create the list of properties files at build time and substitute them into the Zelix script.

Obfuscation and localization

There are a few extra gotchas that need to be handled when obfuscating an RCP application. The main one is that your localization classes (if you are using the new NLS class) need to be excluded from obfuscation. Fortunately, this can be done with one exclusion in the Zelix script:

exclude *.* + extends org.eclipse.osgi.util.NLS;

This line causes any class that extends the NLS class to be excluded from obfuscation.

A second issue also relates to NLS classes. When Zelix does flow-control obfuscation (for more information on this feature, see the Zelix site), it adds a static variable to various classes. This is usually not a problem, but with NLS classes it is. The NLS static initializer attempts to resolve the these static variables which it thinks are localized strings, and this initialization fails for these new variables added by Zelix. This doesn’t cause visible runtime problems, but it does pollute the logs. But again, this problem can be solved by adding a single exclude to the Zelix properties file:

obfuscateFlowExclude *.* extends org.eclipse.osgi.util.NLS *;


Try it for yourself

That ends this brief summary of my obfuscation process. If you would like to try this out for yourself, you can download the scripts that I use. Applying them to your own build process should be pretty straightforward, but if you have any questions, feel free to ask me questions.

Download obfuscation scripts

In my opinion, the best way to work with PDE Build is to get a very simple build running in your environment. Then make small, incremental additions, verifying the integrity of the process after each change. So the immediate problem for a team becomes: How do I get that first build running?

To help with this, I’ve put together a simple feature-based RCP product build. Just download and import the sample projects. If you follow the instructions in the included readme.txt file, you should have a build running fairly quickly. Before you know it, you’ll be seeing that always elusive BUILD SUCCESSFUL message flash across your screen!

Note: If you want to run this sample build using Eclipse 3.2, you’ll need to work around a PDE Build defect. For instructions on how to do this, check out the “Problems parsing feature file in a product build” section on the PDE Build wiki page.

Update – October 6, 2009: The sample projects have now been updated to work with Eclipse 3.5.1