Modular Mind

I believe the next few years are going to be an extremely exciting and productive time for the Eclipse ecosystem. I say this not only because I’m an optimist at heart, but also because there are large scale forces pushing things in this direction.

The modular convergence

The way I see it, some of the large pieces of the software puzzle are converging on a single way of seeing the world. The reason for this convergence is the increasingly obvious benefits of modular software development. So what’s converging?

• OSGi is at the heart of this this process. It’s the glue that holds everything else together.
• Eclipse, meaning the entire ecosystem and not simply the IDE, has obviously been using OSGi for quite a while, but this usage is becoming increasingly explicit.
• Maven has become increasingly focused on OSGi due to obvious synergies in versioning, dependency management and provisioning. I think it’s fair to say that OSGi is going to be a core part of Maven going forward.
• Spring has also made a big bet on OSGi, starting with Spring DM and Spring dm Server. Spring also uses the Eclipse IDE as the basis for it’s development tools and Maven as it’s build solution.

As these technologies continue to become more interrelated, the benefits of using them together is going to increase dramatically. I don’t think it’s too far-fetched to say that these technologies will coalesce to form something like a new new LAMP-style stack for Java development.

What does this mean for Eclipse developers?

It’s in the interest of the Eclipse community to promote this convergence in any way it can. As developers, we should be looking for ways to speed up the process and also take advantage of the resulting synergies. Specifically:

• Eclipse should embrace OSGi as explicitly as possible, in many ways this is already happening. We’re seeing more and more OSGi standard services shipping with Eclipse runtimes and also increasing usage of OSGi services, particularly in the work being done by the e4 team. Another good example of the renaming of the Plug-in Development Environment to the Bundle Development Environment. These changes are a good start, but let’s see how far we can push this.
• When it comes to Maven, the story is unfortunately different. Builds are causing a lot of pain for Eclipse developers, and the quickly multiplying build tools (PDE Build, Buckminster, Athena, B3) are causing some confusion. My questions is: if Maven and Eclipse are both converging based on a shared interest in OSGi, why not focus on Maven as the solution to our build problems?
• Spring is a technology that is not commonly used in Eclipse projects, but it’s focus on OSGi-based modularity would make it a perfect fit. This would also be true for those consuming Eclipse projects, such as Rich Client Platform developers. There’s actually not a lot of technical work to do here, just some evangelization. Spring and the various Eclipse projects have a lot to offer each other, and we should start to take more advantage of this.

Speeding it up

In my opinion, the decision to make OSGi the foundation of the Eclipse ecosystem was one of the best decisions the Eclipse team has ever made. This decision has made it possible for Eclipse to participate in the convergence now taking place, but there’s more work to do. The actions we take now can slow the process down or speed it up. I say we take every opportunity to integrate with those who share an interest in modularity and OSGi. We’ll all be better off for it in the end.

What would I like for Christmas? Thanks for asking! Well, besides a Mac mini, what I’d really like is a simple update manager for Eclipse RCP applications.

Up to now, we’ve had two options for updating our applications: wiring in the Update Manager or coding something from scratch. The problem is that the Update Manager is overkill for most RCP apps and the API is fairly complicated.

What I would like to have is an alternative to the Update Manager that provides very basic update functionality. Call it Update Manager Lite, if you like. For those of you running OS X, what I’m thinking of is something like the Sparkle framework that many Mac applications use.

Would this be of use to anyone else? I’ve opened a Bugzilla entry suggesting this and comments would be appreciated.

Update: There’s already a Bugzilla entry for this feature, so the above entry has been marked as a dup. Please direct any comments you have to the original Bugzilla entry. Thanks to Susan McCourt for pointing me in the right direction!

It’s Friday afternoon and I’m in a contemplative mood. For some reason, I’ve been thinking about my first manager, who I wasn’t very fond of at the time. I’m sure I was a real pain in his backside as well, being a fresh-out-of-college know-it-all.

One of the things that drove me nuts about him was the way he mangled metaphors and sayings. He liked to tell me “never put the horse before the cart” and stuff like that. But one saying he used will always stick with me. He’d say:

Patrick, software is like a bowl full of jello. And our job is to get our arms around that bowl!

I remember thinking that I’d rather not get my arms around a bowl full of jello. What good would that do? It would still be a bowl full of jello, and I’ve never liked jello anyway. But what also struck me at the time was that this really is what software is like to people who don’t leverage information hiding. Spaghetti is almost too clean a metaphor (at least you can pull out some noodles if you want). Jello really captures the nasty, sticky grossness of badly encapsulated software.

Even though I didn’t like this manager at the time, he did inadvertently start me down the path to software modularity. Hopefully by focusing on modularity, we can stop dealing with bowls of jello and at least work with jello shots :-)

While there are a ton of benefits to be gained from adopting OSGi, it’s not a trivial task to migrate your existing code. Class loader issues can bite you and perhaps the biggest pain-point is the migration of third-party libraries.

Third-party libraries are a problem because while bundle repositories are growing in size, there are still a lot of JARs out there not packaged as OSGi bundles.

So what can we do to make it easier to adopt OSGi?

The Knoplerfish answer

Well it turns out that the Knoplerfish OSGi framework has solutions to many of these issues. Among the features offered are:

• Automatic runtime generation of manifests for non-bundle JARs
• Automatic patching of code that uses inappropriate (for OSGi anyway) class loading mechanisms
• Ability to execute static main methods inside of the framework

These solutions are covered in detail in this presentation given last year.

Why are these solutions not more popular?

I’m wondering why these types of solutions are not discussed and suggested more. Are there similar features in Equinox and Felix that I’m not aware of? And if not, are there any plans to implement them?

I’d love to hear from others whether features like these would be useful to you or not. I’d also be interested to hear what else could be done to make OSGi adoption easier. In my opinion, the difficulty of migrating applications to OSGi is one of the main things holding it back. What can we do to fix this?

I love Eclipse RCP. I’ve devoted the last 6 years of my life to developing with RCP as well as teaching it to others. In my opinion it’s one of the most important (and underused) technologies for developing UI applications. Having said all that, the name is horrible and it’s time to change it.

My preference would be to rename Eclipse RCP as part of the Eclipse 4.0 release. There are few times in the life of a project when it changes enough to merit a new name, and for Eclipse RCP this will be one of those times. The work being done as part of the e4 project represents a significant evolution of the platform and will make it useful to a far larger audience. A new name (and branding) will go a long way to encourage the adoption of this technology.

If this approach was taken, we would have a year to select a new name and marketing approach. This post is a bit long, but here is where I would start.

RCP is the user interface of OSGi

The adoption curve for OSGi is turning up and developers are quickly coming to see the benefits of modular architectures. Eclipse RCP is perfectly placed to serve as the UI layer for modular software. The name and branding for RCP should reflect this focus on modularity and on its close relationship to OSGi.

I’ve seen a number of projects switch from RCP to Flex because the decision-makers thought they were choosing between two UI toolkits. They were really deciding between a UI toolkit (Flex) and a modular application framework (RCP). This is a problem.

RCP is moving beyond rich clients

When Eclipse 4.0 is released RCP will be much more than a tool for creating rich client applications. RCP and RAP are converging and there should be common naming scheme that unites them. One approach would be to have a base name for the technology that takes a modifier for each targeted architecture, something like “X Web” and “X RC”.

To use “Eclipse” in the name or not

This is a general issue with Eclipse projects. There is always a tension when a set of projects evolves around a spectacularly successful product. On the one hand, it can be beneficial to leverage the success of that product to promote other projects. But it can also lead to a great deal of confusion, and this is particularly true with Eclipse RCP.

To make this work well requires a rigorous approach to project naming and marketing so that the benefits of name recognition are not overwhelmed by a lack of clarity. In my opinion, the best example of this is the job done by the Apache Foundation. While the Apache HTTP server still exists, the word Apache has been successfully rebranded to apply to a whole host of projects. Each of those projects is uniformly named Apache X and benefits from this association.

The Eclipse Foundation has, unfortunately, not done as good a job. Eclipse is still, well, Eclipse. Most developers assume that other Eclipse Foundation projects are somehow related to the IDE and therefore unapplicable to their use cases. Individual projects are named in a wide variety of ways, sometimes using the Eclipse name (Eclipse RCP), sometimes not (BIRT), and sometimes using it as part of an acronym (EMF). Taken as a whole, the project names in the Eclipse ecosystem are extremely confusing.

On a side note (and I’m sure this is a minority opinion), I think that the umbrella names for the release train (Europa, Galileo, etc.) reinforce the idea that Eclipse (and all of it’s projects) are about the IDE.

In any case, to successfully use the Eclipse name the following two things would need to occur:

1. The adoption of a common naming scheme “Eclipse X” for all projects.
2. The application of this scheme to Eclipse itself (e.g. Eclipse Workbench)

Without this approach, my opinion is that using the Eclipse name in association with a rebranded RCP is not a good option. My vote would be for a stand-alone name.

Getting started

So much work and thought is going into e4 and there is enormous potential here. Let’s give this technology the name and branding that it needs to reach a wide audience.

I’ve purposely shied away from suggesting names here, but I’ve cross-posted to a Bugzilla entry where potential names can be discussed.

Note: If you agree with this post, please consider commenting on the Bugzilla entry (even a “+1″ would help).

I’ve seen a number of blog posts and tweets lately asking some version of the question Why is OSGi important? If you’re one of the many people looking around at the increasing usage of OSGi and wondering whether it matters to you, here’s my answer.

I’m going to start by making a pretty audacious claim, which is that OSGi is one of the most important technologies to have arisen in the last 20 years. This does not mean, however, that OSGi is a revolutionary technology. In fact, OSGi is so important because it represents the logical next step in the long-term evolution of software development.

Where we’re coming from

To understand what I mean, let’s go back 20 to 30 years to the time when object oriented languages first became popular. One of the main reasons we adopted OO at that time was because it allowed us to hide many of the implementation details of our code.

Moving from procedural languages to OO languages allowed us to develop classes exposing a contract defined by a set of public methods.

The result was that much of our code was invisible outside of its class, and this had profound implications for the way we develop software. By accepting the apparent restriction of visibility we gained immense freedom. We gained the freedom to reuse classes without knowing their implementation details. We gained the freedom to refactor our code without worrying about the consumers of a class.

Can you imagine what a pain it would be if you had to develop software without information hiding?

Where we’re headed

Now imagine what it would be like if you could hide not only the methods within a class but entire sets of classes within a JAR. Imagine that JARs could define public contracts the same way classes do, and that these contracts would be enforced both during development and at runtime. Imagine that we could achieve all of the benefits of information hiding (managing complexity, code reuse, testability, refactoring, etc.) at an entirely new level.

OSGi makes this possible by offering up the standard Java package as a new unit of information hiding. When our code is running inside of an OSGi framework, each package in a JAR can be either exposed or hidden from consumers of that JAR.

Just as a class has a small set of public methods representing its contract with consumers, a modularized JAR (a bundle in OSGi terms) has a small set of exported packages representing its public contract. The bulk of our code lives in internal packages hidden from other JARs.

Imagine being able to rename classes, split or combine classes, move classes from one package to another, move entire packages from one JAR to another, all without having to worry about impacting the consumers of a JAR. So many of these types of refactorings are skipped now out of fear. Package level information hiding gives us the confidence we need to perform these refactorings, allowing us to react with agility to the changing needs of our users.

Modularity is inevitable

Whether OSGi in particular succeeds or not, JAR level information hiding is inevitable. The benefits are simply too great to ignore, and in 5 or 10 years we’ll all be wondering how we could have possibly lived without it.

Currently, OSGi is the only tool we have to accomplish this. Luckily for us it’s a well though-out, well tested, standards-based solution. I can’t think of one reason (besides perhaps its name) to develop an alternative to OSGi. It’s here. It works. Let’s use it.

It’s time for OSGi

Steve McConnell has a great quote that really gets at the heart of what OSGi is trying to achieve.

In Code Complete, he writes:

Software development has advanced in large part by increasing the granularity of the aggregations that we have to work with.

Because this granularity of aggregation is so critical, the move from unmodular to modular practices is as important as the move from procedural to object-oriented practices. For 20 years we’ve been limited to using the class as our unit of abstraction. As successful as that has been, it’s time to move on to modules. Its time for OSGi.