Why Java Modularity Matters

These notes represent a summary of my presentation “Why Java Modularity Matters”. They are provided in the hope that those who attend the presentation will be able to spend less time taking notes and more time listening to me. 🙂

You can also view the PowerPoint slides, if you wish.

What this talk is about

The purpose of this talk is convince you that Java modularity represents an important evolution in the way we develop software. Modularity offers huge benefits to both software developers and end users:

Modularity benefits software developers by increasing the granularity of the abstractions we deal with. These benefits are comparable to those gained by moving from a procedural-based language to an object-oriented one.
Modularity benefits end users by allowing us to flexibly assemble applications that meet end users’ specific needs. Modularity allows us to go beyond the “one shoe fits all” mindset imposed upon us by current technologies.

A (very) short history of software development

The history of software development is often seen as a battle between clients and servers. At first, code executed on centralized mainframes. With client/server architectures, code execution was decentralized. Web applications resulted in code execution being recentralized. Lately, a small but growing number of apps are being developed using “rich client” technology, again decentralizing code execution. Rich internet applications attempt to split the difference.

The Eclipse RCP is often evaluated in terms of this history, but to do so misses the point. The power of Eclipse RCP has more to do with modularity.

An alternative history

An alternative history of software development is based on the idea that progress has been made by increasing the granularity of the abstractions we can deal with. Things began with ones and zeroes, advanced to human-readable languages, procedures and finally to object-orientated languages. For the last 20 years, the class has been our unit of abstraction.

Why is abstraction so important?

Managing complexity

First, abstraction allows us to manage complexity. Human intelligence is aided by “chunking”, or breaking knowledge into compartments that hide internal complexity. Abstraction is key because it enables encapsulation, or information hiding. Without information hiding, it would be almost impossible to develop software to handle complex problems.

Refactoring

Second, abstraction allows for refactoring. Software is developed incrementally and must be improved along the way. Information hiding allows us to refactor internal implementations without breaking external APIs.

Modules are the next level of abstraction

For years we have been using the class as our highest-level abstraction. What if we could create abstractions that represented collections of classes? This talk is about how modules work and how they make this new level of abstraction possible.

Java Modularity big picture

In this talk we’ll be discussing OSGi, but Java Modularity extends beyond OSGi and will eventually be a concept familiar to all Java developers. The Spring Framework is incorporating modularity (using OSGi) and there are JSRs attempting to come up with a general technique.

What is OSGi?

OSGi stands for the Open Services Gateway Initiative.

OSGi is a standard defining a module system for Java. It defines the way in which classes can be grouped into modules and also how those modules interact via dependencies and services.

OSGi as a standard is maintained by the OSGi Alliance, but this organization does not supply an implementation. The most commonly used OSGi implementations are Equinox (created by the Eclipse Foundation) and Felix (created by the Apache Foundation).

OSGi allows for the dynamic assembly of modules into an application. This means that modules can come and go without restarting the application or JVM.

OSGi is service-oriented, meaning that it provides a service-based mechanism for module interaction. OSGi services are sometimes described as a SOA in a JVM, and makes possible the same type of decoupling as SOA.

Equinox and Eclipse

Equinox is the Eclipse Foundation implementation of the OSGi standard. It provides the foundation upon which all other Eclipse technologies rest, including the Eclipse IDE itself.

What is a bundle (during development)?

A bundle is the OSGi term for a module. When talking specifically about OSGi I will use the term bundle. When developing inside of Eclipse, a bundle is simply a project (usually a Java project) that can contain source code and other artifacts.

What makes a bundle unique is the additional information placed in a manifest file located in a META-INF directory. This is really just the standard manifest file that is placed in all JAR files. OSGi leverages this manifest file to provide extra information about the content of a JAR.

What is a bundle (at runtime)?

A bundle is distributed as a JAR file. OSGi bundles can be described as “smart” JARs, because they have some understanding of their contents. These smart JARs provide their metadata to the OSGi framework and this metadata allows these JARs to interact in a modular way.

The basic point here is that modules are nothing special from a distribution point of view. They are simply JARs with some extra properties added to their manifest file.

Bundle Manifest

If you haven’t seen a manifest file before, it is simply a collection of name value pairs. The names associated with OSGi are defined by the OSGi specification. If using Eclipse, there is a Manifest Editor which makes it much easier to maintain these files.

Bundle Identity

Manifest files first allow you define the bundle’s identity. The most important aspect of a bundle’s identity is it’s ID. This ID must be unique, although OSGi does allow you to run multiple versions of the same bundle at the same time.

Unlike standard JAR versions, bundle versions are important at runtime and can be used by the OSGi framework to make decisions about bundle relationships. Versioning is a powerful part of the OSGi framework.

Bundle Lifecycle

A bundle can include an Activator class that manages the bundle’s life-cycle. An Activator contains a start method that is run when the bundle is activated. You can use this method to do any type of initialization logic you require.

An Activator contains a stop method that is run when the bundle is deactivated (usually when an application shuts down). Cleanup work can be performed there.

A bundle’s life-cycle is composed of a number of states.

A bundle is deployed if it is available to the OSGi framework (e.g. in a directory known to the framework).
A bundle moves to an installed state when it is cataloged by a running OSGi framework or if a request is made to install it.
A bundle moves to a resolved state if all of the bundle’s dependencies are met. If a bundle’s dependencies cannot be met, the bundle will stay in an installed state and the OSGi framework will carry on.
When a request is made for a class inside a bundle, it will be activated and first enters a starting state. This is when a bundle is actually loaded into memory and also when the Activator’s start method will fire.
After the start method completes, the bundle is in an active state and will stay in this state until a request is made to uninstall the bundle.
When this request is made, the bundle briefly enters a stopping state during which the Activator’s stop method will fire.
When the stop method completes, the bundle is in an uninstalled state.

Bundle Dependencies

Bundle dependencies can be declared by ID or by package. The basic idea though is that a bundle needs to specify the location of the classes that it depends upon. The Eclipse compiler includes warnings and errors for bundles with missing dependencies.

Bundle Encapsulation

Bundle encapsulation is based on exporting packages. Any Java package in a bundle can be exported, making it available to other bundles expressing dependencies. Exporting a package does not result in the visibility of sub-packages.

Bundle encapsulation is really the key to achieving the benefits of modularity from a software developers perspective. Bundle encapsulation allows us to hide packages from other modules just as classes can hide non-public methods from other classes.

Properly constructed bundles present a public API represented by a small set of exported packages. They hide a hopefully much larger set of internal implementation classes which cannot be referred to by other bundles. This information hiding at the bundle level allows for much more drastic refactoring than was ever possible before.

As an aside, many of the design principles associated with classes can also be applied to modules. Principles such maximizing cohesion and minimizing coupling are key to effective module design.

Dual use jars

What if you have code you want to share between OSGi and non-OSGi applications. The solution is to add the extra manifest information to the shared JAR so that it can be used in both situations. Bundle’s are just JARs, so it’s very easy to use code in both environments.

Note however that running a bundle in a non-OSGi environment exposes your internal packages to others. Incorporating a naming element such as “internal” into your package structure can help to inform developers to avoid importing non-exported classes.

Third party JARs are another example of dual use code. Third party JARs can be turned into bundles by adding extra metadata. It is also increasingly possible to locate third party libraries that are already turned into bundles. Check out the Orbit and SpringSource repositories to see which libraries are available.

Services

OSGi services provide a mechanism for bundle interaction that does not require dependencies.

A service is defined by a Java interface. The bundle containing the interface is considered the service definer.

Any bundle can then provide an implementation for the service by implementing the interface and then registering itself with the service registry.

Any bundle can consume a service by accessing it through the service registry. The service is looked up by the interface provided by the service definer.

Note that the service provider and service consumer need to depend on the service definer, but the service provider and consumer have no specific knowledge of each other. In this way, services allow for the decoupling of bundles.

OSGi servers

The most basic thing you can do with a bundle is run it in an OSGi server. An OSGi server is basically an OSGi framework running on top of a JVM. Adding your own bundles to the mix allows you to create your own application servers running modular code. You can take this as far as you like (e.g. IBM Websphere is an OSGi application).

OSGi web servers

OSGi servers can also run web applications. For instance, both Jetty and Tomcat can be run inside of an OSGi server. This means that you can deploy your web applications as bundles. The web space may become one of the fastest growing area of OSGi development.

While deploying web applications on an OSGi server currently is non-trivial task, new products such as SpringSource dm Server will make the process much easier. Other vendors, such as BEA and JBoss, are also moving to adopt OSGi.

Eclipse Rich Client Platform

The Eclipse Rich Client Platform provides a way to create rich client applications based on the OSGi framework. Eclipse RCP is really the visual expression of Java Modularity, in that it allows user interfaces themselves to be composed from modules.

Bundles can contribute user interface elements such as views, editors, menu items and wizards. A rich client application can by dynamically assembled from these pieces, which is an incredibly powerful and flexible way of developing software.

Eclipse Rich Ajax Platform

The Eclipse Rich Ajax Platform essentially renders a rich client application using Javascript running in a browser. The underlying OSGi framework allows a single set of code to be rendered either using RCP or RAP simply by replacing a set of user interface plug-ins in your application.

This ability to render a user interface in multiple ways is made possible by Java modularity and is a core design principle guiding the development of Eclipse technologies in the future.

Future – e4

The Eclipse Foundation has launched an initiative called e4 to re-imagine what Eclipse related technologies will be like in the future.

There are many elements to this initiative, but some of the key ones relate to the construction of user interfaces. The idea is that a UI created declaratively can be rendered for different platforms and styled using stylesheets.

The combination of this technology with a strong modular framework will make Eclipse RCP an incredibly powerful tool for application development.

Use cases: Shared workbench

As I mentioned earlier, Java Modularity provides great benefits to end-users as well. This is because modular software can be packaged and distributed in ways not possible before.

The first use case that illustrates this is that of a workbench containing multiple applications or “tools”. Eclipse developers will recognize this as the model supported by the Eclipse IDE itself, but we can leverage this model for our own applications.

Another example of this use case is the OneBench software developed by JPMorganChase. This RCP application provides a core workbench providing many global services. Individual “applications” can then be contributed to the workbench by providing the modules that define them. Individual applications can leverage the workbench and global services and so can be developed and deployed much more quickly.

Use cases: License control

Another use case enabled by modularity is the support for license control. It is common (especially for commercial software) to license parts of their software separately. These parts might represent different domains (stocks, commodities, forex) or different levels of functionality (basic, professional, etc.).

These licensable parts correlate well to modules. And the dynamic nature of OSGi allows you to upgrade a client by simply supplying a new module. They may not even have to restart their application.

Use cases: 80/20

The final use case illustrating the power of modularity is the case where you have an application that needs to be slightly modified for each customer. In this case, Java Modularity allows you to separate the part unique to each customer in it’s own module.

What’s the big idea?

The big idea of this talk is that Java Modularity matters.

Modularity in general and OSGi specifically are not fringe technologies useful in highly specialized circumstances. Quite the opposite. A strong argument can be made that in the future we will all be using these technologies (or something similar) to develop our applications.

The benefits of Java Modularity, both to software developers and to end-users, are too great to ignore.