Inspired by my experience with the ICD extension in Ethiopia, which was hard coded into DHIS version 1.3.0.17, I was determined to make DHIS 2 sufficiently extensible to be able to make the ICD extension without making a fork. On the planning session we had about the upcoming course INF5750 I said that I wanted to look into different ways of making DHIS 2 extensible. In this section I am going to present my experiences from this task.
Initially I had a plan to make DHIS 2 extensible through the use of plug-ins. Plug-ins are encapsulated pieces of software designed to be plugged into a mother application. This plug-ins provide additional features to the mother application. Plug-ins are usually placed in one or more specified directories where the plug-ins are automatically discovered at application start-up. The plug-ins can also be specified in an configuration file or a start-up script. I and Aasen, my coworker during the duration of the course, decided to start looking for existing plug-in frameworks. The mother application needs to be designed with plug-ins in mind, and a plug-in framework provides the sockets where plug-ins can be plugged into. Our initial plan was to find a plug-in framework and find out how this can be used with the mother application to provide the necessary extensibility to be able to make an ICD plug-in.
We looked into the following plug-in frameworks. This frameworks are pure plug-in frameworks. Pure plug-in frameworks are designed to have just a minimal mother application which basically just starts up the plug-in framework, discover and start plug-ins. All the functionality is provided by the plug-ins. These plug-ins will also provide sockets which other applications can plug into. These extension sockets are commonly called extension points. Other plug-ins can subscribe to an extension point. The plug-in providing the extension point is responsible for providing the necessary features and resources to the plug-ins subscribing to the extension point. Such an application naturally needs a minimum number of plug-ins to be able to operate.
For some of the frameworks like JPF you can make the mother application as large as you want, it is not necessary to make a pure plug-in application from it. JPF is a minimal plug-in framework, it only provides the minimal functionality necessary. Eclipse is an application in its own right, providing many useful extension points. None of these plug-in frameworks, except Emersion, was made with web applications in mind. Emersion is designed for web applications and uses JPF as its plug-in framework.
Emersion did not really fit our need, but we decided to use it as a starting point to make our own server side plug-in framework. The DHIS 2 application was at this time in its infancy of development, so we concluded that it would be too difficult to build a plug-in framework into the evolutionary prototype being developed. This would have required us to convince all the other developers to build their modules using unfamiliar concepts like extensions points. Besides we had not yet proved that it was feasible or desirable to make DHIS 2 using a pure server side plug-in framework. For this reasons we decided to make a throwaway prototype as a proof of concept. We gave this prototype the unimaginative name plug-in-demo.
For reasons I will explain in the following paragraphs it became a real challenge to make plug-in-demo. The development of plug-in-demo was more than enough work for the course. We never came around to implementing extension points in the real DHIS 2 application, and we could plainly forget about making an ICD plug-in. I continued to work on plug-in-demo after the course had ended.
One of the major issues that created complexity for the plug-in-demo was the need to embed a servlet container. A servlet container is a web server built to support servlets, which provide dynamic content using the Java language. JPF needed to be booted before the servlet container, and needed to run as long as the servlet container was running. The alternative would be to start JPF each time a user accessed a web page, which is a huge overhead that would make the application unbearably slow.
Embedding a servlet container within a JPF context created class loader issues which was difficult to debug . In Java the class loader is responsible for loading classes when a class instance is requested. The class loader needs to have access to the description of the classes which the application needs. This created problems for plug-in-demo because JPF and the servlet container were running within different class loader contexts. Even if JPF had access to the description of a class it did not mean that the servlet container had access. We managed to resolve this, but it was hard to debug this problems and could possibly create problems in the future.
A sever problem with using JPF was that the application became difficult to test. The creation of unit tests is considered a important practise for the creation of stable applications. An unit test is a script that test a single unit of code, like a class. The test should be done in isolation, the dependencies the class has on other classes should be satisfied without calling the other classes. For example if a class depend on data from a database the test script should not call the database, but provide the data by other means. It was really difficult to satisfy the dependency a class had on JPF. I wasted a lot of time trying to find a solution to this without avail.
An equally important testing issue are integration tests. This test the interaction between different parts of an application. This kind of tests uses the actual classes and databases used by the application. In the JPF context this means testing the plug-ins when they are actually working together. To do this the mother application and all the plug-ins has to be deployed to their assigned places, and the JPF framework has to be started. This I had to do manually by building and deploying all the plug-ins in the plug-in-demo application, and then start the application. If the application did not start I had too look in the log files created by the application to try an locate the bug, fix the bug and build and deploy the application again. This is time consuming.
In addition to the embedding and testing issues there was issues with integrating the other frameworks used by DHIS 2, and also plug-in-demo, with JPF. I spent a lot of time finding and implementing ways for JPF to integrate with Spring. To this I found solutions I am quite happy with. Spring is like JPF an all-encompassing framework affecting the architecture of the application, and require the developer to get familiar with unfamiliar concepts.
The forth and final challenge I met in building plug-in-demo was the challenge of using Maven to build the application. It was already decided to use Maven as build tool for DHIS 2. The responsibility of a build tools is, among other things, to compile all the source files, link the application to all of its dependencies and install or deploy the application. Maven also offer support for executing automatic tests. To build an application based on JPF I had to make a plug-in to Maven. First we used Maven 1, and I made a reasonably functioning plug-in to that version. Later on the Maven project finished a total remake of the Maven application called Maven 2. The DHIS 2 project started to use Maven 2. It was more difficult to make a JPF plug-in for Maven 2 than it had been for Maven 1. The design of Maven 2 had not considered applications of the JPF kind. I managed to make a JPF plug-in to Maven 2, but it was only a really ugly hack.
Reflecting on how difficult it was for me to make this framework and the thought of having to decide on where to place extension points, I decided that it would create more complexity using a pure plug-in framework than it would give value in better extensibility. Neither did I find a satisfactory solution to the Maven 2 build issue. There are other approaches for making an application extensible, which is not so all-encompassing.
There exists many different way of making an application extensible. Some extensibility is achieved through Object Oriented Programming (OOP) and even more is achieved through an Inversion of Control (IoC) container which is the primary function of Spring. OOP offers the possibility of extending classes and implementing interfaces. If a class is doing almost what you want it to, but not quite, you can extend the class to make the necessary changes. The new class can act as its parent class through polymorphism. It is also possible to make a new implementation of an interface. A class usually interacts with other classes and when a class is instantiated it also needs access to instances of the classes its depends on. An IoC container has the job of inserting this dependencies. By using an IoC container it is easy to change which implementation of an interface is handed to a class, or to insert a sub-class of the depended class. Common to this forms of extensions is that they require some configuration file or source file changes in the core application.
The existing DHIS 2 application uses OOP and Spring to facilitate extensibility. In addition it provides an Application Programming Interface (API). This is a handy way to provide extensibility by providing a way for other application to interact with DHIS 2. Other applications can talk to the business layer of DHIS 2. Having an API do not in itself solve the problem of having visually separate extension to DHIS. An API can be a handy way for plug-ins to talk to the mother application, however. An API is not as likely to change as its implementation.
An other way to achieve extensibility is through hooks. Hooks can be implemented as a global array of function pointers or object references. The extension insert its own function or object into this array, and this array is iterated through at some point in the core application. I have seen the concept of hooks being used in other web applications, a hook is similar to an extension point. Hooks are more light weight than a plug-in framework, therefore it is no need to embed a servlet container. Hooks are easier to test because they do not require any framework, and the integration and build problems are history for the same reason. Each time a user request a web page which access a part of the application providing hooks, all the extensions using this hook has to be included. For the mother application to include the extension some minimal configuration or source file changes has to be made. Automatic discovery is also possible. The discovery algorithm would have to be run every time a user access web pages with hooks, however. Each individual hook will most likely be more difficult to implement, and it will probably require more effort to include an extension of this sort into the mother application. Figuring out which hooks to provide will be just as difficult as figuring out which extension pints to provide.
Hopefully the local adaptations in the upcoming version of DHIS can be done using extensions, in place of forking or visually separate applications feeding on the DHIS database. This remains to be seen.