Real-Time Java for the Enterprise

When people hear about real-time computing they often make the mistake of thinking that these have to be fast systems and are nearly always used for control of mechanical systems. It is true that in most cases the response times required are fast, but speed is not what defines a real-time system. The true core of a real-time environment is that the system will guarantee to perform some task within a pre-defined time so that the system's behaviour becomes fully deterministic.

For enterprise class applications what are the advantages (and possibly disadvantages) of deployment on a real-time system? In many cases there is no great advantage. Once the non-functional requirements of a system (load capacity, average response time, peak-load response time, and so on) have been met the application can be deployed and the customer is satisfied. Should something happen that makes a request to, say, an HR application take rather longer than usual the user simply has to wait longer; there is no measurable impact. However, for many financial, enterprise class applications failing to do something within a given time can easily be equated to a cost, possibly a very large one. Financial markets, by their very definition, are extremely volatile and computer trading systems mean that prices can change at the rate of many times a second. If one part of a system decides to make a trade based on the current price and the actual trade gets delayed for whatever reason even a small change can result in a significant loss of money. If this delay happens frequently the system soon becomes less of an advantage and more of a liability.

Java, the language, and the Java Enterprise Edition platform have proved to be very popular for the development of enterprise applications. The ease-of-development, performance and reliability all make Java extremely attractive to developers. However, the Java platform does not support real-time applications, and even running a Java application on a real-time operating system will not make the application deterministic. The biggest obstacle to deterministic behaviour of a Java application is the way memory management is handled by the Java virtual machine (JVM). Unlike earlier languages like C and C++ Java uses a garbage collector to reclaim memory that is no longer being used by an application. The reason for this is to eliminate ‘memory leaks', where a developer forgets to explicitly free memory that is no longer being used. Since the amount of memory in a machine is finite, if this error occurs in some kind of loop eventually the system will run out of memory. (That's not to say you can completely eliminate memory leaks in Java; if a developer maintains references to objects the garbage collector will still not be able to reclaim the memory).

The garbage collector uses a background thread to monitor the heap space where objects are stored. The garbage collector identifies objects that are no longer referenced and reclaim the unused memory. Much of this work involves copying objects from one address to another within the heap. To prevent the possible corruption of data all application threads (referred to as mutator threads) must be paused when this is happening. On modern JVMs these pauses will not be noticeable for well tuned desktop applications. Even for most enterprise applications the use of the concurrent-mark-sweep collector (also known as the low-pause collector) will reduce pauses to levels that are acceptable for most applications. However, for certain classes of enterprise applications like those described earlier, non-deterministic behaviour and therefore Java is not suitable.

To solve this problem a Java Specification Request (JSR) was created to design and implement a real-time specification for Java (RTSJ). This was, in fact, the very first JSR created under the Java Community Process when it was started in 1998. The expert group had several guiding principals in the creation of the specification. Three of these principals are worth noting when considering the applicability to enterprise applications:

1. Backwards compatibility, so that any class file generated by a Java compliant compiler would run on a RTSJ virtual machine.
2. No syntactic extensions. No new keywords or syntactic changes would be made to the Java language. This is partly a result of the first point, but is also essential to enable Java developers to easily migrate to RTSJ and use existing development tools such as NetBeans.
3. Predictable execution. This would be the first priority when making design decisions. The impact of this is that sometimes the typical computing performance may be reduced. This is one reason that an RTSJ system should be considered carefully for general enterprise applications.

The RTSJ specification describes eight areas of extended semantics for Java:

1. Scheduling
2. Memory management
3. Synchronization
4. Asynchronous event handling
5. Asynchronous transfer of control
6. Asynchronous thread termination
7. Physical memory access
8. Exceptions

Threads within the RTSJ can be one of three types, non real-time, soft real-time and hard real-time. A non real-time thread is somewhat obvious, in that it does not have a specific deadline when some action must be completed. It can be scheduled when convenient by the JVM and any impact of the garbage collector is immaterial. Soft real-time threads are ones that do have a deadline for completion of an action. However, there is a certain amount of leeway so that if the action gets completed shortly after when it should have done everything will still continue without any problems. For hard real-time threads they absolutely must complete the action by the deadline; if they do not an unrecoverable error has occurred. RTSJ applications can have threads of all three types running simultaneously. The choice of what type of thread to use is down to the designer of the application and the level of importance attached to the work of a thread. The diagram below shows the relationship between these types of threads:

Diagram 1. Threads in the RTJS

Typically threads will need to exchange data for the application to work. Since a hard real-time thread cannot rely on a non real-time thread to deliver results in real-time wait-free data transfer queues are used to ensure that a hard real-time thread is not blocked by other threads.

To create a new non real-time thread the existing Thread class is used unchanged. For soft real-time threads the RealtimeThread class is used, which is a sub-class of Thread. The constructor of this class takes optional priority and release parameters to define how the JVM should schedule the thread. For hard real-time threads the NoHeapRealtimeThread class is used which is a sub-class of RealtimeThread. The name of the class used for hard real-time threads probably gives you a clue about how it is implemented by the JVM; we'll discuss this more when we get to memory management.

The simplest way to convert an existing application to be a real-time application is to simply change all places where a new thread is started to use the RealtimeThread class instead. This would only make the application a soft real-time application, but shows how easy it can be.

The standard Java Thread class includes the concept of a priority so that more important threads can be given preference to lower priority ones. In RTSJ this is extended further and the specification says that there must be at least twenty eight distinct priority levels supported. Since the priority is represented as an integer an implementation could provide many more. The specification also says that threads of a higher priority will always run in preference to those of a lower priority and if a thread with a higher priority becomes runnable it will pre-empt the currently running thread. The classes used to represent this information are shown in diagram 2:

Diagram 2. Thread execution priorities

The PriorityParameters class encapsulates the integer value of the threads priority. If there are multiple threads with the same priority an importance can be associated with the thread to indicate which thread should be given preference.

One of the key concepts of a real-time system is that it should be possible, in advance, to determine whether the system can meet the requirements of the application. In order to do that, through rate monotonic analysis, the system must know details about the tasks. These are provided to the RTSJ through release parameters. The class hierarchy is shown in diagram 3:

Diagram 3. Release parameters

The ReleaseParameters class contains the cost of the thread in terms of time (which is platform specific and may change if the application is moved to a different platform), the deadline for when this task must be completed and handlers for both a cost overrun and a missed deadline. The types of task that will be processed can be either periodic, in which case they happen repeatedly at a given rate, or aperiodic in which case they could happen at any time. If a hard real-time system contains aperiodic tasks it is impossible to analyse for correctness so the concept of a sporadic task is included. This treats an aperiodic task as if it was periodic, giving it a minimum frequency and therefore making it possible to analyse the system. Making an aperiodic task sporadic has no impact to the way the application runs, it just enables the system to determine whether it can meet the real-time requirements.

The RTSJ must also implement what is called priority inversion avoidance. Diagram 4 illustrates the problem:

Diagram 4. Priority inversion

A low priority thread (at priority P3) acquires the lock on an object and is pre-empted by a higher priority thread (at priority P1), as the specification mandates. The P1 thread then needs the same object but cannot acquire the lock until the P3 thread has released it. Other tasks that are lower priority (P2) prevent the P3 thread from running and releasing the lock. The net effect is that P2 priority threads run in preference to the P1 thread which is not what the specification requires. There are two ways to avoid this. The first is priority inheritance. The system detects that the P3 thread is holding a lock required by the P1 thread and elevates its priority to P1 until the lock is released. This works with no changes to the code by the developer. The second way is priority ceiling emulation. Here the developer must realise that the thread holding the lock needs its priority raised and must put explicit calls in the code to do this. This method is optional in the RTSJ.

For memory management RTSJ uses the concept of a memory area. The class structure for this is shown in diagram 5.

Diagram 5. Real time memory areas

Since garbage collection causes non-deterministic pauses all hard real-time threads must use memory that is not subject to garbage collection. There are two ways to do this: scoped memory and immortal memory.

Scoped memory is memory that has a lifetime defined by the developer of an application. A developer will create a scoped memory area of a specific size and when objects are instantiated the space for them will be allocated from this area. There are two types of scoped memory, linear and variable which refers to the time required to instantiate an object. The LTMemory class represents an area of memory where the time to instantiate an object is the combination of a fixed allocation time and a variable initialisation time. Since the initialisation time is directly proportional to the size of the object the time is linear rather than constant. The VTMemory class represents an area of memory where the allocation mechanism can use any algorithm and hence can vary in time. Once all objects in the scoped memory are no longer referenced the memory is freed to be used again. One scenario for the use of scoped memory is to create a scoped memory area and provide it for use by a NoHeapRealTimeThread. When the thread execution finishes all objects are known to no longer be referenced and the area can be freed.

Immortal memory, as the name suggests, never gets collected in any way. Immortal memory is shared by all threads and should only be used for objects that the developer knows will exist until the JVM terminates.

The RTSJ specification also allows developers to do something that is impossible in standard Java: access physical memory directly. Since this is mostly applicable to embedded real-time applications and not enterprise applications we won't discuss this further in this article.

As you can see the RTSJ provides a range of capabilities to allow an enterprise application to run as a combination of non real-time, soft real-time and hard real-time threads. For financial applications where time is money the critical sections can be coded to run as hard real-time threads and avoid the non-deterministic behaviour of the garbage collector. With the release of RTSJ 2.0 a reference implementation is now available that runs on the freely available, open source Solaris 10 operating environment which has a real-time scheduler. (To provide real-time functionality in the JVM the operating system underneath must also support the concepts of real-time).

The obvious next step for enterprise applications is to use a real-time capable application server. Engineers at Sun have ported the GlassFish open source application server to RTSJ (in only five hours). IBM also has a real-time WebSphere product so progress has already been made. When evaluating the use of real-time Java for your enterprise application remember that, in this situation, nothing is free. You may get guaranteed response time, but this will come at the cost of overall throughput of the system. However, if your Java application really, really must respond within a certain time regardless of what the garbage collector does, RTSJ is now a possibility rather than a probability.

Links

About the Author

Simon Ritter specialises in looking at emerging technologies including grid computing, RFID, wireless sensor networks, robotics and wearable computing. Simon has been in the IT business since 1984 and holds a Bachelor of Science degree in Physics from Brunel University in the U.K.Originally working in the area of UNIX development for AT&T UNIX System Labs and then Novell, Simon joined Sun in 1996 and started working with Java technology; he has spent time doing both Java development and consultancy.