Type Annotations in Java 8: Tools and Opportunities

In previous versions of Java, developers could write annotations only on declarations. With Java 8, annotations can now also be written on any use of a type such as types in declarations, generics, and casts:

@Encrypted String data;
List<@NonNull String> strings;
myGraph = (@Immutable Graph) tmpGraph;

At first glance, type annotations aren't the sexiest feature of the new Java release. Indeed, annotations are just syntax! -- tools are what give annotations their semantics (i.e., their meaning and behavior). This article introduces the new type annotation syntax and practical tools to boost productivity and build higher-quality software.

In the financial industry, our fluctuating market and regulatory environments mean that time to market is more important than ever. Sacrificing security or quality, however, is not an option: simply confusing percentage points and basis points can have serious consequences. The same story is playing out in every other industry.

As a Java programmer, you're probably already using annotations to improve the quality of your software. Consider the @Override annotation, which was introduced back in Java 1.5. In large projects with non-trivial inheritance hierarchies, it's hard to keep track of which implementation of a method will execute at runtime. If you're not careful, when modifying a method declaration, you might cause a subclass method to not be called. Eliminating a method call in this manner can introduce a defect or security vulnerability. In response, the @Override annotation was introduced so that developers could document methods as overriding a superclass method. The Java compiler then uses the annotations to warn the developer if the program doesn't match their intentions. Used this way, annotations act as a form of machine-checked documentation.

Annotations have also played a central role in making developers more productive through techniques such as metaprogramming. The idea is that annotations can tell tools how to generate new code, transform code, or behave at run-time. For example, the Java Persistence API (JPA), also introduced in Java 1.5, allows developers to declaratively specify the correspondence between Java objects and database entities using annotations on declarations such as @Entity. Tools such as Hibernate use these annotations to generate mapping files and SQL queries at run-time.

In the case of JPA and Hibernate, annotations are used to support the DRY (Don't Repeat Yourself) principle. Interestingly, wherever you look for tools for supporting development best-practices, annotations are not hard to find! Some notable examples are reducing coupling with Dependency Injection and separating concerns with Aspect Oriented Programming.

This raises the question: if annotations are already being used to improve quality and boost productivity, why do we need type annotations?

The short answer is that type annotations enable more --- they allow more kinds of defects to be detected automatically and give you more control of your productivity tools.

Type Annotation Syntax

In Java 8, type annotations can be written on any use of a type, such as the following:

@Encrypted String data
List<@NonNull String> strings
MyGraph = (@Immutable Graph) tmpGraph;

Introducing a new type annotation is as simple as defining an annotation with the ElementType.TYPE_PARAMETER target, ElementType.TYPE_USE target, or both targets:

@Target({ElementType.TYPE_PARAMETER, ElementType.TYPE_USE})
public @interface Encrypted { }

The ElementType.TYPE_PARAMETER target indicates that the annotation can be written on the declaration of a type variable (e.g., class MyClass<T> {...}). The ElementType.TYPE_USE target indicates that the annotation can be written on any use of a type (e.g., types appearing in declarations, generics, and casts).

Once annotations on types are in the source code, like annotations on declarations, they can both be persisted in the class file and made available at run-time via reflection (using the RetentionPolicy.CLASS or RetentionPolicy.RUNTIME policy on the annotation definition). There are two primary differences between type annotations and their predecessors. First, unlike declaration annotations, type annotations on the types of local variable declarations can also be retained in class files. Second, the full generic type is retained and is accessible at run-time.

Although the annotations can be stored in the class file, annotations don't affect the regular execution of the program. For example, a developer might declare two File variables and a Connection variable in the body of a method:

File file = ...;
@Encrypted File encryptedFile = ...;
@Open Connection connection = ...;

When executing the program, passing either of these files to the connection’s send(...) method will result in same method implementation being called:

// These lines call the same method
connection.send(file);
connection.send(encryptedFile);

As you'd expect, the lack of runtime effect implies that, while the types of parameters can be annotated, methods cannot be overloaded based on the annotated types:

public class Connection{
     void send(@Encrypted File file) { ... } 
     // Impossible:
     // void send( File file) { ... }
     . . .
}

The intuition behind this limitation is that the compiler doesn't have any way of knowing the relationship between annotated and un-annotated types, or between types with different annotations.

But wait! -- there's an @Encrypted annotation on the variable encryptedFile corresponding to the parameter file in the method signature; but where is the annotation in the method signature corresponding to the @Open annotation on the connection variable? In the call connection.send(...), the connection variable is referred to as the method's "receiver". (The "receiver" terminology is from the classic Object Oriented analogy of passing messages between objects). Java 8 introduces a new syntax for method declarations so that type annotations can be written on a method's receiver:

void send(@Open Connection this, @Encrypted File file)

Again, since annotations don't affect execution, a method declared with the new receiver parameter syntax has the same behavior as one using the traditional syntax. In fact, currently the only use of the new syntax is so that a type annotation can be written on the type of the receiver.

A full explanation of the type annotation syntax, including syntax for multi-dimensional arrays, can be found on the JSR (Java Specification Request) 308 website.

Detecting Defects with Annotations

Writing annotations in the code serves to emphasize the errors in buggy code:

@Closed Connection connection = ...;
File file = ...;
…
connection.send(file); // Bad!: closed and unencrypted!

However, the above code will still compile, run, and crash --- Java’s compiler does not check user-defined annotations. Instead, the Java platform exposes two APIs, the Java Compiler Plug-in and Pluggable Annotations Processing APIs, so that 3rd-parties can develop their own analyses.

In the previous examples, the annotations were, in effect, qualifying what values variables could contain. We could imagine other ways of qualifying the File type: @Open File, @Localized File, @NonNull File; we could imagine these annotations qualifying other types too, such as @Encrypted String. Because type annotations are independent of the Java type system, concepts expressed as annotations can be re-used for many types.

But how might these annotations be checked automatically? Intuitively, some annotations are subtypes of other annotations, and their usage can be type-checked. Consider the problem of preventing SQL injection attacks caused by the database executing user-provided (tainted) input. We might think of data as being either @Untainted or @MaybeTainted, corresponding to whether the data is guaranteed to be free from user input:

@MaybeTainted String userInput;
@Untainted String dbQuery;

The @MaybeTainted annotation can be thought of as a supertype of the @Untainted annotation. There are a couple ways of thinking about this relation. First, the set of values that might be tainted must be a superset of the values which we know aren't tainted (a value that is certainly untainted can be an element of the set of values that may be tainted). Conversely, the @Untainted annotation provides a strictly stronger guarantee than the @MaybeTainted annotation. So let's see if our subtyping intuition works in practice:

userInput = dbQuery; // OK
dbQuery = "SELECT FROM * WHERE " + userInput; // Type error!

The first line checks out -- we can't get in trouble if we make the assumption that an untainted value is actually tainted. Our subtyping rule reveals a bug in the second line: we're trying to assign the supertype to the more-restrictive subtype.

The Checker Framework

The Checker Framework is a framework for checking Java annotations. First released in 2007, the framework is an active open-source project led by JSR 308 specification co-lead, Prof. Michael Ernst. The Checker Framework comes prepackaged with a broad array of annotations and checkers for detecting defects such as null-pointer dereferences, unit of measure mismatches, security vulnerabilities, and threading/concurrency errors. Because the checkers use type-checking under the hood, their results are sound --- a checker won’t miss any potential errors, where a tool using just heuristics might. The framework uses the compiler API to run these checks during compilation. As a framework, you can also quickly create your own annotation checkers to detect application-specific defects.

The goal of the framework is to detect defects without forcing you to write a lot of annotations. It does this primarily through two features: smart defaults and control-flow sensitivity. For example, when detecting null pointer defects, the checker assumes that parameters are non-null by default. The checker can also make use of conditionals to determine that dereferencing an expression is safe.

void nullSafe(Object nonNullByDefault, @Nullable Object mightBeNull){
     nonNullByDefault.hashCode(); // OK due to default
     mightBeNull.hashCode(); // Warning!
     if (mightBeNull != null){
        mightBeBull.hashCode(); // OK due to check
     }
}

In practice, defaults and control-flow sensitivity mean that you rarely have to write annotations in the body of methods --- the checker can infer and check the annotations automatically. By keeping the semantics for annotations out of the official Java compiler, the Java team has ensured that 3rd party tool designers and users can make their own design decisions. This allows customized error checking to meet a project’s individual needs.

The ability to define your own annotations also enables what you might consider domain-specific type-checking. For example, in finance, interest rates are often quoted using percentages while the difference between rates is often described using basis points (1/100th of 1%). Using the Checker Framework’s Units Checker, you could define two annotations @Percent and @BasisPoints to make sure you don’t mix up the two:

BigDecimal pct = returnsPct(...); // annotated to return @Percent 
requiresBps(pct); // error: @BasisPoints is required

Here, because the Checker Framework is control-flow sensitive, it knows that pct is a @Percent BigDecimal at the time of the call to requiresBps(pct) based on two facts: First, returnsPct(...) is annotated to return a @Percent BigDecimal; second, pct hasn’t been reassigned before the call to requiresBps(...). Often developers use naming conventions to try to prevent these kinds of defects. What the Checker Framework gives you is a guarantee that these defects don’t exist, even as the code changes and grows.

The Checker Framework has already been run on millions of lines of code, exposing hundreds of defects in even well-tested software. Perhaps my favorite example: when the framework was run on the popular Google Collections library (now Google Guava), it revealed null-pointer defects that even extensive testing and heuristic-based static analysis tools had not.

These kinds of results are achievable without cluttering the code. In practice, verifying a property with the Checker Framework requires only 2-3 annotations per thousand lines of code!

For those of you using Java 6 or Java 7, you can still use the Checker Framework to improve the quality of your code. The framework supports type annotations written as comments (e.g., /*@NonNull*/ String). Historically, the reason for this is that the Checker Framework was co-developed with JSR 308 (the type annotation specification) beginning back in 2006.

While the Checker Framework is the best framework for taking advantage of the new syntax for error checking, it’s not the only one right now. Both Eclipse and IntelliJ are type-annotation-aware:

Boosting Productivity with Type Annotations

The main driver behind the new type annotation feature was error checking. Perhaps not surprisingly, error checking tools have the best current and planned support for annotations. However, there are very compelling applications for type annotations in productivity tools as well. To get a feeling for why, consider these examples of how annotations are used:

The annotations are declarative specifications of (1) how tools should generate code or auxiliary files, and (2) how the tools should impact the runtime behavior of the program. Using annotations in these ways can be considered metaprogramming. Some frameworks, such as Lombok, take metaprogramming with annotations to the extreme, resulting in code that barely looks like Java anymore.

Let’s first consider Aspect Oriented Programming (AOP). AOP aims to separate concerns such as logging and authentication from the main business logic of the program. With AOP, you run a tool at compile-time which adds additional code to your program based on a set of rules. For example, we could define a rule which automatically adds authentication based on type annotations:

void showSecrets(@Authenticated User user){
     // Automatically inserted using AOP:
     if (!AuthHelper.EnsureAuth(user)) throw . . .;
}

As before, the annotation is qualifying the type. Instead of checking the annotations at compile-time, however, the AOP framework is being used to automatically perform verification at runtime. This example shows type annotations being used to give you more control over how and when the AOP framework modifies the program.

Java 8 also supports type annotations on local declarations which are persisted in the class file. This opens up new opportunities for performing fine-grained AOP. For example, adding tracing code in a disciplined way:

// Trace all calls made to the ar object
@Trace AuthorizationRequest ar = . . .;

Again, type annotations give more control when metaprogramming with AOP. Dependency injection is a similar story. With Spring 4, you could finally use generics as a form of qualifier:

@Autowired private Store<Product> s1;
@Autowired private Store<Service> s2;

Using generics eliminated the need to introduce classes such as ProductStore and ServiceStore, or use fragile name-based injection rules.

With type annotations, it’s not hard to imagine (read: this is not implemented in Spring yet) using annotations to further control injection:

@Autowired private Store<@Grocery Product> s3;

This example demonstrates type annotations serving as a tool for separating concerns, keeping the project’s type hierarchy clean. This separation is possible because type annotations are independent from the Java type system.

The Road Ahead

We’ve seen how the new type annotations can be used to both detect/prevent program errors and boost productivity. However, the real potential for type annotations is in combining error-checking and metaprogramming to enable new development paradigms.

The basic idea is to build runtimes and libraries that leverage annotations to automatically make programs more efficient, parallel, or secure, and to automatically enforce that developers use those annotations correctly.

A great example of this approach is Adrian Sampson’s EnerJ framework for energy efficient computing via approximate computing. EnerJ is based on the observation that sometimes, such as when processing images on mobile devices, it makes sense to trade off accuracy for energy savings. A developer using EnerJ annotates data that is non-critical using the @Approx type annotation. Based on these annotations, the EnerJ runtime takes various short-cuts when working with that data. For example, it might store and perform calculations on the data using low-energy approximate hardware. However having approximate data moving through the program is dangerous --- as a developer you don’t want control flow to be affected by approximate data. Therefore, EnerJ uses the Checker Framework to enforce that no approximate data can flow into data used in control flow (e.g., in an if-statement).

The applications of this approach aren’t limited to mobile devices. In finance, we often face a trade-off between accuracy and speed. In these cases, the runtime can be left to control the number of Monte Carlo paths or convergence criteria, or even run computation on specialized hardware, based on the current demands and available resources.

The beauty of this approach is that the concern of how the execution is performed is kept separate from the core business logic describing what computation to perform.

Conclusion

In Java 8, you can write annotations on any use of a type in addition to being able to write annotations on declarations. By themselves, annotations don’t affect program behavior. However, by using tools such as the Checker Framework, you can use type annotations to automatically check and verify the absence of software defects and boost your productivity with metaprogramming. While it will take some time for existing tools to take full advantage of type annotations, the time is now to start exploring how type annotations can improve both your software quality and your productivity.

Acknowledgements

I thank Michael Ernst, Werner Dietl, and the NYC Java Meetup for providing feedback on the presentation on which this article is based. I thank Scott Bressler, Yaroslav Dvinov, and Will Leslie for reviewing a draft of this article.

About the Author

Todd Schiller is the head of FinLingua, a financial software development and consulting company. FinLingua's consulting practice helps development teams adopt domain-specific language, metaprogramming, and program analysis techniques. Todd is an active member of the software engineering research community; his research on specification and verification has been presented at premier international conferences including ICSE and OOPSLA.