The Java Evolution of Eclipse Collections

30 Second Introduction – What is Eclipse Collections?

Eclipse Collections is a drop in replacement for the Java Collections framework. It has JDK-compatible List, Set and Map implementations with a rich API, as well as additional types not found in the JDK such as Bags, Multimaps and BiMaps. Eclipse Collections also has a full complement of primitive containers. It was developed internally at Goldman Sachs for 10 years before being open sourced in 2012 as GS Collections. In 2015, it was migrated to the Eclipse foundation, and since then, all active development for the framework was done under the Eclipse Collections name and repository. If you’d like some good introductory literature, take a look at Donald Raab’s InfoQ articles, "GS Collections By Example" part I and part II.

The Domain

Before we dive into any details or code examples, let’s walk through the domain that we will use in this article for our code snippets. As you can see in the diagram below:

(Click on the image to enlarge it)

We have a list of people (type Person), each person can have a list of Pets, and each pet is of a certain PetType enum.

Version 8 for Java 8

Prior to the Eclipse Collections release, EC was compatible with Java versions 5 - 7. You could also use Java 8 and leverage lambdas and method references when using the rich API, and in fact it worked quite well.

But that’s all you really got. Eclipse Collections was compatible with Java 8, but it did not use or embrace it. Now, starting with Eclipse Collections, we have made the design decision to be compatible with Java 8+ in order to start leveraging some of the cool Java 8 features in our own codebase.

Optional

Optional is one of the most popular new features for Java 8. From the Javadoc, "A container object which may or may not contain a non-null value. If a value is present, isPresent() will return true and get() will return the value". Basically, Optional helps protect us from NullPointerExceptions by forcing us to handle potentially null items. So, where can we use this in Eclipse Collections? RichIterable.detectWith() is a perfect fit. detectWith accepts a Predicate argument and returns the first element in the collection that satisfies that condition. If it does not find any element, it returns null. Thus, in 8.0 we introduced detectWithOptional(). Instead of returning the element or null, it returns an Optional which is then left to the user to handle. See the code example below (taken from our kata tutorial materials):

Person person = this.people.detectWith(Person::named, "Mary Smith");
//null pointer exception
Assert.assertEquals("Mary", person.getFirstName());
Assert.assertEquals("Smith", person.getLastName());

Here, we want to find Mary Smith. When we call detectWith, person gets set to null as we couldn’t find anyone that satisfied the predicate. Thus, the code throws a NullPointerException.

Person person = this.people.detectWith(Person::named, "Mary Smith");
if (person == null)
{
      person = new Person("Mary", "Smith");
}
Assert.assertEquals("Mary", person.getFirstName());
Assert.assertEquals("Smith", person.getLastName());

Next, pre Java 8, we could always use a null check as seen above. But Java 8 provides an Optional, so let’s use it!

Optional<Person> optional = 
          this.people.detectWithOptional(Person::named, "Mary Smith");
Person person = optional.orElseGet(() -> new Person("Mary", "Smith"));
Assert.assertEquals("Mary", person.getFirstName());
Assert.assertEquals("Smith", person.getLastName());

Here, instead of returning null, detectWithOptional returns an Optional wrapper around Person. It’s now up to me to decide how I want to handle this potentially null value. In my code, I’m going to call orElseGet() to create a new Person instance if it is null. The test passes, and we’ve avoided any exceptions!

Collectors

If you use streams in your code, you’ve most likely used a Collector before. A Collector is a way to implement a mutable reduction operation. For instance, Collectors.toList() allows us to accumulate items from a stream into a list. The JDK has several "built in" Collectors that can be found on the Collectors class. See below some Java 8 (no Eclipse Collections) examples.

List<String> names = this.people.stream()
                         .map(Person::getFirstName)
                         .collect(Collectors.toList());
// Output:
// [Bob, Ted, Jake]
int total = this.people.stream().collect(
          Collectors.summingInt(Person::getNumberOfPets));
// Output:
// 4

Since we can now leverage streams using Eclipse Collections, we should have our own built in Collectors as well – Collectors2. Many of these Collectors are for Eclipse Collections specific data structures; features that you can’t get from the JDK out of the box – things like toBag(), toImmutableSet(), etc.

(Click on the image to enlarge it)

This chart scratches the surface of the Collectors2 API. The top boxes are all different data structures you can put the result of a Collectors2 into, and the bullets beneath are some of the different APIs available in order to do so. You can see we have type support for both JDK as well as Eclipse Collections data structures, and also primitive collections. You can even use our familiar collect(), select(), reject() etc. API via Collectors2.

There is also the possibility of interop with Collectors and Collectors2; the two are not mutually exclusive. See below in this example we are using JDK 8 Collectors, but then using EC 8.0 Collectors2 for convenience:

Map<Integer, String> people = this.people
   .stream()
   .collect(Collectors.groupingBy(
      Person::getNumberOfPets,
      Collectors2.makeString()));

Map<Integer, String> people2 = this.people
   .stream()
   .collect(Collectors.groupingBy(
      Person::getNumberOfPets,
      Collectors.mapping(
         Object::toString,
         Collectors.joining(","))));
// Output: {1=Ted, Jake, 2=Bob}

The two code snippets produce the same output, but notice the subtle difference: Eclipse Collections offers the makeString() functionality, which creates a comma separated collection of elements represented as a String. In order to do this using just Java 8, it requires a bit more work by calling Collectors.mapping(), transforming each object to its toString value, and joining it by a comma.

Default Methods

For a framework like Eclipse Collections, default methods are a great new addition to the JDK. We can implement new APIs on some of our highest sitting interfaces without having to change many of the implementations below. reduceInPlace() is one of the new methods we added to RichIterable – what does it do?

    /**
     * This method produces the equivalent result as 
     * {@link Stream#collect(Collector)}.
     * <p>
     * <pre>
     * MutableObjectLongMap<Integer> map2 =
     *     Lists.mutable
                .with(1, 2, 3, 4, 5)
                .reduceInPlace(
                  Collectors2.sumByInt(
                    i -> Integer.valueOf(i % 2), Integer::intValue));
     * </pre>
     * @since 8.0
     */
    default <R, A> R reduceInPlace(Collector<? super T, A, R> collector)
    {
        A mutableResult = collector.supplier().get();
        BiConsumer<A, ? super T> accumulator = collector.accumulator();
        this.each(each -> accumulator.accept(mutableResult, each));
        return collector.finisher().apply(mutableResult);
    }

reduceInPlace is the equivalent of using Collector on a stream. But why did we need to add this to Eclipse Collections? The reason is pretty interesting; once we go into the Immutable or Lazy API that Eclipse Collections offers, we can no longer leverage the streaming API. At that point, we can’t get the same functionality that we get by using Collectors because stream() is no longer available to us and neither are the subsequent API calls; this is where reduceInPlace comes into play.

As shown below, once we call .toImmutable() or .asLazy() on a collection, we can no longer call .stream(). So, if we want to leverage collectors, we now can use .reduceInPlace() and achieve the same result.

Primitive Collections

We’ve had the benefit of primitive collections since GS Collections 3.0. Eclipse Collections adds memory optimized collections for all primitive types, with similar interfaces to the Object types as well as symmetry amongst the primitive types.

(Click on the image to enlarge it)

As shown above, there are several benefits to using primitive collections. Memory savings are huge as you can avoid boxing your non primitive types. Beginning with Java 8, we have three primitive types (int, long, and double) that use specialized primitive streams and lambda expressions. In Eclipse Collections, if you want the same lazy evaluation, we offer that API directly on all eight primitive types. Let’s take a look at some code examples:

Streams - like Iterator

IntStream stream = IntStream.of(1, 2, 3);
Assert.assertEquals(1, stream.min().getAsInt());
Assert.assertEquals(3, stream.max().getAsInt());

java.lang.IllegalStateException: stream has already been operated upon or closed
	at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:229)
	at java.base/java.util.stream.IntPipeline.reduce(IntPipeline.java:474)
	at java.base/java.util.stream.IntPipeline.max(IntPipeline.java:437)

LazyIntIterable lazy = IntLists.mutable.with(1, 2, 3).asLazy();
Assert.assertEquals(1, lazy.min());
Assert.assertEquals(3, lazy.max()); //reuse!

Above, we are creating an IntStream of 1, 2, and 3, and trying to call min() and max() on it. Java 8 streams are like an iterator in that you cannot reuse them. Eclipse Collections LazyIterables allow for reuse. Let’s take a look at a more complex example:

List<Map.Entry<Integer, Long>> counts = this.people.stream().flatMap(
                                  person -> person.getPets().stream())
       .collect(Collectors.groupingBy(Pet::getAge, Collectors.counting()))
       .entrySet()
       .stream()
       .filter(e -> e.getValue().equals(Long.valueOf(1)))
       .collect(Collectors.toList());
// Output: [3=1, 4=1]

MutableIntBag counts2 = this.people.asLazy()
       .flatCollect(Person::getPets)
       .collectInt(Pet::getAge)
       .toBag()
       .selectByOccurrences(IntPredicates.equal(1));
// Output: [3, 4]

Here, we want to filter pet ages appearing only once. Since Java 8 does not have a Bag data structure (which maps items to their counts) , we must group our collections by counting into a map. Notice that once we have called collectInt() on our pets, we have now moved onto the primitive collections and API. When we call .toBag(), we have a specialized primitive IntBag. selectByOccurrences() is a Bag specific API that allows us to filter out items in our Bag based on the number of occurrences they have.

Java 9 - What’s next?

As we know, Java 9 introduces many interesting changes to the Java ecosystem, like the new module system and internal API encapsulation. Eclipse Collections must change in order to be compatible.

In our 8.2 release, we had to modify any of our methods that used reflection in order to get the project building. You can see an example here in ArrayListIterate:

public final class ArrayListIterate
{
   private static final Field ELEMENT_DATA_FIELD;
   private static final Field SIZE_FIELD;
   private static final int MIN_DIRECT_ARRAY_ACCESS_SIZE = 100;

   static
   {
       Field data = null;
       Field size = null;
       try
       {
           data = ArrayList.class.getDeclaredField("elementData");
           size = ArrayList.class.getDeclaredField("size");
           data.setAccessible(true);
           size.setAccessible(true);
       }
       catch (Exception ignored)
       {
           data = null;
           size = null;
       }
       ELEMENT_DATA_FIELD = data;
       SIZE_FIELD = size;
   }

In this example, once we call data.setAccessible(true), an exception would be thrown. As a workaround, we simply set data and size to null in order to move on. Unfortunately, we can’t use these fields to optimize our iteration patterns anymore, but we are now Java 9 compatible as this fix solved our reflection issues.

There are workarounds for reflection if you are not yet ready to migrate your current code. You can avoid these exceptions being thrown by adding a command line argument, however as a framework we did not want to put this burden on our users. All reflection issues have been proactively solved so you can get started coding in Java 9!

Conclusion

Eclipse Collections continues to grow and evolve with the ever changing Java landscape. If you haven’t already done so, please try it out with your Java 8 code and see the new features described above in action! If you’re new to the framework and need a place to start, here are some resources for you:

More information:

Eclipse Collections

Training material

Reference guide

You can also take a look at the full video of The Java Evolution of Eclipse Collections here.

Happy coding and enjoy!

About the Author

Kristen O'Leary is an associate at Goldman Sachs in the platform group, which is responsible for many of the firm’s technology tools and frameworks. She has contributed several container, API, and performance enhancements to Eclipse Collections. She has also taught classes internally and externally about the framework.