The Parallel Universe Blog

May 01, 2014

Not Your Father's Java: An Opinionated Guide to Modern Java Development, Part 1

This is part 1 in a three-part series: part 2, part 3

More working, useful code has been written in the Java programming language than in any other in history, with the possible exceptions of C and COBOL. When Java was released almost 20 years ago, it took the software world by storm. It was a simpler, safer, alternative to C++, and some time later its performance caught up, too (depending on the exact usage, a large Java program can be slightly slower, as fast, or a little faster than a comparable C++ codebase). It offered truly tremendous productivity benefits over C++, while sacrificing very little (if anything at all) in return.

Java is a blue-collar language – the working person’s trusty tool – adopting only tried and true idioms, and adding features only if they solve major pain points. Whether or not Java has stayed true to its mission or not is an open question, but it certainly tries not to let current fashions sway it too far off course. Java has been used to write code for anything from smart-cards, through embedded devices, and all the way to mainframes. It is even being used to write mission- and safety-critical hard realtime software.

And yet, in recent years, the Java programming language has gained some noteriety as well, especially among web startups. Java is verbose relative to languages like Ruby or Python, and its web frameworks used to require extensive amounts of XML configuration, especially when compared to configuration-free frameworks like Rails. In addition, Java’s widespread use in large enterprise companies led to the adoption of programming patterns and practices that might have a place in a very large programming team working for a company with extensive bureaucracy, but do not belong in a fast-moving-things-breaking startup.

But all the while, Java has changed. The language recently acquired lambda expression and traits; libraries provide it with true lightweight threads – just like Erlang’s and Go’s. And, most importantly, a more modern, lightweight approach now guides API, library and framework design, replacing all the old heavyweight, XML-laden ones.

Another thing has happened in the Java ecosystem in the past few years: a bunch of good implementations of alternative languages for the JVM have started gaining popularity; some of those languages are quite good (my personal favorites are Clojure and Kotlin). But even with those languages as viable (and sometimes recommended) options, Java does have several advantage over other JVM languages, among them: familiarity, support, maturity, and community. With modern tools and modern libraries, Java actually has a lot going for it. It is not surprising, thenrefore, that many Silicon Valley startups, once they grow a bit, come back to Java, or, at the very least – to the JVM.

This opinionated, introductory guide is intended for the Java programmer (all 9 million of them) who wants to learn how to write modern, lean Java, or for the Python/Ruby/Javascript programmer who’s heard (or may have experienced) bad things about Java and is curious to see how things have changed and how they can get Java’s awesome performance, flexibility and monitoring without sacrificing too much coolness.

The JVM

For those unfamiliar with Java terminology, Java is conceptually made of three parts: Java, the programming language, the Java runtime libraries, and the Java Virtual Machine, or JVM. If you’re familiar with Node.js, Java the language is analogous to JavaScript, the runtime libraries are analogous to Node.js itself, and the JVM would be analogous to V8. The JVM and runtime libraries are packaged together into what is known as the Java Runtime Environment, or the JRE (although often when people say “JVM” they actually mean the entire JRE). The Java Development Kit, or the JDK, is a version of the JRE that includes development tools like javac, the Java compiler, and various monitoring and profiling tools. The JRE comes in several flavors, like those made for embedded devices, but in this blog post series, we will only be referring to the JRE made for server (or desktop) machines, known as Java SE (Standard Edition).

There are quite a few implementations of the JVM (or the JRE) – some are open-source and some are commercial. Some are highly specific: for example, there are JVMs for hard-realtime embedded software, and those made for huge RAM sizes (in the hundreds of gigabytes). But we will be using HotSpot, the free, “common” JVM implementation made by Oracle, which is also available as part of the open-source OpenJDK.

Java was built for the JVM, and the JVM was built for Java (recently, though, the JVM has undergone some modifications specifically with other programming languages in mind). But what is the JVM? This talk by Cliff Click explains what the JVM does, but put simply, the JVM is an abstraction-implementation magic machine. It takes nice, simple, and useful abstractions, like infinite memory and polymorphism – which sound costly to implement – and implements them so efficiently that they can easily compete with runtimes that don’t provide these useful abstractions. More specifically, the JVM has the best garbage collection implementations in widespread production use, and its JIT allows it to inline and optimize virtual method calls (which are at the core of the most useful abstractions in most programming languages), making them extremely cheap while preserving all of their usefulness. The JVM’s JIT (Just-In-Time compiler) is basically a highly advanced profile guided optimizing compiler running concurrently with your application.

The JVM also hides many of the idiosyncracies of the underlying hardware/OS platform, like the memory model (how and when code running on different CPU cores sees changes to variables made on other cores) and access to timers. It also offers dynamic runtime-linking of all code, hot code swapping, and monitoring of pretty much everything that’s going on in the JVM itself, and in the Java libraries.

That is not to say that the JVM is perfect. Right now its missing the possibility to embed complex structs inside arrays (this is scheduled to be resolved in Java 9), and proper tail-call optimization. Nevertheless, the JVM is so mature, well-tested, fast, flexible, and allows for such detailed runtime profiling and monitoring, that I wouldn’t consider running a critical, non-trivial server process on anything else.

But enough theory. Before we go any further, you should download and install the latest JDK here, or, if you prefer, use your OS’s package manager to install a recent version of the OpenJDK.

The Build

We will start our tour of modern Java with the build tool. Java has had several build tools over its longish history (Ant, Maven), and yes, most of them were based on XML. But the modern Java developer uses Gradle (which has recently become Android’s official build tool). Gradle is a mature, heavily developed, modern Java build tool, that uses a DSL built on top of the Groovy language to specify the build process. It combines the simplicity of Maven with the power and flexibility of Ant, while throwing away all that XML. But Gradle is not without its faults: while it makes the most common things easy and declaratives, there are quite a few things that are quite, though not very, common, but still require dropping down to imperative Groovy.

So let’s create a new modern Java project with Gradle. First, we’ll download Gradle here, and install it. Now, we’ll create our project, which we shall call JModern, by first creating the jmodern directory, changing into that directory, and running

gradle init --type java-library

Gradle creates a skeleton project, with some stub classes (Library.java and LibraryTest.java) which we will need to delete:

Gradle init directory structure

Our source code goes into src/main/java/ while our test code goes in src/test/java/. Let’s call our main class jmodern.Main (so its source file is src/main/java/jmodern/Main.java), which for now will be a variation of Hello World, but in order to have some fun with Gradle, we will use Google’s Guava library as well. Use your favorite text editor to create src/main/java/jmodern/Main.java, which will initially consist of this code:

package jmodern;

import com.google.common.base.Strings;

public class Main {
    public static void main(String[] args) {
        System.out.println(triple("Hello World!"));
        System.out.println("My name is " + System.getProperty("jmodern.name"));
    }

    static String triple(String str) {
        return Strings.repeat(str, 3);
    }
}

Let’s also create a small unit-test suite in src/test/java/jmodern/MainTest.java:

package jmodern;

import static org.hamcrest.CoreMatchers.*;
import static org.junit.Assert.*;
import org.junit.Test;

public class MainTest {
    @Test
    public void testTriple() {
        assertThat(Main.triple("AB"), equalTo("ABABAB"));
    }
}

Now, we’ll modify build.gradle in the main project directory to be:

apply plugin: 'java'
apply plugin: 'application'

sourceCompatibility = '1.8'

mainClassName = 'jmodern.Main'

repositories {
    mavenCentral()
}

dependencies {
    compile 'com.google.guava:guava:17.0'

    testCompile 'junit:junit:4.11' // A dependency for a test framework.
}

run {
    systemProperty 'jmodern.name', 'Jack'
}

The build file sets jmodern.Main as the main class, it declares Guava as a dependency, and sets the value of the jmodern.name system property, which we read in our program. When we run:

gradle run

Gradle will download Guava from Maven Central, compile our program, and run it with Guava on the classpath, and jmodern.name set to "Jack". That’s it.

Now, for kicks, let’s run the unit tests:

gradle build

The test report, now found in build/reports/tests/index.html, looks like this:

Gradle test report

The IDE

Some people say that IDEs are there to hide problems with the programming language. Well, I don’t have an opinion about that, but having a good IDE always helps, regardless of the programming language you’re using, and Java’s got the best around. While the choice of an IDE is not as important as anything else in this article, of the “big three” Java IDEs: Eclipse, IntelliJ IDEA, and NetBeans, you should really use either IntelliJ or NetBeans. IntelliJ is probably the most powerful of the three, while NetBeans is the most intuitive and easiest to get started with (and, in my opinion, the best looking). Also, NetBeans has the best Gradle support thanks to the Gradle plugin (which can be installed by going to Tools -> Plugins -> Available Plugins). Eclipse is (still?) the most popular of the three. I abandoned it some years ago, and from what I hear it’s become kind of a mess, but if you’re a long-time Eclipse user and are happy with it, that’s OK, too.

Here’s how our little project looks in NetBeans, after installing the Gradle plugin:

NetBeans

What I like best about NetBeans’ Gradle support is that the IDE takes not only the project dependencies from the build file, but all other configurations as well, so we only need to specify them once – in the build file. If you’re adding new dependencies to the build file while the project is open in NetBeans, you’ll want to right-click the project and select “Reload Project”, so that NetBeans can download the dependencies. If you then right-click the “Dependencies” node of the project in the IDE and choose “Download Sources”, NetBeans will download the dependencies’ source code and Javadoc, so you can step into the third-party library code in the debugger, or see API documentation as you type.

Documenting Your Code in Markdown

Java has long had really good API documentation with Javadoc, and Java developers are accustomed to writing Javadoc comments. But the modern Java developer likes Markdown, and would like to write spice up their Javadoc with Markdown. To do that, we will use the Pegdown Doclet project (a Doclet is a Javadoc plugin) by making the following additions to our build file: Before the dependencies section, we will add

configurations {
    markdownDoclet
}

and we’ll add this line to dependencies:

markdownDoclet 'ch.raffael.pegdown-doclet:pegdown-doclet:1.1.1'

Finally, put this somewhere in the build file:

javadoc.options {
    docletpath = configurations.markdownDoclet.files.asType(List) // gradle should relly make this simpler
    doclet = "ch.raffael.doclets.pegdown.PegdownDoclet"
    addStringOption("parse-timeout", "10")
}

Now we can use Markdown in our Javadoc comments, complete with syntax highlighting!

You might want to turn off your IDE’s comment formatting (in Netbeans: Preferences -> Editor -> Formatting, choose Java and Comments, and uncheck Enable Comments Formatting). IntelliJ has a plugin that renders our Markdown Javadocs in the IDE.

To test our setup, let’s add a fancy Markdown Javadoc to a method called randomString (which we’ll write in the next section):

/**
 * ## The Random String Generator
 *
 * This method doesn't do much, except for generating a random string. It:
 *
 *  * Generates a random string at a given length, `length`
 *  * Uses only characters in the range given by `from` and `to`.
 *
 * Example:
 *
 * ```java
 * randomString(new Random(), 'a', 'z', 10);
 * ```
 *
 * @param r      the random number generator
 * @param from   the first character in the character range, inclusive
 * @param to     the last character in the character range, inclusive
 * @param length the length of the generated string
 * @return the generated string of length `length`
 */
public static String randomString(Random r, char from, char to, int length) ...

Then, generate the javadocs with gradle javadoc, which will put the html files in build/docs/javadoc/. Our doc will look like this:

Markdown Javadoc

I don’t use markdown in comments often, as they don’t render well in IDEs. But this does make life much easier when you want to include code examples in your Javadoc.

Write Succinct Code with Java 8

The recent release of Java brought the biggest change to the language since its original release with the addition of lambda expressions. Lambda expressions (with type inference) address one of the biggest issues people have had with the Java language, namely unjustified verbosity when doing simple stuff. To see how much lambda expressions help, I took the most infuriatingly verbose, simple data manipulation example I could think of, and wrote it in Java 8. It generates a list of random “student names” (just random strings), groups them by their first letter, and prints out a nicely formatted student directory. So now let’s run our program after changing our Main class to this:

package jmodern;

import java.util.List;
import java.util.Map;
import java.util.Random;
import static java.util.stream.Collectors.*;
import static java.util.stream.IntStream.range;

public class Main {
    public static void main(String[] args) {
        // generate a list of 100 random names
        List<String> students = range(0, 100).mapToObj(i -> randomString(new Random(), 'A', 'Z', 10)).collect(toList());

        // sort names and group by the first letter
        Map<Character, List<String>> directory = students.stream().sorted().collect(groupingBy(name -> name.charAt(0)));

        // print a nicely-formatted student directory
        directory.forEach((letter, names) -> System.out.println(letter + "\n\t" + names.stream().collect(joining("\n\t"))));
    }

    public static String randomString(Random r, char from, char to, int length) {
        return r.ints(from, to + 1).limit(length).mapToObj(x -> Character.toString((char)x)).collect(Collectors.joining());
    }
}

Java infers the types of all lambdas’ arguments, but everything is still type safe, and if you’re using an IDE, you’ll get autocomplete and refactoring for all type-inferred variables. Java does not infer types for local variables (like the auto keyword in C++ or var in C# or Go) because that would arguably hurt code readability. But that doesn’t mean you have to manually type the types (heh). For example, type Alt+Enter in NetBeans on this line: students.stream().sorted().collect(Collectors.groupingBy(name -> name.charAt(0))) and the IDE will assign the result to a local variable of the appropriate type (in this case, Map<Character, String>).

If we wanted to go a little crazier with the functional style, we could write the main method like so:

public static void main(String[] args) {
    range(0, 100)
            .mapToObj(i -> randomString(new Random(), 'A', 'Z', 10))
            .sorted()
            .collect(groupingBy(name -> name.charAt(0)))
            .forEach((letter, names) -> System.out.println(letter + "\n\t" + names.stream().collect(joining("\n\t"))));
}

Not your father’s Java, indeed (look ma, no types!), but I would say that taking this too far would certainly go against the spirit of the language.

Even though Java has lambdas, it doesn’t have function types. Instead, lambda expressions are eventually converted to an appropriate functional interface, namely an interface with a single abstract method. This automatically makes a lot of legacy code work beautifully with lambdas. For example, the Arrays.sort method has always taken an instance of the Comparator interface, which simply specifies the single abstract int compare(T o1, T o2) method. In Java 8, a lambda expression can be used to sort an array of strings according to their third character:

Arrays.sort(array, (a, b) -> a.charAt(2) - b.charAt(2));

Java 8 also added the ability to include method implementations in interfaces (which turns them into what is known as “traits”). For example, the FooBar interface below contains two methods, one abstract (foo) and the other (bar) with a default implementation. The useFooBar method, well, uses a FooBar:

interface FooBar {
    int foo(int x);
    default boolean bar(int x) { return true; }
}

int useFooBar(int x, FooBar fb) {
    return fb.bar(x) ? fb.foo(x) : -1;
}

Even though FooBar has two methods, only one of them (foo) is abstract, so it is still a functional interface, and can be created with a lambda expression. For example, the call:

useFooBar(3, x -> x * x)

will return 9.

Simple Lightweight Concurrency with Fibers

For people like me, who are interested in concurrent data structures, the JVM is paradise. On the one hand, it gives you low-level access to the CPU’s concurrency primitives like CAS instructions and memory fences, while on the other it gives you a platform-neutral memory model combined with world-class garbage collectors; the combination is everything you want when building high-performance concurrent data structures. But for those who use concurrency not because they want to but becuase they have to in order to scale their software – namely, everyone else – the Java’s concurrency story is problematic.

True, Java was designed for concurrency from the get-go, and places a lot of emphasis on its concurrency constructs in every release. It’s got state-of-the-art implementations of very useful concurrent data structures (like ConcurrentHashMap, ConcurrentSkipListMap, and ConcurrentLinkedQueue) – not even Erlang and Go have those – and is usually 5 years or more ahead of C++ when it comes to concurrency, but using all this stuff correctly and efficiently is pretty damn hard. First we had threads and locks, and those worked fine for a while, until we needed more concurrency and that approach didn’t scale very well. Then we had thread pools and events: those scale quite well, but can even be harder to reason about, especially in a language that does not protect against racy mutation of shared state. Besides, if your problem is that kernel threads don’t scale well, then asynchronous handling of events is a bad idea. Why not simply fix threads? That’s precisely the approach taken by Erlang and (much later) Go: lightweight, user-mode threads. Those allow mapping domain concurrency (like number of concurrent users) directly to program concurrency (lightweight threads). They allow for a simple, familiar, blocking programming style without sacrificing scalability, and for efficient use of synchronization constructs simpler than locks and semaphores.

Quasar is an open-source library made by us, that adds true lightweight threads to the JVM (in Quasar they’re called fibers), where they can work naturally alongside plain (OS) threads. Quasar also has CSP mechanisms just like Go’s, and a very Erlang-like actor system. Fibers are certainly the modern developer’s weapon of choice when it comes to concurrency. They are simple, elegant and very performant. Let’s play with them for a bit.

First, we’ll setup the build. Merge the following into build.gradle:

configurations {
    quasar
}

dependencies {
    compile "co.paralleluniverse:quasar-core:0.5.0:jdk8"
    quasar "co.paralleluniverse:quasar-core:0.5.0:jdk8"
}

run {
    jvmArgs "-javaagent:${configurations.quasar.iterator().next()}" // gradle should make this simpler, too
}

This will be our new Main.java (if you’re using NetBeans, you’ll want to right-click the project and select “Reload Project” after adding the new dependencies):

package jmodern;

import co.paralleluniverse.fibers.Fiber;
import co.paralleluniverse.strands.Strand;
import co.paralleluniverse.strands.channels.Channel;
import co.paralleluniverse.strands.channels.Channels;

public class Main {
    public static void main(String[] args) throws Exception {
        final Channel<Integer> ch = Channels.newChannel(0);

        new Fiber<Void>(() -> {
            for (int i = 0; i < 10; i++) {
                Strand.sleep(100);
                ch.send(i);
            }
            ch.close();
        }).start();

        new Fiber<Void>(() -> {
            Integer x;
            while((x = ch.receive()) != null)
                System.out.println("--> " + x);
        }).start().join(); // join waits for this fiber to finish
    }
}

We now have two fibers communicating via a channel.

Strand.sleep, and all of the Strand class’s methods, work equally well whether we run our code in a fiber or a plain Java thread. Let’s now replace the first fiber with a plain (heavyweight) thread:

new Thread(Strand.toRunnable(() -> {
    for (int i = 0; i < 10; i++) {
        Strand.sleep(100);
        ch.send(i);
    }
    ch.close();
})).start();

and this works just as well (of course, we could have millions of fibers running in our app, but only up to a few thousand threads).

Now, let’s try channel selection (which mimics Go’s select statement):

package jmodern;

import co.paralleluniverse.fibers.Fiber;
import co.paralleluniverse.strands.Strand;
import co.paralleluniverse.strands.channels.Channel;
import co.paralleluniverse.strands.channels.Channels;
import co.paralleluniverse.strands.channels.SelectAction;
import static co.paralleluniverse.strands.channels.Selector.*;

public class Main {
    public static void main(String[] args) throws Exception {
        final Channel<Integer> ch1 = Channels.newChannel(0);
        final Channel<String> ch2 = Channels.newChannel(0);

        new Fiber<Void>(() -> {
            for (int i = 0; i < 10; i++) {
                Strand.sleep(100);
                ch1.send(i);
            }
            ch1.close();
        }).start();

        new Fiber<Void>(() -> {
            for (int i = 0; i < 10; i++) {
                Strand.sleep(130);
                ch2.send(Character.toString((char)('a' + i)));
            }
            ch2.close();
        }).start();

        new Fiber<Void>(() -> {
            for (int i = 0; i < 10; i++) {
                SelectAction<Object> sa
                        = select(receive(ch1),
                                receive(ch2));
                switch (sa.index()) {
                    case 0:
                        System.out.println(sa.message() != null ? "Got a number: " + (int) sa.message() : "ch1 closed");
                        break;
                    case 1:
                        System.out.println(sa.message() != null ? "Got a string: " + (String) sa.message() : "ch2 closed");
                        break;
                }
            }
        }).start().join(); // join waits for this fiber to finish
    }
}

Starting with Quasar 0.6.0 (in development), you can use lambda expressions directly in the select statement (to try this at home, you’ll need to change Quasar’s version in the build file from 0.5.0 to 0.6.0-SNAPSHOT and add maven { url "https://oss.sonatype.org/content/repositories/snapshots" } to the repositories section), so the code running in the last fiber could also be written so:

for (int i = 0; i < 10; i++) {
    select(
        receive(ch1, x -> System.out.println(x != null ? "Got a number: " + x : "ch1 closed")),
        receive(ch2, x -> System.out.println(x != null ? "Got a string: " + x : "ch2 closed")));
}

Now let’s try some high-performance IO with fibers:

package jmodern;

import co.paralleluniverse.fibers.*;
import co.paralleluniverse.fibers.io.*;
import java.io.IOException;
import java.net.InetSocketAddress;
import java.nio.*;
import java.nio.charset.*;

public class Main {
    static final int PORT = 1234;
    static final Charset charset = Charset.forName("UTF-8");

    public static void main(String[] args) throws Exception {
        new Fiber(() -> {
            try {
                System.out.println("Starting server");
                FiberServerSocketChannel socket = FiberServerSocketChannel.open().bind(new InetSocketAddress(PORT));
                for (;;) {
                    FiberSocketChannel ch = socket.accept();
                    new Fiber(() -> {
                        try {
                            ByteBuffer buf = ByteBuffer.allocateDirect(1024);
                            int n = ch.read(buf);
                            String response = "HTTP/1.0 200 OK\r\nDate: Fri, 31 Dec 1999 23:59:59 GMT\r\n"
                                            + "Content-Type: text/html\r\nContent-Length: 0\r\n\r\n";
                            n = ch.write(charset.newEncoder().encode(CharBuffer.wrap(response)));
                            ch.close();
                        } catch (IOException e) {
                            e.printStackTrace();
                        }
                    }).start();
                }
            } catch (IOException e) {
                e.printStackTrace();
            }
        }).start();
        System.out.println("started");
        Thread.sleep(Long.MAX_VALUE);
    }
}

What have we done here? First, we launch a fiber that will loop forever, accepting TCP connection attempts. For each accepted connection, it spawns another fiber that reads the request, sends a response and then terminates. While this code is blocking on IO calls, under the covers it uses async EPoll-based IO, so it will scale as well as any async IO server (we’ve greatly improved IO performance in Quasar 0.6.0-SNAPSHOT).

But enough writing Go in Java. Let’s try Erlang.

Fault-Tolerant Actors and Hot Code Swapping

The actor model, (semi-)popularized by the Erlang language, is intended for the writing of fault-tolerant, highly maintainable applications. It breaks the application into independent fault-containment units – actors – and formalizes the handling of and recovery from errors.

Before we start playing with actors, we’ll need to add this dependency to the dependencies section in the build file: compile "co.paralleluniverse:quasar-actors:0.5.0".

Now let’s rewrite our Main class yet again, this time the code is more complicated as we want our app to be fault tolerant:

package jmodern;

import co.paralleluniverse.actors.*;
import co.paralleluniverse.fibers.*;
import co.paralleluniverse.strands.Strand;
import java.util.Objects;
import java.util.concurrent.ThreadLocalRandom;
import java.util.concurrent.TimeUnit;

public class Main {
    public static void main(String[] args) throws Exception {
        new NaiveActor("naive").spawn();
        Strand.sleep(Long.MAX_VALUE);
    }

    static class BadActor extends BasicActor<String, Void> {
        private int count;

        @Override
        protected Void doRun() throws InterruptedException, SuspendExecution {
            System.out.println("(re)starting actor");
            for (;;) {
                String m = receive(300, TimeUnit.MILLISECONDS);
                if (m != null)
                    System.out.println("Got a message: " + m);
                System.out.println("I am but a lowly actor that sometimes fails: - " + (count++));

                if (ThreadLocalRandom.current().nextInt(30) == 0)
                    throw new RuntimeException("darn");

                checkCodeSwap(); // this is a convenient time for a code swap
            }
        }
    }

    static class NaiveActor extends BasicActor<Void, Void> {
        private ActorRef<String> myBadActor;

        public NaiveActor(String name) {
            super(name);
        }

        @Override
        protected Void doRun() throws InterruptedException, SuspendExecution {
            spawnBadActor();

            int count = 0;
            for (;;) {
                receive(500, TimeUnit.MILLISECONDS);
                myBadActor.send("hi from " + self() + " number " + (count++));
            }
        }

        private void spawnBadActor() {
            myBadActor = new BadActor().spawn();
            watch(myBadActor);
        }

        @Override
        protected Void handleLifecycleMessage(LifecycleMessage m) {
            if (m instanceof ExitMessage && Objects.equals(((ExitMessage) m).getActor(), myBadActor)) {
                System.out.println("My bad actor has just died of '" + ((ExitMessage) m).getCause() + "'. Restarting.");
                spawnBadActor();
            }
            return super.handleLifecycleMessage(m);
        }
    }
}

Here we have a NaiveActor spawning an instance of a BadActor, which occasionally fails. Because our naive actor watches its protege, it will be notified of its untimely death, and re-spawn a new one.

In this example, Java is rather annoying, especially when it comes to testing the type of a message with instanceof and casting objects from one type to another. This is much better done in Clojure or Kotlin (I’ll post a Kotlin actor example one day), with their pattern matching. So, yes, all this type-checking and casting is certainly bothersome, and if this type of code encourages you to give Kotlin a try – you should certainly go for it (I have, and I like Kotlin a lot, but it has to mature before it’s fit for use in production). Personally, I find this annoyance rather minimal.

But let’s get back to substance. A crucial component of actor-based fault-tolerant systems, is reducing downtime not only caused by application erros, but also by maintenance. We will explore the JVM’s manageabity in part 2 of this guide, but now we’ll play with actor hot code swapping.

There are several ways to perform actor hot code swapping (e.g. via JMX, which we’ll learn about in part 2), but now we’ll do it by monitoring the file system. First, create a subdirectory under the project’s directory, which we’ll call modules. Then add the following line to build.gradle’s run section:

systemProperty "co.paralleluniverse.actors.moduleDir", "${rootProject.projectDir}/modules"

Now, in a terminal window, start the program (gradle run, remember?). While the program is running, let’s go back to the editor, and modify our BadActor class a bit:

@Upgrade
static class BadActor extends BasicActor<String, Void> {
    private int count;

    @Override
    protected Void doRun() throws InterruptedException, SuspendExecution {
        System.out.println("(re)starting actor");
        for (;;) {
            String m = receive(300, TimeUnit.MILLISECONDS);
            if (m != null)
                System.out.println("Got a message: " + m);
            System.out.println("I am a lowly, but improved, actor that still sometimes fails: - " + (count++));

            if (ThreadLocalRandom.current().nextInt(100) == 0)
                throw new RuntimeException("darn");

            checkCodeSwap(); // this is a convenient time for a code swap
        }
    }
}

We add the @Upgrade annotation because that’s the class we want to upgrade, and modify the code so that now the actor fails less often. Now, while our original program is still running, let’s rebuild our program’s JAR, by running gradle jar in a new terminal window. For those unfamiliar with Java, JAR (Java Archive) files are used to package Java modules (we’ll discuss modern Java packaging and deployment in part 2). Finally, in that second terminal, copy build/libs/jmodern.jar into our modules directory. In Linux/Mac:

cp build/libs/jmodern.jar modules

You’ll see the running program changing (depending on your OS, this can take up to 10 seconds). Note that unlike when we restarted BadActor after it failed, when we swap the code, its internal state (the value of counter) is preserved.

Designing fault-tolerant applications with actors is a big subject, but I hope you’ve now got a little taste of what’s possible.

Advanced Topic: Pluggable Types

Before signing off, we’ll venture into dangerous territory. The tool we’ll play with in this section cannot be added to the modern Java developer’s toolbelt just yet, as using it is still too cumbersome, and it would greatly benefit from IDE integration, which is currently very sketchy. Nevertheless, the possibilities it opens are so cool, that if the tool continues to be developed and fleshed out, and if it’s not overused in a frenzy, it might prove invaluable, and that is why it’s included here.

One of the potentially most powerful (and probably least discussed) new features in Java 8, is type annotations and pluggable type systems. The Java compiler now allows adding annotations wherever it allows specifying a type (we will shortly see an example). This, combined with the ability to plug annotation processors into the compiler, opens the door to pluggable type systems. These optional type systems, that can be turned on and off, can add powerful type-based static verification to Java code. The Checker framework is a library that allows (advanced) developers write their own pluggable type systems, complete with inheritence, type inference and more. It also comes pre-packaged with quite a few type systems, that verify nullability, tainting, regular expressions, physical units, immutability and more.

I haven’t been able to get Checker to work well with NetBeans, so for this section, we’ll continue without our IDE. First, let’s modify build.gradle a bit. We’ll merge the following:

configurations {
    checker
}

dependencies {
    checker 'org.checkerframework:jdk8:1.8.1'
    compile 'org.checkerframework:checker:1.8.1'
}

into the respective configurations and dependencies sections.

Then, we’ll put this somewhere in the build file:

compileJava {
    options.fork = true
    options.forkOptions.jvmArgs = ["-Xbootclasspath/p:${configurations.checker.asPath}:${System.getenv('JAVA_HOME')}/lib/tools.jar"]
    options.compilerArgs = ['-processor', 'org.checkerframework.checker.nullness.NullnessChecker,org.checkerframework.checker.units.UnitsChecker,org.checkerframework.checker.tainting.TaintingChecker']
}

(as I said, cumbersome).

The last line says that we would like to use Checker’s nullness type system, the physical units type system, and the tainted data type system.

Now let’s run a few experiments. First, let’s try the nullability type system, which is supposed to prevent null pointer exceptions:

package jmodern;

import org.checkerframework.checker.nullness.qual.*;

public class Main {
    public static void main(String[] args) {
        String str1 = "hi";
        foo(str1); // we know str1 to be non-null

        String str2 = System.getProperty("foo");
        // foo(str2); // <-- doesn't compile as str2 may be null
        if (str2 != null)
            foo(str2); // after the null test it compiles
    }

    static void foo(@NonNull String s) {
        System.out.println("==> " + s.length());
    }
}

The Checker framework developers were kind enough to annotate the entire JDK for nullability return types, so you should be able to pass the return value of library methods that never return null as a @NonNull parameter (but I haven’t tried).

Next, let’s try the units type system, supposed to prevent unit conversion errors:

package jmodern;

import org.checkerframework.checker.units.qual.*;

public class Main {
    @SuppressWarnings("unsafe") private static final @m int m = (@m int)1; // define 1 meter
    @SuppressWarnings("unsafe") private static final @s int s = (@s int)1; // define 1 second

    public static void main(String[] args) {
        @m double meters = 5.0 * m;
        @s double seconds = 2.0 * s;
        // @kmPERh double speed = meters / seconds; // <-- doesn't compile
        @mPERs double speed = meters / seconds;

        System.out.println("Speed: " + speed);
    }
}

Cool. According to the Checker documentation, you can also define your own physical units.

Finally, let’s try the tainting type system, which helps you track tainted (potentially dangerous) data obtained, say, as a user input:

package jmodern;

import org.checkerframework.checker.tainting.qual.*;

public class Main {
    public static void main(String[] args) {
        // process(parse(read())); // <-- doesn't compile, as process cannot accept tainted data
        process(parse(sanitize(read())));
    }

    static @Tainted String read() {
        return "12345"; // pretend we've got this from the user
    }

    @SuppressWarnings("tainting")
    static @Untainted String sanitize(@Tainted String s) {
        if(s.length() > 10)
            throw new IllegalArgumentException("I don't wanna do that!");
        return (@Untainted String)s;
    }

    // doesn't change the tainted qualifier of the data
    @SuppressWarnings("tainting")
    static @PolyTainted int parse(@PolyTainted String s) {
        return (@PolyTainted int)Integer.parseInt(s); // apparently the JDK libraries aren't annotated with @PolyTainted
    }

    static void process(@Untainted int data) {
        System.out.println("--> " + data);
    }
}

Checker gives Java pluggable (can be turned on or off) intersection types (you can have @m int or @m double), with type inference (e.g. a null check turns a @Nullable into a @NonNull), and type annotations can even be added to pre-compiled libraries with the help of a tool. Not even Haskell can do that!

Checker isn’t ready for primetime yet, but when it is, if used wisely, it could become one of the modern Java developer’s most powerful tools.

Wrapping Up (For Now)

We’ve seen how with the changes made in Java 8, along with modern tools and libraries, Java bears little resemblance to the Java of old. While the language still shines in large applications, the language and ecosystem now nicely compete with newer “simple” languages, which are less mature, less tested, less platform-independent, have much smaller ecosystems and almost always poorer performance than Java. We have learned how the modern Java programmer writes code, but we have hardly begun to unleash the full power of Java and the JVM. In particular, we are yet to see Java’s awesome monitoring and profiling tools, or its new, lean, web microframeworks. We will visit those topics in the upcoming blog posts.

In case you want to get a head start, in part 2 we will be discussing modern Java packaging (with Capsule, which is a little like npm, only much cooler), monitoring and management (with VisualVM, JMX, Jolokia and Metrics), profiling (with Java Flight Recorder, Mission Control, and Byteman), and benchmarking (with JMH). In part 3, we will discuss writing lightweight, scalable HTTP services with Dropwizard and Comsat, Web Actors, and dependency injection with JSR-330.

Discuss on Hacker News

Part 2

Join our mailing list

Sign up to receive news and updates.

Tags: , ,

comments powered by Disqus