The Parallel Universe Blog

May 15, 2014

An Opinionated Guide to Modern Java, Part 3: Web Development

By Ron

This is part 3 in a three-part series: part 1, part 2

Welcome to OGMJ, part 3. After playing with modern Java code in part 1, and exploring deployment, management, monitoring and benchmarking of JVM applications in part 2, it is time to delve into modern Java web development. But before we do, let me address some reader responses from last time.

Last week we’ve seen how the JVM takes monitoring seriously, and how it exposes every aspect of its runtime behavior. One of the readers mentioned a tool, which I’ve had the pleasure to use a few times but neglected to mention it in last week’s installment. This tool, JITWatch, it is called, helps to analyze the deepest reaches of the JVM, and is therefore only recommended to the most performance-conscious expert Java (or other JVM language) developers. It takes the output logs generated by adding these JVM options – -XX:+UnlockDiagnosticVMOptions -XX:+TraceClassLoading -XX:+LogCompilation -XX:+PrintAssembly – and turns them into explorable insight into how, and when, the JVM optimizes your code. It shows which methods are compiled to machine code (it even displays the generated machine code if the -XX:+PrintAssembly is available, on certain JVM builds), which are inlined, why certain methods have not been inlined, and much, much more. Usage information, as well as screenshots, can be found on the project’s wiki.

Some readers have opined that Capsule is somehow less standard than OS packages. This is not true, as capsules are stateless executables, and require no installation whatsoever; as such, they don’t even perform the same function as OS packages, which standardize installation. If your application requires some stateful installation (i.e., installation that requires user-guided configuration at install time), then Capsule is not for you. Others have expressed concern over capsules relying on the availability of a Maven repository at runtime. To this I say that, obviously, software comes at different points on an availability/”mission-criticalness” spectrum, and different applications might choose different tradeoffs when it comes to safety vs. convenience. You can create a capsule with no support for automatic upgrades, or a capsule with no external dependencies at all (and embed all dependencies directly in the capsule). You’ll still get automatic choice of a Java runtime and automatic JVM configuration when you launch the capsule. If you do choose to use external Maven dependencies, I think that there is no reason to suspect accidental inclusion of a wrong dependency version – or other such mishaps – any more than when dependency resolution is done at build time. On the contrary: in the former case, the dependencies are explicitly specified in the capsule, and the capsule can be asked to print out its entire dependency tree. Also, if your organization’s Maven release repository is used as a host for capsule artifacts, there is no reason not to treat it as a devops server, and make sure that it’s as available as other servers needed at runtime (especially because Maven repository software isn’t particularly known to crash).

Now let’s get back to the matter at hand.

Introduction to Modern Java Web Development

Because Java web servers are about as old as the web itself, and there are long, successful, traditions and practices associated with them – traditions which we will soon commence on dismantling – this is perhaps as good a time as any to explain what I mean by “modern” in this series’s title.

In this context, I take “modern” to mean “in accordance with current general software development trends”. These trends aren’t completely arbitrary, and they are consistent with one another. The birth of a very large number of small, young, fast-moving startups, has given rise to a preference of lean development approaches. Those require easier onboarding, fewer installation, deployment and configuration steps, and a convergence of the roles of development and operations. The growing popularity of the cloud encourages certain approaches towards resource management, namely virtualization, either at the hypervisor level or the OS level. OS-level deployment/resource allocation also supports the proliferation of polyglot architectures, which seek to use the right (but possibly different) tool for each job.

Traditional Java web servers – or, in their full-featured incarnation, application servers – have one particular distinguishing feature: the ability to run multiple applications in a single JVM instance. They provide a runtime that is separate from the application. It is upgraded, and installed separately. It can be launched separately. Applications are deployed to a fully configured, possible already running, runtime. While this approach has worked well for quite some time – and you may have good reasons to continue employing it – it is far from modern. Allocating resources among the different applications occupying the server is not simple (if possible at all), and it is certainly incompatible with current trends of using hypervisor- and OS containers to host applications. Tools and techniques designed for hypervisor/OS containers are not effective with multi-app servers. Even if these servers are used to host a single application, their operation is most certainly not modern: installing and configuring the web- or app- server is usually not trivial, and deploying applications to said servers take multiple steps, and may be cumbersome.

The modern approach, that used by practically all other languages and runtime platforms, and increasingly adopted in the Java world, is that of the single-app server. In this kind of server, the web container is embedded in the application (rather than the application being deployed to a web container). This allows for simple deployment, management and configuration, and resource allocation at the OS/hypervisor level. This is why, when modern Java is concerned, application servers (and by that I mean any servlet or full-featured Java EE server designed to run more than one application) – are dead.

Now, the tools and techniques we will survey here are far from comprehensive. Especially when it comes to web, and web-related, development, tools, libraries and frameworks proliferate. Part of the reason for that is that, unlike fields such as embedded development or mainframe applications, web development is popular among all those fast-moving startups I mentioned, as well as hobbyists. These populations are early adopters and like to experiment with new techniques and invent new ones – sometimes in order to get some technological edge, sometimes for learning purposes, and sometimes for more gratuitous self gratification. The outcome is hundreds of libraries, all achieving similar goals, but in slightly different ways. This happens in the Java world, just as it does in other language ecosystems.

Also, we will not discuss “full” web frameworks at all, and by that I mean large MVC frameworks, template systems, or any framework designed for applications that render HTML on the server. There are a few reasons for that: One, I’ve never used any of those, so I certainly cannot comment on their suitability or “modernness”; two, the topic is a complex one, requires much discussion, and has been done elsewhere (see here and here); three, it seems like web development is moving towards client-side rendering and “single-page apps” (with client side frameworks like Angular et al.), essentially adopting an architecture similar to old client-server apps, and relying on HTTP services to transfer data and commands to and from the server. That transition is not complete – in particular, it depends on mobile phone browsers improving their JavaScript performance – but it is near certain that we’ll be seeing less and less HTML generated dynamically on the server. We will, therefore, only discuss frameworks and libraries for HTTP “data” services.

HTTP Services with JAX-RS and Dropwizard

One of the things setting Java apart from other languages is the work of the JCP (Java Community Process), whose work it is to specify standard APIs (even for libraries not part of the language specification or even of the standard runtime library), which are then implemented by various commercial or open-source projects. These JSRs (Java Specification Requests), are made by expert groups, and are often known to take an eternity to fully mature and become a standard. But when JSRs are successful, they are very useful as almost all libraries catering to the related field, would implement the standard API, which makes switching implementations, if not trivial than at least less painful.

Standards are more important for server implementation (where the framework is much more pervasive in the code), than for clients (where each call is more-or-less independent and can be replaced). You can use three different HTTP clients, and 3 different JDBC APIs in the same method, but your server usually runs on a single framework. For that reason, you should prefer a standard server API over a non-standard one, unless the non-standard one gives you some very substantial advantages, or otherwise fits your particular use case significantly better. Mere API aesthetics should not sway you in favor of a non-standard API.

A lightweight web-service server, then, should best implement the standard APIs. When it comes to HTTP services, there are a couple of pertienent APIs. The first is the good-old Servlet API (currently under JSR-315 for Servlet 3.0, and JSR-340, for Servlet 3.1). Almost all Java web servers implement the Servlet API, some of them are “modern” (in the sense we discussed earlier), the most popular of which is Jetty. Unlike traditional Java web servers, Jetty is not a standalone web-app container, but a web-server library embedded in the application. It was built to be modern. But even more traditional web servers, like Tomcat, now have modern embedded modes. Because Servlets are a relatively low-level HTTP server API, we won’t be directly working with them here, so let’s move on to the next standard API, JAX-RS (currently, in version 2.0, specified in JSR-339). There are several implementations of JAX-RS, like Apache CXF, RESTEasy and Restlet, but the most popular one seems to be Jersey.

A JAX-RS implementation is usually used by applying it on top of a Servlet server. It would therefore be natural to build a modern Java web service microframework by putting together Jetty and Jersey, and that’s pretty much the tool we’ll be playing with next: Dropwizard.

So Dropwizard takes Jetty, Jersey, a JSON library called Jackson, Metrics – the modern performance monitoring library we presented in part 2 (which happens to have been created by Coda Hale, the man behind Dropwizard) – and a bunch of other libraries, and combines them into a complete, simple, modern Java web service microframework.

We’ll now write our first modern Java web service with Dropwizard. If you haven’t read part 1, I suggest you do so now, to familiarize yourself with the basic usage of Gradle, the build too we’ll be using.

We’ll create a new jmodern-web directory, cd into it, and create a Gradle project with gradle init --type java-library, and delete the stub source files Gradle creates (src/main/java/Library.java and src/test/java/LibraryTest.java).

Then, we’ll edit build.gradle to say:

apply plugin: 'java'
apply plugin: 'application'

sourceCompatibility = '1.8'

mainClassName = 'jmodern.Main'
version = '0.1.0'

repositories {
    mavenCentral()
}

configurations {
    capsule
}

dependencies {
    compile 'io.dropwizard:dropwizard-core:0.7.0'
    capsule 'co.paralleluniverse:capsule:0.4.0'
    testCompile 'junit:junit:4.11'
}

task capsule(type: Jar, dependsOn: classes) {
    archiveName = "jmodern-web.jar"

    from jar // embed our application jar
    from { configurations.runtime } // embed dependencies

    from(configurations.capsule.collect { zipTree(it) }) { include 'Capsule.class' } // we just need the single Capsule class

    manifest {
        attributes(
            'Main-Class'  :   'Capsule',
            'Application-Class'   : mainClassName,
            'Application-Version' : version,
            'Min-Java-Version' : '1.8.0',
            'JVM-Args' : run.jvmArgs.join(' '),
            'System-Properties' : run.systemProperties.collect { k,v -> "$k=$v" }.join(' '),
        )
    }
}

Our src/main/java/jmodern/Main.java file would be:

package jmodern;

import io.dropwizard.Application;
import io.dropwizard.*;
import io.dropwizard.setup.*;
import java.util.*;
import java.util.concurrent.atomic.AtomicLong;
import javax.ws.rs.*;
import javax.ws.rs.core.*;

public class Main extends Application<Configuration> {
    public static void main(String[] args) throws Exception {
        new Main().run(new String[]{"server"});
    }

    @Override
    public void initialize(Bootstrap<Configuration> bootstrap) {
    }

    @Override
    public void run(Configuration configuration, Environment environment) {
        environment.jersey().register(new HelloWorldResource());
    }

    @Path("/hello-world")
    @Produces(MediaType.APPLICATION_JSON)
    public static class HelloWorldResource {
        private final AtomicLong counter = new AtomicLong();

        @GET
        public Map<String, Object> sayHello(@QueryParam("name") String name) {
            Map<String, Object> res = new HashMap<>();
            res.put("id", counter.incrementAndGet());
            res.put("content", "Hello, " + (name != null ? name : "World"));
            return res;
        }
    }
}

This is pretty much the simplest Dropwizard service possible. The sayHello method returns a map, which is automatically changed into a JSON object. To run this, type gradle run at the shell, or build a capsule first with gradle capsule and run it with java -jar build/libs/jmodern-web.jar. To test the service point your browser at http://localhost:8080/hello-world, and then at http://localhost:8080/hello-world?name=Modern+Developer.

Now let’s improve our service a bit by taking advantage of other Dropwizard features:

package jmodern;

import com.codahale.metrics.*;
import com.codahale.metrics.annotation.*;
import com.fasterxml.jackson.annotation.*;
import com.google.common.base.Optional;
import io.dropwizard.Application;
import io.dropwizard.*;
import io.dropwizard.setup.*;
import java.util.concurrent.ThreadLocalRandom;
import java.util.concurrent.atomic.AtomicLong;
import javax.ws.rs.*;
import javax.ws.rs.core.*;
import org.hibernate.validator.constraints.*;

public class Main extends Application<Main.JModernConfiguration> {
    public static void main(String[] args) throws Exception {
        new Main().run(new String[]{"server", System.getProperty("dropwizard.config")});
    }

    @Override
    public void initialize(Bootstrap<JModernConfiguration> bootstrap) {
    }

    @Override
    public void run(JModernConfiguration cfg, Environment env) {
        JmxReporter.forRegistry(env.metrics()).build().start(); // Manually add JMX reporting (Dropwizard regression)

        env.jersey().register(new HelloWorldResource(cfg));
    }

    // YAML Configuration
    public static class JModernConfiguration extends Configuration {
        @JsonProperty private @NotEmpty String template;
        @JsonProperty private @NotEmpty String defaultName;

        public String getTemplate()    { return template; }
        public String getDefaultName() { return defaultName; }
    }

    // The actual service
    @Path("/hello-world")
    @Produces(MediaType.APPLICATION_JSON)
    public static class HelloWorldResource {
        private final AtomicLong counter = new AtomicLong();
        private final String template;
        private final String defaultName;

        public HelloWorldResource(JModernConfiguration cfg) {
            this.template = cfg.getTemplate();
            this.defaultName = cfg.getDefaultName();
        }

        @Timed // monitor timing of this service with Metrics
        @GET
        public Saying sayHello(@QueryParam("name") Optional<String> name) throws InterruptedException {
            final String value = String.format(template, name.or(defaultName));
            Thread.sleep(ThreadLocalRandom.current().nextInt(10, 500));
            return new Saying(counter.incrementAndGet(), value);
        }
    }

    // JSON (immutable!) payload
    public static class Saying {
        private long id;
        private @Length(max = 10) String content;

        public Saying(long id, String content) {
            this.id = id;
            this.content = content;
        }

        public Saying() {} // required for deserialization

        @JsonProperty public long getId() { return id; }
        @JsonProperty public String getContent() { return content; }
    }
}

We’ve made a few improvement. First, the JSON object is now modeled as an immutable Java class. Second, we’ve added a random sleep to our service, and the @Timed anontation, which will automatically monitor and report its latency with the Metrics library. Finally, we’ve made our HelloWorld service configurable, using DropWizard YAML configuration. While this is probably excessive for a simple “Hello, World” service, this serves as the basis for much more robust applications. The extra code has bought us monitoring, configurability and type safety. To create the configuration we’ll need to create a configuration class, and made a couple of changes to our build file.

The file src/main/resurces/jmodern.yml will contain our configuration:

template: Hello, %s!
defaultName: Stranger

Next, add this to build.gradle so that when we run the server with gradle run, it will locate the config file:

run {
    systemProperty "dropwizard.config", "build/resources/main/jmodern.yml"
}

Finally, we want to include the default configuration file in the capsule, so to the capsule section we’ll add:

from { sourceSets.main.resources }

and we’ll change the System-Properties manifest attribute to use our config file by default:

'System-Properties' : (run.systemProperties + ["dropwizard.config": '$CAPSULE_DIR/jmodern.yml']).collect { k,v -> "$k=$v" }.join(' '),

We’ll now build our deployment capsule with gradle capsule, and launch the server with java -jar build/libs/jmodern-web.jar. You can now test our improved service browser at http://localhost:8080/hello-world, and http://localhost:8080/hello-world?name=Modern+Developer.

What if we want to override the default? Create the following foo.yml file in the project’s directory:

template: Howdy, %s!
defaultName: fella

To use the new configuration, we override the dropwizard.config system property:

java -Ddropwizard.config=foo.yml -jar build/libs/jmodern-web.jar

We can fire up VisualVM (see part 2) and take a look at (the many!) metrics our server reports, and in particular, our service methods timing:

Dropwizard VisulaVM

When we point our browser to port 8081, we see Dropwizard’s admin console:

Dropwizard Admin

Going to http://localhost:8081/metrics, returns all collected metrics as a JSON object:

Dropwizard Metrics JSON

And that’s that! The configuration file can also be used to configure a lot of Dropwizard’s internals, set the logging level and more. See the Dropizard documentation for details.

All in all, Dropwizard is a lean, fun, modern microframework, that gives you simple deployment, easy configuration, and superb monitoring out of the box. Another, library with a similar goal is Spring Boot. Unfortunately, Boot does not use the JAX-RS standard API, but there’s a project that seeks to rectify that.

Dropwizard has a great out-of-the-box experience, but more advanced users might find it confining (some of Dropwizard’s components are hard to replace with others: its choice of a logging engine, for example). Those users might find it worthwhile to assemble Jersey, Jetty and other libraries of their choosing, and work out the plumbing themselves, to build a lightweight server that’s the best fit for their organization. Doing so should not require a lot of work, and the work necessary is only required once for all of your projects. Dropwizard is an excellent starting point, and if it works for you (which it should, in most cases), you can safely stick with it. We will be using Dropwizard for most of the examples in this post, but everything we do is possible using Jetty alone, or combined with Jersey. Dropwizard simply adds simple, consistent configuration and automatic monitoring without little extra work.

HTTP Clients

Add the following dependency to the build file:

compile 'io.dropwizard:dropwizard-client:0.7.0'

Add the following imports to jmodern.Main:

import io.dropwizard.client.*;
import com.sun.jersey.api.client.Client;

and the following two lines to the JModernConfiguration class:

@Valid @NotNull @JsonProperty JerseyClientConfiguration httpClient = new JerseyClientConfiguration();
public JerseyClientConfiguration getJerseyClientConfiguration() { return httpClient; }

We’ll instantiate the client and register a new service, which we’ll call Consumer, in these two lines, added to the run method:

Client client = new JerseyClientBuilder(env).using(cfg.getJerseyClientConfiguration()).build("client");
env.jersey().register(new ConsumerResource(client));

Finally, this will be our consumer service:

@Path("/consumer")
@Produces(MediaType.TEXT_PLAIN)
public static class ConsumerResource {
    private final Client client;

    public ConsumerResource(Client client) {
        this.client = client;
    }

    @Timed
    @GET
    public String consume() {
        Saying saying = client.resource(UriBuilder.fromUri("http://localhost:8080/hello-world").queryParam("name", "consumer").build())
                .get(Saying.class);
        return String.format("The service is saying: %s (id: %d)",  saying.getContent(), saying.getId());
    }
}

Notice how the returned JSON is deserialized into a Saying object; it can also be returned as a map, a string, and probably other types as well (Dropwizard is using an outdated version of the Jersey JAX-RS client, but the new API is similar). And because Dropwizard supports the Jersey JAX-RS client out-of-the-box, it automatically supplies performance metrics of outgoing requests1.

To test our new service, start up our application (gradle run, remember) and point your browser at http://localhost:8080/consumer.

So the JAX-RS standard also specifies a client API. But, as we’ve said before, when it comes to client APIs we can let ourselves use non-standard APIs as well. An interesting HTTP client API is Retrofit by Square. As you’ve seen, JAX-RS Client can automatically serialize and deserialize Java objects to JSON objects (or XML). Retrofit takes this automatic Java/REST translation (which, BTW, is not always a good thing; domain translations are often particularly leaky abstractions, but they’re helpful if you keep yourself constrained to simple protocols) a step further, and translates the service target URL – not just the payload – to Java interfaces. Unfortunately, Retrofit uses the same annotation names as JAX-RS (server), only defined in a different package, which would make our example a bit ugly. Luckily, Retrofit has a clone/derivatice called Feign, by Netflix. The differences between Feign and Retrofit are not entirely clear to me. Although it seems that Retrofit is more widely adopted (it is older), while Feign is more easily customizable. In any case, the two are extremely similar, and can be used pretty much interchangeably.

To try Feign out, add the following dependencies to build.gradle:

compile 'com.netflix.feign:feign-core:6.1.2'
compile 'com.netflix.feign:feign-jaxrs:6.1.2'
compile 'com.netflix.feign:feign-jackson:6.1.2'

and these imports to Main:

import feign.Feign;
import feign.jackson.*;
import feign.jaxrs.*;

Instead of the JAX-RS client initialization and the consumer service registration in the run method, we’ll create a Feign.Builder:

Feign.Builder feignBuilder = Feign.builder()
        .contract(new JAXRSModule.JAXRSContract()) // we want JAX-RS annotations
        .encoder(new JacksonEncoder()) // we want Jackson because that's what Dropwizard uses already
        .decoder(new JacksonDecoder());
env.jersey().register(new ConsumerResource(feignBuilder));

Our consumer service will now look like this:

@Path("/consumer")
@Produces(MediaType.TEXT_PLAIN)
public static class ConsumerResource {
    private final HelloWorldAPI hellowWorld;

    public ConsumerResource(Feign.Builder feignBuilder) {
        this.hellowWorld = feignBuilder.target(HelloWorldAPI.class, "http://localhost:8080");
    }

    @Timed
    @GET
    public String consume() {
        Saying saying = hellowWorld.hi("consumer");
        return String.format("The service is saying: %s (id: %d)",  saying.getContent(), saying.getId());
    }
}

Finally, we’ll add the HelloWorldAPI interface, which puts the REST API in Java terms (you can put the interface definition somewhere in our Main class; no need to create a new Java file):

interface HelloWorldAPI {
    @GET @Path("/hello-world")
    Saying hi(@QueryParam("name") String name);

    @GET @Path("/hello-world")
    Saying hi();
}

This interface uses JAX-RS annotations to specify how method calls translate to HTTP requests. The code that actually performs this translation is automatically generated by Feign (or Retrofit).

After launching our server app, visit http://localhost:8080/consumer to test the new consumer service.

If you want to see how more complex REST APIs translate to Java, this very simple example demonstrates consuming the GitHub API with Retrofit, and here’s the same example using Feign. Both Retrofit and Feign are very feature-rich, and allow great control over how requests are translated and executed. At this point, I would recommend Retrofit over Feign because Retrofit is more mature, and it makes use of the efficient NIO networking API under the hood, while Feign uses the slow HttpURLConnection API (a better transport mechanism could be plugged into Feign, but I haven’t found any).

There are other, lower level HTTP client APIs (like Apache HTTP Client, which is also directly supported by Dropwizard), but in most circumstances, the higher-level APIs we’ve tried – JAX-RS Client or Retorfit/Feign – work best.

Database Access

The JDK includes a standard API for (relational) database access called JDBC (Java Database Connectivity). Practically all SQL databases support JDBC. But JDBC is a very low-level API, and can sometimes be tiresome to use. Java also has a standard high-level database access API – an ORM API actually – called JPA (Java Persistance API), specified by JSR-220 and JSR-317. Well known implementations of JPA include Hibernate, OpenJPA and EclipseLink. Do yourself a big favor and don’t use any of them if you can help it. Not that they don’t work – they most certainly do, and they are all fine implementations – but they’re often more trouble than they’re worth. Full blown ORMs encourage a complex object graph and a complex schema, which often results in extremely complex, automatically generated, SQL statements, which are then hard to optimize. Plus, such complex ORMs are not known for their terrific performance.

Using JDBC directly is often better, but perhaps the best approach is to use one of the tools we will now present. They are somewhere between the low-level JDBC, and a fullblown ORM. They are not standard, which means each has its own API and there aren’t competing, interchangeable, implementations for a standard one, but as we’ve said, going off the standards path is OK for client APIs. In our examples, we’ll be using the H2 embedded database, in in-memory mode.

We’ll start off with JDBI, which is also directly supported by Dropwizrd. To use JDBI effectively, you’ll need to trade off optimal schema and simple code, until you get to a nice middle ground (JDBI is not ideal for very complex schemas).

We’ll add these dependencies:

compile 'io.dropwizard:dropwizard-db:0.7.0'
compile 'io.dropwizard:dropwizard-jdbi:0.7.0'
runtime 'com.h2database:h2:1.4.178'

and these imports:

import io.dropwizard.db.*;
import io.dropwizard.jdbi.*;
import org.skife.jdbi.v2.*;
import org.skife.jdbi.v2.util.*;

We then need to add a DataSource factory to JModernConfiguration:

@Valid @NotNull @JsonProperty private DataSourceFactory database = new DataSourceFactory();
public DataSourceFactory getDataSourceFactory() { return database; }

And the following to the run method, which will connect to the database and register our new DbService:

DBI dbi = new DBIFactory().build(env, cfg.getDataSourceFactory(), "db");
env.jersey().register(new DBResource(dbi));

In order to configure the database, we need to add the following to jmodern.yml:

database:
  driverClass: org.h2.Driver
  url: jdbc:h2:mem:test
  user: u
  password: p

Finally, let’s create our database resource (if you’re finding the code changes hard to follow, the full listing is here):

@Path("/db")
@Produces(MediaType.APPLICATION_JSON)
public static class DBResource {
    private final DBI dbi;

    public DBResource(DBI dbi) {
        this.dbi = dbi;

        try (Handle h = dbi.open()) {
            h.execute("create table something (id int primary key auto_increment, name varchar(100))");
            String[] names = { "Gigantic", "Bone Machine", "Hey", "Cactus" };
            Arrays.stream(names).forEach(name -> h.insert("insert into something (name) values (?)", name));
        }
    }

    @Timed
    @POST @Path("/add")
    public Map<String, Object> add(String name) {
        try (Handle h = dbi.open()) {
            int id = h.createStatement("insert into something (name) values (:name)").bind("name", name)
                    .executeAndReturnGeneratedKeys(IntegerMapper.FIRST).first();
            return find(id);
        }
    }

    @Timed
    @GET @Path("/item/{id}")
    public Map<String, Object> find(@PathParam("id") Integer id) {
        try (Handle h = dbi.open()) {
            return h.createQuery("select id, name from something where id = :id").bind("id", id).first();
        }
    }

    @Timed
    @GET @Path("/all")
    public List<Map<String, Object>> all(@PathParam("id") Integer id) {
        try (Handle h = dbi.open()) {
            return h.createQuery("select * from something").list();
        }
    }
}

For those of you who know JDBC, this is both familiar and different. JDBI has a fluent interface, and the operations return Java collections, which are then conveniantly and automatically serialized to JSON objects. All in all, this is like a fun, modern, JDBC.

Fire up the application and point your browser at http://localhost:8080/db/all to see all entries, or at http://localhost:8080/db/item/2 to see the second entry. Then, you can also a new entry by entering

curl --data Velouria http://localhost:8080/db/add

into the console.

JDBI can also go a step further, and just like Retrofit, provide us with a custom interface tailored to our DB usage. We’ll even get a bit tricker, by mapping table rows to Java objects.

This will be our domain object (full code listing is here)

public static class Something {
    @JsonProperty public final int id;
    @JsonProperty public final String name;

    public Something(int id, String name) {
        this.id = id;
        this.name = name;
    }
}

The @JsonProperty annotations will ensure that returning this class from one of our service methods will automatically serialize it as a JSON object, but in order for JDBI to work with Something we also need to create a ResultSetMapper which transforms a JDBC ResultSet into a Something object:

public static class SomethingMapper implements ResultSetMapper<Something> {
    public Something map(int index, ResultSet r, StatementContext ctx) throws SQLException {
        return new Something(r.getInt("id"), r.getString("name"));
    }
}

Now, for the fun part. This is our DAO class (or SQL Object in JDBI parlance) – JDBI SQL Object is to the database what Retrofit is for REST:

@RegisterMapper(SomethingMapper.class)
interface ModernDAO {
    @SqlUpdate("insert into something (name) values (:name)")
    @GetGeneratedKeys
    int insert(@Bind("name") String name);

    @SqlQuery("select * from something where id = :id")
    Something findById(@Bind("id") int id);

    @SqlQuery("select * from something")
    List<Something> all();
}

And now, our new and improved database resource can be written like this:

@Path("/db")
@Produces(MediaType.APPLICATION_JSON)
public static class DBResource {
    private final ModernDAO dao;

    public DBResource(DBI dbi) {
        this.dao = dbi.onDemand(ModernDAO.class);

        try (Handle h = dbi.open()) {
            h.execute("create table something (id int primary key auto_increment, name varchar(100))");
            String[] names = { "Gigantic", "Bone Machine", "Hey", "Cactus" };
            Arrays.stream(names).forEach(name -> h.insert("insert into something (name) values (?)", name));
        }
    }

    @Timed
    @POST @Path("/add")
    public Something add(String name) {
        return find(dao.insert(name));
    }

    @Timed
    @GET @Path("/item/{id}")
    public Something find(@PathParam("id") Integer id) {
        return dao.findById(id);
    }

    @Timed
    @GET @Path("/all")
    public List<Something> all(@PathParam("id") Integer id) {
        return dao.all();
    }
}

JDBI isn’t a full ORM solution: it does not automatically generate SQL statements, nor does it automatically marshal full object graphs, but it does get us to a database access sweet-spot, with a lot less weight than any of the JPA implementations.

When using JDBI, Dropwizard will automatically add a health check (http://localhost:8081/healthcheck) that tests the database connectivity, and instrument our DAO with performance metrics:

Dropwizard DAO Metrics

The next database access library we will look at, jOOQ, is similar to JDBI’s fluent API (it doesn’t have an analogous API to JDBI’s SQL objects), but takes a different approach: it uses chains of method calls, rather than strings, to generate SQL statements (and it can generate SQL compatible with various database implementations).

We’ll add this dependency

compile 'org.jooq:jooq:3.3.2'

and these imports:

import org.jooq.Record;
import org.jooq.RecordMapper;
import static org.jooq.impl.DSL.*;

In the run method, we’ll now register our DB resource like so (the full code listing is here):

DataSource ds = cfg.getDataSourceFactory().build(env.metrics(), "db"); // Dropwizard will monitor the connection pool
env.jersey().register(new DBResource(ds));

And our new DBResource looks like this:

@Path("/db")
@Produces(MediaType.APPLICATION_JSON)
public static class DBResource {
    private final DataSource ds;
    private static final RecordMapper<Record, Something> toSomething =
            record -> new Something(record.getValue(field("id", Integer.class)), record.getValue(field("name", String.class)));

    public DBResource(DataSource ds) throws SQLException {
        this.ds = ds;

        try (Connection conn = ds.getConnection()) {
            conn.createStatement().execute("create table something (id int primary key auto_increment, name varchar(100))");

            String[] names = { "Gigantic", "Bone Machine", "Hey", "Cactus" };
            DSLContext context = using(conn);
            Arrays.stream(names).forEach(name -> context.insertInto(table("something"), field("name")).values(name).execute());
        }
    }

    @Timed
    @POST @Path("/add")
    public Something add(String name) throws SQLException {
        try (Connection conn = ds.getConnection()) {
            // this does not work
            int id = using(conn).insertInto(table("something"), field("name")).values(name).returning(field("id"))
                       .fetchOne().into(Integer.class);
            return find(id);
        }
    }

    @Timed
    @GET @Path("/item/{id}")
    public Something find(@PathParam("id") Integer id) throws SQLException {
        try (Connection conn = ds.getConnection()) {
            return using(conn).select(field("id"), field("name")).from(table("something"))
                    .where(field("id", Integer.class).equal(id)).fetchOne().map(toSomething);
        }
    }

    @Timed
    @GET @Path("/all")
    public List<Something> all(@PathParam("id") Integer id) throws SQLException {
        try (Connection conn = ds.getConnection()) {
            return using(conn).select(field("id"), field("name")).from(table("something")).fetch().map(toSomething);
        }
    }
}

Now, jOOQ does not yet implement DDL (SQL statements like create table), so you’ll notice we’re creating the table using JDBC. That’s OK because jOOQ is implemented as a JDBC wrapper, and requires some JDBC anyway (I haven’t been able to get the add method to work (probably because of the auto-generated primary key). jOOQ people: if you’re reading this, please help).

This example really doesn’t do jOOQ justice, as its greatest strength is the ability to generate classes from your schema, and perform all the operations we’ve done above – and far more complex ones – in a completely typesafe manner. jOOQ is a little too clever for my own tastes, but if your schema is complex, it can be an invaluable tool.

Dependency Injection

Dependency injection is one of those programming patterns that, depending on whom you ask, can be said to be invaluable or completely useless. I believe that DI can be extremely useful in complex codebases; for simple ones it’s largely unnecessary. Java has a simple standard DI API specified by JSR-330. JSR-330 has the following implementations: Spring IoC, Guice, Dagger, Sisu (which builds on top of Guice), and HK2. Each of these is developed by a major company or organization. Given this selection, people are often faced with a paradox of choice. Don’t fear: if you stick to the JSR-330 standard, or deviate slightly, you can change your DI solution at any point. If you want your application to be completely configurable by the user (in the form of XML files2), choose Spring (that’s what why we’ve chosen Spring for Galaxy); if not, then start with Dagger, and only switch to something else if and when it no longer satisfies your needs.

We’re going to take Dagger for a spin. First, let’s add the Dagger dependencies:

compile 'com.squareup.dagger:dagger:1.2.1'
compile 'com.squareup.dagger:dagger-compiler:1.2.1'

To keep things uncluttered, we’ll leave only the HelloWorldResource. This time, however, instead of manually creating the service and passing it the configuration object, we’ll use Dagger to read our configuration from the YAML file, and inject them into our service.

This is the service:

@Path("/hello-world")
@Produces(MediaType.APPLICATION_JSON)
public static class HelloWorldResource {
    private final AtomicLong counter = new AtomicLong();
    @Inject @Named("template") String template;
    @Inject @Named("defaultName") String defaultName;

    HelloWorldResource() {
    }

    @Timed // monitor timing of this service with Metrics
    @GET
    public Saying sayHello(@QueryParam("name") Optional<String> name) throws InterruptedException {
        final String value = String.format(template, name.or(defaultName));
        Thread.sleep(ThreadLocalRandom.current().nextInt(10, 500));
        return new Saying(counter.incrementAndGet(), value);
    }
}

Note the @Inject and @Named annotations. These are part of the JSR-330 standard, so our service code will remain unchanged no matter which DI tool we use. To actually wire and inject the dependencies we use Dagger specific code. Dagger specifies the dependency wiring configuration in a module class. Here’s ours:

@Module(injects = HelloWorldResource.class)
class ModernModule {
    private final JModernConfiguration cfg;

    public ModernModule(JModernConfiguration cfg) {
        this.cfg = cfg;
    }

    @Provides @Named("template") String provideTemplate() {
        return cfg.getTemplate();
    }

    @Provides @Named("defaultName") String provideDefaultName() {
        return cfg.getDefaultName();
    }
}

One of the coolest things about Dagger is that it verifies that all dependencies are satisfied at compile time using an annotation processor. If we forget to define provideDefaultName for example, this is what appears in NetBeans as we type:

NetBeans Dagger error

To obtain a fully configured instance of HelloWorldResource, this is what we put in the run method of our application:

ObjectGraph objectGraph = ObjectGraph.create(new ModernModule(cfg));
env.jersey().register(objectGraph.get(HelloWorldResource.class));

You can look at the entire (short) application here.

You’ll notice that the ModernModule class replicates some of the behavior of JModernConfiguration. It would have been great to simply annotate JModernConfiguration with @Module, and the getTemplate and getDefaultName methods with @Provides. Unfortunately, Dagger forbids subclassing in modules.

Advanced Topic: Blocking vs. Non-blocking or Synchronous vs. Asynchronous

At this point we’ll take some time for a more theoretical discussion on blocking vs. nonblocking APIs. A blocking, or synchronous, API is one whose methods block the calling thread until they complete. Of course, the concept of blocking (or nonblocking) only makes sense when these methods can potentially take a long time to complete (say tens of milliseconds to tens of seconds). Another type of API, which is often called nonblocking but here we’ll call them semi-blocking (or semi-asynchronous), has methods that don’t block the calling thread for the duration of the operation. They only initiate an operation and return a future object. The future is then used to block and wait for the operation to complete at some other convenient time. Finally, a third type of API, a true non-blocking, or asynchronous, API, also does not block the calling thread. Its methods take an additional parameter – a callback function which is code that will be executed (on some unknown thread) when the operation completes. Sometimes, Java APIs mix the last two types, both taking a callback and returning a future.

Let me put this clearly: asynchronous APIs are always more complicated than blocking APIs (that’s true even using languages and libraries that attempt to make callbacks easier with composable promises, comprehensions, monads, and other functional shenanigans3). The problem with asynchronous code is especially bad in languages like Java – and basically all other JVM languages, perhaps with the exception of Clojure – which supports multi-threading, but does not restrict side-effects. We’ll not get into detail here, but using nonblocking APIs in such languages requires great discipline, and a clear understanding of complex concurrency issues. Blocking APIs have none of these problems.

Why would anyone use asynchronous APIs over synchronous ones, then? The answer is simple: performance. To delve a little deeper, kernel threads have a non-negligible task switching costs (not to speak of thread stack sizes that can quickly swallow up your RAM, which would be better used to store data caches). Modern web applications often delegate actual processing to myriad services, some do offline map-reduce, others may do some online processing, leaving the main function of the client-facing web server to that of a coordinator: it calls many other services and assembles data from many sources. It hardly does any processing, but it performs lots of IO operations – some can be done in parallel, and others need to be called consecutively. This means that the web server generates many thread-scheduling events (threads blocking and unblocking) for relatively little CPU “work” done, and the thread scheduling overhead imposed by the operating system become onerous. So people put their code into all sort of unnatural contortions simply to get around the kernel thread scheduling performance hit, and some modern web frameworks/libraries lovingly embrace nonblocking APIs (we haven’t discussed any of them because, as we’ll see, they’re all severely misguided :)).

This is the wrong approach. In order to get around an unsuitable implementation, people abandon a suitable abstraction (threads), instead of simply fixing the implementation. Lightweight (or user-level) threads – used in Erlang, Go, or on the JVM via the Quasar library – let you use simple, blocking APIs, without any of their performance issues4.

It is a rare occurrence in computer science – usually full of tradeoffs and caveats – that one approach almost always beats another, but this is one: asynchronous code has many disadvantages and absolutely zero advantages relative to synchronous code. Even an imperfect implementation of lightweight threads is leaps and bounds better than asynchronous programming, particularly when the language does not guard against shared state mutations. There are probably some exceptions to this rule (after all, in CS, even the absolute isn’t absolutely so), but they are far fewer than those when a goto statement is recommended.

Synchronous and asynchronous are duals (each can be transformed into the other using “constant time” transformations), but synchronous is the better abstraction for humans, and I can prove this. Let’s examine two APIs:

interface Sync {
    Object pull();
}

and:

interface Async {
    void push(Callback cb);
}

interface Callback {
    void got(Object obj);
}

Now let’s implement Async using Sync:

Async syncToAsync(Sync sync) {
    return new Async() {
        public void push(final Callback cb) {
            new Thread(() -> {
                  for(;;)
                      cb.got(sync.pull());
              }).start();
        }
    }
}

Now, in your favorite programming language, implement the opposite, i.e. transform an Async to a Sync. This will be trickier, and will invariably require introducing some intermediate data storage, like a queue or a buffer. Of course, you’ll need to take into account that Callback.got can be called on any thread, so you’ll need to mind your concurrency with that data structure. The transformation from Async to Sync is, therefore, not only less trivial, but introduces unnecessary data storage: unnecessary because it is probably already built into the system (in the form of the IO buffers, for example). So Async is trivially implemented using Sync, but the opposite transformation is both wasteful, longer, and requires managing concurrency. Again, this is less of an issue in languages that restrict, or manage, side effects (like Clojure or Haskell).

The Comsat project integrates standard (and non-standard but good) Java web-related API with Quasar fibers (lightweight threads). Comsat’s next release will support the tools discussed in this post (with the possible exceptions of jOOQ and Retrofit/Feign), so that you can write the same simple blocking code, but get all the performance and scalability advantages of obtuse asynchronous code. In a future blog post we’ll show how Comsat lets you keep your code but make your application more scalable and resistent to cascading failures.

Advanced Topic: Interactive Web Services with Web Actors

While generally you should stick to standard server APIs, sometimes an alternative provides significant advantages. One of the topics not covered here is that of interactive web services that make use of server-push using WebSocket or SSE (Server Sent Events). While Java’s standard APIs support both, working with WebSockets in particular can cause complex concurrency issues, because the standard Java WebSocket APIs (JSR-356) is asynchronous. This means that WebSocket messages might arrive concurrently with, say HTTP requests from the same user. And in any event, the asynchronous API requires managing mutable shared state, which sucks. Comsat, therefore, provides an API called Web Actors, which assigns an actor to each user session, which receives – synchronously, i.e. with pull rather than push – all messages from a given web session, and makes state management a cinch. To learn more about Web Actors read the introductory blog post, this post on writing an MMO with web actors, or the Comsat documentation.

Conclusion

This concludes the Opinionated Guide to Modern Java (although I might post an addendum summarizing your responses). I hope you have enjoyed reading it as much as I have writing it. I hope I have been able to convey that the Java ecosystem is not only huge, but also vibrant and going with the times. Lambdas and streams replace verbose data manipulation code; Markdown replaces HTML; fibers, channels and actors replace locks and callbacks; simple, embedded servers replace heavyweight, unwieldy application servers. And underneath all that is the powerful, flexible JVM, with its emphasis on performance and unmatched monitoring, and its support for runtime code injection and replacement.

Discuss on Hacker News


  1. Although the performance metrics instrumentation for the JAX-RS client seems very lacking in comparison to Dropwizards support for the Apache HTTP Client

  2. Clarification necessary: Spring does not require XML. But if you want XML – as you might, say, if you want the user to wire your app’s components – then Spring is the DI tool to use.

  3. Perhaps with the sole exception of languages where all side effects are computed lazily anyway.

  4. Asynchronous APIs must exist for the sole purpose of being used under the covers by lightweight thread implementations.

Join our mailing list

Sign up to receive news and updates.

Tags: ,

comments powered by Disqus