Skip to content

Author archive for: gerben

Prepare yourself for Java 9 with docker and maven

As you may know, Java 9 is coming (feature complete : 2015/12/10, general availability : 2016/09/22, more info on the jdk9 project page).

docker-logo

It’s time to prepare yourself, but maybe you’ll want to avoid polluting your environment … There’s a solution : docker !

In this post, we’ll prepare a docker container providing :

  • the current version of jdk 9
  • apache maven 3.3.3

A docker container for JDK 9

To start, I took a Dockerfile defined in this article : Build OpenJDK 9 using Docker.

I refactored it to group the package installations and added a dev user to avoid working with root.

Here’s the result :

FROM phusion/baseimage:latest

# Pre-install
RUN \
apt-get update && \
apt-get install -y \
libxt-dev zip pkg-config libX11-dev libxext-dev \
libxrender-dev libxtst-dev libasound2-dev libcups2-dev libfreetype6-dev \
mercurial ca-certificates-java build-essential wget && \
rm -rf /var/lib/apt/lists/*

# User
RUN export uid=1000 gid=1000 && \
mkdir -p /home/javadev && \
echo “javadev:x:${uid}:${gid}:JavaDev,,,:/home/javadev:/bin/bash” >> /etc/passwd && \
echo “javadev:x:${uid}:” >> /etc/group && \
echo “javadev ALL=(ALL) NOPASSWD: ALL” > /etc/sudoers.d/javadev && \
chmod 0440 /etc/sudoers.d/javadev && \
chown ${uid}:${gid} -R /home/javadev

ENV JAVA_HOME=/opt/java-bin
ENV PATH=$JAVA_HOME/bin:$PATH

# We need JDK8 to build
RUN \
wget –no-check-certificate –header “Cookie: oraclelicense=accept-securebackup-cookie” http://download.oracle.com/otn-pub/java/jdk/8u65-b17/jdk-8u65-linux-x64.tar.gz

RUN \
tar zxvf jdk-8u65-linux-x64.tar.gz -C /opt

# Let’s get JDK9
RUN \
cd /tmp && \
hg clone http://hg.openjdk.java.net/jdk9/jdk9 openjdk9 && \
cd openjdk9 && \
sh ./get_source.sh

RUN \
cd /tmp/openjdk9 && \
bash ./configure –with-cacerts-file=/etc/ssl/certs/java/cacerts –with-boot-jdk=/opt/jdk1.8.0_65

RUN \
cd /tmp/openjdk9 && \
make clean images

RUN \
cd /tmp/openjdk9 && \
cp -a build/linux-x86_64-normal-server-release/images/jdk \
/opt/openjdk9

RUN \
cd /tmp/openjdk9 && \
find /opt/openjdk9 -type f -exec chmod a+r {} + && \
find /opt/openjdk9 -type d -exec chmod a+rx {} +

ENV PATH /opt/openjdk9/bin:$PATH
ENV JAVA_HOME /opt/openjdk9

# Maven
RUN mkdir /apache-maven
RUN curl -fSL http://apache.mirrors.ovh.net/ftp.apache.org/dist/maven/maven-3/3.3.3/binaries/apache-maven-3.3.3-bin.tar.gz -o maven.tar.gz \
&& tar -xvf maven.tar.gz -C apache-maven –strip-components=1 \
&& rm maven.tar.gz*

ENV PATH /opt/openjdk9/bin:/opt/apache-maven/bin:$PATH

USER javadev
WORKDIR /home/javadev
VOLUME /home/javadev

(you can find it on github : docker-jdk9-maven)

Building the container is quite easy, launch :

> docker build -t gcastel/openjdk9 .

from the Dockerfile folder.

Using the container

Is everything working ?
JDK9 is a work in progress, you could face some errors.

If everything is ok, we can test the container :

> docker run -it gcastel/openjdk9 java -version
openjdk version “1.9.0-internal”
OpenJDK Runtime Environment (build 1.9.0-internal-_2015_10_20_08_31-b00)
OpenJDK 64-Bit Server VM (build 1.9.0-internal-_2015_10_20_08_31-b00, mixed mode)

Now you’re ready to compile your projects using this container, for instance by mounting a local workdir (localdir) as the home directory :

> docker run -v localdir:/home/javadev -it gcastel/openjdk9 bash

OK, now let’s try using maven with a java 9 functionality (the new Process API).

I pushed a small test project on github.

The main class is quite simple :

package fr.gcastel;

public class TestProcessAPI {
  public static void main(String[] args) throws Exception {
        System.out.println("Your pid is " + ProcessHandle.current().getPid());
  }
}

We display the current PID using the java 9 Process API.

No need to add any dependency, we’ll only inform maven that we are using java 9 via the pom.xml:

  <plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-compiler-plugin</artifactId>
    <version>2.3.2</version>
    <configuration>
      <source>1.9</source>
      <target>1.9</target>
    </configuration>
  </plugin>

Let’s check :

> mvn compile

And if everything is ok :

> cd target/classes
> java fr.gcastel.TestProcessAPI
Your pid is 24

One more thing : if you want to help the java community, prepare a dedicated jenkins to compile your project with JDK9 as soon as the release candidate is available.
It coul help to avoid problems such as the one of java 7 with lucene.

References:

ReST service and resource creation

Consuming ReST services has become a common task, but resource creation is a somewhat more complex task. (Disclaimer : I won’t introduce new concepts, I’m only trying to gather information concerning resource creation).

Resource creation : POST or PUT ?

First question : to create a new resource, should I use a PUT ?

Well, you must ask yourself : Who is responsible for generating the resource id ?

The client provides the ID ? Use a PUT

When using a PUT, the provided URI gives the ID.

$ curl -XPUT 'http://localhost:8080/restservice/resources/42' -d '{
    "name" : "mybeautifulresource"
}'

The server provides the ID ? Use a POST

When using a POST, it’s up to the server to provide the ID.

$ curl -XPOST 'http://localhost:8080/restservice/resources' -d '{
    "name" : "mybeautifulresource"
}'

But ? Where is the ID ? Well … in the server response (for instance, Location: gives you the resource URI)

HTTP/1.1 201 - Created
Server: Pouet server v.1.02
Location: http://localhost:8080/restservice/resources/42
...
{
    "id" : "42",
    "name" : "mybeautifulresource"
}

Response content

In the response, some fields may be useful for the client.

  • Location, provides the resource URI
  • The ETag matching the version of this resource enables client-side caching.
  • The Content-Type and the content provides the resource itself

Let’s add those fields, with JAX-RS :

@Context
UriInfo uriInfo;
...

@PUT
@Path("{id}")
@Consumes("application/json")
public Response addResource(@PathParam("id") String id, Data inputdata) {
  ...
  return Response.created(uriInfo.getRequestUri())
          .tag(Integer.toString(newResource.hashCode()))
          .entity(newResource)
          .type(MediaType.APPLICATION_JSON_TYPE)
          .build();
}

OK, now I want to use this ETag field correctly

You have to check the client-provided ETag (“If-None-Match” field in the client request) against your server resource. Your server should return the content only if the resource has changed.

The request object can check this tag using the evaluatePreconditions method.

If the provided ResponseBuilder is null, you must return the resource content. Else, you can return the built response.

    @GET
    @Path("{id}")
    public Response getApp(@PathParam("id") String id, @Context Request request) {
        Data resource = fetchData(id);
        EntityTag etag = new EntityTag(Integer.toString(resource.hashCode()));
        Response.ResponseBuilder builder = request.evaluatePreconditions(etag);

        if (builder == null) {
            builder = Response.ok(resource);
            builder.tag(etag);
        }
        return builder.build();
    }

Now it’s up to you :)

Elasticsearch : MasterNotDiscoveredException and discovery.zen.minimum_master_nodes

elasticsearch-logo-icon-lg
There’s no need to present Elasticsearch, this open source, distributed and very accessible search engine.

Elasticsearch is easy to use and configure, but building a cluster can be complex, especially when it comes to “master node” management.

The “master node” is a cluster node that deals with shards allocation among the cluster. So, there must be one and only one master at a time on the cluster.

————— Master nodes and split-brain —————

Let’s imagine a network failure prevents your nodes to reach each other.
Some nodes won’t be able to communicate with the master node and will trigger a new master election.

This election result will cause a split-brain situation : the cluster is split in 2 parts with a different master for each (which can cause data loss).

There’s a way to avoid this situation : change the discovery.zen.minimum_master_nodes parameter, as defined :

The discovery.zen.minimum_master_nodes allows to control the minimum number of master eligible nodes a node should “see” in order to operate within the cluster.

To solve the “split-brain” problem, you must define discovery.zen.minimum_master_nodes with the value (nodes in the cluster) / 2 + 1. If there’s 8 nodes in the cluster, you will set this value to 5.

5 voting nodes are then necessary to elect a new master. 2 nodes won’t have enough votes to get simultaneously elected.

————— MasterNotDiscoveredException —————

Well, everything is working and now … you want to remove some of your nodes.

Let’s say you have 8 nodes on your cluster and you want to restrict it to 4 nodes. We start with discovery.zen.minimum_master_nodes at 5.

We must modify discovery.zen.minimum_master_nodes to set it to the new minimum voting node number (4 / 2 + 1 => 3). This can be done with curl via the settings API :

curl -XPUT localhost:9200/_cluster/settings -d '{
    "persistent" : {
        "discovery.zen.minimum_master_nodes" : 3
    }
}'

If we stop our nodes, the cluster will gracefully continue his work, right ?
Well … yes, if we don’t shut the master node down.

If the master node is among the stopped nodes, your requests will return MasterNotDiscoveredException:

curl -XGET "http://localhost:9200/_cluster/health?pretty=true"
{
  "error" : "MasterNotDiscoveredException[waited for [30s]]",
  "status" : 503
}

————— What happened ? —————

Let me sum up :

  • Our cluster had 8 nodes
  • “discovery.zen.minimum_master_nodes” was 5
  • We want to switch to 4 nodes
  • We modified “discovery.zen.minimum_master_nodes” to 3 (4 / 2 + 1)
  • We stopped some nodes to have get our 4 nodes => everything is ok
  • If the master node is among the stopped node => MasterNotDiscoveredException

No master has been elected, as if the cluster wasn’t using our configuration.

Here’s my analysis :

  • The dynamic setting of “discovery.zen.minimum_master_nodes” via the “/_cluster/settings” API is applied by the master node. He is in charge of dispatching this parameter among the nodes.
  • If the master node is stopped, he can’t provide dynamic parameters values to the other nodes, so they use their default setting (from elasticsearch.yml)

In conclusion, if you want to modify the “discovery.zen.minimum_master_nodes” setting, set it in the elasticsearch.yml file to prevent unexpected results.