Spring Rest Web Service Client and Server with RestController and RestTemplate

Spring offers features to setup a Restful web service that responds with JSON on the server using the RestController, but also give a framework for accessing the webservice with the RestTemplate. In this Spring Rest API tutorial we will explore how to use and unit test these within Eclipse, by building and running examples.

Embedded Tomcat and Junit

To make it easy to follow along we will setup our code to run in an Eclipse IDE environment and use Junit and Embedded Tomcat. We will run junit tests within Eclipse, that will start an embedded tomcat instance, deploy the code, and then execute the tests to ensure everything is working as expected.

This should make things easier to understand as everything that is needed to run and validate the code, should be self contained in each Junit test. If you need to setup your dev environment, see my post here on setting up java dev environment with Eclipse.

Adding Maven Dependencies

Update Your Pom.xml

The pom.xml shown below includes the required dependencies were are going to need for this tutorial.

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
	xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
	xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
	<modelVersion>4.0.0</modelVersion>

	<groupId>com.colwil</groupId>
	<artifactId>resttutorial</artifactId>
	<version>0.0.1</version>
	
	<properties>
		<junit-version>4.12</junit-version>
		<hamcrest-version>1.3</hamcrest-version>
		<tomcat-version>9.0.21</tomcat-version>
		<tomcat-juli-version>9.0.0.M6</tomcat-juli-version>
		<commons-io-version>2.6</commons-io-version>
		<selenium-java-version>3.141.59</selenium-java-version>
		<htmlunit-driver-version>2.52.0</htmlunit-driver-version>
		<spring-version>5.1.8.RELEASE</spring-version>
		<spring-securitycore-version>5.1.5.RELEASE</spring-securitycore-version>
	</properties>

	<dependencies>

		<dependency>
			<groupId>junit</groupId>
			<artifactId>junit</artifactId>
			<version>${junit-version}</version>
		</dependency>
		<dependency>
			<groupId>org.hamcrest</groupId>
			<artifactId>hamcrest-all</artifactId>
			<version>${hamcrest-version}</version>
		</dependency>
		<dependency>
			<groupId>org.apache.tomcat.embed</groupId>
			<artifactId>tomcat-embed-core</artifactId>
			<version>${tomcat-version}</version>
		</dependency>
		<dependency>
			<groupId>org.apache.tomcat.embed</groupId>
			<artifactId>tomcat-embed-jasper</artifactId>
			<version>${tomcat-version}</version>
		</dependency>
		<dependency>
			<groupId>commons-io</groupId>
			<artifactId>commons-io</artifactId>
			<version>${commons-io-version}</version>
		</dependency>
		<dependency>
			<groupId>org.seleniumhq.selenium</groupId>
			<artifactId>selenium-htmlunit-driver</artifactId>
			<version>${htmlunit-driver-version}</version>
		</dependency>
    		<dependency>
        	<groupId>org.springframework</groupId>
        	<artifactId>spring-webmvc</artifactId>
        	<version>${spring-version}</version>
    	</dependency>	
		<dependency>
			<groupId>org.springframework.security</groupId>
			<artifactId>spring-security-core</artifactId>
			<version>${spring-securitycore-version}</version>
		</dependency>
		<dependency>
			<groupId>org.springframework.security</groupId>
			<artifactId>spring-security-web</artifactId>
			<version>${spring-securitycore-version}</version>
		</dependency>
		<dependency>
		    <groupId>org.springframework.security</groupId>
		    <artifactId>spring-security-config</artifactId>
		    <version>${spring-securitycore-version}</version>
		</dependency>
	  </dependencies>
		
</project>

Lets go through what each of the dependencies are for to ensure that we understand what each of them are doing.

Junit

Junit is the unit test framework. We are going to be running our code using junit tests, so you should be able to take these and run them on your own environment and immediately see if the code is working for you or not

Hamcrest

Hamcrest is a matching framework that is used alongside Junit, and is used to help make you unit tests more expressive and more readable.

Tomcat Embedded

Tomcat is the Apache Foundations application server that implements Java servlets, jsps and other parts of the java specification. This open source app server is great to learn on, and we will be using the embedded version, that allows us to run Tomcat within our Junit tests. This means we can set the whole thing up from without our code, and so no manual deployment or setup will be required. All we have to do is run our Junit tests.

Apache Commons IO

Commons IO are some general IO Utility class

Selenium

Selenium is automated web browsing software that is often used when test web pages.

HtmlUnit

HtmlUnit is a simulated browser that also is often used for testing web pages, and is part of Selenium.

Spring

The Spring Framework provide lots of different modules that provide additonal features on top of core java, that offer facilities that make java easier to work with.

If we build our POM for the project, then the dependencies will be available to us

Testing With Embedded Tomcat

We will be using the embedded version of tomcat as our java container and to test our rest client and server functions. To confirm that our setup works as expected lets first test that our java project has been setup and built properly, by accessing a jsp and a servlet.

Lets start with a simple Java project with 1 unit test, 1 helper class to manage embedded tomcat, and 1 jsp

So the project will look something like this. EmbeddedTomcat.java will manage our embedded tomcat startup, shutdown and deployment

EmbeddedTomcat.java

package com.colwil.tomcat;

import java.io.File;

import org.apache.catalina.WebResourceRoot;
import org.apache.catalina.core.StandardContext;
import org.apache.catalina.startup.Tomcat;
import org.apache.catalina.webresources.DirResourceSet;
import org.apache.catalina.webresources.StandardRoot;
import org.apache.commons.io.FileUtils;
import org.apache.tomcat.util.scan.StandardJarScanFilter;
import org.apache.tomcat.util.scan.StandardJarScanner;

public class EmbeddedTomcat {
    private Tomcat tomcat;

    public int PORT = 8080;
    String separator = System.getProperty("file.separator");

At the top of our class we keep a variable that defines the port, and also we get the default system seperator in case we need that late and ensure that we are using the right one irrespective on which system we are running on.

Deploy() method

In the deploy method, which we call before we start tomcat, we setup a number of different things.

    public void deploy(String appName) {
	tomcat = new Tomcat();

	StandardContext ctx = (StandardContext) tomcat.addWebapp("",
		System.getProperty("user.dir") + separator + "src/main/webapp");
	// Disable Tomcat Jar Scanning to stop errors
	((StandardJarScanner) ctx.getJarScanner()).setScanManifest(false);
	// Disable default pluggability to speed up startup times
	StandardJarScanFilter filter = new StandardJarScanFilter();
	filter.setDefaultPluggabilityScan(false);
	((StandardJarScanner) ctx.getJarScanner()).setJarScanFilter(filter);

	altPath(ctx);
    }

1. First we get an instance of the Tomcat class then we add a web app to tomcat. This would normally be where applications that run in an application would be deployed to and then the container would access that when users access the server.

addWebapp

On the first line we call addWebapp, and the tomcat docs above show that we use this to map the contact and specify the base directory. As we passed blank as the context, this will use the default which is the root context. We would access this at http://localhost:8080/. For the base directory we are using the user.dir system property. When running from eclipse that will effectively give the root of the project, then we add a separator and then “src/main/webapp”, which will be where we put our jsp. This also returns a context that we can use to apply more config to the webapp.

setScanManifest(false)

The next line has a comment of //Disable Tomcat Jar Scanning. Jar scanning is used to enable auto config and initialisation of tomcat. For our purposes we dont need it, but also it introduces some errors in the startup of embedded tomcat, and there are some classes the scanning expects that cant be found. So the easiest solution here is to disable the jar scanning.

setDefaultPluggabilityScan(false)

Similar to the above, tomcat does a lot of scanning of jars for pluggability. This process can take a long time and slow down the startup of the embedded server, which is particularly annoying if you need to work through stopping and starting the server as you develop your code. So we disable the pluggability and startup times for tomcat embedded improve dramatically.

altPath(ctx)

altPath(ctx);
}

private void altPath(StandardContext ctx) {
// Declare an alternative location for your "WEB-INF/classes" dir
// Servlet 3.0 annotation will work
File additionWebInfClasses = new File("target/classes");
WebResourceRoot resources = new StandardRoot(ctx);
resources.addPreResources(
    new DirResourceSet(resources, "/WEB-INF/classes", additionWebInfClasses.getAbsolutePath(), "/"));
ctx.setResources(resources);
}

The altPath method is one we created to add the location of our non jsp artifacts to tomcat embedded so it can find them on the classpath of the webapp when it needs to.

start() method

public void start() {
try {

    tomcat.getConnector().setPort(PORT);
    String baseDir = System.getProperty("user.dir");
    tomcat.setBaseDir(baseDir);
    tomcat.getHost().setAppBase(baseDir);
    tomcat.getHost().setDeployOnStartup(true);
    tomcat.getHost().setAutoDeploy(true);

    tomcat.start();

} catch (Exception e) {
    tomcat.getServer().await();
    try {
    tomcat.stop();
    } catch (Exception e1) {
    // TODO Auto-generated catch block
    e1.printStackTrace();
    }
    throw new RuntimeException(e);

}
}

The start method is where we configure some additional tomcat settings that start tomcat embedded.

1. tomcat.getConnector().setPort(PORT); // Set the tomcat listener port. This is very important as I spent 2 days down a rabbit whole where i couldnt access tomcat and it was due to getting this line incorrect.

It was only after reading the above in the documentation for Tomcat, and realising that I wasnt calling getConnector as is specified above, that I was able to fix the issue I was having. My problem was that I initially called tomcat.setPort(PORT) by mistake, which then doesnt trigger the default connector, so nothing works, when I should have called tomcat.getConnector().setPort(PORT);

2. String baseDir = System.getProperty(“user.dir”); // we get the base directory which is the root of the project that we get as a System property

3. tomcat.getHost().setAppBase(baseDir); // getHost will return the default host that is setup by Tomcat, which is localhost unless you set it otherwise, but is good enough for our needs. setAppBase then sets the root location for this host, that everything else will be relative to.

4. tomcat.getHost().setDeployOnStartup(true); // This tells tomcat to deploy and make the webapps available as soon as the server starts.

5. tomcat.getHost().setAutoDeploy(true); // Seems to be a duplicate of setDeplyOnStartup. We will test later if either or both of these are needed.

6. tomcat.start(); // Starts the server

stop() method

public void stop() {
try {
    tomcat.stop();
    tomcat.destroy();
    // Delete tomcats temporary folders
    FileUtils.deleteDirectory(new File("work"));
    FileUtils.deleteDirectory(new File("tomcat.8080"));
} catch (Exception e) {
    throw new RuntimeException(e);
}
}

In this method we shutdown the server and delete any temporary files/directories that tomcat created.

So with what we have here in out helper class we have enough to get up and running, so lets look at our junit test class.

Building The Junit Test

RestTest.java

package com.colwil.rest;

import static org.hamcrest.CoreMatchers.is;
import static org.junit.Assert.assertThat;

import org.junit.AfterClass;
import org.junit.BeforeClass;
import org.junit.Test;
import org.openqa.selenium.By;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.htmlunit.HtmlUnitDriver;

import com.colwil.tomcat.EmbeddedTomcat;
import com.gargoylesoftware.htmlunit.WebClient;

public class RestTest {
    static EmbeddedTomcat tomcat = new EmbeddedTomcat();
    static WebDriver browser;
    static WebClient client;

    @BeforeClass
    public static void setUp() {

	tomcat.deploy("");
	tomcat.start();

	// create a new browser so that we test with an invalid password
	browser = new HtmlUnitDriver();
	client = new WebClient();
    }

As shown above we have static references to an instance of our helper class EmbeddedTomcat. We also have references to WebDriver and WebClient of Selenium that we will use to act as clients when we interact with tomcat. These are static so that we can access them from the @BeforeClass and @AfterClass tagged methods.

setup() method

The setup() method is annotated with the @BeforeClass, so junit will run this method first. This allows us to do any global setup we need before the test methods run, such as in our case, configuring and starting a tomcat instance.

	tomcat.deploy("");
	tomcat.start();

So her we call the methods we saw earlier to do just that

	// create new clients
	browser = new HtmlUnitDriver();
	client = new WebClient();

then we create our browser clients so that we can interact with Tomcat and check the results.

tearDown() method

    @AfterClass
    public static void tearDown() {
	tomcat.stop();
	browser.close();
	client.close();
    }

The tearDown() method is annotated with the @AfterClass method, so junit will run this method last after all the tests are complete. This allows us to tidy up any instances we need to. Not necessarily important when you have one test class, but when you build up multiple test lasses you want to make sure you dont leave any mess before the next class runs, otherwise you an get weird side effects. So its good practice in Junit to ensure you close down any resources you have opened up in a test.

In this case we shutdown Tomcat and close our browser clients.

Tomcat Console Output

When we are running our unit tests, the output from tomcat will be logged to the eclipse console. Lets create a unit test that does nothing, and see what gets logged by Tomcat

    @Test
    public void doNothing() throws Exception {
    }

In eclipse, right click the method name and then click Run As… Junit Test. The Junit window should pop and and run, with green to indicate success (pretty easy as we havent tested anything yet!). The console should show output similar to below from Tomcat starting and stopping

Jul 05, 2019 8:13:20 AM org.apache.coyote.AbstractProtocol init
INFO: Initializing ProtocolHandler ["http-nio-8080"]
Jul 05, 2019 8:13:21 AM org.apache.catalina.core.StandardService startInternal
INFO: Starting service [Tomcat]
Jul 05, 2019 8:13:21 AM org.apache.catalina.core.StandardEngine startInternal
INFO: Starting Servlet engine: [Apache Tomcat/9.0.21]
Jul 05, 2019 8:13:21 AM org.apache.catalina.startup.ContextConfig getDefaultWebXmlFragment
INFO: No global web.xml found
Jul 05, 2019 8:13:22 AM org.apache.jasper.servlet.TldScanner scanJars
INFO: At least one JAR was scanned for TLDs yet contained no TLDs. Enable debug logging for this logger for a complete list of JARs that were scanned but no TLDs were found in them. Skipping unneeded JARs during scanning can improve startup time and JSP compilation time.
Jul 05, 2019 8:13:22 AM org.apache.catalina.core.ApplicationContext log
INFO: No Spring WebApplicationInitializer types detected on classpath
Jul 05, 2019 8:13:22 AM org.apache.coyote.AbstractProtocol start
INFO: Starting ProtocolHandler ["http-nio-8080"]
Jul 05, 2019 8:13:24 AM org.apache.coyote.AbstractProtocol pause
INFO: Pausing ProtocolHandler ["http-nio-8080"]
Jul 05, 2019 8:13:25 AM org.apache.catalina.core.StandardService stopInternal
INFO: Stopping service [Tomcat]
Jul 05, 2019 8:13:25 AM org.apache.coyote.AbstractProtocol stop
INFO: Stopping ProtocolHandler ["http-nio-8080"]
Jul 05, 2019 8:13:25 AM org.apache.coyote.AbstractProtocol destroy
INFO: Destroying ProtocolHandler ["http-nio-8080"]

Hopefully you are seeing something similar to this, and this will indicate you are able to start and shutdown Embedded Tomcat from a Junit test case.

Java Specifications and Servlet 3.0

The Java specifications detail how particular aspects of Java should work. As various companies and organisations can implement Java, the specifications define the rules around the standards and requirements for doing that for different aspects and versions of Java.

The Servlet specification describes how aspects of java web applications should work. One of the big shifts in the Servlet 3.0 specification was to make it simpler to add Servlets and Jsps to your container, with less configuration required.

When defining Servlets and Jsps previously, you needed to have config setup in the web.xml deployment descriptor file in your WEB-INF directory. With Servlet 3.0 and Java EE 6, you can now configure your Servlets and Jsps with little or no config in the web.xml.

This allows your Servlets and Jsps to be more self contained and makes development quicker and easier. Annotations like @WebServlet, @WebFilter, @WebListener to set the properties right in your java class.

Testing a JSP

To check that our embedded tomcat is working as expected, lets test a JSP to confirm its accessible and gives the output we can expect. In our EmbeddedTomcat.java helper class, we add a method that will return the appropriate URL for accessing tomcat, shown below.

 
    public String getApplicationUrl(String target) {
	String protocol = "http";
	String url = String.format("%s://%s:%d/%s", protocol, getHostName(), PORT, target);
	System.out.println("Application URL is:" + url);
	return url;
    }

    private String getHostName() {
	// TODO Auto-generated method stub
	return tomcat.getHost().getName();
    }

The getApplicationUrl method should return us a formatted URL to use, based on the protocol, hostname and target. We will use that url when we access our JSP.

@Test
public void testJsp() throws Exception {

String url = tomcat.getApplicationUrl("");
browser.get(url);
System.out.println(browser.getPageSource());
assertThat("Element can be found on the jsp page", browser.findElement(By.id("name")).getText(),
    is("Hello World from the JSP!"));
}

Here is testJsp() our unit test.

1. The first step here is to get the url by passing it a target string
2. Then we use our client the access the page
3. On the next line we print the returned page to the console
4. Then we use a Junit assertion to confirm that a specific element on the page exists with the expected content

Heres the jsp we will be checking for

<html>
  <head>
    <title>App</title>
  </head>
  <body>
    <span id="name">Hello World from the JSP!</span>
  </body>
</html>

Our jsp is in src/main/webapp

This is what we get back on the console if we run this unit test

Also our Junit test complete successfully, so is shown in green. This means that our browser found the name element, and returned the expected test.

Testing a Servlet

So now we have proved that we can run a jsp in embedded tomcat, lets also ensure that we can run a servlet.

Whereas the Jsp was in the src/main/webapp directory, our Servlet will be on the normal classpath. These classes end up in target/classes. If you remember we had some logic to add the classpath of the project to the classpath of our webapp, as shown again below.

	altPath(ctx);
    }

    private void altPath(StandardContext ctx) {
	// Declare an alternative location for your "WEB-INF/classes" dir
	// Servlet 3.0 annotation will work
	File additionWebInfClasses = new File("target/classes");
	WebResourceRoot resources = new StandardRoot(ctx);
	resources.addPreResources(
		new DirResourceSet(resources, "/WEB-INF/classes", additionWebInfClasses.getAbsolutePath(), "/"));
	ctx.setResources(resources);
    }

Currently we have no Servlet defined, so if we create a Junit test that tries to access that Servlet it should come back with an error message.

@Test
public void testServlet() throws Exception {
String url = tomcat.getApplicationUrl("hello", true);
browser.get(url);
System.out.println(browser.getPageSource());
assertThat("Element can be found on the servlet page", browser.findElement(By.id("name")).getText(),
    is("Hello World from the Servlet!"));
}

Our Servlet will be called “hello” and we expect to find the text “Hello World from the Servlet”, so lets see what happens if we run this without creating the servlet.

Application URL is:http://localhost:8080/hello
<?xml version="1.0" encoding="UTF-8"?>
<html lang="en">
  <head>
    <title>
      HTTP Status 404 – Not Found
    </title>
    <style type="text/css">
      h1 {font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;font-size:22px;} h2 {font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;font-size:16px;} h3 {font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;font-size:14px;} body {font-family:Tahoma,Arial,sans-serif;color:black;background-color:white;} b {font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;} p {font-family:Tahoma,Arial,sans-serif;background:white;color:black;font-size:12px;} a {color:black;} a.name {color:black;} .line {height:1px;background-color:#525D76;border:none;}
    </style>
  </head>
  <body>
    <h1>
      HTTP Status 404 – Not Found
    </h1>
    <hr class="line"/>
    <p>
      <b>
        Type
      </b>
       Status Report
    </p>
    <p>
      <b>
        Message
      </b>
       /hello
    </p>
    <p>
      <b>
        Description
      </b>
       The origin server did not find a current representation for the target resource or is not willing to disclose that one exists.
    </p>
    <hr class="line"/>
    <h3>
      Apache Tomcat/9.0.21
    </h3>
  </body>
</html>

So as you can see we get a 404 page back from Tomcat as it cant find anything to respond to this request. We also get an error in Junit as the results dont match what our test is expecting.

So lets create the Servlet now.

So on our classpath we create a class called HelloServlet in the servlet package. Note that this doesnt match the name “hello” that we are looking for in our Junit test. If we look at the source code we will see how that links up to what we are looking for.

package servlet;

import java.io.IOException;

import javax.servlet.ServletException;
import javax.servlet.ServletOutputStream;
import javax.servlet.annotation.WebServlet;
import javax.servlet.http.HttpServlet;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;

@WebServlet(name = "MyServlet", urlPatterns = { "/hello" })
public class HelloServlet extends HttpServlet {

    private static final long serialVersionUID = 1L;

    @Override
    protected void doGet(HttpServletRequest req, HttpServletResponse resp) throws ServletException, IOException {
	ServletOutputStream out = resp.getOutputStream();
	out.write("<html><body><h2 id=\"name\">Hello World from the Servlet!</h2></body></html>".getBytes());
	out.flush();
	out.close();
    }

}

This is a very simple Servlet that writes out some HTML. The key thing to not in here is the @WebServlet annotation. This does the configuration of the Servlet from within the Servlet itself. In our example we are configuring

1. The name of the Servlet – name = “MyServlet”
2. The url path that the Servlet will repond to – urlPatterns = { “/hello” }

The urlPatterns parameter is the link to what we specify in the URL to invoke the Servlet, so if we run the unit test now, we should see that this Servlet will respond. Also the class will get compiled when we run the unit test so we will see the class on the classpath in target/classes

The output from the log also shows that the Servlet responded

And our Junit test was successful, so we got the text back that we were expecting in the browser.

Rest Web Services With Spring

REST or Representational State Transfer web services are a subset of traditional web services that aim to simplify the web service concept by providing a standard list of operations. Basically REST took web services and made them much simpler to do. This allows us to have a relatively simple and standard way to do client/server processing.

Spring

The Spring Framework has been around in the Java world for a long time with the aim of providing an abstraction layer to make it easier to create large applications without getting buried in lots of complex code. Through Springs various modules it allows you to create simple code as its provides a layer on top of the various Java technologies that otherwise can be quite complex and cumbersome to deal with

Spring MVC

The Model View Controller pattern is often used when creating UI workflows. Spring MVC implements such a pattern for the Java World, in an easy and flexible way.

Spring MVC plays a role in both in both traditional UI type applications when a view is needed to updated a browser, and non traditional service applications such as a REST web service, where the response will typically be some data.

DispatchServlet

Spring MVC has the concept of the DispatchServlet. The DispatchServlet acts as the front door to your web app and determines which classes, methods and/or views are required to fulfil the request, based on the configuration.

We can configure the DispatchServlet in our application either by adding config to web.xml, or as I prefer, to do it through code. Doing it with code gives you more flexibility to set things up at runtime, so allow you to customise it to your own needs more. Heres our WebInitializer class that configures the DispatchServlet.

WebInitializer.java

package hello;

import javax.servlet.ServletContext;
import javax.servlet.ServletException;
import javax.servlet.ServletRegistration.Dynamic;

import org.springframework.web.WebApplicationInitializer;
import org.springframework.web.context.support.AnnotationConfigWebApplicationContext;
import org.springframework.web.servlet.DispatcherServlet;

public class WebInitializer implements WebApplicationInitializer {

    @Override
    public void onStartup(ServletContext servletContext) throws ServletException {

	AnnotationConfigWebApplicationContext ctx = new AnnotationConfigWebApplicationContext();
	ctx.register(WebConfig.class);
	ctx.setServletContext(servletContext);

	Dynamic servlet = servletContext.addServlet("dispatcher", new DispatcherServlet(ctx));
	servlet.setLoadOnStartup(1);
	servlet.addMapping("/");
    }
}

1. To configure the DispatchServlet, we must use a class that implements the WebApplicationInitializer interface as shown above.
2. In our example we use the AnnotationConfigWebApplicationContext which allows us to configure the DispatcherServlet using another annotated class rather than another xml file. Springs ApplicationContext is the main source of configuration within Spring, so has lots of additional features, but for what we are doing here we can keep it simple.
3. Then we register that class to the context, in this case WebConfig.class.
4. set the context
5. Then create and add the DispatcherServlet and map it to “/”

WebConfig.java

package hello;

import org.springframework.context.annotation.ComponentScan;
import org.springframework.context.annotation.Configuration;
import org.springframework.web.servlet.config.annotation.EnableWebMvc;

@Configuration
@EnableWebMvc
@ComponentScan(basePackages="hello")
public class WebConfig {

}

Heres our annotated class for configuring the DispatchServlet. In our case its very simple with @Configuration, @EnableWebMvc @ComponentScan(basePackages = “hello”), which indicates which package on the classpath that Spring will use to search for controllers.

Creating and Testing A Rest Web Service

With our WebInitializer and WebConfig in place, lets create a Junit test that is calling a rest API from our java code, to see if we can access a REST web service (Even though we havent created one yet).

@Test
public void testRestServerText() throws Exception {
String url = tomcat.getApplicationUrl("helloworld");

Page page = client.getPage(url);
WebResponse response = page.getWebResponse();
String content = response.getContentAsString();
System.out.println(content);
assertThat("Can get content from a simple rest controller", content,
    is("Hello from Spring and Embedded Tomcat!"));
}

Our testRestServerText() method will look for a java backend web service at \helloworld and check for some simple text in the response, as \helloworld is action as our java api endpoint. If we run this now we can see that Tomcat found our DispatcherServlet ‘dispatcher’ during server startup

INFO: 1 Spring WebApplicationInitializers detected on classpath
Jul 06, 2019 9:34:48 PM org.apache.catalina.core.ApplicationContext log
INFO: Initializing Spring DispatcherServlet 'dispatcher'
Jul 06, 2019 9:34:48 PM org.springframework.web.servlet.FrameworkServlet initServletBean
INFO: Initializing Servlet 'dispatcher'

We can also see that the helloworld was not found, and gave us a 404 error and a failed unit test

Application URL is:http://localhost:8080/helloworld
Jul 06, 2019 9:34:56 PM org.springframework.web.servlet.DispatcherServlet noHandlerFound
WARNING: No mapping for GET /helloworld
Jul 06, 2019 9:34:56 PM com.gargoylesoftware.htmlunit.WebClient printContentIfNecessary
INFO: statusCode=[404] contentType=[text/html]

@RestController

As mentioned earlier the DispatchServlet of Spring MVC acts as the gatekeeper and then directs requests to the relevant controllers. The one we will use for our REST web services is the @RestController. The difference with a @RestController is that is returns a response directly rather than letting that be dealt with by a view.

In our WebConfig.java, we specified

@ComponentScan(basePackages = “hello”)

so as long as we put a controller within that package hierarchy, the DispatchServlet should be able to find it.

HelloWorldController.java

package hello.controller;

import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RestController;

@RestController
@RequestMapping("/helloworld")
public class HelloWorldController {
    @GetMapping
    public String sayHello() {
	return "Hello from Spring and Embedded Tomcat!";
    }
}

Heres our simple controller. We have the @RestController annotation. This tells Spring 2 things:

1. This is a @Controller so is available to DispatchServlet to handle requests
2. This specifies @ResponseBody, so the controller should respond with an object and not delegate to any view.

The @RequestMapping tells Spring expose the request that this class will map to and the @GetMapping on the sayHello() method tells sprint to map GET requests to this method.

If we run our Junit test again we get the response below

Application URL is:http://localhost:8080/helloworld
Hello from Spring and Embedded Tomcat!

And our Junit test is successful as we get the response we expected.

Returning JSON Objects from the @RestController

In our last example, the HelloWorld example has a return type of String, and so was returning plain text. If we replace that with an object, Spring will automatically convert the output of the @RestController to a JSON format, giving us a simple restful web services JSON example.

Below is our Greeting.java POJO that we are going to return the output of an instance of that from a webservice.

package hello.pojo;

public class Greeting {

    private final long id;
    private final String content;

    public Greeting(long id, String content) {
	this.id = id;
	this.content = content;
    }

    public long getId() {
	return id;
    }

    public String getContent() {
	return content;
    }

    @Override
    public String toString() {
	return "Greeting [id=" + id + ", content=" + content + "]";
    }

}

Lets also create a new @RestController GreetingController.java to act as the controller for this new object.

package hello.controller;

import java.util.concurrent.atomic.AtomicLong;

import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RequestParam;
import org.springframework.web.bind.annotation.RestController;

import hello.pojo.Greeting;

@RestController
public class GreetingController {

    private static final String template = "Hello, %s!";
    private final AtomicLong counter = new AtomicLong();

    @RequestMapping("/greeting")
    public Greeting greeting(@RequestParam(value = "name", defaultValue = "World") String name) {
	Greeting greeting = new Greeting(counter.incrementAndGet(), String.format(template, name));
	return greeting;
    }
}

In this case we have mapping /greeting to greeting() method. Notice here that this method returns an instance of the Greeting object. It also takes in a parameter called name, but will set a default of ‘World’ if that parameter is not passed.

testRestServerJson()

Heres our unit test for checking the JSON output coming from the /greeting REST service.

    @Test
    public void testRestServerJson() throws Exception {
	String url = tomcat.getApplicationUrl("greeting?name=Dave");

	Page page = client.getPage(url);
	WebResponse response = page.getWebResponse();
	String content = response.getContentAsString();
	System.out.println(content);
	assertThat("Can get json content from a rest controller", response.getContentType(), is("application/json"));
	assertThat("Can get json content from a rest controller", content,
		startsWith("{\"id\":1,\"content\":\"Hello, Dave!\"}"));

	Greeting greeting = new Gson().fromJson(content, Greeting.class);
	System.out.println(greeting);
	assertThat("Can format a pojo from json", greeting.toString(), is("Greeting [id=1, content=Hello, Dave!]"));

	url = tomcat.getApplicationUrl("greeting?name=Steve");
	content = client.getPage(url).getWebResponse().getContentAsString();
	greeting = new Gson().fromJson(content, Greeting.class);
	assertThat("Can format a pojo from json", greeting.toString(), is("Greeting [id=2, content=Hello, Steve!]"));

    }

We have some extract checks here as we are doing more. We want to validate that we are actually getting JSON format back which we can do with the below.

ssertThat("Can get json content from a rest controller", response.getContentType(), is("application/json"));

We can also check that the content is in JSON format too.

	assertThat("Can get json content from a rest controller", content,
		startsWith("{\"id\":1,\"content\":\"Hello, Dave!\"}"));

We then use Gson to convert from JSON to an instance of the required object

assertThat("Can format a pojo from json", greeting.toString(), is("Greeting [id=1, content=Hello, Dave!]"));

and also that the Greeting object got created as expected.

We can also see that the Name parameter we used is getting returned in the repose Json object, but lets try calling /greeting with a different name to confirm.

url = tomcat.getApplicationUrl("greeting?name=Steve");
content = client.getPage(url).getWebResponse().getContentAsString();
greeting = new Gson().fromJson(content, Greeting.class);
assertThat("Can format a pojo from json", greeting.toString(), is("Greeting [id=2, content=Hello, Steve!]"));

Lets run our unit test now and see what we get back.

So our unit tests passed which means that the responses we got back we as we expected which is good. And if we look at the console output we can see that the Greeting object is being returned as JSON from the Rest Web Service, and we can convert that back to a Greeting instance from the JSON.

Application URL is:http://localhost:8080/greeting?name=Dave
{"id":1,"content":"Hello, Dave!"}
Greeting [id=1, content=Hello, Dave!]
Application URL is:http://localhost:8080/greeting?name=Steve

Securing Our Rest Web Services

Spring Web Application Security

At the moment our restful web services are wide open and anyone can access them, so lets look into how we can secure those using facilities provided by Spring. We can apply security using spring starting with the AbstractSecurityWebApplicationInitializer. By extending this we can tell Spring that we want to configure some security for our application.

package hello.security;

import org.springframework.security.web.context.AbstractSecurityWebApplicationInitializer;

public class SecurityWebApplicationInitializer extends AbstractSecurityWebApplicationInitializer {

    public SecurityWebApplicationInitializer() {
	super(SecurityJavaConfig.class);
    }
}

Here we are telling spring to find out security configuration by using SecurityJavaConfig.class

In our SecurityJavaConfig class, we can setup our Basic Authentication in Spring by defining a user and password with a role, then configuring the role and setting the URL for the Basic Authentication.

We can also create a custom entry point so that we return a sensible message that is appropriate for a Rest web service, instead of returning a UI response.

package hello.security;

import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.security.config.annotation.web.builders.HttpSecurity;
import org.springframework.security.config.annotation.web.configuration.EnableWebSecurity;
import org.springframework.security.config.annotation.web.configuration.WebSecurityConfigurerAdapter;
import org.springframework.security.config.http.SessionCreationPolicy;
import org.springframework.security.core.userdetails.User;
import org.springframework.security.core.userdetails.UserDetails;
import org.springframework.security.core.userdetails.UserDetailsService;
import org.springframework.security.crypto.factory.PasswordEncoderFactories;
import org.springframework.security.crypto.password.PasswordEncoder;
import org.springframework.security.provisioning.InMemoryUserDetailsManager;

@Configuration
@EnableWebSecurity
public class SecurityJavaConfig extends WebSecurityConfigurerAdapter {

    @Bean
    public UserDetailsService userDetailsService() {
	PasswordEncoder encoder = PasswordEncoderFactories.createDelegatingPasswordEncoder();
	UserDetails user = User.withUsername("user").password(encoder.encode("password")).roles("USER").build();
	InMemoryUserDetailsManager manager = new InMemoryUserDetailsManager();
	manager.createUser(user);
	return manager;
    }

    @Override
    protected void configure(HttpSecurity http) throws Exception {
	http.csrf().disable().authorizeRequests().antMatchers("/*").hasRole("USER").and().httpBasic()
		.authenticationEntryPoint(new CustomBasicAuthenticationEntryPoint()).and().sessionManagement()
		.sessionCreationPolicy(SessionCreationPolicy.STATELESS); // We don't need sessions to be created.
    }

}

Heres our CustomBasicAuthenticationEntryPoint

package hello.security;

import java.io.IOException;
import java.io.PrintWriter;

import javax.servlet.ServletException;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;

import org.springframework.security.core.AuthenticationException;
import org.springframework.security.web.authentication.www.BasicAuthenticationEntryPoint;

public class CustomBasicAuthenticationEntryPoint extends BasicAuthenticationEntryPoint {

    @Override
    public void commence(final HttpServletRequest request, final HttpServletResponse response,
	    final AuthenticationException authException) throws IOException, ServletException {
	// Authentication failed, send error response.
	response.setStatus(HttpServletResponse.SC_UNAUTHORIZED);
	response.addHeader("WWW-Authenticate", "Basic realm=" + getRealmName() + "");

	PrintWriter writer = response.getWriter();
	writer.println("HTTP Status 401 : " + authException.getMessage());
    }

    @Override
    public void afterPropertiesSet() throws Exception {
	setRealmName("TEST_REALM");
	super.afterPropertiesSet();
    }
}

This sets the status code and header, and returns the security error as a message.

If we run our unit test again, we get the following logged in the console.

INFO: HTTP Status 401 : Full authentication is required to access this resource

and our unit test fails with a 401 error, as we havent provided any credentials on our request.

Passing Credentials With Selenium

As we are using Selenium to act as our browser/client at the moment, we need to pass the credentials via the Selenium browser to access our Restful web services now, as we have secured them with Basic Authentication using Spring. Lets create 2 methods getBrowser() and getWebClient() to setup the credentials for the 2 browsers we are using.

	// create a new clients
	browser = getBrowser();
	client = getWebClient();

The getBrowser method adds the username and password to the WebDriver we are using by overridding the modifyWebClient method so it can add the credentials.

    private static HtmlUnitDriver getBrowser() {
	// create a new browser so that we test with an invalid password
	return new HtmlUnitDriver() {
	    protected WebClient modifyWebClient(WebClient client) {
		getCredentialsProvider(client);
		return client;
	    }

	    private void getCredentialsProvider(WebClient client) {
		DefaultCredentialsProvider creds = new DefaultCredentialsProvider();
		creds.addCredentials(username, password);
		client.setCredentialsProvider(creds);
	    }
	};
    }

The getClient method does similar to the WebClient browser.

    private static WebClient getWebClient() {
	WebClient client = new WebClient();
	client.setCredentialsProvider(getCredentialsProvider());
	return client;
    }

    private static DefaultCredentialsProvider getCredentialsProvider() {
	DefaultCredentialsProvider creds = new DefaultCredentialsProvider();
	creds.addCredentials(username, password);
	return creds;
    }

If we run out unit test again, we can check that we now get out JSON response back from our REST web service.

Application URL is:http://localhost:8080/greeting?name=Dave
{"id":1,"content":"Hello, Dave!"}
Greeting [id=1, content=Hello, Dave!]
Application URL is:http://localhost:8080/greeting?name=Steve

Just to confirm, lets change the password in our Junit test to an invalid one to check that the Basic Authentication is really working. In our SecurityJavaConfig we had set the username and password to user/password, so if we change those in our unit test to something invalid it should fail.

So now instead of getting our JSON response, we get an error in the console and our unit test fails with a 401 error.

INFO: HTTP Status 401 : Bad credentials

So we have confirmed that we can setup Basic Authentication and confirm that its actually checking the user and password that we provide before letting us access the rest web services.

Additional Security With SSL

We have added Basic Authentication to ensure that people have to pass their credentials to access the REST web services. But the one disadvantage is that currently the username and password is being passed in clear text.

In a production server environment, this would be a real security risk. The best way to ensure that no one can sniff the connection and grab your credentials is to encrypt the conversation between the browser and the server by configuring and enabling SSL.

We wont be configuring the checking of the certs between the client and the server though, as we are running both the client and the server in the same JVM so it doesnt really work as the system properties clash.

Configuring SSL For Embedded Tomcat

To setup SSL in Embedded Tomcat we need to first create a keystore for embedded tomcat, and stick a certificate in it. We can do that with the keytool command as shown below.

keytool -genkeypair -alias localhost -keyalg RSA -keysize 2048 -keystore keystore.jks -validity 3650

This will create the keystore file and we will stick this in the root of our eclipse project and point to that in our config.

We also need to add this self signed cert to java so it will trust it.

keytool -export -alias localhost -keystore keystore.jks -rfc -file localhost.cer

keytool -import -file <path to cert>\localhost.cer -alias localhost -keystore "<path to jre>\jre\lib\security\cacerts"

Now that we have the keystore file we need to configure our tomcat to enable the SSL. We should create a new method to do the SSL configuration.

We also want to setup a new variable to hold the SSL port, 8443, that we will configure.

    public int PORT = 8080;
    public int SSLPORT = 8443;

We will configure the settings on tomcat as below

    private void configureSsl(Tomcat tomcat, String baseDir) {
	Connector connector = new Connector();
	connector.setPort(SSLPORT);
	connector.setSecure(true);
	connector.setScheme("https");
	connector.setAttribute("keyAlias", "<your computer name>");
	connector.setAttribute("keystorePass", "keystore");
	connector.setAttribute("keystoreType", "JKS");
	connector.setAttribute("keystoreFile", baseDir + separator + "keystore.jks");
	connector.setAttribute("clientAuth", "false");
	connector.setAttribute("protocol", "HTTP/1.1");
	connector.setAttribute("sslProtocol", "TLS");
	connector.setAttribute("maxThreads", "200");
	connector.setAttribute("protocol", "org.apache.coyote.http11.Http11AprProtocol");
	connector.setAttribute("SSLEnabled", true);
	tomcat.getService().addConnector(connector);
    }

We will also update our getApplicationUrl method to allow us to specify if we are sending our request to the SSL port or not.

    public String getApplicationUrl(String target, boolean ssl) {
	String protocol = "http";
	int port = getPort(ssl);
	if (ssl) {
	    protocol = "https";
	}

	String url = String.format("%s://%s:%d/%s", protocol, getHostName(), port, target);
	System.out.println("Application URL is:" + url);
	return url;
    }

    public int getPort(boolean ssl) {
	int port = PORT;
	if (ssl) {
	    port = SSLPORT;
	}
	return port;
    }

We will pass a boolean from our Junit test to this to indicate if we are using SSL, so it will return a properly formatted URL.

String url = tomcat.getApplicationUrl("greeting?name=Dave", true);

So if we run our Junit test again we can confirm it will return a response if we run with SSL

Application URL is:https://localhost:8443/greeting?name=Dave
{"id":1,"content":"Hello, Dave!"}
Greeting [id=1, content=Hello, Dave!]
Application URL is:https://localhost:8443/greeting?name=Steve

We can prove that its actually running with SSL enabled by changing our URL to non SSL and see if it still works.

We dont have the SSL port enabled, so instead of returning the results we expect, tomcat denies our request as the http port is not enabled .

Application URL is:http://localhost:8080/greeting?name=Dave

org.apache.http.conn.HttpHostConnectException: Connect to localhost:8080 [localhost/127.0.0.1, localhost/0:0:0:0:0:0:0:1] failed: Connection refused: connect

Using Springs RestTemplate To Build A Web Service Client

So far we have just used Selenium to act as a dummy browser to check the responses back from the RESTful web services we built. But in the real world you will probably want to use something a bit more powerful for processing your web service calls.

The perfect solution for this is Springs RestTemplate class. This is a good helper, that encaupsulates a lot of the functionality you will likely need for talking to RESTful web services. Its supports the various REST methods and makes it easy to handle the responses coming from the the server.

To try this lets take the last unit test we did and create a new one using RestTemplate instead of the Selenium WebClient.

    @Test
    public void testRestTemplate() throws Exception {
	String url = tomcat.getApplicationUrl("greeting?name=Dave", true);

	RestTemplate restTemplate = new RestTemplate();
	Greeting greeting = restTemplate.getForObject(url, Greeting.class);

	System.out.println(greeting);
	assertThat("Can format a pojo from json", greeting.toString(), is("Greeting [id=1, content=Hello, Dave!]"));
    }

As you can see we are calling restTemplate.getForObject(url, Greeting.class) which should call the web service and then convert the JSON response into the Greeting object. But unfortunately we are getting a 401 message back as we havent provided any authentication details on the RestTemplate.

To add authentication to our RestTemplate call we need to add the credentials to it. To do that first we need to a HttpRequestFactory to the constructor for RestTemplate. We also need to build a HttpClient that we can add the credentials to.

private HttpComponentsClientHttpRequestFactory getRequestFactory() {
HttpClientBuilder builder = HttpClients.custom();
CloseableHttpClient httpClient = getHttpClient(builder);

HttpComponentsClientHttpRequestFactory clientHttpRequestFactory = new HttpComponentsClientHttpRequestFactory();
clientHttpRequestFactory.setHttpClient(httpClient);
return clientHttpRequestFactory;
}

The getRequestFactory() method creates the Factory. That uses a builder to setup a HttpClient to pass the credentials as shown below.

    private CloseableHttpClient getHttpClient(HttpClientBuilder builder) {
	RequestConfig config = RequestConfig.custom().setAuthenticationEnabled(true).build();
	CredentialsProvider credentialsProvider = new BasicCredentialsProvider();
	UsernamePasswordCredentials credentials = new UsernamePasswordCredentials(username, password);
	credentialsProvider.setCredentials(AuthScope.ANY, credentials);

	CloseableHttpClient httpClient = builder.setDefaultRequestConfig(config)
		.setDefaultCredentialsProvider(credentialsProvider).build();
	return httpClient;
    }

So if we change our creation of the RestTemplate to use the RequestFactory

	RestTemplate restTemplate = new RestTemplate(getRequestFactory());

If we run the testRestTemplate() Junit test again, this time we get the proper response from the Rest web service, as we are now able to authenticate.

Application URL is:https://localhost:8443/greeting?name=Dave
Greeting [id=1, content=Hello, Dave!]

We can also see that the RestTemplate must be automatically converting the JSON response into our Greeting object as we have not specified any code in the unit test to convert that, apart from telling the RestTemplate what POJO object to convert it to.

    @Test
    public void testRestTemplate() throws Exception {
	String url = tomcat.getApplicationUrl("greeting?name=Dave", true);

	RestTemplate restTemplate = new RestTemplate(getRequestFactory());
	Greeting greeting = restTemplate.getForObject(url, Greeting.class);

	System.out.println(greeting);
	assertThat("Can format a pojo from json", greeting.toString(), is("Greeting [id=1, content=Hello, Dave!]"));
    }

Testing Your REST Web Service Client with MockRestServiceServer

Once you have established what the response is from the server, and you have built a client, you may want to test your client code to ensure it still handles your RESTful web services responses properly.

The MockRestServiceServer allows you to set up the requests to expect and the mock responses to provide, that allows you to test your client code without having to access a server. By using a mock server provided by using MockRestServiceServer, you can shorten your development and testing time when you are changing or implementing you code, as you are reliant on a real server all the time.

Heres our method for testing the REST Webservice using MockRestServiceServer Mock Server.

    @Test
    public void testRestTemplateWithMockRestServer() throws Exception {

	String url = tomcat.getApplicationUrl("greeting?name=Dave", true);

	RestTemplate restTemplate = new RestTemplate(getRequestFactory());
	mockRestServer(restTemplate);

	Greeting greeting = restTemplate.getForObject(url, Greeting.class);

	System.out.println(greeting);
	assertThat("Can format a pojo from json", greeting.toString(), is("Greeting [id=1, content=Hello, Dave!]"));

The only additional line we have added versus the previous unit test where we were calling embedded tomcat rather than using a mock server is here.

	mockRestServer(restTemplate);

Our MockRestServer method is where we setup our expected requests and our responses for the mock server:

    private void mockRestServer(RestTemplate restTemplate) throws Exception {
	MockRestServiceServer mockServer = MockRestServiceServer.bindTo(restTemplate).bufferContent().build();
	mockServer.reset();

	Greeting greeting = new Greeting(1, "Hello, Dave!");
	String greetingJson = new Gson().toJson(greeting);

	HttpHeaders headers = new HttpHeaders();
	mockServer.expect(ExpectedCount.once(), requestTo(new URI("https://localhost:8443/greeting?name=Dave")))
		.andExpect(method(HttpMethod.GET)).andRespond(withStatus(HttpStatus.OK).body(greetingJson)
			.headers(headers).contentType(MediaType.APPLICATION_JSON));

This method will build the mock server attached to the RestTemple we are going to use, and reset it to make sure its starting afresh. We can then apply matcher to the mock server to tell it what to expect. In this case we are saying that if we receive a GET request on the URL

https://localhost:8443/greeting?name=Dave

that the mock server should respond with HTTP Status Ok, and the JSON message and also set the content type to indicate the response is JSON.

So effectively we can run the same test as previously, but testing the client code against the mock server, and not touch the real embedded tomcat server.

Using Jackson To Convert JSON Objects

Although in our previous examples we used Gson for converting between JSON and Java, another is Jackson. Jackson is a java library for working with JSON. It allows you to convert between Java POJOs and JSON messages, plus has other tools that aid in the processing of JSON with Java. Jackson includes annotions that you can use in your code to allow more control of the parsing and conversion process.

JSON To Java Example Using Jackson

As we have done previously, lets walk through a quick example of using Jackson. In this case rather than using our embedded tomcat server, we will use an external RESTful web service to provide a more real world example of the case in action. We will create a new Junit Test class for this as we dont need Tomcat running for this test each time.

CoinMarketCap is a large crytocurrency data provider. They have a cryptocurrency ticker that provides REST out. We will call their API by accessing the URL https://api.coinmarketcap.com/v1/ticker/ . If we access the URL we see that we get a JSON message similar to below.

As you can see, this JSON message returns multiple cryptocurrencies. Let setup some code to return that JSON message first of all. Initially lets grab the results as a String.

package com.colwil.rest;

import static org.hamcrest.CoreMatchers.containsString;
import static org.junit.Assert.assertThat;

import org.junit.Test;
import org.springframework.web.client.RestTemplate;

public class RestTestExternal {

    @Test
    public void testJackson() throws Exception {
	String url = "https://api.coinmarketcap.com/v1/ticker/";

	RestTemplate restTemplate = new RestTemplate();
	String currencies = restTemplate.getForObject(url, String.class);

	System.out.println(currencies);
	assertThat("Retrieves currencies", currencies, 
           containsString("Bitcoin"));
    }
}

So all we have done here is access the CoinMarketCap.com ticker and grabbed the results as a String. To make this useful of course we want to convert this into a Java POJO.

When making use of JSON data, its very likely you may be provided with a lot of fields ,but you may not want to use all of them. So lets create our POJO only using the name, symbol, rank and price_usd. Heres our POJO below.

package crypto;

public class Currency {

    String name;
    String symbol;
    String rank;
    String price_usd;

}

We dont have any getters and setters but in eclipse its easy to add those by selecting the fields, doing right click, click Generate Getters and Setters, then on the window below, select the fields and press generate.

And we end up with our POJO with getters and setters

package crypto;

public class Currency {

    String name;
    String symbol;
    String rank;
    String price_usd;

    public String getName() {
	return name;
    }

    public void setName(String name) {
	this.name = name;
    }

    public String getSymbol() {
	return symbol;
    }

    public void setSymbol(String symbol) {
	this.symbol = symbol;
    }

    public String getRank() {
	return rank;
    }

    public void setRank(String rank) {
	this.rank = rank;
    }

    public String getPrice_usd() {
	return price_usd;
    }

    public void setPrice_usd(String price_usd) {
	this.price_usd = price_usd;
    }

}

Now lets add the Jackson dependency to our POM and rebuild so we can use it in our code.

So I have modified our testJackson test as below

    @Test
    public void testJackson() throws Exception {
	String url = "https://api.coinmarketcap.com/v1/ticker/";

	RestTemplate restTemplate = new RestTemplate();
	String json = restTemplate.getForObject(url, String.class);

	ObjectMapper mapper = new ObjectMapper();
	Currency currency = mapper.readValue(json, Currency.class);

	System.out.println(currency);
	assertThat("Retrieved currency", currency.toString(), containsString("Bitcoin"));
    }

As you can see there are a couple of lines in there that related specifically to Jackson parsing.

ObjectMapper mapper = new ObjectMapper();

~The Jackson ObjectMapper is the class that handles reading and writing the JSON, and here were a creating an instance of it to use.

Currency currency = mapper.readValue(json, Currency.class);

readValue is the method used to deserialize the JSON string and create the POJO.

Now when we run our Junit test it fails with this error.

com.fasterxml.jackson.databind.exc.MismatchedInputException: Cannot deserialize instance of `crypto.Currency` out of START_ARRAY token
 at [Source: (StringReader); line: 1, column: 1]

There is reason for that and if we look at the format of the JSON message again we can see why.

[
    {
        "id": "bitcoin", 
        "name": "Bitcoin", 
        "symbol": "BTC", 
        "rank": "1", 
        "price_usd": "10673.597655", 
        "price_btc": "1.0", 
        "24h_volume_usd": "22517294625.9", 
        "market_cap_usd": "190174424658", 
        "available_supply": "17817275.0", 
        "total_supply": "17817275.0", 
        "max_supply": "21000000.0", 
        "percent_change_1h": "-0.07", 
        "percent_change_24h": "-6.57", 
        "percent_change_7d": "-5.89", 
        "last_updated": "1563116428"
    }, 
    {
        "id": "ethereum", 
        "name": "Ethereum", 
        "symbol": "ETH", 
        "rank": "2", 
        "price_usd": "239.250065882", 
        "price_btc": "0.02245193", 
        "24h_volume_usd": "7911763913.22", 
        "market_cap_usd": "25574955311.0", 
        "available_supply": "106896335.0", 
        "total_supply": "106896335.0", 
        "max_supply": null, 
        "percent_change_1h": "-0.14", 
        "percent_change_24h": "-11.69", 
        "percent_change_7d": "-19.13", 
        "last_updated": "1563116482"
    }, 

The message starts with a “[” symbol. This indicates an array. So what this is telling us is to expect multiple entries. So in this case we are going to get multiple currencies. So we need to use an array of currency objects rather than a single object.

	Currency[] currencies = mapper.readValue(json, Currency[].class);

We run our modified code as per below

    @Test
    public void testJackson() throws Exception {
	String url = "https://api.coinmarketcap.com/v1/ticker/";

	RestTemplate restTemplate = new RestTemplate();
	String json = restTemplate.getForObject(url, String.class);

	ObjectMapper mapper = new ObjectMapper();
	Currency[] currencies = mapper.readValue(json, Currency[].class);

	System.out.println(currencies[0]);
	assertThat("Retrieved currency", currencies[0].toString(), containsString("Bitcoin"));
    }

but we get another error

com.fasterxml.jackson.databind.exc.UnrecognizedPropertyException: Unrecognized field "id" (class crypto.Currency), not marked as ignorable (4 known properties: "symbol", "name", "rank", "price_usd"])

If we look back at our JSON data we can see that the currency starts with an “id” field, but Jackson is complaining as we havvent told it what to do with that, and it currently expects all fields to be used. If we add an annotation

@JsonIgnoreProperties(ignoreUnknown = true)
public class Currency {

to our Currency class, this will tell Jackson to ignore any properties in the JSON message that it doesnt have matching getter/setters for in the POJO. So if we run our unit test again lets see what we get.

Currency [name=Bitcoin, symbol=BTC, rank=1, price_usd=10618.3367488]

The unit test passes and we see our list of fields.

So we have seen that we can use Jackson to do JSON mapping in Java. But the Spring RestTemplate does actually do its own conversion using its own internal version of Jackson if you dont explicity use Jackson yourself. So if we change our code to use the RestTemplate to do the conversion it still works.

    @Test
    public void testJackson() throws Exception {
	String url = "https://api.coinmarketcap.com/v1/ticker/";

	RestTemplate restTemplate = new RestTemplate();
	Currency[] currencies = restTemplate.getForObject(url, Currency[].class);

	System.out.println(currencies[0]);
	assertThat("Retrieved currency", currencies[0].toString(), containsString("Bitcoin"));
    }

We can additional prove this is using Jackson by adding a new field to our currency object.

    String identifier;

    public String getIdentifier() {
	return identifier;
    }

    public void setIdentifier(String identifier) {
	this.identifier = identifier;
    }

If we run out unit tests they dont fail (RestTemplate internal version of Jackson ignores unknown elements by default) but our new field isnt populated as we dont have a matching field in the JSON data.

Currency [name=Bitcoin, symbol=BTC, rank=1, price_usd=10630.4903779, identifier=null]

We can add a Jackson annotation to our Currncy class to tell Jackson where to get this from. If we add @JsonAlias({ “id” }), this tells Jackson to get the “identifier” field in the POJO from the “id” in the JSOn message.

So lets rerun our test and see what happens

Currency [name=Bitcoin, symbol=BTC, rank=1, price_usd=10664.3742598, identifier=bitcoin]

As you can see out identifier field is now populated with the id field from the JSON message.

    {
        "id": "bitcoin", 
        "name": "Bitcoin", 
        "symbol": "BTC", 
        "rank": "1", 
        "price_usd": "10673.597655", 
        "price_btc": "1.0", 
        "24h_volume_usd": "22517294625.9", 
        "market_cap_usd": "190174424658", 
        "available_supply": "17817275.0", 
        "total_supply": "17817275.0", 
        "max_supply": "21000000.0", 
        "percent_change_1h": "-0.07", 
        "percent_change_24h": "-6.57", 
        "percent_change_7d": "-5.89", 
        "last_updated": "1563116428"
    },