Spring Cloud 1.1: RxJava

The next big thing that is going to happen in Java world in upcoming future is standardization of the reactive programming API in JVM. Java 9 is going to most likely be bundled with Reactive Streams API. Also some existing libraries like RxJava 2.x are moving towards the same standardized API. From what you could learn from last SpringOne is that Spring 5 have plans for seamless integration with reactive paradigm.

Question is do you really need to wait to easily facilitate all of this goodies? The answer is not necessary, especially that you can already use libraries like RxJava.

Spring MVC integration

One nice kind of integration that you can get out of the box thanks to upcoming Spring Cloud Netflix 1.1.x release is support for Observable return types in your controller methods, which makes it convenient if you return Observable calls to remote service and perform some aggregation on the results.

With Spring Cloud Netflix 1.1.x on classpath you can transform your Spring MVC REST controllers into fallowing:

@RestController
public class ProductController {
    
    private final ProductService productService;

    public ProductController(ProductService productService) {
        this.productService = productService;
    }

    @ResponseStatus(OK)
    @RequestMapping(method = GET)
    public Observable<Product> getProductList() {

        return productService.getProducts();
    }
}

You still need to configure TaskExecutor to be able to use this functionality as you would with standard DefferedResult, ListenableFuture or Callable return types.

Hystrix and Javanica

Hystric commands are capable of handling Observables return types as well.

Future possible integrations

I’ve happen to find this interesting pull request that brings the RxJava integration directly to Spring Data commons module, sadly it hasn’t been merged yet. Personally I think that it would be really convenient to call multiple queries at the same time through such API. Maybe this issue simply needs some extra support from community?

If you would be more interested in this topic I highly advice visiting the RxJava project page and some articles on the web.

Spring Cloud: Zuul routes health

When I had described the Go CD Health check plugin I’ve briefly talk about it’s use cases and that it can be set up for automatic delay of acceptance or stress tests. It’s time to describe a bit more in details and how it can be used with Spring Cloud application in particular. We wanted to run the tests of system user interface, but to do that we needed all the backing systems to be available. Since the application works in reverse proxy setup, we had simply await for the depending services to register with ‘UP’ state in our discovery service. So what we wanted to do, is to adjust the application health endpoint to make it aware whether the routes to backend services are available.

This required to register custom Spring Boot’s Actuator health indicator. Actually I had prepared a small utility project that will do this:

https://github.com/jmnarloch/zuul-route-health-spring-cloud-starter

It can be turn on and off through settings ‘zulu.health.enabled’ flag:

zuul:
  ignoredServices: '*'
  health:
    enabled: true
  routes:
    resource:
      path: /api/**
      serviceId: rest-service
    oauth2:
      path: /uaa/**
      serviceId: oauth2
      stripPrefix: false

So you can configure it for your test/dev/stg environments and disable it for production, mostly because cascading the system failures might not be such a good idea. Though for our use case this worked more then well.

The extensions has it’s limitations, at this point it only checks the explicitly configures services (the routes with specified serviceId) and verifies whether they are accessible from the discovery service.

Health check in practice
zuul-health

Spring Cloud: Eureka, Zuul and OAuth2 – scaling out authorization server

We are going to touch here a very practical problem, scaling out the Spring OAuth2 authorization server and describing a bit more in detail how this can be done using the Spring Cloud itself. Your authorization server might be requested multiple times during single request processing so you would rather want it to be able to match the demand.

When I first approach the issue I didn’t fully understood which of functionalities Spring Cloud provides and made some false assumptions, so it might be worth describing how to do a basic setup, what is happening under the hood and what you might expect.

In Spring OAuth2 2.0.x you have three very helpful annotations that can enable different functionaries within your application:

  • @EnableOAuth2Resource
  • @EnableOAuth2Sso
  • @EnableOAuth2Client

I want to cover the first two in greater detail.

For each of the annotation there is a corresponding properties that must be to specified. The minimal resource server configuration needed to verify the token and retrieve the user information is to specify the userInfoUri property.

spring:
  oauth2:
    resource:
      userInfoUri: http://localhost:9999/uaa/user

This will allow for the incoming requests to be verified using Barer Token authentication.

So most likely you are going to enable resource server on probably every microservice that you are going to develop. On one of your edge services you may want to enable the single sign on. The @EnableOAuthSso will allow you to turn automatic redirection of your unauthicated user towards the authorization server, where they will be able to log in.

Similar you will have to specify some extra properties like the address of the authorization server token and authorization uri together with client credentials. We can also specify which paths within our application should be secured.

spring:
  oauth2:
    sso:
      home:
        secure: true
        path: /,/**/*.html
    client:
      accessTokenUri: http://localhost:9999/uaa/oauth/token
      userAuthorizationUri: http://localhost:9999/uaa/oauth/authorize
      clientId: edge
      clientSecret: secret

Now this is a minimal setup that is going work on a single node (of course you can put the authorization server behind any load balancer).

This is not all, if you import spring-cloud-security module (in version at least 1.0.2) you can expect fallowing. On each resource server the auto configuration class is going to setup OAuth2RestTemplate that will propagate the authentication token and if you happen to have discovery client enabled (by importing Eureka client or Consul and setting it up correctly) it will be also be capable of discovering and distributing the load towards registered services.

Also you can achieve the same for the Feign clients through custom extension.

Finally if you set spring.oauth2.resource.loadBalanced to true, you will configure client load balancing for retrieval of the user information from authorization servers (when you run up multiple nodes and register them in the discovery service):

spring:
  oauth2:
    resource:
      loadBalanced: true
      userInfoUri: http://oauth2/uaa/user

This feature isn’t specially documented, and I happen to found it by going through the Spring Cloud Security source code.

This is already neat, but what about our single sign-on? At the moment we had to specify a single address.

In the project that I had been working on we have addressed the issue in two different ways in two bit different configurations. In one of the applications that was a single-page application we completely resigned from the build in server side redirection and used instead AngularJS and simple AJAX POST for authentication. For that to work we had to either enable CORS (which the authorization server dosen’t do, but you can find a interesting article how to do this) or configure a reverse proxy – like Zuul. We had chosen the latter. Through simple setup you can route the authorization AJAX request through Zuul towards the authorization server:

zuul:
  ignoredServices: '*'
  routes:
    oauth2:
      path: /uaa/**
      serviceId: oauth2
      stripPrefix: false

Finally if you want to use Spring’s single sign-on you can do that, by setting up the Zuul proxy as your edge gateway and correctly configure the route to the auto discovered nodes replacing accessTokenUri and userAuthorizationUri with the edge gateway address (preferable the DNS name). Of course there wouldn’t be any reason not to use any other load balancer instead, especially if your cloud provider offers one out of the box. Although the build in Zuul proxy auto discovery is very tempting to use here.

So to sum up, for your “internal” communication the resource servers can use the build in Spring Cloud load balancing support with help of the discovery service. When the “external” world needs to communicate with your OAuth2 server, you should put load balancer in front of it, with Netflix Zuul being natural candidate making also the solution portable across different cloud providers as well as containers, like Docker for instance.

With such setup you are now able to add as many authorization server nodes as you want to.

Java 8 functional interfaces and varags functions

Fallowing the article about simple Partial Function application in Java, I had made a simple experiment with handling the varags methods using lambdas and functional interfaces. For functions that we can represent in a form of SAM interface:

@FunctionalInterface
public interface VarArgFunction<R, T> {

    R apply(T... args);
}

As mentioned in the previous post, you can assign in fact the variable argument method to any number argument interfaces as long as the arguments are all of the same type:

Function f = Functions::funcVarArgs;
Function1<String, String> f1 = Functions::funcVarArgs;
Function2<String, String, String> f2 = Functions::funcVarArgs;

But also we can have our function “assigned” to the vararg interface:

VarArgFunction<String, String> varargs = Functions::funcVarArgs;

The Java 8 default methods turn out to be a very powerful together with lambda functions, because we can now define instance methods for that interface, let’s try to implement similar partial function application as previously for standard functions.

    @SuppressWarnings("unchecked")
    default VArgFunction<R, T> arg(T arg) {

        return (T... args) -> {
            final T[] arguments = (T[]) new Object[args.length + 1];
            arguments[0] = arg;
            System.arraycopy(args, 0, arguments, 1, args.length);
            return apply(arguments);
        };
    } 

Unfortunately this is the best we can get with expanding the original arguments array. We will need to create a new array and copy all of the original values into it. The benefit of that is that we can now add the function arguments at runtime for instance in loop, and we are not bound by any constraints, except of the heap memory size:


for(String arg : args) {
  varargs = varargs.arg(arg);
}

String result = varargs.apply()

Fallowed by immediate method invocation to get the computation result.

Unfortunately this implementation has one significant problem, since it will create for every arg() call new object array instance the running time of apply method will be quadratic – O(N^2). The arg itself is performing it’s work in O(1) time, but when the function gets evaluated it will be cascading the apply calls with creating for each individual call new array.

Now can we do better then that?

Actually a simple wrapper around the interface will do the job, since there is already Partial class in the library let’s expand it with one extra definition:

static <R, T> VArgFunction<R, T> vargFunction(VArgFunction<R, T> function) {

        return new VArgFunction<R, T>() {

            private final List<T> args = new LinkedList<>();

            public R apply(T... args) {
                this.args.addAll(Arrays.asList(args));
                final T[] argsArray = (T[]) Array.newInstance(args.getClass().getComponentType(), args.length);
                return function.apply(this.args.toArray(argsArray));
            }

            public VArgFunction<R, T> arg(T arg) {
                args.add(arg);
                return this;
            }
        };
    }

This time we are wrapping around the original functional interface and store the arguments in collection. We have effectively reduced the running time of apply method to linear time O(N) – proportional to number of arguments applied to the function.

This way we still provide the base functionality through default method implementation, though we can provide more specialized implementation through wrapper interface around the original lambda expression.

If you like to you can grab the code and see if you can find some interesting applications for running the variable argument function through functional interfaces.

https://github.com/jmnarloch/funava

Spring Cloud: Feign OAuth2 authentication

It’s worth to describe one additional use case for Spring Cloud Feign clients in microservice oriented architecture: authentication.

You may quite fast face the fact that your requests are being send across multiple services and that they may require to be aware of the user on behalf of whom the requests are being processed. Afterwards you can use that information to perform some user specific operations or simply perform authorization and verify if the user is permitted to perform specific actions.

Fortunately Spring Cloud Security module comes here with aid and whenever you use RestTemplate and OAuth2 authentication this information will be propagated with any remote call that you perform. The module will configure for you OAuth2RestTemplate that can be injected and used as normal RestOperations/RestTemplate. Providing that you had enable Spring’s OAuth2 context, which happens if you have enabled the resource server or enabled the OAuth2 client, using @EnableOAuth2Client.

Unfortunately this does not apply to your Feign clients, but we are going to change this through two simple steps. We are going to support OAuth Bearer token authentication.

First let’s define our custom RequestInterceptor.

public class OAuth2FeignRequestInterceptor implements RequestInterceptor {

    private static final String AUTHORIZATION_HEADER = "Authorization";

    private static final String BEARER_TOKEN_TYPE = "Bearer";

    private static final Logger LOGGER = LoggerFactory.getLogger(OAuth2FeignRequestInterceptor.class);

    private final OAuth2ClientContext oauth2ClientContext;

    public OAuth2FeignRequestInterceptor(OAuth2ClientContext oauth2ClientContext) {
        Assert.notNull(oauth2ClientContext, "Context can not be null");
        this.oauth2ClientContext = oauth2ClientContext;
    }

    @Override
    public void apply(RequestTemplate template) {

        if (template.headers().containsKey(AUTHORIZATION_HEADER)) {
            LOGGER.warn("The Authorization token has been already set");
        } else if (oauth2ClientContext.getAccessTokenRequest().getExistingToken() == null) {
            LOGGER.warn("Can not obtain existing token for request, if it is a non secured request, ignore.");
        } else {
            LOGGER.debug("Constructing Header {} for Token {}", AUTHORIZATION_HEADER, BEARER_TOKEN_TYPE);
            template.header(AUTHORIZATION_HEADER, String.format("%s %s", BEARER_TOKEN_TYPE,
                    oauth2ClientContext.getAccessTokenRequest().getExistingToken().toString()));
        }
    }
}

We are going to use here Spring OAuth2 OAuth2ClientContext, that will store request bound OAuth token details.

Next step: we need to register that in our application, using Spring Boot we can leverage auto configuration features and register our class in META-INF/spring.factories:

org.springframework.boot.autoconfigure.EnableAutoConfiguration=\
io.jmnarloch.spring.cloud.feign.OAuth2FeignAutoConfiguration

Finally the configuration itself:

@Configuration
@ConditionalOnClass({ Feign.class })
@ConditionalOnProperty(value = "feign.oauth2.enabled", matchIfMissing = true)
public class OAuth2FeignAutoConfiguration {

    @Bean
    @ConditionalOnBean(OAuth2ClientContext.class)
    public RequestInterceptor oauth2FeignRequestInterceptor(OAuth2ClientContext oauth2ClientContext) {
        return new OAuth2FeignRequestInterceptor(oauth2ClientContext);
    }
}

Basically that’s it. From now on every call that will be made will also go with Authorization header.

As always there is a Spring Cloud starter that you can use:

compile ('io.jmnarloch:feign-oauth2-spring-cloud-starter:1.0.0')

The source code is available at Github:

https://github.com/jmnarloch/feign-oauth2-spring-cloud-starter

Spring Cloud: Feign SPDY/HTTP2

I like to experiment with different things, so this time let’s see how we can use alternative transport protocols with our Feign clients.

Spring Cloud’s Feign clients by default work over either java.net API or HttpClient (since Spring Cloud 1.1) depending on the version and the settings, but we aren’t going to use any of those. Instead we are going to switch to Square’s OkHttp.

We do this because HttpClient does not support any alternative protocol except for HTTP 1.0/1.1 as the documentation of HttpClient 4.5.1 states:

Standards based, pure Java, implementation of HTTP versions 1.0 and 1.1

Though support for HTTP2 is planned to be added in future.

The OkHttp in contrary supports both SPDY and HTTP2 while still being able to use HTTP 1.1 to communicate with any “older” server that does not “speak” with latest transport protocols.

Feign setup

First let’s make sure we are using the latest Feign version:

configurations.all {
    resolutionStrategy {
        eachDependency { DependencyResolveDetails details ->
            if (details.requested.group == 'com.netflix.feign') {
                details.useVersion "8.10.1"
            }
        }
    }
}

Let’s setup our project and add the required dependencies:

compile ('org.springframework.cloud:spring-cloud-starter-feign:1.0.3.RELEASE')
compile ('com.netflix.feign:feign-core:8.10.1')
compile ('com.netflix.feign:feign-okhttp:8.10.1')

If you are using Spring Cloud 1.1 you will have to first disable the HttpClient support:

feign.httpclient.enabled=false

Now we need to configure Feign to work over OkHttp. This is matter only of proper configuring Spring’s bean.

    @Configuration
    @ConditionalOnClass(com.squareup.okhttp.OkHttpClient.class)
    public class OkHttpClientAutoConfiguration {

        @Autowired(required = false)
        private com.squareup.okhttp.OkHttpClient httpClient;

        @Bean
        @ConditionalOnMissingBean(OkHttpClient.class)
        public Client feignClient() {
            if (httpClient != null) {
                return new OkHttpClient(httpClient);
            }
            return new OkHttpClient();
        }
    }

If you are using Ribbon and discovery service you are going also need to configure the load balanced client. In the end you going to end up with setup like this:


@Configuration
@ConditionalOnClass({ com.squareup.okhttp.OkHttpClient.class, Feign.class, ILoadBalancer.class })
@ConditionalOnProperty(value = "feign.okhttp.enabled", matchIfMissing = true)
@AutoConfigureBefore(FeignAutoConfiguration.class)
public class OkHttpClientAutoConfiguration {

    @Autowired(required = false)
    private com.squareup.okhttp.OkHttpClient httpClient;

    @Resource(name = "cachingLBClientFactory")
    private LBClientFactory lbClientFactory;

    @Bean
    public Client feignClient() {
        RibbonClient.Builder builder = RibbonClient.builder();

        if (httpClient != null) {
            builder.delegate(new OkHttpClient(httpClient));
        } else {
            builder.delegate(new OkHttpClient());
        }

        if (lbClientFactory != null) {
            builder.lbClientFactory(lbClientFactory);
        }

        return builder.build();
    }
}

Alternativly, you may want to simply use the Spring Cloud starter that will do this for you:

https://github.com/jmnarloch/feign-okhttp-spring-cloud-starter

by importing it into your project:

compile ('io.jmnarloch:feign-okhttp-spring-cloud-starter:1.0.0')

Go Continuous Delivery: Health check plugin

This is another Go Continuous Delivery plugin to your build pipeline toolbelt. It’s name is quite descriptive so let’s discuss first it’s motivation. As you may know you can use Go not only for building your applications, but also for automatic deployments. Nevertheless you shouldn’t stop there yet. The Go build pipelines are ideal to orchestrate fully featured automated test workflows. So it’s not uncommon that after the deployment stages you will most likely configure your acceptance tests using for instance tool’s like Selenium or Protractor or similar. That will work on real setup with real data in addition to any unit or integration tests that you may had run durring the application build. The acceptance tests can be then immediately fallowed by stress tests or performence benchmarks using for example Gatling.

At my current project we had reach the stage when both the build and deployment was fully automatic, but any other pipeline stages require user interaction and manual triggering, mostly because there is delay between the point in time that the application was started and it becomes fully operational. Mostly because in mean time it may perform a lot of hard work: like establishing the database connections and migrating the database schema for instance. Registration in the discovery service and propagation of the registry among clients is also not instant. Due to all of this factors we had been running our tests after manually checking that the application has completed initalization and is ready to process requests.

But that is now history.

We had leverage the application health information in order to be able to tell whether it is in state that it can process the requests and we had added health checks to our build stages to automate monitoring them. Despite that this could be easly “implemnted” for instance in deployment script, this idea seemed like it had a large reusability potential. So is has been implemented as a Go plugin.

https://github.com/jmnarloch/gocd-health-check-plugin

gocd_healthcheck_screen

You can configure the health check task in any of your build stages. The task will delay the pipeline to the configured amount of seconds, in them same time polling your application health endpoint untill your application starts to report that is in “UP” state. If that does not happen a timeout will occur and the build will fail.

You will still need to adjust your application behaviour and define additional smart health checks, that will for instance await that your Zulu proxy (if you are using Spring Cloud and Zulu in particular) will be able to route your requests, but this can be done easly.

The implementation is reactive and uses RxJava/RxNetty under the hood, though intially it has been implemented in Project Reactor, it turn out that it may not work so well in Apache Felix the – OSGi container that is used by the Go internally.