Spring Cloud: Sock.js + STOMP + Zuul = No WebSockets

This time I wanted to share my experience when I worked on setting up Zuul proxy in front of Websocket service.

Let’s start by stating that Zuul does not support WebSocket protocol: https://github.com/spring-cloud/spring-cloud-netflix/issues/163. So probably this post should have ended here. Despite that, I wanted to see what I will be able to achieve having already all the setup in place, with Zuul reverse proxies in front and WebSocket enabled service in back.

Sock.js

Our application fronted, a single web page, was using Sock.js. The library that has one crucial functionality, especially useful in this particular case, it implements multiple fallbacks protocols when WebSocket protocol is not supported. Initially it allowed to same API to work with different browsers, hiding all communication details. We facilitate this functionality as a workaround to communicate with our backing server through Zuul.

STOMP

STOMP is messaging protocol that can be used on top of WebSocket (or HTTP) to communicate with more message orientated manner. Both on the client side and on the server side this is going to hide the details about routing the messages.

Spring Messaging

Spring Framework for a quite some time has been supporting both Sock.js and STOMP so we are going to use that.

Spring Integration

My goal was to connect our system internals, with the application UI. The backend already was exposing REST API. So I wanted to introduce a lightweight “WebSocket enabled” “fronted” that could be scaled separately and could simply forward the messages from fronted. We didn’t want to split the application logic across different modules, simply because they support different communication protocols, instead we have connected the web socket module with the core services through messaging system.

Spring Integration does integrate on top of Spring WebSocket support, although there are some rough edges. I wasn’t able to accomplish my goal as easly as advertised in reference: http://docs.spring.io/spring-integration/docs/4.2.1.RELEASE/reference/html/web-sockets.html and I’ve ended with construct quite similar to the below code:

@Component
@MessagingGateway
public class NotificationEndpoint {

    @Resource(name = "inboundNotification")
    private DirectChannel notificationChannel;

    @MessageMapping("/notification")
    public void notification(String payload, Principal principal) {
        
        notificationChannel.send(new GenericMessage<>(payload, createHeaders(principal)));
    }
}

AMQP

Eventually we were using RabbitMQ in two bit different roles, first it was used as STOMP messaging relay. Allowing to scale out the WebSocket nodes. The second usage was to pass around the messages through AMQP, so that notification from backend could be forwarded through Sock.js and STOMP to UI.

Spring OAuth

We had been using OAuth2 for authentication and for that we needed to propagate the user authorization token through Sock.js to be able to authenticate. Due to the issue: https://github.com/spring-projects/spring-security-oauth/issues/478 you will have to use version 2.0.8 or otherwise you will be receiving 401 – Unauthorized status.

Zuul

The Zuul setup initially was pretty standard, required only to configure proper route to discovered service.

zuul:
  ignoredServices: '*'
  routes:
    resource:
      path: /ws/**
      serviceId: ws-frontend-service

Unfortunately the first tests that we performed, revealed that the client can not maintain the connection, which were continuously closed. Fortunately Sock.js was design in a way that it requires the server to send hearth beats. The solution was to properly set the Ribbon/Hystrix timeouts as described here. Eventually we had set the timeouts to the double of Sock.js heartbeat delays:

hystrix.command.default.execution.isolation.thread.timeoutInMilliseconds: 50000
ribbon:
  ConnectTimeout: 3000
  ReadTimeout: 50000

Summary

Overall I could be satisfied with such solution, though it’s not perfect and long polling will be resource consuming (keeping open the connection on the proxy and occupying the application server thread – the web socket connections generally are handled by separate server thread pools). The nice side effect of above stack is that it should allow transparently move to different transport protocol, Sock.js and STOMP nicely hide all the communication details. But if you need real web socket support you will have to resign from Zuul. Hopefully with release of Zuul 2 this matter will change.

One comment

  1. bilak · September 14

    Hi Jakub, how did you solved the connection header which is always overwritten by zuul and set to Keep-Alive instead of Upgrade? Thanks

    Like

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s