Spring Cloud: Fixing Eureka application status

Spring Cloud integrates seamlessly with Netflix Eureka, you can use it for discovering your services. It also has integration with Ribbon – client load balancer, which will be used whenever you will use RestTemplate or Fegin clients to make request to one of your registered services.

Although just recently I’ve discovered, that in case of Spring Cloud (version 1.0.3), applications registered in Eureka does not entirely reflect the actual application state. I’m referring here to Spring Boot Actuator’s health checks, those by default are not being propagated to Eureka. Eureka always happily announces that all applications are in UP state. As a result you may experience sitauation that despite the fact that your application is being in fact unhealthy, for instance due to backend service connectivity problem, other applications will be still sending traffic towards it. Generally this isn’t desired situation, so I would prefer that Eureka would have the accurate and up to date application status.

How to fix this?

Fortunetly there is a simple solution for that. Eureka expects the it’s client will configure HealthCheckHandler, that will be used with each Eureka heart beat to determine the current status. So let’s define our custom implementation that will do the job. Additionally it turns out that the default statuses defined by Actuator and Eureka matches each other perfectly, so everything that needs to be done is to determine current aggregated application status, map it to Eureka’s and return it.

Let’s jump directly to the implementation.

public class EurekaHealthCheckHandler implements HealthCheckHandler, ApplicationContextAware, InitializingBean {

    private static final Map<Status, InstanceInfo.InstanceStatus> healthStatuses = new HashMap<Status, InstanceInfo.InstanceStatus>() {{
        put(Status.UNKNOWN, InstanceInfo.InstanceStatus.UNKNOWN);
        put(Status.OUT_OF_SERVICE, InstanceInfo.InstanceStatus.OUT_OF_SERVICE);
        put(Status.DOWN, InstanceInfo.InstanceStatus.DOWN);
        put(Status.UP, InstanceInfo.InstanceStatus.UP);
    }};

    private final CompositeHealthIndicator healthIndicator;

    private ApplicationContext applicationContext;

    public EurekaHealthCheckHandler(HealthAggregator healthAggregator) {
        Assert.notNull(healthAggregator, "HealthAggregator must not be null");

        this.healthIndicator = new CompositeHealthIndicator(healthAggregator);
    }
    
    @Override
    public void setApplicationContext(ApplicationContext applicationContext) throws BeansException {
        this.applicationContext = applicationContext;
    }

    @Override
    public void afterPropertiesSet() throws Exception {

        final Map<String, HealthIndicator> healthIndicators = applicationContext.getBeansOfType(HealthIndicator.class);
        for (Map.Entry<String, HealthIndicator> entry : healthIndicators.entrySet()) {
            healthIndicator.addHealthIndicator(entry.getKey(), entry.getValue());
        }
    }

    @Override
    public InstanceInfo.InstanceStatus getStatus(InstanceInfo.InstanceStatus instanceStatus) {

        return getHealthStatus();
    }

    protected InstanceInfo.InstanceStatus getHealthStatus() {
        final Status status = healthIndicator.health().getStatus();
        return mapToInstanceStatus(status);
    }

    protected InstanceInfo.InstanceStatus mapToInstanceStatus(Status status) {
        if(!healthStatuses.containsKey(status)) {
            return InstanceInfo.InstanceStatus.UNKNOWN;
        }
        return healthStatuses.get(status);
    }
}

As you can see we have implemented our own HealthCheckHandler. Going from top you will notice our mapping between coresponding statuses. The next important part is CompositeHealthIndicator instance toward we delegated establishing the status of entire application based on any individual health check that has been registered within the application context. The remaining part becomes trivial.

We only need to register our health check handler.

@Configuration
public class EurekaHealthCheckHandlerConfiguration {

    @Autowired(required = false)
    private HealthAggregator healthAggregator = new OrderedHealthAggregator();

    @Bean
    @ConditionalOnMissingBean
    public EurekaHealthCheckHandler eurekaHealthCheckHandler() {
        return new EurekaHealthCheckHandler(healthAggregator);
    }
}

This will end the good old times where every application was in green (UP) state and force us to deal with real life problems 😉

 

Update: Sneak peaking Spring Cloud Netflix 1.1

The behavior described above is going to be an option for Eureka clients starting from Spring Cloud 1.1.

In order to turn it on you will need to set ‘eureka.client.healthcheck.enabled’:

eureka.client.healthcheck.enabled: true