Oct 30, 20226 min read

Implementing resilient applications with API Gateway (Circuit breaker)

Everything fails, accept it!🤦

In a distributed system, a client service usually calls other services to retrieve data and there is the possibility that one of those services can fail due to some unexpected scenarios if there is a tough issue with a particular microservice, then it will be unavailable for a long time that’s unlikely to succeed and it is not recoverable after it tries again and again with some delay (that’s how traditional Retry strategy works) because the caller service has no idea about its health and waiting for the other service to respond continuously. As a result, the network resources will be spent by delivering low performance and a bad user experience.

A circuit breaker is a rescuer🦸

In such a situation, the service should be coded to accept that the operation has failed and handle the failure accordingly. The Circuit Breaker pattern comes as a savior that can prevent a service from repeatedly trying to execute an operation that’s likely to fail.GIF

“Your code can not just wait forever for a response that might never come, sooner or later, it needs to give up. Hope is not a design method.” -Michael T. Nygard, from the book Release It!

One solution can be implementing the circuit breaker pattern for each microservice by using proven .NET libraries like Polly which is a very time-consuming and sometimes even challenging task.

Another recommended approach is to take advantage of an API Gateway pattern that can be considered as a single entry point means all the external requests will go through it only as a Circuit breaker mechanism that should be also implemented as a proxy for operations that might fail. The proxy should monitor the number of recent failures that have occurred, and use this information to decide whether to allow the operation to proceed, or simply return an exception immediately.

Circuit breaker in an API Gateway

By now, the circuit breaker pattern is pretty well-known, it has many benefits in a Microservices architecture and it can improve the stability and resiliency of an application. You can learn this design pattern from many other resources like this one.

In this post, let’s examine primarily how to apply the circuit breaker for ASP.NET Core WEB APIs with the help of Apache APISIX API Gateway.

Apache APISIX API Breaker Plugin

Apache APISIX offers 💁 a special api-breaker Plugin 🔌 that implements and uses circuit breaker functionality to invoke, handle failures and prevent upstream services from constant retry attempts if the service fails or performs slowly. It determines the health of targets based on their behavior in responding to requests. From the usage point of view, it is very easy to enable the plugin (there is no need to add anything in code) and you can make use of it with the combination of other built-in plugins as well.

Circuit breaker states

As you may already know, the circuit breaker works with three different internal states. Let’s go through each state and understand how the APISIX API Breaker plugin addresses them accordingly.

Closed State

In the initial state, we can assume that the system is healthy and sending connections to the upstream service (backend). The api-breaker plugin routes the requests to the backend and counts the number of failures in each period as an interceptor. You can also define it with unhealthy.failures the plugin’s attribute (it counts the number of consecutive failures for the upstream service to be considered unhealthy).

☝️ Note that it is also possible to configure HTTP status codes with unhealthy.http_statuses properties for the upstream to be considered unhealthy. See all available plugin’s properties.

If the number of recent failures exceeds a specified threshold within a given time period (max_breaker_sec attribute to set the value in seconds), the proxy state is placed into the Open state by api-breaker plugin.

Open State

When api-breaker plugin moves to the “Open” state, requests from the upstream service will fail immediately, and an exception will be returned directly from the API gateway as its shown in the below diagram:

Half-Open State

In this state, api-breaker the plugin only allows a fixed number of requests to pass through the upstream service. If these requests are successful (with specified HTTP status codes in the configuration field healthy.http_statuses for the number of successes defined in the healthy.successes field), it’s assumed that the fault that was previously causing the failure has been fixed and the circuit breaker switches to the healthy state (Closed state) and the number of failures counter is reset.

API Breaker Plugin Demo

🙋🏼 With all the theoretical knowledge in mind, let’s jump into the demo of using api-breaker plugin offered by Apache APISIX and see the result in action for our existing sample project ASP.NET Core WEB API with a single endpoint (retrieves all products list).

Prerequisites

☝️ If you followed the previous blog post about Manage .NET Microservices APIs with Apache APISIX API Gateway, make sure you have read it and completed the steps (To run APISIX, etcd, and ASP.NET WEB API) before continuing with a demo session.

Demo scenario

Take a very simple scenario of an online marketplace website where there is the main webpage we offer all/some products available in the shop with their corresponding name, price, and all other meta info (clients can search for products or sort them). While we are exchanging information between UI and the backend API Server there can be a case the upstream service is failing and we obviously should show an error message indicating that the service is temporarily unavailable to the user. You can handle it directly on UI code but it is not an ideal solution. Especially, it can be tough in case UI collects and aggregates data from different microservices. At this moment, API Gateway comes into the picture.

Create an Upstream

The first step we need to create a new upstream for our backend product service. The following command creates a sample upstream (that’s our API Server):

curl "http://127.0.0.1:9080/apisix/admin/upstreams/1" -H "X-API-KEY: edd1c9f034335f136f87ad84b625c8f1" -X PUT -d '
{
 "type": "roundrobin",
 "nodes": {
 "productapi:80": 1
 }
}

Configure the API Breaker Plugin

Now let’s start with adding api-breaker plugin to Apache APISIX declarative configuration file config.yaml under /apisix_conf folder in the project or you can simple check-out the api-circuit-breaker branch in the repo (it is already configured and you can find example commands we show in this demo under /cmds folder) and run the solution from that branch. Because in the current project, we have not registered yet the plugin we are going to use for this demo. We appended api-breaker plugin’s name to the end of plugins list:

plugins:
 - http-logger
 - ip-restriction
 …
 - api-breaker

Next, we can directly request Admin API to reload the latest plugin code without restarting Apache APISIX. See the command to reload the newly added plugin:

curl http://127.0.0.1:9080/apisix/admin/plugins/reload -H 'X-API-KEY: edd1c9f034335f136f87ad84b625c8f1' -X PUT

Enable the API Breaker Plugin

Next, we configure the api-breaker plugin on a specific Route. The following configuration is an example of how to add circuit breaker capabilities to the upstream we created in the previous step:

curl "http://127.0.0.1:9080/apisix/admin/routes/1" -H 'X-API-KEY: edd1c9f034335f136f87ad84b625c8f1' -X PUT -d '
{
    "name": "Route for circuit breaker",
    "methods": ["GET"],
    "uri": "/api/products",
    "upstream_id": "1",
    "plugins": {
        "api-breaker": {
            "break_response_code": 502,
                  "max_breaker_sec": 30,
            "unhealthy": {
                "http_statuses": [500, 503],
                "failures": 3
            },
            "healthy": {
                "http_statuses": [200],
                "successes": 1
            }
        }
    }
}'

In the plugin configuration example above ⤴️, the circuit breaker policy is configured so it breaks or opens the circuit when there have been three consecutive faults (500, 503 HTTP statuses return) when retrying the HTTP requests. When that happens, the upstream service moves to an unhealthy state and the circuit will break for 30 seconds. In that period, calls will be failed immediately api-breaker rather than actually be placed. Any single HTTP response code of 200 will restore its healthy status again.

How to validate API Breaker🙎

It is very straightforward to validate this scenario. One stupid option can be just stopping productapi container from docker 🤪 which causes the 503 Service Temporarily Unavailable error and we can see api-breaker takes the control😏 or another option update the code in the ProductController.cs file so that the endpoint returns 500 Internal Server error.

For example:

        [HttpGet]
        public IActionResult GetAll()
        {
            throw new Exception();

            return Ok(_productsService.GetAll());
        }

We request 3 times the API gateway and starting from the 4th call, you can expect to get 502 Bad Gateway error immediately from the gateway as we set break_response_code to 502 in the api-breaker plugin configuration.

Example curl command:

curl http://127.0.0.1:9080/api/products -i

Related resources

➔ Manage .NET Microservices APIs with Apache APISIX API Gateway.

➔ API Gateway Caching for ASP.NET Core WEB API