TechOnDiapers
Implementing Resilience Patterns in ASP.NET Core Web APIs: Circuit Breaker, Retry, and Timeout Policies

Implementing Resilience Patterns in ASP.NET Core Web APIs: Circuit Breaker, Retry, and Timeout Policies

Learn how to use Circuit Breaker, Retry, and Timeout policies in ASP.NET Core Web APIs

Opaluwa Emidowo-ojo's photo
Opaluwa Emidowo-ojo
·

15 min read

Welcome back! We have a brand-new technical tutorial today. Still discussing Web APIs, we are going to learn how to implement resilience in our APIs. Web APIs often interact with other services or databases, and sometimes, things can go wrong—like a server going down or a network connection being slow.

If your API isn’t resilient, these issues can cause errors or downtime, frustrating users. By making your API resilient, you help ensure that it remains reliable and available, even when things don’t go as planned.

Pre-requisites

To fully benefit from this article, readers should have the following prerequisites:

  • Basic Understanding of C# and ASP.NET Core

    • Familiarity with C# programming language.

    • Understanding of how to create and run a simple ASP.NET Core Web API.

  • Development Environment Setup

    • Visual Studio or Visual Studio Code installed on your machine.

    • .NET Core SDK installed.

  • Familiarity with Dependency Injection

    • Basic knowledge of dependency injection (DI) and how it’s used in ASP.NET Core.

  • NuGet Package Management

    • Understanding how to install and manage NuGet packages in an ASP.NET Core project.

  • Basic Knowledge of HTTP and REST APIs

    • Understanding of how HTTP requests and responses work.

    • Familiarity with RESTful principles and API endpoints.

  • Exposure to Error Handling in ASP.NET Core

    • Basic understanding of error handling and exception management in ASP.NET Core applications.

  • Understanding of Application Logging

    • Familiarity with logging practices in ASP.NET Core for monitoring and troubleshooting.

Table of Contents

  • Introduction to Resilience in Web APIs

  • Getting Started with ASP.NET Core and Polly

  • Understanding Circuit Breaker Pattern

  • Implementing Retry Policies

  • Setting Up Timeout Policies

  • Combining Resilience Policies for Robust APIs

  • Testing and Monitoring Resilience

  • Common Pitfalls and Troubleshooting

  • Conclusion

Introduction to Resilience in Web APIs

What is Resilience in Software Applications?
Resilience in software means building your application to handle unexpected problems, like temporary server issues or network glitches, without crashing or affecting the user experience. It's about ensuring your app can "bounce back" from failures and continue working smoothly.

Why Resilience Matters in Web APIs
Web APIs often interact with other services or databases, and sometimes, things can go wrong—like a server going down or a network connection being slow. If your API isn’t resilient, these issues can cause errors or downtime, frustrating users. By making your API resilient, you help ensure that it remains reliable and available, even when things don’t go as planned.

Overview of Common Resilience Patterns
There are several strategies, or "patterns," that developers use to make their applications more resilient:

  • Circuit Breaker: Temporarily stops requests to a service that’s failing and tries again after a while.

  • Retry: Automatically tries a request again if it fails the first time.

  • Timeout: Limits how long your API waits for a response from another service, so it doesn’t get stuck.

These patterns help your API handle problems gracefully and keep running smoothly for users.

Getting Started with ASP.NET Core and Polly

  • Setting Up an ASP.NET Core Project
    First, create a new ASP.NET Core Web API project. You can use Visual Studio or the .NET CLI. Here’s how you can create a new project using the .NET CLI:
dotnet new webapi -n MyResilientApi
cd MyResilientApi

Once your project is created, run it to ensure everything is set up correctly:

dotnet run

  • Installing and Configuring Polly

    Next, you need to install Polly via NuGet. You can do this using the .NET CLI or the NuGet Package Manager in Visual Studio.

    Using the .NET CLI:

dotnet add package Polly
dotnet add package Microsoft.Extensions.Http.Polly

After installing Polly, you can configure it in your Startup.cs or Program.cs (depending on your project template). Here's how to add Polly policies to an HTTP client:

using Microsoft.Extensions.DependencyInjection;
using Polly;
using Polly.Extensions.Http;

public class Startup
{
    public void ConfigureServices(IServiceCollection services)
    {
        services.AddHttpClient("MyHttpClient")
            .AddPolicyHandler(GetRetryPolicy())
            .AddPolicyHandler(GetCircuitBreakerPolicy());

        services.AddControllers();
    }

    private IAsyncPolicy<HttpResponseMessage> GetRetryPolicy()
    {
        return HttpPolicyExtensions
            .HandleTransientHttpError()
            .OrResult(msg => msg.StatusCode == System.Net.HttpStatusCode.NotFound)
            .WaitAndRetryAsync(3, retryAttempt => TimeSpan.FromSeconds(Math.Pow(2, retryAttempt)));
    }

    private IAsyncPolicy<HttpResponseMessage> GetCircuitBreakerPolicy()
    {
        return HttpPolicyExtensions
            .HandleTransientHttpError()
            .CircuitBreakerAsync(2, TimeSpan.FromMinutes(1));
    }

    public void Configure(IApplicationBuilder app, IWebHostEnvironment env)
    {
        if (env.IsDevelopment())
        {
            app.UseDeveloperExceptionPage();
        }

        app.UseRouting();
        app.UseEndpoints(endpoints =>
        {
            endpoints.MapControllers();
        });
    }
}

  • Introduction to Polly’s Resilience Policies

    Here are some common Polly policies that you can use in your ASP.NET Core project:

Retry Policy

Retries a failed request a specified number of times with an increasing delay between each retry:

private IAsyncPolicy<HttpResponseMessage> GetRetryPolicy()
{
    return HttpPolicyExtensions
        .HandleTransientHttpError()
        .OrResult(msg => msg.StatusCode == System.Net.HttpStatusCode.NotFound)
        .WaitAndRetryAsync(3, retryAttempt => TimeSpan.FromSeconds(Math.Pow(2, retryAttempt)));
}

Circuit Breaker Policy

Breaks the circuit (stops making requests) after a specified number of failures, and then resets after a certain period:

private IAsyncPolicy<HttpResponseMessage> GetCircuitBreakerPolicy()
{
    return HttpPolicyExtensions
        .HandleTransientHttpError()
        .CircuitBreakerAsync(2, TimeSpan.FromMinutes(1));
}

Timeout Policy

Limits how long a request can take before it times out:

private IAsyncPolicy<HttpResponseMessage> GetTimeoutPolicy()
{
    return Policy.TimeoutAsync<HttpResponseMessage>(10); // 10-second timeout
}

  • Using Polly Policies in an HTTP Client

    Once you’ve defined your Polly policies, you can apply them to an HTTP client as shown in the Startup.cs example above. Here’s how you would make a request using this configured client:

public class MyService
{
    private readonly IHttpClientFactory _httpClientFactory;

    public MyService(IHttpClientFactory httpClientFactory)
    {
        _httpClientFactory = httpClientFactory;
    }

    public async Task<string> GetDataAsync()
    {
        var client = _httpClientFactory.CreateClient("MyHttpClient");
        var response = await client.GetAsync("https://api.example.com/data");
        response.EnsureSuccessStatusCode();

        return await response.Content.ReadAsStringAsync();
    }
}

Understanding Circuit Breaker Pattern

What is the Circuit Breaker Pattern?
The Circuit Breaker Pattern is a way to prevent a service from repeatedly trying to perform an action that is likely to fail. Think of it like a fuse in your home—if something goes wrong, the circuit "breaks" to stop further damage. In software, this pattern temporarily stops requests to a service when it detects multiple failures, giving it time to recover.

When and Why to Use a Circuit Breaker
You use a Circuit Breaker when you have a service that might fail due to things like network issues, server overload, or other transient errors. The pattern is helpful because it prevents your application from wasting resources on repeated failed attempts. Instead, it waits for the service to stabilize before trying again, which helps keep your application running smoothly.

Implementing a Circuit Breaker in ASP.NET Core
In ASP.NET Core, you can implement the Circuit Breaker pattern using a library like Polly. Polly makes it easy to define rules for when to "trip" the circuit breaker (like after a certain number of failures) and how long to wait before trying again. This helps your application handle errors gracefully and ensures it remains responsive, even when some services are temporarily unavailable.

Implementing Retry Policies

What is a Retry Policy?

A retry policy is a strategy to automatically try an operation again when it fails. Instead of giving up after the first failure, a retry policy makes your application more resilient by allowing it to attempt the operation multiple times, with the hope that the issue (like a temporary network glitch) resolves itself.

Scenarios Where Retry Makes Sense

Retry policies are useful in situations where failures are likely temporary and can be resolved with a second or third attempt. For example, if your app is calling a web service and the service is temporarily unavailable, retrying after a brief pause might result in a successful call. It’s also useful for handling intermittent network issues or transient faults, which are problems that are not permanent.

How to Add a Retry Policy Using Polly

Polly is a popular .NET library that makes it easy to add retry policies to your code. With Polly, you can define a retry policy that automatically retries a failed operation a specific number of times, with an optional delay between each attempt. Here's a basic example of how you might use Polly to retry an API call:

var retryPolicy = Policy
    .Handle<HttpRequestException>() // Handle specific exceptions
    .WaitAndRetry(3, retryAttempt => TimeSpan.FromSeconds(2)); // Retry 3 times with a 2-second delay

retryPolicy.Execute(() =>
{
    // Your operation here, e.g., an API call
    var response = httpClient.GetAsync("https://example.com").Result;
    response.EnsureSuccessStatusCode();
});

In this example, if the HttpRequestException is thrown, Polly will wait for 2 seconds and then retry the operation, up to three times. This gives your application a better chance to succeed if the failure was just a temporary issue.

Setting Up Timeout Policies

What is a Timeout Policy?

A timeout policy is a rule you set up to manage how long your application will wait for a response from a service or operation before giving up. Think of it like setting a timer—if the service doesn’t respond within the allowed time, your application stops waiting and moves on. This helps prevent your application from getting stuck or slowing down when something takes too long.

The Importance of Handling Timeouts

Handling timeouts is crucial for keeping your application running smoothly and efficiently. Without a timeout policy, your app might hang or become unresponsive if a service takes too long to reply. By setting timeouts, you can:

  • Improve User Experience: Users won’t have to wait indefinitely if something goes wrong.

  • Prevent Resource Blockage: Your application won’t waste resources waiting for a response.

  • Manage Failures Gracefully: You can handle situations where a service is slow or down more effectively.

Implementing Timeout Policies in Your Web API

Here’s a simple way to add timeout policies to your ASP.NET Core Web API using the Polly library:

  • Install Polly: First, you need to add Polly to your project. You can do this via NuGet Package Manager with the following command:

      dotnet add package Polly

  • Configure Timeout Policy: In your Startup.cs file or wherever you configure services, add Polly’s timeout policy to your HttpClient. Here’s an example:

      public void ConfigureServices(IServiceCollection services)
  {
      services.AddHttpClient("MyApiClient")
          .AddPolicyHandler(Policy.TimeoutAsync<HttpResponseMessage>(TimeSpan.FromSeconds(10)));
  }

    In this example, TimeSpan.FromSeconds(10) means the application will wait up to 10 seconds for a response. If it takes longer, Polly will stop waiting and move on.

  • Use the Configured HttpClient: When you make HTTP requests using the named client, the timeout policy will automatically be applied:

      public class MyService
  {
      private readonly HttpClient _httpClient;

      public MyService(IHttpClientFactory httpClientFactory)
      {
          _httpClient = httpClientFactory.CreateClient("MyApiClient");
      }

      public async Task<string> GetDataAsync()
      {
          var response = await _httpClient.GetAsync("https://example.com/api/data");
          response.EnsureSuccessStatusCode();
          return await response.Content.ReadAsStringAsync();
      }
  }

By setting up a timeout policy, you help ensure your application remains responsive and handles delays gracefully, improving overall performance and user satisfaction.

Combining Resilience Policies for Robust APIs

When building reliable APIs, it's important to handle failures gracefully. By combining different resilience policies—like circuit breakers, retries, and timeouts—you can make your API more robust. Here’s how to do it:

How to Combine Circuit Breaker, Retry, and Timeout Policies

  • Understanding the Policies:

    • Circuit Breaker: Stops requests to a failing service to avoid overloading it.

    • Retry: Tries a request again if it fails.

    • Timeout: Limits how long a request can take before giving up.

  • Combining Policies with Polly:

    • Step 1: Install Polly: Add the Polly NuGet package to your project to use these policies.

    • Step 2: Define Policies: Create each policy separately. For example:

        var timeoutPolicy = Policy.TimeoutAsync<HttpResponseMessage>(10); // 10 seconds timeout
  var retryPolicy = Policy.Handle<HttpRequestException>().RetryAsync(3); // Retry 3 times
  var circuitBreakerPolicy = Policy.Handle<HttpRequestException>().CircuitBreakerAsync(5, TimeSpan.FromMinutes(1)); // Break after 5 failures

    • Step 3: Combine Policies: Use Polly’s WrapAsync method to combine them. Policies can be combined in a sequence that matches your needs:

        var combinedPolicy = Policy.WrapAsync(timeoutPolicy, retryPolicy, circuitBreakerPolicy);

  • Apply the Combined Policy: Use the combined policy when making HTTP requests:

      var httpClient = new HttpClient();
  var response = await combinedPolicy.ExecuteAsync(() => httpClient.GetAsync("https://example.com"));

Best Practices for Chaining Policies in Polly

  • Order Matters: The order of policies in the WrapAsync method is important. For instance, timeout policies should generally be applied first, followed by retry and then circuit breaker policies.

  • Keep It Simple: Avoid overcomplicating the policy chain. Only include policies that address specific needs to maintain readability and manageability.

  • Monitor and Adjust: Regularly monitor how your policies are performing and adjust them based on real-world usage and failure patterns.

Real-World Examples of Combined Resilience Strategies

  • API Call with Timeout and Retry: For an API that sometimes has slow responses but should keep retrying, you might use a timeout policy to limit how long you wait and a retry policy to try again if the call fails:

      var timeoutPolicy = Policy.TimeoutAsync<HttpResponseMessage>(5); // 5 seconds timeout
  var retryPolicy = Policy.Handle<HttpRequestException>().RetryAsync(2); // Retry 2 times
  var combinedPolicy = Policy.WrapAsync(timeoutPolicy, retryPolicy);
  var response = await combinedPolicy.ExecuteAsync(() => httpClient.GetAsync("https://example.com"));

  • Service with Circuit Breaker and Timeout: For a service that can fail frequently and needs to be protected from overloading, use a circuit breaker to stop requests after several failures and a timeout to ensure requests don’t hang indefinitely:

      var timeoutPolicy = Policy.TimeoutAsync<HttpResponseMessage>(10); // 10 seconds timeout
  var circuitBreakerPolicy = Policy.Handle<HttpRequestException>().CircuitBreakerAsync(3, TimeSpan.FromMinutes(1)); // Break after 3 failures
  var combinedPolicy = Policy.WrapAsync(timeoutPolicy, circuitBreakerPolicy);
  var response = await combinedPolicy.ExecuteAsync(() => httpClient.GetAsync("https://example.com"));

By combining these resilience policies effectively, you can build APIs that handle failures gracefully and provide a better experience for your users.

Testing and Monitoring Resilience

How to Test Resilience Policies

  • Simulate Failures

    • Use Tools: Tools like Polly's fault injection can help you simulate failures, such as network outages or timeouts, to see how your API handles them.

    • Manual Testing: You can manually test by disrupting network connections or causing delays to check if your API correctly applies resilience patterns.

  • Write Unit Tests

    • Create Test Cases: Write unit tests to check if resilience policies (like circuit breakers or retries) are triggered correctly.

    • Use Mocking: Tools like Moq can be used to simulate transient faults and verify that your policies respond as expected.

  • Verify Policy Behavior

    • Check Logs: Review application logs to ensure that resilience policies are logging events as they should.

    • Test Different Scenarios: Test how your API behaves under various failure conditions to ensure that policies like retries or circuit breakers work correctly.

Monitoring API Performance and Resilience

  • Monitor Logs

    • Check Log Files: Regularly review logs to see if resilience policies are being triggered and if there are any unexpected errors.

    • Use Log Analysis Tools: Tools like ELK Stack or Azure Monitor can help you analyze and visualize log data.

  • Track Metrics

    • Measure Response Times: Monitor response times to see if your policies are improving API performance.

    • Track Failure Rates: Keep an eye on the rate of failed requests to ensure your resilience policies are handling faults effectively.

  • Set Up Alerts

    • Configure Alerts: Set up alerts for critical issues, such as high error rates or circuit breaker trips, so you can quickly address problems.

    • Use Monitoring Tools: Tools like Prometheus or Application Insights can help you set up and manage alerts.

Tools and Techniques for Continuous Improvement

  • Use Performance Profilers

    • Analyze Performance: Tools like Visual Studio Profiler or dotTrace can help you find bottlenecks and optimize performance.

  • Apply Best Practices

    • Review and Refine Policies: Regularly review your resilience policies and adjust them based on performance data and new insights.

    • Stay Updated: Keep up with updates and best practices for resilience patterns and tools to ensure you are using the latest techniques.

  • Conduct Regular Reviews

    • Review Policies: Periodically assess the effectiveness of your resilience policies and make improvements based on performance and monitoring data.

    • Learn from Issues: Analyze any issues or failures to understand their causes and enhance your resilience strategies.

By following these steps, you can ensure that your API is resilient, performs well, and continues to improve over time.

Common Pitfalls and Troubleshooting

Common Mistakes When Implementing Resilience Patterns

  • Not Configuring Policies Properly:

    • Pitfall: Using default settings without customizing them for your needs.

    • Solution: Review and adjust settings like retry counts and timeouts based on your application's requirements.

  • Ignoring Logging and Monitoring:

    • Pitfall: Failing to log or monitor resilience patterns can make it hard to spot issues.

    • Solution: Implement logging to track when policies are triggered and monitor their performance.

  • Overusing Resilience Policies:

    • Pitfall: Applying too many policies can complicate your code and reduce performance.

    • Solution: Use only the necessary resilience patterns for your specific use cases.

  • Not Handling Policy Failures:

    • Pitfall: Assuming policies will always succeed without considering how to handle their failures.

    • Solution: Implement fallback strategies to manage scenarios where policies fail.

How to Troubleshoot Issues with Polly

  • Verify Policy Configuration:

    • Issue: Policies not working as expected.

    • Troubleshooting: Check that policies are configured correctly in your code and review the settings.

  • Check Logs for Errors:

    • Issue: Unexpected behavior or failures.

    • Troubleshooting: Look at your logs to identify if and when resilience policies are being triggered and what errors are occurring.

  • Review Policy Execution:

    • Issue: Policies not applying during API calls.

    • Troubleshooting: Ensure that policies are correctly injected into your services and used in the right places.

  • Test Policies Independently:

    • Issue: Uncertain if policies are working.

    • Troubleshooting: Create unit tests to validate the behavior of your resilience policies in isolation.

Tips for Fine-Tuning Your Resilience Strategy

  • Adjust Policy Settings Based on Load:

    • Tip: Tune retry counts and timeouts according to your application's traffic and response times.

  • Monitor and Review Performance:

    • Tip: Regularly review the performance impact of your policies and adjust as needed to balance resilience and efficiency.

  • Simplify Policy Configurations:

    • Tip: Keep your resilience patterns straightforward and only add complexity when necessary.

  • Document Your Resilience Strategies:

    • Tip: Maintain clear documentation on how and why you use specific resilience patterns to help with future troubleshooting and maintenance.

By being aware of these common pitfalls, knowing how to troubleshoot issues effectively, and applying these tips for fine-tuning, you can better implement and manage resilience patterns in your ASP.NET Core Web APIs.

Conclusion

Recap of Key Concepts

In this guide, we learned how to make your ASP.NET Core Web APIs more reliable using resilience patterns. Here’s a quick summary of what we covered:

  • Resilience Patterns: These are techniques that help your API handle errors and failures gracefully.

  • Circuit Breaker: This pattern prevents your API from repeatedly trying to access a failing service, giving it time to recover.

  • Retry Policy: This pattern automatically retries a failed request a few times before giving up, which helps handle temporary issues.

  • Timeout Policy: This pattern ensures that requests don’t hang forever by setting a maximum time limit for responses.

Next Steps for Learning and Improving API Resilience

Now that you have a basic understanding, here’s how you can continue improving your API’s resilience:

  • Practice Implementing Patterns: Try adding these resilience patterns to a sample project and see how they work in practice.

  • Monitor and Test: Use logging and monitoring tools to track how your API handles failures and test the resilience patterns to ensure they work as expected.

  • Learn More: Explore advanced topics and other resilience patterns to build even more robust APIs.

Additional Resources for Further Learning

To deepen your understanding of API resilience and related topics, check out these resources:

  • Polly Documentation: Learn more about Polly, the library used for implementing resilience patterns.

  • ASP.NET Core Resilience: Explore official ASP.NET Core documentation and tutorials on making APIs resilient.

  • Online Courses and Tutorials: Look for beginner-friendly courses or tutorials on resilience patterns and best practices for building reliable APIs.

Feel free to reach out if you have questions or need further guidance as you continue your journey in improving API resilience.
I hope you found this guide helpful and learned something new. Stay tuned for the next article in the Mastering C# series: Implementing Microservices Architecture with ASP.NET Core Web APIs

Happy coding!

Timeout policiesWeb APItechnologyResiliencecircuit breakerBeginner Developersbackendsoftware developmentSoftware EngineeringTechnical writing