Introduction

In a monolithic application, transactions are straightforward: if something fails, you roll back the entire operation. But in a microservices architecture, where each service owns its database (and may even use different technologies), this approach breaks down. Services might be written in different languages, use NoSQL databases, or avoid databases entirely.

In such an environment, we cannot guarantee that an operation will succeed immediately or at all. This is where the Saga pattern comes in. It provides a way to coordinate transactions across multiple services, ensuring consistency even when direct rollbacks aren’t possible.

What Is the Saga Pattern?

The Problem

Let’s illustrate the problem with an online shop:

1. A customer adds items to their cart and creates an order.

2. The Order Service saves the order and triggers a payment.

3. The Accounting Service processes the payment.

4. The Warehouse Service reserves the items.

5. The Delivery Service schedules shipping.

It can be represented in the following flow:

But what if the warehouse runs out of stock after payment?
Now, the system is in an inconsistent state:

• The customer’s money is taken (Accounting).

• The order exists (Order Service).

• The items can’t be shipped (Warehouse).

• The delivery is created (Delivery).

This mismatch between the actual and desired state is the problem the Saga pattern solves.

Compensating Transactions

The Saga pattern addresses this by defining compensating transactions. These are not true transactions but new operations that reverse the effect of previous steps, and they may not perfectly undo the original action.

Additionally compensating actions may introduce their own issues if they fail or are implemented incorrectly. A retry or escalating mechanism needs to be implemented.

Look at the actions from our example and their compensating transactions in the table below:

Service	Action	Compensating transaction
Order	CreateOrder	CancelOrder
Accounting	CompletePayment	RefundPayment
Warehouse	BookInventory	ReleaseInventory
Delivery	CreateDelivery	CancelDelivery

These compensating actions ensure the system can roll back to a consistent state, even if the original flow fails.

For a better understanding of how it works better to look at the graph representation of the flow with compensating transactions.

On the left we have the normal flow of events and on the right the compensating transactions for these events. It is worth to draw such diagrams when you work with sagas, because it can clearly show you all compensating transactions in the system and when they are used.

When we hear the word “transaction" we think about the transaction in the relational database which either executes completely or revert all changes. Compensating transactions rely on events and they completely lack of isolation. Thus they cannot be considered as ACID-compliant, but rather ACD.

There are two common ways to implement the Saga pattern: Choreography and Orchestration.

Choreography saga

In a Choreography-based Saga, services communicate asynchronously via events, acting as both producers and consumers. There is no central controller so that each service reacts to events and publishes its own.

How it works:

1. A user makes a request to create an order. The Order Service creates the order in its database and publishes the event OrderCreated.

2. The Message Broker sends the OrderCreated event to the Warehouse and Delivery services.

3. The Order Service makes a request to the Accounting Service to process the payment from the user. As soon as the payment is successfully processed the service stores the billing information regarding the order in its database and publishes the event OrderPaid.

4. The Warehouse Service receives the OrderPaid event. It tries to book inventory, but there are no inventory left and instead of publishing the InventoryBooked event it publishes the InventoryBookingFailed event.

5. All services receive the InventoryBookingFailed event and apply compensating actions:

   • Order: Cancels the order.

   • Accounting: Refunds money to the customer.

   • Delivery: Cancels the delivery.

Pros:

• Loose coupling: Services only need to know about the events they produce/consumed.

• Resilience: No single point of failure.

• Scalability: A decentralized nature avoids bottlenecks.

Cons:

• Complex debugging: Distributed logic makes it hard to trace issues.

• Eventual consistency: Services may temporarily disagree on the system state.

When to use:

• Simple workflows with a few services.

• Systems where scalability and loose coupling are priorities.

Implementation

In this implementation, events are the backbone of the saga. Let's define them:

public record OrderCreatedEvent(
    Guid OrderId,
    Guid CustomerId,
    List<OrderItem> Items,
    DateTime CreatedAt);

public record OrderPaidEvent(
    Guid OrderId,
    decimal Amount,
    DateTime PaidAt);

public record InventoryBookedEvent(
    Guid OrderId,
    List<InventoryItem> BookedItems);

public record InventoryBookingFailedEvent(
    Guid OrderId,
    string Reason);

The following events are used:

• OrderCreated begins the flow when the order is created by the OrderService. This event is used as an indicator for other services that a new order has been created and they should wait further instructions.

• OrderPaid is published be the AccountingService when the order has been successfully paid for by the customer.

• InventoryBooked is an indicator that the warehouse has the necessary products and has successfully booked them for the order.

• InventoryBookingFailed indicates that something has happened during the inventory booking process at the warehouse and the flow should be reverted. This event starts the compensating actions.

The code-snippet for OrderService is below:

public class OrderService
{
    private readonly IOrderRepository _orderRepository;
    private readonly IEventBus _eventBus;

    public async Task<Guid> CreateOrder(CreateOrderRequest request)
    {
        var order = new Order
        {
            Id = Guid.NewGuid(),
            CustomerId = request.CustomerId,
            Items = request.Items,
            Status = OrderStatus.Created,
            CreatedAt = DateTime.UtcNow
        };

        await _orderRepository.Save(order);
        await _eventBus.Publish(new OrderCreatedEvent(
            order.Id,
            order.CustomerId,
            order.Items,
            order.CreatedAt));

        return order.Id;
    }
}

CreateOrder is the first step of the saga so it does not publish any failure events. In this step other services know nothing about the order.

The WarehouseService handles the OrderPaid event and tries to book the inventory for the order:

public class WarehouseService
{
    private readonly IInventoryRepository _inventoryRepository;
    private readonly IEventBus _eventBus;

    public async Task Handle(OrderPaidEvent @event)
    {
        // Check if inventory is available
        var isAvailable = await _inventoryRepository.CheckAvailability(@event.Items);
        if (!isAvailable)
        {
            await _eventBus.Publish(new InventoryBookingFailedEvent(
                @event.OrderId,
                "Insufficient stock"));
            return;
        }

        // Book inventory
        await _inventoryRepository.BookItems(@event.OrderId, @event.Items);
        await _eventBus.Publish(new InventoryBookedEvent(
            @event.OrderId,
            @event.Items));
    }
}

When the product items are not available in the stock we publish the InventoryBookingFailed event, otherwise we book the items and publish InventoryBooked.

The OrderService should react to the InventoryBookingFailed event and cancel the order. To do this, it changes the status of the order from Created to Cancelled.

public async Task Handle(InventoryBookingFailedEvent @event)
{
    await _orderRepository.CancelOrder(@event.OrderId);
}

The Cancelled status makes the order is invalid. In this implementation it is important to define which status transitions are available. For example, Created → Cancelled is allowed, but Cancelled → Created is not allowed.

Orchestration saga

An Orchestration saga is suitable when you need a central orchestrator for your flow due to its complexity. In this case it is responsible for triggering the compensating transactions and retries.

How it works:

1. A user makes a request to create an order.

2. The Orchestrator sends the request to the Order Service and receives a successful response.

3. The Orchestrator sends the request for payment to the Account Service and receives a successful response.

4. The Orchestrator sends the request to the Warehouse Service for booking the inventory and a receives failure response.

5. The Orchestrator sends compensating transactions to the Account and Order services and if the delivery was created than to the Delivery service as well.

Pros:

• Ease of debugging. The flow is centralized in one place.

Cons:

• Additional coupling: Services are tightly coupled to the orchestrator which must know the contracts for each service to be able to communicate with them.

• Moderate resilience: The orchestrator is the single point of failure, but this can be mitigated with the following strategies:

  • State persistence. Store the Saga’s state to restore the orchestrator after a crash.

  • Requests Retry. If a call to the service fails than a retry mechanism is used.

  • Additional Orchestrator Instance. Deploy an additional instance of the orchestrator which can start processing the requests as soon as the main orchestrator fails.

While orchestration-based Saga introduces tight coupling to the orchestrator, scalability challenges, and a single point of failure, these issues are manageable with the right design. The trade-off is often worth it for complex workflows, where the orchestrator’s centralized control simplifies debugging and coordination.

Implementation

An Orchestrator can be implemented using various technologies and approaches: event-based, direct call or a mix of both. Let’s look at a direct call Saga orchestrator:

public class OrderSagaOrchestrator
{
    private readonly ISagaStateRepository _stateRepository;
    private readonly IOrderClient _orderClient;
    private readonly IPaymentClient _paymentClient;
    private readonly IInventoryClient _inventoryClient;
    private readonly IDeliveryClient _deliveryClient;

    public async Task Handle(CreateOrderRequest request)
    {
        // Step 1: Create order
        var order = await _orderClient.CreateOrder(request);
    
        var sagaState = new OrderSagaState { OrderId = order.OrderId, OrderCreated = true };
        await _stateRepository.Save(sagaState);

        try
        {
            // Step 2: Process payment
            await _paymentClient.ProcessPayment(order.OrderId, order.Amount);
            sagaState.PaymentProcessed = true;
            await _stateRepository.Update(sagaState);

            // Step 3: Book inventory
            await _inventoryClient.BookInventory(order.OrderId, order.Items);
            sagaState.InventoryBooked = true;
            await _stateRepository.Update(sagaState);
            
            // Step 4: Create delivery
            await _deliveryClient.CreateDelivery(order.OrderId, order.Address);
            sagaState.DeliveryScheduled = true;
            await _stateRepository.Update(sagaState);
        }
        catch (Exception ex)
        {
            sagaState.FailureReason = ex.Message;
            await _stateRepository.Update(sagaState);
            await Compensate(sagaState);
        }
    }

    private async Task Compensate(OrderSagaState sagaState)
    {
        await _orderClient.CancelOrder(sagaState.OrderId);
        
        if (sagaState.PaymentProcessed)
            await _paymentClient.RefundPayment(sagaState.OrderId);

        if (sagaState.InventoryBooked)
            await _inventoryClient.ReleaseInventory(sagaState.OrderId);
            
        if(sagaState.DeliveryScheduled)
            await _deliveryClient.CancelDelivery(sagaState.OrderId);
    }
}

public class OrderSagaState
{
    public Guid OrderId { get; set; }
    public bool OrderCreated { get; set; }
    public bool PaymentProcessed { get; set; }
    public bool InventoryBooked { get; set; }
    public bool DeliveryScheduled { get; set; }
    public string FailureReason { get; set; }
}

There are a few differences when comparing this with the Choreography saga:

• OrderSagaState contains the state of the saga, so it is possible to track the saga state using a database.

• The flow is in one place. If something has happened it is possible to check which state is next and which steps were processed.

Comparison Table: Choreography vs. Orchestration

Aspect	Choreography	Orchestration
Coupling	Loose (services only know about events).	Tight to the orchestrator, but not to each other.
Scalability	High (no central bottleneck).	Moderate (orchestrator can be a bottleneck, but is scalable with additional effort).
Resilience	High (no single point of failure).	Moderate (orchestrator is a single point of failure, but can be made resilient).
Debugging	Hard (distributed logic).	Easier (centralized logic).
Complexity	High (services must handle compensating logic).	Moderate (orchestrator manages flow).

Problems and Considerations

Idempotency

Regardless of the implementation the idempotency is a key element of the saga. If an event is redelivered due to network issue or other reasons the service must handle it without side effects.

Imagine a situation when the RefundPayment event arrives multiple times. The Accounting Service should process it only once, and all others attempts should be skipped.

public class AccountingService
{
    private readonly IPaymentService _paymentService;
    private readonly IPaymentRepository _paymentRepository;

    public async Task Handle(RefundPaymentEvent @event)
    {
        // Check if this refund was already processed
        var existingPayment = await _paymentRepository.GetByOrderId(@event.OrderId);
        if (existingPayment == null || existingPayment.State == PaymentState.Refunded)
            return; // Idempotency: Skip if already processed
            
        await _paymentService.RefundOrder(@event.OrderId);
        await _paymentRepository.MarkPaymentRefunded(@event.OrderId);
    }
}

Eventual Consistency

Eventual consistency means the system will eventually reach a consistent state, but not necessarily immediately. The Choreography Saga relies on events thus it cannot guarantee that all of them will be processed immediately due to its asynchronous nature. The orchestration saga has a similar issue and even if it uses direct communication with services via HTTP or gRPC it cannot guarantee that services will react immediately.

Irreversible Steps

Some operations cannot be truly reversible (for example, sending a confirmation email or shipping a real product). Such actions are supposed to be implemented as late in the flow as possible. For example, instead of sending the confirmation email to the customer after the initial creation of the order, send it only when the saga is completed.

Error Handling and Retries

Services can temporarily fail due to various issues, but this should not break the whole flow. For example, when the Warehouse Service is down, the saga should wait until it is online and then continue. Otherwise, the saga may hang indefinitely, which may introduce cascade failures in other services.

Solutions

• Exponential backoff for retries: Do not retry with a constant interval between attempts. Use increasing delays to avoid overwhelming the system.

• Circuit breakers: Temporarily stop retries if a service is repeatedly failing.

• Dead-letter queues: If an event cannot be processed correctly after a few attempts, it should be moved to the DLQ and the development team should be notified.

• Timeouts: If a request takes too long, it is better to fail it with a timeout. This helps prevent overloading the service further.

Summary

The Saga pattern is the de facto standard for managing distributed transactions in microservices, where traditional ACID transactions cannot span multiple services. By breaking workflows into local transactions paired with compensating actions, Sagas ensure your system can recover from failures and maintain eventual consistency - even when services operate independently.

Sagas are not a silver bullet, but they are the most practical solution for coordinating transactions in microservices. Start with Choreography if your workflow is simple and services are loosely coupled. Use Orchestration for complex workflows where centralized control outweighs the added complexity.

Links

Polly: The .NET resilience library

ACID

Choreography Pattern in Microservices

Orchestration Pattern in Microservices

Image credits: Jane__ml