Help me design a microservice architecture for a global e-commerce site. I need to…
- User
Help me design a microservice architecture for a global e-commerce site. I need to implement the saga pattern for handling distributed transactions across Order, Inventory, and Payment services, including the rollback logic if Payment fails after Inventory is already reserved.
Assistant
This is a comprehensive request about designing a microservice architecture with distributed transactions using the Saga pattern. I'll provide a thorough, practical solution with:
- Architecture overview
- Saga pattern implementation (both choreography and orchestration)
- Code examples
- Error handling and rollback logic
- Best practices
Let me create a detailed solution focusing on the Order, Inventory, and Payment services scenario.
I'll provide:
- Service definitions
- Saga orchestrator
- Event handling
- Compensation (rollback) logic
- Database transactions
- Error handling
- Complete working examples
Microservice Saga Pattern Implementation
I'll design a complete solution using Orchestration Saga (more suitable for complex flows) with detailed rollback logic.
1. Architecture Overview
┌─────────────────────────────────────────────────────┐ │ Order Saga Orchestrator Service │ │ (Coordinates the entire transaction flow) │ └────────────┬────────────────────────────────────────┘ │ ┌────────┼────────┐ ▼ ▼ ▼ ┌────────┐┌────────┐┌──────────┐ │ Order ││Inventory││ Payment │ │Service ││Service ││ Service │ └────────┘└────────┘└──────────┘2. Service Models & Enums
// shared/types.ts enum SagaStatus { PENDING = 'PENDING', ORDER_CREATED = 'ORDER_CREATED', INVENTORY_RESERVED = 'INVENTORY_RESERVED', PAYMENT_PROCESSED = 'PAYMENT_PROCESSED', COMPLETED = 'COMPLETED', FAILED = 'FAILED', COMPENSATING = 'COMPENSATING', COMPENSATED = 'COMPENSATED' } enum CompensationStatus { PENDING = 'PENDING', IN_PROGRESS = 'IN_PROGRESS', COMPLETED = 'COMPLETED', FAILED = 'FAILED' } interface OrderSagaData { sagaId: string; orderId: string; userId: string; items: OrderItem[]; totalAmount: number; status: SagaStatus; compensationStatus: CompensationStatus; createdAt: Date; failureReason?: string; reservationId?: string; paymentId?: string; } interface OrderItem { productId: string; quantity: number; price: number; } interface StepResult<T> { success: boolean; data?: T; error?: Error; timestamp: Date; }3. Order Service
// order-service/order.service.ts import { EventEmitter2 } from '@nestjs/event-emitter'; import { Injectable, Logger } from '@nestjs/common'; @Injectable() export class OrderService { private readonly logger = new Logger(OrderService.name); constructor( private eventEmitter: EventEmitter2, private prisma: PrismaService ) {} async createOrder( userId: string, items: OrderItem[] ): Promise<{ orderId: string; sagaId: string }> { const orderId = uuidv4(); const sagaId = uuidv4(); try { // Create order in PENDING state await this.prisma.order.create({ data: { id: orderId, userId, status: 'PENDING', items: { create: items.map(item => ({ productId: item.productId, quantity: item.quantity, price: item.price })) }, sagaId } }); // Emit saga started event this.eventEmitter.emit('order.created', { sagaId, orderId, userId, items, totalAmount: this.calculateTotal(items), timestamp: new Date() }); return { orderId, sagaId }; } catch (error) { this.logger.error(`Failed to create order: ${error.message}`); throw error; } } async confirmOrder(sagaId: string, orderId: string): Promise<void> { await this.prisma.order.update({ where: { id: orderId }, data: { status: 'CONFIRMED', sagaId } }); this.logger.log(`Order ${orderId} confirmed for saga ${sagaId}`); } async cancelOrder( sagaId: string, orderId: string, reason: string ): Promise<void> { await this.prisma.order.update({ where: { id: orderId }, data: { status: 'CANCELLED', cancellationReason: reason } }); this.logger.warn( `Order ${orderId} cancelled for saga ${sagaId}: ${reason}` ); } private calculateTotal(items: OrderItem[]): number { return items.reduce((sum, item) => sum + item.price * item.quantity, 0); } }4. Inventory Service
// inventory-service/inventory.service.ts @Injectable() export class InventoryService { private readonly logger = new Logger(InventoryService.name); constructor( private eventEmitter: EventEmitter2, private prisma: PrismaService ) {} async reserveInventory( sagaId: string, orderId: string, items: OrderItem[] ): Promise<StepResult<{ reservationId: string }>> { const reservationId = uuidv4(); try { // Check all items in stock const availability = await this.checkAvailability(items); if (!availability.allAvailable) { throw new Error( `Insufficient inventory: ${availability.unavailableItems.join(', ')}` ); } // Create reservation record const reservation = await this.prisma.reservation.create({ data: { id: reservationId, orderId, sagaId, status: 'RESERVED', reservedItems: { create: items.map(item => ({ productId: item.productId, quantity: item.quantity, reservedAt: new Date() })) } } }); // Update inventory stock with transaction await this.prisma.$transaction( items.map(item => this.prisma.inventoryStock.update({ where: { productId: item.productId }, data: { reserved: { increment: item.quantity } } }) ) ); this.eventEmitter.emit('inventory.reserved', { sagaId, orderId, reservationId, items, timestamp: new Date() }); return { success: true, data: { reservationId }, timestamp: new Date() }; } catch (error) { this.logger.error( `Failed to reserve inventory for saga ${sagaId}: ${error.message}` ); this.eventEmitter.emit('inventory.reservation.failed', { sagaId, orderId, error: error.message, timestamp: new Date() }); return { success: false, error, timestamp: new Date() }; } } async releaseReservation( sagaId: string, reservationId: string ): Promise<void> { try { const reservation = await this.prisma.reservation.findUnique({ where: { id: reservationId }, include: { reservedItems: true } }); if (!reservation) { throw new Error(`Reservation ${reservationId} not found`); } // Release inventory with transaction await this.prisma.$transaction( reservation.reservedItems.map(item => this.prisma.inventoryStock.update({ where: { productId: item.productId }, data: { reserved: { decrement: item.quantity } } }) ) ); // Mark reservation as cancelled await this.prisma.reservation.update({ where: { id: reservationId }, data: { status: 'CANCELLED', releasedAt: new Date() } }); this.eventEmitter.emit('inventory.released', { sagaId, reservationId, timestamp: new Date() }); this.logger.log(`Inventory released for reservation ${reservationId}`); } catch (error) { this.logger.error( `Failed to release reservation ${reservationId}: ${error.message}` ); throw error; } } private async checkAvailability( items: OrderItem[] ): Promise<{ allAvailable: boolean; unavailableItems: string[] }> { const unavailableItems: string[] = []; for (const item of items) { const stock = await this.prisma.inventoryStock.findUnique({ where: { productId: item.productId } }); const available = (stock?.quantity || 0) - (stock?.reserved || 0); if (available < item.quantity) { unavailableItems.push( `${item.productId} (requested: ${item.quantity}, available: ${available})` ); } } return { allAvailable: unavailableItems.length === 0, unavailableItems }; } }5. Payment Service
// payment-service/payment.service.ts @Injectable() export class PaymentService { private readonly logger = new Logger(PaymentService.name); constructor( private eventEmitter: EventEmitter2, private prisma: PrismaService, private stripeService: StripeService ) {} async processPayment( sagaId: string, orderId: string, userId: string, amount: number ): Promise<StepResult<{ paymentId: string; transactionId: string }>> { const paymentId = uuidv4(); try { // Verify payment method exists const paymentMethod = await this.prisma.paymentMethod.findFirst({ where: { userId, isDefault: true } }); if (!paymentMethod) { throw new Error('No default payment method found'); } // Create payment record with PENDING status const payment = await this.prisma.payment.create({ data: { id: paymentId, orderId, userId, sagaId, amount, status: 'PENDING', paymentMethod: paymentMethod.id } }); // Attempt charge through payment gateway const chargeResult = await this.stripeService.charge({ amount, currency: 'USD', customerId: userId, idempotencyKey: paymentId // Ensures idempotency }); // Update payment record with transaction details const updatedPayment = await this.prisma.payment.update({ where: { id: paymentId }, data: { status: 'SUCCESS', transactionId: chargeResult.id, processedAt: new Date() } }); this.eventEmitter.emit('payment.processed', { sagaId, orderId, paymentId, transactionId: chargeResult.id, amount, timestamp: new Date() }); return { success: true, data: { paymentId, transactionId: chargeResult.id }, timestamp: new Date() }; } catch (error) { this.logger.error( `Payment processing failed for saga ${sagaId}: ${error.message}` ); // Mark payment as failed await this.prisma.payment.update({ where: { id: paymentId }, data: { status: 'FAILED', failureReason: error.message } }); this.eventEmitter.emit('payment.failed', { sagaId, orderId, paymentId, error: error.message, timestamp: new Date() }); return { success: false, error, timestamp: new Date() }; } } async refundPayment( sagaId: string, paymentId: string ): Promise<void> { try { const payment = await this.prisma.payment.findUnique({ where: { id: paymentId } }); if (!payment) { throw new Error(`Payment ${paymentId} not found`); } if (payment.status !== 'SUCCESS') { this.logger.warn( `Cannot refund payment ${paymentId} with status ${payment.status}` ); return; } // Process refund through payment gateway const refundResult = await this.stripeService.refund({ chargeId: payment.transactionId, amount: payment.amount, idempotencyKey: `refund-${paymentId}` // Ensures idempotency }); // Update payment record await this.prisma.payment.update({ where: { id: paymentId }, data: { status: 'REFUNDED', refundedAt: new Date(), refundTransactionId: refundResult.id } }); this.eventEmitter.emit('payment.refunded', { sagaId, paymentId, refundTransactionId: refundResult.id, timestamp: new Date() }); this.logger.log(`Payment ${paymentId} refunded for saga ${sagaId}`); } catch (error) { this.logger.error( `Failed to refund payment ${paymentId}: ${error.message}` ); throw error; } } }6. Saga Orchestrator (The Heart)
// saga-orchestrator/order-saga.orchestrator.ts @Injectable() export class OrderSagaOrchestrator { private readonly logger = new Logger(OrderSagaOrchestrator.name); constructor( private eventEmitter: EventEmitter2, private onEvent: OnEvent, private prisma: PrismaService, private orderService: OrderService, private inventoryService: InventoryService, private paymentService: PaymentService, private sagaStateManager: SagaStateManager, private deadLetterQueue: DeadLetterQueueService ) { this.setupEventListeners(); } private setupEventListeners(): void { // Listen for saga initiation this.onEvent('order.created', (event) => this.handleOrderCreated(event) ); // Listen for inventory result this.onEvent('inventory.reserved', (event) => this.handleInventoryReserved(event) ); this.onEvent('inventory.reservation.failed', (event) => this.handleInventoryReservationFailed(event) ); // Listen for payment result this.onEvent('payment.processed', (event) => this.handlePaymentProcessed(event) ); this.onEvent('payment.failed', (event) => this.handlePaymentFailed(event) ); // Listen for compensation events this.onEvent('inventory.released', (event) => this.handleInventoryReleased(event) ); this.onEvent('payment.refunded', (event) => this.handlePaymentRefunded(event) ); } /** * Step 1: Order Created - Initiate Saga */ private async handleOrderCreated(event: any): Promise<void> { const { sagaId, orderId, userId, items, totalAmount } = event; this.logger.log(`[Saga ${sagaId}] Starting - Order ${orderId} created`); try { // Store saga state await this.sagaStateManager.createSagaState({ sagaId, orderId, userId, items, totalAmount, status: SagaStatus.ORDER_CREATED, compensationStatus: CompensationStatus.PENDING }); // Step 2: Reserve Inventory this.logger.log(`[Saga ${sagaId}] Step 1: Reserving inventory`); await this.inventoryService.reserveInventory(sagaId, orderId, items); } catch (error) { this.logger.error( `[Saga ${sagaId}] Failed to reserve inventory: ${error.message}` ); await this.compensate(sagaId, `Inventory reservation failed: ${error.message}`); } } /** * Step 2: Inventory Reserved - Proceed to Payment */ private async handleInventoryReserved(event: any): Promise<void> { const { sagaId, orderId, reservationId, items } = event; this.logger.log( `[Saga ${sagaId}] Step 1 Complete: Inventory reserved (${reservationId})` ); try { // Update saga state await this.sagaStateManager.updateSagaState(sagaId, { status: SagaStatus.INVENTORY_RESERVED, reservationId }); // Get user's total amount const sagaState = await this.sagaStateManager.getSagaState(sagaId); // Step 3: Process Payment this.logger.log(`[Saga ${sagaId}] Step 2: Processing payment`); await this.paymentService.processPayment( sagaId, orderId, sagaState.userId, sagaState.totalAmount ); } catch (error) { this.logger.error( `[Saga ${sagaId}] Failed during payment processing: ${error.message}` ); // First compensate inventory await this.compensate(sagaId, `Payment processing failed: ${error.message}`); } } /** * Inventory Reservation Failed - Compensate */ private async handleInventoryReservationFailed(event: any): Promise<void> { const { sagaId, orderId, error } = event; this.logger.warn( `[Saga ${sagaId}] Inventory reservation failed: ${error}` ); await this.compensate( sagaId, `Inventory reservation failed: ${error}`, false // No compensation needed for inventory yet ); } /** * Step 3: Payment Processed - Complete Saga */ private async handlePaymentProcessed(event: any): Promise<void> { const { sagaId, orderId, paymentId, transactionId } = event; this.logger.log( `[Saga ${sagaId}] Step 2 Complete: Payment processed (${paymentId})` ); try { // Update saga state await this.sagaStateManager.updateSagaState(sagaId, { status: SagaStatus.PAYMENT_PROCESSED, paymentId }); // Confirm order const sagaState = await this.sagaStateManager.getSagaState(sagaId); await this.orderService.confirmOrder(sagaId, orderId); // Mark saga complete await this.sagaStateManager.updateSagaState(sagaId, { status: SagaStatus.COMPLETED }); this.logger.log( `[Saga ${sagaId}] COMPLETED: Order ${orderId} confirmed` ); this.eventEmitter.emit('saga.completed', { sagaId, orderId, timestamp: new Date() }); } catch (error) { this.logger.error( `[Saga ${sagaId}] Failed to complete saga: ${error.message}` ); await this.compensate(sagaId, `Failed to confirm order: ${error.message}`); } } /** * Payment Failed - Compensate All Steps */ private async handlePaymentFailed(event: any): Promise<void> { const { sagaId, orderId, paymentId, error } = event; this.logger.error( `[Saga ${sagaId}] Payment failed: ${error} - INITIATING ROLLBACK` ); await this.compensate( sagaId, `Payment failed: ${error}` ); } /** * Compensation Logic - Rollback All Steps */ private async compensate( sagaId: string, failureReason: string, withInventoryCompensation: boolean = true ): Promise<void> { this.logger.warn(`[Saga ${sagaId}] Initiating compensation: ${failureReason}`); try { await this.sagaStateManager.updateSagaState(sagaId, { status: SagaStatus.COMPENSATING, compensationStatus: CompensationStatus.IN_PROGRESS, failureReason }); const sagaState = await this.sagaStateManager.getSagaState(sagaId); // Compensation Order (Reverse of execution): // 1. Refund Payment (if processed) // 2. Release Inventory Reservation (if reserved) // 3. Cancel Order // Step 1: Refund Payment if (sagaState.paymentId) { this.logger.log(`[Saga ${sagaId}] Compensating: Refunding payment`); try { await this.paymentService.refundPayment(sagaId, sagaState.paymentId); } catch (error) { this.logger.error( `[Saga ${sagaId}] Failed to refund payment: ${error.message}` ); await this.deadLetterQueue.enqueue({ type: 'COMPENSATION_FAILED', sagaId, step: 'payment_refund', error: error.message }); } } // Step 2: Release Inventory if (withInventoryCompensation && sagaState.reservationId) { this.logger.log(`[Saga ${sagaId}] Compensating: Releasing inventory`); try { await this.inventoryService.releaseReservation( sagaId, sagaState.reservationId ); } catch (error) { this.logger.error( `[Saga ${sagaId}] Failed to release inventory: ${error.message}` ); await this.deadLetterQueue.enqueue({ type: 'COMPENSATION_FAILED', sagaId, step: 'inventory_release', error: error.message }); } } // Step 3: Cancel Order this.logger.log(`[Saga ${sagaId}] Compensating: Cancelling order`); await this.orderService.cancelOrder( sagaId, sagaState.orderId, failureReason ); // Mark compensation complete await this.sagaStateManager.updateSagaState(sagaId, { status: SagaStatus.COMPENSATED, compensationStatus: CompensationStatus.COMPLETED }); this.logger.log(`[Saga ${sagaId}] Compensation completed`); this.eventEmitter.emit('saga.compensated', { sagaId, orderId: sagaState.orderId, failureReason, timestamp: new Date() }); } catch (error) { this.logger.error( `[Saga ${sagaId}] Compensation failed: ${error.message}` ); // Mark as failed and send to dead letter queue await this.sagaStateManager.updateSagaState(sagaId, { status: SagaStatus.FAILED, compensationStatus: CompensationStatus.FAILED }); await this.deadLetterQueue.enqueue({ type: 'SAGA_FAILED', sagaId, error: error.message, timestamp: new Date() }); } } private async handleInventoryReleased(event: any): Promise<void> { this.logger.log(`[Saga ${event.sagaId}] Inventory released`); } private async handlePaymentRefunded(event: any): Promise<void> { this.logger.log(`[Saga ${event.sagaId}] Payment refunded`); } }7. Saga State Manager
// saga-orchestrator/saga-state.manager.ts @Injectable() export class SagaStateManager { constructor(private prisma: PrismaService) {} async createSagaState(state: Partial<OrderSagaData>): Promise<void> { await this.prisma.sagaState.create({ data: { sagaId: state.sagaId!, orderId: state.orderId!, userId: state.userId!, items: state.items!, totalAmount: state.totalAmount!, status: state.status || SagaStatus.PENDING, compensationStatus: CompensationStatus.PENDING, createdAt: new Date() } }); } async getSagaState(sagaId: string): Promise<OrderSagaData> { const state = await this.prisma.sagaState.findUnique({ where: { sagaId } }); if (!state) { throw new Error(`Saga state not found for sagaId: ${sagaId}`); } return state as OrderSagaData; } async updateSagaState( sagaId: string, updates: Partial<OrderSagaData> ): Promise<void> { await this.prisma.sagaState.update({ where: { sagaId }, data: { ...updates, updatedAt: new Date() } }); } async getAllPendingSagas(): Promise<OrderSagaData[]> { return this.prisma.sagaState.findMany({ where: { status: { in: [SagaStatus.PENDING, SagaStatus.COMPENSATING] } } }) as Promise<OrderSagaData[]>; } }8. Dead Letter Queue Service
// saga-orchestrator/dead-letter-queue.service.ts @Injectable() export class DeadLetterQueueService { private readonly logger = new Logger(DeadLetterQueueService.name); constructor( private prisma: PrismaService, private emailService: EmailService ) { this.startRetryWorker(); } async enqueue(failedEvent: { type: string; sagaId: string; step?: string; error: string; }): Promise<void> { await this.prisma.deadLetterMessage.create({ data: { messageId: uuidv4(), messageType: failedEvent.type, sagaId: failedEvent.sagaId, step: failedEvent.step, payload: failedEvent, status: 'PENDING', retryCount: 0, createdAt: new Date() } }); // Alert support team await this.emailService.sendAlert({ subject: `Saga Compensation Failed: ${failedEvent.sagaId}`, body: `Error: ${failedEvent.error}`, severity: 'HIGH' }); this.logger.error( `Message enqueued to DLQ: ${failedEvent.sagaId} - ${failedEvent.error}` ); } private startRetryWorker(): void { setInterval(async () => { const failedMessages = await this.prisma.deadLetterMessage.findMany({ where: { status: 'PENDING', retryCount: { lt: 3 } }, orderBy: { createdAt: 'asc' }, take: 10 }); for (const message of failedMessages) { try { // Attempt recovery based on message type await this.attemptRecovery(message); } catch (error) { this.logger.error( `Retry failed for message ${message.messageId}: ${error.message}` ); await this.prisma.deadLetterMessage.update({ where: { messageId: message.messageId }, data: { retryCount: { increment: 1 } } }); } } }, 60000); // Retry every minute } private async attemptRecovery(message: any): Promise<void> { // Implement recovery logic based on message type this.logger.log(`Attempting recovery for ${message.messageType}`); } }9. Error Handling & Timeouts
// saga-orchestrator/saga-timeout.handler.ts @Injectable() export class SagaTimeoutHandler { private readonly logger = new Logger(SagaTimeoutHandler.name); private readonly SAGA_TIMEOUT_MS = 5 * 60 * 1000; // 5 minutes constructor( private sagaStateManager: SagaStateManager, private orderSagaOrchestrator: OrderSagaOrchestrator ) { this.startTimeoutChecker(); } private startTimeoutChecker(): void { setInterval(async () => { const pendingSagas = await this.sagaStateManager.getAllPendingSagas(); for (const saga of pendingSagas) { const elapsedTime = Date.now() - saga.createdAt.getTime(); if (elapsedTime > this.SAGA_TIMEOUT_MS) { this.logger.warn( `Saga ${saga.sagaId} timed out - initiating compensation` ); // Use reflection to call compensate (it's private) await this.orderSagaOrchestrator['compensate']( saga.sagaId, 'Saga timeout exceeded' ); } } }, 30000); // Check every 30 seconds } }10. Module Setup
// order-saga/order-saga.module.ts import { Module } from '@nestjs/common'; import { EventEmitterModule } from '@nestjs/event-emitter'; @Module({ imports: [ EventEmitterModule.forRoot(), TypeOrmModule.forFeature([ Order, Reservation, Payment, SagaState, DeadLetterMessage ]), OrderModule, InventoryModule, PaymentModule ], providers: [ OrderSagaOrchestrator, SagaStateManager, SagaTimeoutHandler, DeadLetterQueueService ], exports: [OrderSagaOrchestrator, SagaStateManager] }) export class OrderSagaModule {}11. Database Schema (Prisma)
// prisma/schema.prisma model Order { id String @id @default(cuid()) userId String status String sagaId String @unique items OrderItem[] createdAt DateTime @default(now()) updatedAt DateTime @updatedAt cancellationReason String? @@index([userId, status]) @@index([sagaId]) } model OrderItem { id String @id @default(cuid()) orderId String order Order @relation(fields: [orderId], references: [id], onDelete: Cascade) productId String quantity Int price Float @@unique([orderId, productId]) } model Reservation { id String @id @default(cuid()) orderId String @unique sagaId String status String reservedItems ReservedItem[] releasedAt DateTime? createdAt DateTime @default(now()) @@index([sagaId]) } model ReservedItem { id String @id @default(cuid()) reservationId String reservation Reservation @relation(fields: [reservationId], references: [id], onDelete: Cascade) productId String quantity Int reservedAt DateTime @@unique([reservationId, productId]) } model InventoryStock { productId String @id quantity Int reserved Int @default(0) updatedAt DateTime @updatedAt @@index([quantity, reserved]) } model Payment { id String @id @default(cuid()) orderId String userId String sagaId String @unique amount Float status String paymentMethod String transactionId String? @unique refundTransactionId String? refundedAt DateTime? failureReason String? processedAt DateTime? createdAt DateTime @default(now()) @@index([userId, status]) @@index([sagaId]) } model SagaState { sagaId String @id orderId String userId String items Json totalAmount Float status String compensationStatus String failureReason String? reservationId String? paymentId String? createdAt DateTime @default(now()) updatedAt DateTime @updatedAt @@index([status]) @@index([compensationStatus]) } model DeadLetterMessage { messageId String @id messageType String sagaId String step String? payload Json status String retryCount Int @default(0) createdAt DateTime @default(now()) @@index([sagaId, status]) }12. Testing
// order-saga/order-saga.spec.ts import { Test } from '@nestjs/testing'; describe('Order Saga - Distributed Transactions', () => { let orchestrator: OrderSagaOrchestrator; let orderService: OrderService; let inventoryService: InventoryService; let paymentService: PaymentService; let stateManager: SagaStateManager; beforeEach(async () => { const module = await Test.createTestingModule({ providers: [ OrderSagaOrchestrator, OrderService, InventoryService, PaymentService, SagaStateManager, { provide: EventEmitter2, useValue: new EventEmitter2() } ] }).compile(); orchestrator = module.get(OrderSagaOrchestrator); orderService = module.get(OrderService); inventoryService = module.get(InventoryService); paymentService = module.get(PaymentService); stateManager = module.get(SagaStateManager); }); describe('Happy Path - Complete Transaction', () => { it('should complete saga when all steps succeed', async () => { const sagaId = uuidv4(); const orderId = uuidv4(); const items = [{ productId: 'PROD1', quantity: 2, price: 50 }]; jest.spyOn(inventoryService, 'reserveInventory') .mockResolvedValue({ success: true, data: { reservationId: uuidv4() }, timestamp: new Date() }); jest.spyOn(paymentService, 'processPayment') .mockResolvedValue({ success: true, data: { paymentId: uuidv4(), transactionId: 'TXN123' }, timestamp: new Date() }); jest.spyOn(orderService, 'confirmOrder') .mockResolvedValue(undefined); // Simulate saga flow await orchestrator['handleOrderCreated']({ sagaId, orderId, userId: 'USER1', items, totalAmount: 100 }); // Verify final state const finalState = await stateManager.getSagaState(sagaId); expect(finalState.status).toBe(SagaStatus.COMPLETED); }); }); describe('Payment Failure Scenario', () => { it('should refund and release inventory when payment fails', async () => { const sagaId = uuidv4(); const orderId = uuidv4(); const reservationId = uuidv4(); const paymentId = uuidv4(); const items = [{ productId: 'PROD1', quantity: 2, price: 50 }]; // Setup saga state await stateManager.createSagaState({ sagaId, orderId, userId: 'USER1', items, totalAmount: 100, status: SagaStatus.INVENTORY_RESERVED, compensationStatus: CompensationStatus.PENDING, reservationId, paymentId }); const refundSpy = jest.spyOn(paymentService, 'refundPayment') .mockResolvedValue(undefined); const releaseSpy = jest.spyOn(inventoryService, 'releaseReservation') .mockResolvedValue(undefined); const cancelSpy = jest.spyOn(orderService, 'cancelOrder') .mockResolvedValue(undefined); // Simulate payment failure await orchestrator['handlePaymentFailed']({ sagaId, orderId, paymentId, error: 'Insufficient funds' }); // Verify compensation happened in order expect(refundSpy).toHaveBeenCalledWith(sagaId, paymentId); expect(releaseSpy).toHaveBeenCalledWith(sagaId, reservationId); expect(cancelSpy).toHaveBeenCalled(); const compensatedState = await stateManager.getSagaState(sagaId); expect(compensatedState.status).toBe(SagaStatus.COMPENSATED); expect(compensatedState.compensationStatus) .toBe(CompensationStatus.COMPLETED); }); it('should handle partial compensation failure', async () => { const sagaId = uuidv4(); const dlqSpy = jest.spyOn( new DeadLetterQueueService( null as any, null as any ), 'enqueue' ); jest.spyOn(paymentService, 'refundPayment') .mockRejectedValue(new Error('Gateway timeout')); // This should capture the error and not throw await expect( orchestrator['compensate'](sagaId, 'Payment failed') ).resolves.not.toThrow(); }); }); describe('Idempotency', () => { it('should handle duplicate payment requests', async () => { const paymentId = uuidv4(); const sagaId = uuidv4(); // First payment attempt const result1 = await paymentService.processPayment( sagaId, 'ORDER1', 'USER1', 100 ); // Duplicate request with same idempotency key const result2 = await paymentService.processPayment( sagaId, 'ORDER1', 'USER1', 100 ); // Should get same result expect(result1.data?.transactionId) .toBe(result2.data?.transactionId); }); }); describe('Timeout Handling', () => { it('should compensate sagas that exceed timeout', async () => { const sagaId = uuidv4(); await stateManager.createSagaState({ sagaId, orderId: 'ORDER1', userId: 'USER1', items: [], totalAmount: 100, status: SagaStatus.PENDING, compensationStatus: CompensationStatus.PENDING, // Set creation time to past createdAt: new Date(Date.now() - 10 * 60 * 1000) // 10 minutes ago }); const compensateSpy = jest.spyOn( orchestrator as any, 'compensate' ); const handler = new SagaTimeoutHandler(stateManager, orchestrator); // Manually trigger timeout check await handler['startTimeoutChecker'](); expect(compensateSpy).toHaveBeenCalled(); }); }); });13. Monitoring & Metrics
// saga-orchestrator/saga-metrics.ts import { Injectable } from '@nestjs/common'; import { Counter, Histogram } from 'prom-client'; @Injectable() export class SagaMetrics { sagasStarted = new Counter({ name: 'sagas_started_total', help: 'Total number of sagas started' }); sagasCompleted = new Counter({ name: 'sagas_completed_total', help: 'Total number of sagas completed successfully' }); sagasCompensated = new Counter({ name: 'sagas_compensated_total', help: 'Total number of sagas that required compensation' }); sagasDuration = new Histogram({ name: 'saga_duration_seconds', help: 'Time taken to complete sagas', buckets: [1, 5, 10, 30, 60, 300] }); compensationDuration = new Histogram({ name: 'compensation_duration_seconds', help: 'Time taken for compensation', buckets: [1, 5, 10, 30] }); trackSagaCompletion(sagaId: string, durationMs: number): void { this.sagasCompleted.inc(); this.sagasDuration.observe(durationMs / 1000); } trackCompensation(compensationMs: number): void { this.sagasCompensated.inc(); this.compensationDuration.observe(compensationMs / 1000); } }Key Improvements & Best Practices
Aspect Implementation Idempotency Unique saga IDs, payment idempotency keys Idempotency Keys Using saga IDs + service operation names Compensation Reverse order: Payment → Inventory → Order Timeouts 5-minute sage timeout with automatic compensation Dead Letter Queue Automatic retry with 3 attempts State Management Persisted saga state for recovery Monitoring Prometheus metrics tracking Error Handling Graceful degradation with alerts Atomicity Database transactions within services Observability Structured logging with saga IDs This architecture ensures that your distributed transactions are reliable, recoverable, and maintainable!
Other models, same or similar prompt
Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.