Skip to content
TheChatArchiveThe Chat Archive
Grok 4.1 Fast· xaiHugging Face Frontier Reasoning Traces

Design a form submission API using domain events, saga orchestrations, and idempotent…

  1. User

    Design a form submission API using domain events, saga orchestrations, and idempotent command handlers.

  2. Assistant

    Form Submission API Design

    This design implements a robust, scalable form submission API using Domain-Driven Design (DDD) principles, CQRS/Event Sourcing, domain events, saga orchestration (choreography via events with a central orchestrator for coordination), and idempotent command handlers. It's suitable for a microservices architecture handling high-volume submissions (e.g., surveys, applications) with async processing across services like validation, payment, notification, and storage.

    Assumptions:

    • Microservices: Form Service (owns aggregates), Validation Service, Processing Service (e.g., business logic/payment), Notification Service.
    • Tech stack: .NET/C#, EventStoreDB (event store), MassTransit (for sagas/events with RabbitMQ/Kafka), Redis (idempotency/ sagas state).
    • Idempotency: Via unique IdempotencyKey (UUID) per submission attempt.
    • Sagas: Orchestrator pattern (FormSubmissionSaga in Form Service) coordinates cross-service transactions.
    • Resilience: Outbox pattern for reliable event publishing, retries, dead-letter queues.

    Key Components

    1. API Layer (RESTful Endpoint)

    POST /api/v1/forms/{formId}/submissions
    Content-Type: application/json
    Headers: X-Idempotency-Key: <UUID>
    
    Body:
    {
      "formData": { ... },  // JSON payload
      "submitterId": "user-123",
      "metadata": { "ip": "...", "userAgent": "..." }
    }
    
    • Response:

      • 202 Accepted: Async processing started (with Location: /api/v1/forms/{formId}/submissions/{submissionId}/status).
      • 409 Conflict: Duplicate idempotency key.
      • 400 Bad Request: Invalid data.
    • Gateway/Controller → SubmitFormCommand (via MediatR or message bus).

    2. Commands & Aggregates

    • SubmitFormCommand:

      public record SubmitFormCommand(
        Guid FormId,
        Guid SubmissionId,  // Generated UUID
        string IdempotencyKey,
        FormData FormData,
        string SubmitterId
      );
      
    • FormSubmission Aggregate (Event-Sourced):

      • Handles SubmitFormCommand.
      • Raises FormSubmitted domain event.
      • States: Draft → Submitted → Validated → Processed → Completed/Failed.

    3. Idempotent Command Handler

    Uses Redis (or DynamoDB) for IdempotencyStore with TTL (e.g., 24h).

    public class SubmitFormCommandHandler : ICommandHandler<SubmitFormCommand>
    {
      private readonly IIdempotencyStore _store;
      private readonly IEventStore _eventStore;
      private readonly IPublisher _publisher;  // Outbox-integrated
    
      public async Task Handle(SubmitFormCommand cmd)
      {
        // 1. Check idempotency
        var existing = await _store.GetAsync(cmd.IdempotencyKey);
        if (existing?.Status == ProcessingStatus.Completed)
          return Result.Success(existing.SubmissionId);  // 200 OK
        if (existing?.Status == ProcessingStatus.InProgress)
          return Result.Pending(existing.SubmissionId);  // 202 Accepted
    
        // 2. Optimistic lock: Set InProgress
        var locked = await _store.TrySetInProgressAsync(cmd.IdempotencyKey, cmd.SubmissionId);
        if (!locked) throw new ConflictException();
    
        try
        {
          // 3. Load/append to aggregate
          var aggregate = await _eventStore.LoadAsync<FormSubmission>(cmd.SubmissionId);
          aggregate.Submit(cmd);  // Raises FormSubmitted event
    
          // 4. Persist events (atomic via outbox)
          await _eventStore.AppendAsync(aggregate);
    
          // 5. Publish events
          await _publisher.Publish(aggregate.DomainEvents);
    
          // 6. Mark Completed
          await _store.SetCompletedAsync(cmd.IdempotencyKey, cmd.SubmissionId);
        }
        catch
        {
          await _store.SetFailedAsync(cmd.IdempotencyKey);  // Allows retry
          throw;
        }
      }
    }
    
    • Idempotency Benefits: Safe retries (e.g., network flakes), at-least-once delivery.

    4. Domain Events

    Published via message broker (topics/exchanges).

    EventPublisherDescriptionSubscribers
    FormSubmittedForm AggregateSubmission received.Saga Orchestrator, Query Updater
    FormValidated / FormValidationFailedValidation ServiceValidation complete.Saga
    FormProcessed / FormProcessingFailedProcessing ServiceBusiness logic done (e.g., payment).Saga
    UserNotified / NotificationFailedNotification ServiceEmails/SMS sent.Saga
    FormSubmissionCompleted / FormSubmissionFailedSagaEnd state.Query Updater, Analytics

    5. Saga Orchestration (FormSubmissionSaga)

    Stateful orchestrator in Form Service (MassTransit Saga):

    • State: SubmissionId (correlation ID), Status (Submitted/Validating/...), timeouts/retries.
    • Persistence: SagaState in DB/Redis (idempotent via correlation).
    public class FormSubmissionSaga : MassTransitStateMachine<FormSubmissionSagaState>
    {
      public State Validating { get; private set; }
      public State Processing { get; private set; }
      public State Notifying { get; private set; }
      public State Completed { get; private set; }
      public State Failed { get; private set; }
    
      Initial<FormSubmissionSagaState> Initial => Initial =>
        ConsumeContext<FormSubmitted>(ctx => ctx.CorrelationId = ctx.Message.SubmissionId)
          .Then<ValidateForm>()  // Publish command
          .TransitionTo(Validating);
    
      During(Validating, FormValidated.Instance, ctx => 
        Publish(new ProcessForm(ctx.Message.SubmissionId))
          .TransitionTo(Processing))
        .Then<FormValidationFailed>(ctx => 
          Publish(new FormSubmissionFailed(ctx.Message.SubmissionId, "Validation failed"))
            .TransitionTo(Failed));
    
      // Similar for Processing → Notifying → Completed
      // Compensations: On timeout/failure, publish compensating events (e.g., Refund if payment was partial).
      // Final: Publish FormSubmissionCompleted.
    }
    
    • Saga Commands (published by orchestrator):

      • ValidateForm
      • ProcessForm
      • SendNotification
    • Cross-Service Handlers (e.g., in Validation Service):

      // Idempotent too!
      public class ValidateFormHandler : ICommandHandler<ValidateForm>
      {
        // Check idempotency, validate, raise FormValidated event.
      }
      
    • Choreography Fallback: Services can subscribe directly for simple flows, but saga ensures ordering/coordination.

    Sequence Diagram (Text-Based)

    Client → API: POST /forms/{formId}/submissions (IdempotencyKey)
    API → CommandBus: SubmitFormCommand
    Handler → Redis: Check/Set InProgress
    Handler → Aggregate: Submit() → FormSubmitted
    Handler → EventStore: Append
    Handler → Broker: Publish FormSubmitted
    Handler → Redis: Set Completed
    API ← 202 Accepted
    
    Broker → Saga: FormSubmitted (corr: SubmissionId)
    Saga → Broker: ValidateForm
    Broker → ValidationSvc: ValidateForm
    ValidationSvc → Broker: FormValidated
    Saga → Broker: ProcessForm
    Broker → ProcessingSvc: ProcessForm → FormProcessed
    Saga → Broker: SendNotification
    Broker → NotificationSvc: SendNotification → UserNotified
    Saga → Broker: FormSubmissionCompleted  // Saga Complete
    

    Query & Status API

    • Read Model: Projection handlers update Marten/Postgres views from events.
    • GET /api/v1/forms/{formId}/submissions/{submissionId} → Status, data.

    Resilience & Ops

    • Outbox: Transactions ensure commands → events atomic.
    • Retries: Exponential backoff, circuit breakers (Polly).
    • Monitoring: OpenTelemetry traces (saga spans), metrics (submission latency, failure rates).
    • Scaling: Stateless handlers, saga state sharded by SubmissionId.
    • Compensations: Saga timeouts → failure events; manual intervention via admin API.

    This design ensures reliability (idempotency + outbox), scalability (async events), and consistency (eventual via sagas). Deployable on Kubernetes with Helm charts for services/broker.

    For implementation code/repos or tweaks (e.g., GraphQL), provide more details!

Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.