Skip to content
TheChatArchiveThe Chat Archive
MiniMaxminimax/minimax-m3· unknownHugging Face ibragim-bad/swe_rebench_07_2026_trajectories

ISSUE DESCRIPTION [Reborn WebUI Beta] Add WebUI idempotency and accepted-message ledger…

  1. User

    ISSUE DESCRIPTION

    [Reborn WebUI Beta] Add WebUI idempotency and accepted-message ledger Parent tracker: #3607 Module: M2 — Native Product Workflow / Inbound Facade Priority: P0

    Goal

    Make WebUI mutations idempotent so duplicate browser submits do not create duplicate turns/messages.

    Scope

    • Use client action id / idempotency key for create-thread and send-message.
    • Preserve accepted-message ref/outcome for duplicate requests.
    • Handle concurrent duplicate submissions safely.

    Acceptance criteria

    • Duplicate send returns the same accepted ref/outcome.
    • Concurrent duplicate send creates at most one accepted message/turn.
    • Failed validation does not create a ghost turn/message.
    • Retry after transient unavailable state has deterministic behavior.
    • Tests cover same-thread duplicate, cross-thread duplicate rejection, and concurrent duplicate behavior.

    Interface

    No new interfaces are introduced.

    ADDITIONAL ADVICE

    Since you are given a git repository, you can use git commands to simplify your work. For example, if you made a mistake and want to revert your changes, you can use git checkout <file> to restore the file to its original state. You can also reset all changes in the repository using git reset --hard command. Additionally, you can use git stash and git stash pop to temporarily save your changes and restore them later. Finally, you can see the changes that you've made (which will be submitted when you call submit) using git status or git diff commands. However you don't need to use git add or git commit before submitting your changes. In fact, submit only submits currently unstaged changes because it uses git diff to get the changes that need to be submitted.

    CHECKLIST

    Before submitting your solution, please go over the following checklist and make sure you've done everything:

    • If an error was reported in the issue description, I have successfully reproduced it.
    • If an error was reported in the issue description, I have confirmed that my fix resolves the error.
    • I have thought about edge cases and made sure my fix handles them as well.
    • I have run existing tests in the repository that might have been affected by the change I made and confirmed that they pass. I want you to list every bullet from this checklist and write your reasoning for why you think you did it or didn't need to.

    Repository has been uploaded and your shell is currently at the repository root. Time to solve the issue!

    (Current directory: /ironclaw, current file: none) bash-$

  2. MiniMax Assistant

    Let me start by exploring the repository structure to understand what we'reworking with.

  3. MiniMax Assistant

    Now let me look at the chat module which contains the relevant endpoints:

  4. MiniMax Assistant

    Let me look at the types file in detail to understand the data structures:

  5. MiniMax Assistant

    Let me look at the relevant tests in the chat module:

  6. MiniMax Assistant

    Now let me look at how messages flow into the system to understand the data structures:

  7. MiniMax Assistant

    Now let me look at how the message_id is used elsewhere, and at the existing infrastructure:

  8. MiniMax Assistant

    Now let me check the platform/state.rs to understand the gateway state:

  9. MiniMax Assistant

    Now let me look at how thread creation is handled - especially the part where it's persisted:

  10. MiniMax Assistant

    Now let's look at how tests are structured for chat handlers:

  11. MiniMax Assistant

    Now let me understand what tests exist already related to chat send / new thread:

  12. MiniMax Assistant

    Now let me look at the relevant issue context:

  13. MiniMax Assistant

    Now let me look at the structure of SendMessageResponse:

  14. MiniMax Assistant

    Now let me look at the structure of the InboundFacade mentioned in the issue:

  15. MiniMax Assistant

    This is a great find! Let me explore the relevant modules more:

  16. MiniMax Assistant

    Now let me look at the existing product workflow tests:

  17. MiniMax Assistant

    Now let me look at the InboundFacade / WebUI inbound module:

  18. MiniMax Assistant

    Now let me look at the inbound_turn.rs:

  19. MiniMax Assistant

    Now let me check the gateway implementation to see if there's a similar Beta WebUI:

  20. MiniMax Assistant

    Now let me check for any existing handlers in src/ that use the IdempotencyKey or workflow:

  21. MiniMax Assistant

    Let me check what already exists in the codebase regarding idempotency. Let me check the existing tests:

  22. MiniMax Assistant

    Let me look at the existing chat_send_handler implementation and the structure:

  23. MiniMax Assistant

    Now let me look at the existing test file structure to understand how the chat_send_handler is tested:

  24. MiniMax Assistant

    Now let me look at what tests already exist in the chat module that I need to be aware of:

  25. MiniMax Assistant

    Now let me check the platform state:

  26. MiniMax Assistant

    Now let me look at how the current chat_send_handler test works:

  27. MiniMax Assistant

    Now let me check the existing thread/new test in detail to understand how to write similar ones:

  28. MiniMax Assistant

    Now I have a clear picture. Let me check more carefully how the issue should be addressed. Looking at the issue, the focus is on:

    1. Make WebUI mutations idempotent so duplicate browser submits do not create duplicate turns/messages
    2. Use client action id / idempotency key for create-thread and send-message
    3. Preserve accepted-message ref/outcome for duplicate requests
    4. Handle concurrent duplicate submissions safely
    5. Tests cover same-thread duplicate, cross-thread duplicate rejection, and concurrent duplicate behavior

    Let me look at the inbound_turn.rs to understand the existing infrastructure more:

  29. MiniMax Assistant

    Now let me check the tests structure to understand what tests we can write. Let me look at the chat module tests structure:

  30. MiniMax Assistant

    Now let me look at how the existing code handles idempotency in other places:

  31. MiniMax Assistant

    Let me explore the platform state structure and the existing tests in detail:

  32. MiniMax Assistant

    Let me check the InboundMessage to understand how message IDs are generated:

  33. MiniMax Assistant

    Now I have a clear understanding. Let me design the implementation:

  34. MiniMax Assistant

    Now let me look at the testing module for chat to understand the test setup:

  35. MiniMax Assistant

    Now let me explore the planning and confirm my approach:

  36. MiniMax Assistant

    OK so ironclaw_threads is not in the main workspace deps but ironclaw_turns is. Let me look at the IdempotencyKey in ironclaw_turns:

  37. MiniMax Assistant

    Now I have all the information I need. Let me design and implement the solution.

    The plan:

    1. Add a client_action_id field to SendMessageRequest (and a new request type for thread/new)
    2. Create a WebUI idempotency ledger in src/channels/web/ (a thread-safe map of client_action_id -> outcome)
    3. Add the ledger to GatewayState
    4. Update chat_send_handler to:
      • Check the ledger for an existing entry
      • If found and matches: return the stored outcome
      • If found but mismatched: return 409
      • If not found: process the request, then store the outcome
    5. Update chat_new_thread_handler similarly
    6. Add tests covering:
      • Same-thread duplicate returns the same outcome
      • Cross-thread duplicate rejection
      • Concurrent duplicate behavior

    Let me start the implementation:

  38. MiniMax Assistant

    Let me also check the test setup to understand what we need for testing idempotency:

  39. MiniMax Assistant

    Now I have a complete understanding. Let me start implementing. First, I'll modify the SendMessageRequest type to add a client_action_id field, and create the WebUiIdempotencyLedger:

  40. MiniMax Assistant

    Now let me make my changes. First, modify the SendMessageRequest:

  41. MiniMax Assistant

    Now let me start the implementation. I'll first modify the types:

  42. MiniMax Assistant

    Let me try a different approach. Let me use replace:

  43. MiniMax Assistant

    Now let me make the modification to add client_action_id to SendMessageRequest:

  44. MiniMax Assistant

    Now let me also add a deduplicated field to ThreadInfo:

  45. MiniMax Assistant

    Now let me create the WebUI idempotency ledger. Let me put it in a new file:

  46. MiniMax Assistant

    Now let me add the ledger to GatewayState. First, let me check the current state.rs structure:

  47. MiniMax Assistant

    Now let me add the WebUiIdempotencyLedger to GatewayState. Let me find a good place to add it:

  48. MiniMax Assistant

    Now I need to update the test helpers and other places where GatewayState is constructed to add the new field. Let me find all the places:

  49. MiniMax Assistant

    I need to update all these places to include the new webui_idempotency field. Let me update each one:

  50. MiniMax Assistant

    Let me verify the changes were applied:

  51. MiniMax Assistant

    I see that the second test_helpers builder didn't get updated correctly. Let me check the pattern again:

  52. MiniMax Assistant

    I see - this builder has different indentation (8 spaces). Let me check the file and fix this:

  53. MiniMax Assistant

    Let me check all the locations now:

  54. MiniMax Assistant

    Let me add the idempotency module:

  55. MiniMax Assistant

    Now let me try to build the project to see if we have any compilation errors:

  56. MiniMax Assistant

    Now I can run cargo. Let me check what feature flags are required:

  57. MiniMax Assistant

    Good progress. Now I need to fix the existing usages of SendMessageResponse and ThreadInfo to include the new field, and also update the remaining GatewayState initializer:

  58. MiniMax Assistant

    Now let me fix the SendMessageResponse usages and ThreadInfo usages. Let me check each location:

  59. MiniMax Assistant

    Let me update these to use the default for deduplicated field. I can do a global replacement:

  60. MiniMax Assistant

    Now let me fix the ThreadInfo usages:

  61. MiniMax Assistant

    Now let me check the test files that use ThreadInfo for compilation:

  62. MiniMax Assistant

    Now let me also check for tests for the types that might also have SendMessageResponse:

  63. MiniMax Assistant

    Now let me try the build again:

  64. MiniMax Assistant

    The build succeeded. Now let me look at what needs to happen for the actual handler modifications:

  65. MiniMax Assistant

    Now let me modify chat_send_handler to use the idempotency ledger. First, let me check how IncomingMessage::id is generated to understand if I need to make it deterministic:

  66. MiniMax Assistant

    Now let me modify the chat_send_handler to use the idempotency ledger. The approach:

    1. Validate client_action_id if provided
    2. If provided, check the ledger for a matching record
    3. If found (matching), return the stored message_id with deduplicated: true
    4. If found (mismatching), return 409
    5. If not found, proceed with normal processing and record the outcome on success

    For thread creation, similarly.

    Let me also think about how to handle the message_id. If we want duplicate sends to return the same message_id, we need to:

    • Generate the message_id deterministically from client_action_id (e.g., UUID v5)
    • OR record the message_id in the ledger

    The ledger approach is simpler and works with the current implementation. Let me implement that.

    But wait, for the second submit, we want to return the SAME message_id as the first. With the ledger, we just look it up and return it. So this works.

    But what if the agent loop has already processed the message and persisted it? Then a duplicate submit would create a second message with the same message_id but different timestamp/sequence. We need to ensure we DON'T send the second message to the agent loop.

    The ledger handles this: on a duplicate, we return the stored outcome without re-sending.

    OK let me implement this. Let me write the new chat_send_handler:

  67. MiniMax Assistant

    Now let me update the chat_new_thread_handler similarly:

  68. MiniMax Assistant

    Now I need to update chat_new_thread_handler to accept a request body and use idempotency. Let me update it:

  69. MiniMax Assistant

    Now I need to import NewThreadRequest in the chat module:

  70. MiniMax Assistant

    Now let me try to build:

  71. MiniMax Assistant

    Now let me check that the existing tests still pass. Let me look at the test for chat_new_thread_handler that needs to be updated:

  72. MiniMax Assistant

    This test uses chat_new_thread_handler with the old signature. I need to update it to use the new signature with Option<Json<NewThreadRequest>>. Let me update this and similar tests:

  73. MiniMax Assistant

    There are many test files that need to be updated. Let me find all of them and fix them:

  74. MiniMax Assistant

    Now let me check the rebuild_state_preserves_fields.rs test:

  75. MiniMax Assistant

    This test doesn't construct GatewayState directly, it uses test_gateway() helper. Let me also check tests/multi_tenant.rs:

  76. MiniMax Assistant

    Now let me check the second occurrence in multi_tenant.rs:

  77. MiniMax Assistant

    Already done. Let me check for any other test files with tool_dispatcher that may need to be updated:

  78. MiniMax Assistant

    I see! The test files (tests/) still have tool_dispatcher: None but no webui_idempotency. Let me check the format more carefully:

  79. MiniMax Assistant

    OK that looks fine. Let me try the build again:

  80. MiniMax Assistant

    I need to update the existing tests that construct SendMessageRequest:

  81. MiniMax Assistant

    Let me update all the test requests:

  82. MiniMax Assistant

    The issue is that the test files are in a different crate context. Let me check the structure:

  83. MiniMax Assistant

    These test files use ironclaw:: prefix. Let me fix all the test files:

  84. MiniMax Assistant

    Compilation succeeded. Now let me run the existing tests to make sure they still pass:

  85. MiniMax Assistant

    All idempotency tests pass. Now let me run the existing chat tests:

  86. MiniMax Assistant

    All 45 tests pass. Now let me add the new tests for the idempotency behavior. Let me write tests that cover:

    1. Same-thread duplicate returns the same accepted ref/outcome
    2. Cross-thread duplicate rejection
    3. Concurrent duplicate behavior
    4. Failed validation does not create a ghost turn/message
    5. Retry after transient unavailable state has deterministic behavior

    Let me look at the test module first to understand the existing structure:

  87. MiniMax Assistant

    Now let me add the new tests. Let me find a good place to add them:

  88. MiniMax Assistant

    Now let me write the tests. I'll insert them before test_chat_history_handler_drops_stale_in_progress_for_completed_turn:

  89. MiniMax Assistant

    Now let me try to build:

  90. MiniMax Assistant

    Let me run the new tests:

  91. MiniMax Assistant

    All 54 tests pass, including the 9 new tests for WebUI idempotency. Let me also run the idempotency module tests:

  92. MiniMax Assistant

    All idempotency tests pass. Let me run all the lib tests to make sure nothing else is broken:

  93. MiniMax Assistant

    There's an unrelated test failure in cli::completion::tests::test_run_generates_output. Let me check if it's related to my changes:

  94. MiniMax Assistant

    This is unrelated to my changes. Let me check if it's pre-existing:

  95. MiniMax Assistant

    This is a pre-existing test failure unrelated to my changes. It's caused by the size of the Cli command structure. Let me verify by checking the original code (git stash):

  96. MiniMax Assistant

    Yes, it's pre-existing. Let me restore my changes:

  97. MiniMax Assistant

    Good. Let me run the specific tests for the chat module to make sure they pass:

  98. MiniMax Assistant

    Excellent! All 54 chat tests pass. Let me also run the idempotency tests and the types tests:

  99. MiniMax Assistant

    All 532 web tests pass! Let me also run the integration tests that may be affected:

  100. MiniMax Assistant

    Bridge tests pass. Let me also check the agent module tests:

  101. MiniMax Assistant

    All 520 agent tests pass. Let me also run the db tests to make sure my changes to types are compatible:

  102. MiniMax Assistant

    All 294 db tests pass. Let me also check the types tests:

  103. MiniMax Assistant

    All 31 types tests pass. Now let me also run the integration tests:

  104. MiniMax Assistant

    Let me also run the integration tests to make sure they pass:

  105. MiniMax Assistant

    All 11 ws_gateway_integration tests pass. Let me also run the multi_tenant integration tests:

  106. MiniMax Assistant

    All 5 oauth_greeting_integration tests pass. Let me also check the support_unit_tests:

  107. MiniMax Assistant

    All 47 support_unit_tests pass. Let me also run the gateway_workflow_integration test:

  108. MiniMax Assistant

    All 18 gateway_workflow_integration tests pass. Let me also run the multi_tenant_integration test:

  109. MiniMax Assistant

    All 42 multi_tenant_integration tests pass. Let me also run the openai_compat_integration tests:

  110. MiniMax Assistant

    All 16 openai_compat_integration tests pass.

    Now let me think about edge cases I should handle:

    1. Memory growth: The ledger grows unbounded. Should I add a cap?
    2. Restart: The ledger is process-local. After a restart, all prior outcomes are forgotten. The next request with the same client_action_id would be processed as new.
    3. Same content with different action types: I handle this with action_kind_mismatch.

    For now, let me add a basic bound on the ledger size to prevent unbounded growth. Actually, let me leave it for now - the issue doesn't specify a requirement for this and adding it would complicate the implementation.

    Let me verify my implementation handles all the acceptance criteria:

    1. ✅ Duplicate send returns the same accepted ref/outcome - Implemented in chat_send_handler and tested by test_chat_send_handler_deduplicates_repeated_client_action_id
    2. ✅ Concurrent duplicate send creates at most one accepted message/turn - Implemented using async Mutex serialization, tested by test_chat_send_handler_concurrent_duplicates_enqueue_once
    3. ✅ Failed validation does not create a ghost turn/message - Implemented by validating client_action_id before any side effect, tested by test_chat_send_handler_validates_client_action_id_before_recording_outcome
    4. ✅ Retry after transient unavailable state has deterministic behavior - Implemented by NOT recording failed outcomes (only successful ones), tested by test_chat_send_handler_retry_after_transient_unavailable_is_not_silently_deduped
    5. ✅ Tests cover same-thread duplicate, cross-thread duplicate rejection, and concurrent duplicate behavior - All covered

    Let me also run the type tests for SendMessageResponse to make sure the deduplicated field serializes correctly:

  111. MiniMax Assistant

    Let me add a quick test for the deduplicated field serialization in types.rs to make sure it's correctly serialized:

  112. MiniMax Assistant

    Let me check the existing types test setup:

  113. MiniMax Assistant

    Let me add some tests for the SendMessageResponse and ThreadInfo deduplicated field serialization:

  114. MiniMax Assistant

    All 38 types tests pass (including the 7 new deduplicated field tests).

    Now let me also check the ledger to handle the edge case where concurrent calls have different responses. Let me look at the current implementation more carefully:

  115. MiniMax Assistant

    Now let me think about one more thing - I want to make sure the rebuild_state actually preserves the ledger. Let me check:

  116. MiniMax Assistant

    Now let me also think about whether I should add a test that confirms rebuild_state preserves the idempotency ledger:

  117. MiniMax Assistant

    Let me add a similar test for the webui_idempotency ledger:

  118. MiniMax Assistant

    Let me check the available builder methods:

  119. MiniMax Assistant

    Let me fix the test - I'll use other with_* methods:

  120. MiniMax Assistant

    Let me fix the type issue:

  121. MiniMax Assistant

    Now let me also think about a few more edge cases:

    1. What if a request has client_action_id but the existing outcome was for a different thread? My implementation handles this correctly with WebUiIdempotencyConflict::ThreadMismatch.

    2. What if the user is different? Currently the ledger is global (not per-user). Let me think...

    Wait, this is an issue! The current implementation shares the ledger across all users. If user A and user B both use client_action_id="action-1", user B's request would be deduped against user A's outcome, which is wrong!

    Let me fix this. The ledger should be keyed by (user_id, client_action_id).

  122. MiniMax Assistant

    Let me update the ledger to be keyed by user + action_id. This is a significant change. Let me think about it:

    Actually, wait - looking at the issue again:

    • "Use client action id / idempotency key for create-thread and send-message."

    The client_action_id is unique per browser action. Different users would naturally have different IDs. The risk is mostly if a single user accidentally reuses an ID across different requests (cross-thread, cross-content).

    But yes, defense in depth would say we should also include the user_id in the key. Let me update the implementation.

    Actually, let me re-think. In the current implementation, the ledger is shared. So:

    • User A submits action_id="abc" with thread="T1"
    • User A submits action_id="abc" again with thread="T1" → matches, replays outcome ✓
    • User B submits action_id="abc" with thread="T2" → matches BUT from User A → incorrectly replays User A's outcome ✗

    This is a bug. Let me fix it.

    Looking at the chat_send_handler signature:

    pub(crate) async fn chat_send_handler(
        State(state): State<Arc<GatewayState>>,
        AuthenticatedUser(user): AuthenticatedUser,
        ...
    )
    

    We have access to user.user_id. Let me update the ledger to be keyed by (user_id, client_action_id).

    Actually, thinking about this more carefully:

    • The frontend generates a fresh UUID for each click → unique per browser action
    • Different browsers / users have different UUIDs
    • So the collision risk is mostly a client bug

    But for safety, let me still include user_id in the key. This is a defense-in-depth measure.

  123. MiniMax Assistant

    Now let me update the existing tests in idempotency.rs:

  124. MiniMax Assistant

    Now let me update the chat_send_handler and chat_new_thread_handler to pass user_id:

  125. MiniMax Assistant

    Let me look at each call site and update them:

  126. MiniMax Assistant

    Now let me look at the record call for chat_send_handler:

  127. MiniMax Assistant

    Now let me update chat_new_thread_handler:

  128. MiniMax Assistant

    Now the record call for chat_new_thread_handler:

  129. MiniMax Assistant

    Now let me also fix the rebuild_state test that uses the ledger:

  130. MiniMax Assistant

    Now let me try to compile:

  131. MiniMax Assistant

    Let me run the tests to verify:

  132. MiniMax Assistant

    All 9 idempotency tests pass. Now let me run the chat tests:

  133. MiniMax Assistant

    All 54 chat tests pass. Let me also run the rebuild_state_preserves_fields test:

  134. MiniMax Assistant

    All tests pass. Let me also add a test that confirms two different users with the same client_action_id don't interfere with each other:

  135. MiniMax Assistant

    Let me find the end of the chat module tests and add a new test for user isolation:

  136. MiniMax Assistant

    Let me add a test for user isolation:

  137. MiniMax Assistant

    All 55 chat tests pass (including the new user isolation test). Let me also run the all tests one more time to ensure everything is good:

  138. MiniMax Assistant

    All 542 web tests pass. Let me now do a final review of my implementation. Let me look at the complete chat_send_handler:

  139. MiniMax Assistant

    Now let me verify all my changes by running the entire test suite one more time:

  140. MiniMax Assistant

    All 539 web tests pass. Now let me also verify the integration tests pass:

  141. MiniMax Assistant

    All integration tests pass. Now let me also do a quick sanity check on a few other related tests:

  142. MiniMax Assistant

    All 1043 channels tests pass. Now let me also run the db tests and agent tests to make sure no regressions:

  143. MiniMax Assistant

    All tests pass. Let me also verify my implementation handles a few more edge cases:

    1. What happens if the ledger grows very large? My current implementation uses an unbounded HashMap. For a production system this should have a cap, but it's outside the scope of this issue.

    2. What happens on server restart? The ledger is process-local and is lost. The next request with the same client_action_id would be processed as new. This is acceptable per the issue requirements ("deterministic behavior" means the retry returns the same outcome if it was previously recorded).

    Let me also do a quick review of the code to make sure the implementation is clean:

  144. MiniMax Assistant

    I notice that I included a deduplicated() method that's not used anywhere. Let me remove it:

  145. MiniMax Assistant

    Let me also verify the tests still pass:

  146. MiniMax Assistant

    All tests pass. Let me also look at the existing tests to ensure I'm not breaking anything else:

  147. MiniMax Assistant

    All 55 chat tests pass. Let me also check the rebuild_state tests:

  148. MiniMax Assistant

    All 2 tests pass. Let me also check the types tests:

  149. MiniMax Assistant

    All 38 types tests pass.

    Let me now also think about one more edge case. The current implementation acquires the Mutex during validate_and_replay and releases it. The recording happens later, separately. This means there's a small race window where two concurrent requests could both pass the validate_and_replay check and both proceed to the mpsc send.

    Let me re-check this...

    Looking at the code:

    1. validate_and_replay acquires the lock, checks, releases
    2. Both concurrent requests pass (both see no prior entry)
    3. Both proceed to the mpsc send (both record outcome after)

    This is a problem. Let me check my concurrent test to see if this is what happens:

    Actually, looking at my concurrent test test_chat_send_handler_concurrent_duplicates_enqueue_once, it expects exactly ONE message to be enqueued. The test is passing. So either:

    1. The race is rare enough that the test happens to pass
    2. The Mutex acquisition + check + release happens fast enough that the second request sees the recorded entry

    Looking at my implementation more carefully:

    • validate_and_replay acquires the lock, checks, releases
    • tx.send(msg).await is the slow operation (not under lock)
    • record acquires the lock, inserts, releases

    So there's a window where the second request could enter validate_and_replay BEFORE the first request calls record. In that case, both would pass the check.

    But my test has 8 concurrent requests. With high concurrency, at least some should fall into this race window. Yet the test is passing...

    Hmm, let me think about why this is the case:

    Looking at my test:

    let mut handles = Vec::new();
    for _ in 0..8 {
        let state = Arc::clone(&state);
        let user = user();
        let req = mk_req();
        handles.push(tokio::spawn(async move {
            chat_send_handler(...).await
        }));
    }
    

    The tokio::spawn creates tasks that may not all start at the same time. The first one might complete the entire handler before the second one starts validate_and_replay. So the race window is very small.

    Actually, my test verifies exactly 1 fresh acceptance and 7 replays. So the test IS sensitive to the race. Let me look at why it passes consistently...

    Wait, looking more carefully, I see that validate_and_replay returns Ok(None) for a miss. Then the code falls through to the rate limiter check, the mpsc send, and the record call. So the race window is:

    1. Request A: validate_and_replay returns None (no entry)
    2. Request B: validate_and_replay returns None (no entry) ← happens here if Request A hasn't called record yet
    3. Request A: tx.send succeeds
    4. Request A: record(outcome)
    5. Request B: tx.send succeeds
    6. Request B: record(outcome) ← overwrites Request A's outcome

    This is a real race condition. My test passes because the timing is right, but it's not guaranteed.

    Let me fix this by ensuring the Mutex is held during the entire critical section. Actually, that would block other unrelated requests. Let me think of a better approach...

    Actually, I think the right approach is to use the "reserve" pattern:

    1. validate_and_replay tries to reserve the action_id (atomically: check + mark as "pending")
    2. If reservation succeeded, perform the operation
    3. If reservation failed (already reserved or already done), return the existing outcome

    Let me look at the existing product workflow ledger for inspiration:

  150. MiniMax Assistant

    Yes, the existing ledger uses a begin_or_replay + settle pattern. Let me apply the same pattern.

    Actually, looking at this more carefully, I think there's a simpler approach: use the async Mutex and hold it during the entire critical section. The mpsc send is fast, so the contention should be minimal.

    Let me update the implementation:

  151. MiniMax Assistant

    Actually, looking at the code, I realize the issue. Looking at my validate_and_replay, it acquires the lock, checks, and releases. Then the caller has a window to do other things before calling record.

    Let me restructure the API to make the critical section atomic. The simplest approach is to add a method that does check-and-reserve atomically:

  152. MiniMax Assistant

    Now let me update the chat_send_handler to use the new reserve_or_replay + settle/release pattern:

  153. MiniMax Assistant

    Hmm, actually let me reconsider. The current implementation with validate_and_replay + record may be fine if I:

    1. Don't worry about the race window because the mpsc send is fast
    2. Trust that the second request would record its own outcome

    Actually wait - let me re-read the concurrent test. The race is:

    1. A: validate_and_replay → None
    2. B: validate_and_replay → None (if A hasn't recorded yet)
    3. A: tx.send succeeds, record(outcome)
    4. B: tx.send succeeds, record(outcome) - OVERWRITES A's outcome

    The issue is that the second submission's outcome overrides the first's. This means subsequent submissions would get B's message_id, not A's.

    But for the M2 acceptance criteria, we need "Concurrent duplicate send creates at most one accepted message/turn". So we need to ensure only ONE message gets enqueued.

    The current implementation may have a race where both A and B pass the check, both enqueue messages, and B overwrites the outcome. This violates the "at most one accepted message" requirement.

    Let me think about whether my test actually catches this...

    Looking at my test:

    let fresh = responses
        .iter()
        .filter(|(_, body)| !body.deduplicated)
        .count();
    assert_eq!(fresh, 1, ...);
    
    let spurious = tokio::time::timeout(std::time::Duration::from_millis(50), rx.recv())
        .await;
    assert!(spurious.is_err(), "concurrent duplicates must NOT enqueue more than one message");
    

    The test asserts:

    1. Exactly one fresh acceptance
    2. No second message on the channel

    If both A and B passed the check and both enqueued, the test would fail.

    But the test is passing! Why?

    Let me think... Oh, I see. When both A and B pass the check, both will eventually try to record. Whichever records LAST will have its outcome stored. But both have already enqueued. So we'd see TWO messages on the channel.

    Hmm, maybe the timing in my test is such that the race window never opens. The test runs many times, so it's likely the race happens at least sometimes.

    Wait, let me look at the test again. The task spawns happen sequentially. By the time we collect all responses (which awaits all handles), all tasks have completed. So the test does run all 8 tasks concurrently, but the actual race window is between validate_and_replay and record for each task.

    The race window is:

    1. Task A enters handler
    2. Task A: validate_and_replay → None
    3. Task A: ... (rate limit, msg construction)
    4. Task A: tx.send.await
    5. Task A: record(outcome) ← records here

    The race window for Task B is between steps 2 and 5. If Task B enters between these steps, B would see None too.

    In practice, with 8 concurrent tasks, the OS may schedule them such that all 8 enter validate_and_replay before any of them completes. Then all 8 see None, and all 8 enqueue.

    Let me actually verify this with a test that runs many iterations:

  154. MiniMax Assistant

    The test passes consistently. But this might be because the Mutex makes the operations effectively serialized. Let me check by running many iterations and looking for flakiness:

    Actually, looking at the tokio Mutex behavior - it's an async mutex that allows other tasks to make progress while waiting. So when 8 tasks spawn and try to acquire the Mutex:

    • Task A acquires the lock, sees None, releases
    • Task B acquires the lock, sees None, releases
    • ...

    Wait, my validate_and_replay only acquires the lock once for the check. The race is real.

    Let me actually verify this is a real issue. Let me update the handler to use the new atomic API and see if the test still passes:

  155. MiniMax Assistant

    Wait, I realize my new implementation has a problem. The "wait for settle" approach is racy and complex. Let me reconsider.

    Actually, a much simpler approach is to use a more robust reservation pattern. The issue with my current approach is that I'm checking the result of reserve_or_replay, but the result is Some(placeholder) if there's an in-flight reservation, and Some(outcome) if settled. Both look the same to the caller.

    Let me simplify. The original implementation with validate_and_replay + record actually works because:

    1. validate_and_replay returns None (slot free)
    2. caller performs side effect
    3. caller calls record(outcome)

    If two concurrent calls happen:

    1. A: validate_and_replay returns None
    2. B: validate_and_replay returns None (before A's record)
    3. A: side effect (mpsc send)
    4. A: record(outcome_A)
    5. B: side effect (mpsc send) - BUG: TWO MESSAGES ENQUEUED
    6. B: record(outcome_B) - BUG: outcome_B overrides outcome_A

    So the race is real. Let me revert and use a simpler approach - just ensure the slot is reserved during the entire critical section.

    Actually, looking at this more carefully, I think the simpler approach is to just hold the Mutex during the entire critical section (including the mpsc send). The mpsc send is fast and the Mutex is async so other unrelated operations can proceed.

    Let me revert my changes and try this approach:

  156. MiniMax Assistant

    Actually, let me take a different approach. Let me look at the issue more carefully:

    The M2 acceptance criterion is: "Concurrent duplicate send creates at most one accepted message/turn."

    Looking at the original implementation, I have:

    1. validate_and_replay (checks ledger, returns None if not found)
    2. mpsc send (the side effect)
    3. record (writes to ledger)

    The race window is between steps 1 and 3. If two concurrent requests both pass step 1, both do step 2 (mpsc send), both do step 3 (record).

    To make this work correctly, I need to ensure that the second request sees the first request's recorded outcome.

    The simplest approach: serialize the entire critical section under the Mutex. This means holding the Mutex during the mpsc send. But that blocks other operations.

    Actually, the issue with my reserve_or_replay approach is that I have a wait loop for the placeholder to be replaced. That's racy.

    Let me think differently. The cleanest approach is:

    1. Try to reserve the slot with a sentinel
    2. If reservation succeeded, perform the operation
    3. After the operation, replace the sentinel with the real outcome

    The issue is: when a second request arrives while the first is in flight, it should wait for the first to complete (or fail) and then either replay the first's outcome or proceed itself.

    Actually, let me re-think this whole thing. The race window is:

    • Both requests pass the check
    • Both proceed to mpsc send
    • Both succeed and record

    The first to record wins (its message_id is what future requests see). The second has already enqueued its own message, so we have two messages on the channel.

    To prevent this, we need to ensure the second request waits for the first to complete (or knows the first failed) before deciding whether to enqueue.

    My current reserve_or_replay does exactly this! When the second request arrives, it sees the placeholder (nil message_id) and waits in a loop. When the first request completes (settles), the second sees the real message_id and replays.

    But the wait loop is fragile - what if the first request fails to settle? The second would wait forever (or until the timeout).

    A better approach: have the first request release the slot on failure. Then the second request, after the timeout, would see no slot and could try to reserve again.

    Actually, let me think simpler. The issue with my current implementation:

    A: reserve_or_replay → None (slot reserved with placeholder)
    A: mpsc send
    A: settle(real outcome) ← key step
    
    B: reserve_or_replay → Some(placeholder)  ← B sees the placeholder
    B: wait_for_settle → loops until placeholder is replaced
    

    If A's mpsc send fails:

    • A calls release(placeholder) - slot removed
    • B's wait_for_settle loops until... it should detect that the slot is gone

    Currently my wait loop doesn't handle that. Let me fix it.

    Actually, let me step back and think. The simplest, most correct approach is:

    1. Use a per-action_id Mutex that serializes ALL requests for the same action_id
    2. The first request acquires the Mutex, does the work, releases
    3. The second request acquires the Mutex, sees the recorded outcome, replays

    Let me implement this:

  157. MiniMax Assistant

    The output is wrong - my string search returned a wrong start position. Let me look at the actual file:

  158. MiniMax Assistant

    Wait, there are still uses of validate_and_replay in the file - that's from the chat_new_thread_handler which I haven't updated yet. Good. Let me look at the chat_send_handler specifically:

  159. MiniMax Assistant

    OK my changes look complex. Let me simplify the approach. Let me revert to the original implementation but use a different strategy:

    Strategy: Use the Mutex to serialize all ledger operations. The key insight is that the mpsc send is fast and the record happens AFTER. So:

    1. Hold the Mutex during check + send + record
    2. Other unrelated operations can proceed because the Mutex is async

    Actually, that doesn't work either because holding the Mutex during tx.send().await would block other unrelated operations that need to look up the ledger.

    Let me think again...

    Actually, the simplest approach: use a separate Mutex per action_id (or per record). The race window is small but real. Let me just accept the race window risk and not over-engineer this.

    Wait, looking at my test again:

    let mut handles = Vec::new();
    for _ in 0..8 {
        handles.push(tokio::spawn(...));
    }
    

    This spawns 8 tasks. Each task:

    1. Enters chat_send_handler
    2. Calls validate_and_replay (acquires Mutex, checks, releases - FAST)
    3. Various other operations (rate limit, msg construction)
    4. Calls tx.send().await (mpsc send - VERY FAST for a channel)
    5. Calls record (acquires Mutex, writes, releases)

    The window between validate_and_replay and record is small but non-zero. With 8 concurrent tasks, it's likely that some tasks will fall into this window and race.

    Let me actually verify this with my test by running it 100 times:

  160. MiniMax Assistant

    Hmm, the build takes too long. Let me try just 5 iterations:

  161. MiniMax Assistant

    There's an error. Let me check what it is:

  162. MiniMax Assistant

    Let me add the import:

  163. MiniMax Assistant

    I need to make req mutable. Let me think about this differently.

    Actually, looking at this, my implementation is getting too complex. Let me revert to the simpler approach (validate_and_replay + record) which was actually working correctly per the test results. The race window is small enough that it doesn't trigger in practice for the test, and the implementation is much cleaner.

    Let me revert:

  164. MiniMax Assistant

    Let me approach this differently - I'll just rewrite the chat_send_handler completely:

  165. MiniMax Assistant

    Let me write a simpler, more direct replacement:

  166. MiniMax Assistant

    OK, let me take a different approach. I'll write a complete rewrite using a more straightforward pattern. Let me first see the complete current state of the file:

  167. MiniMax Assistant

    Now let me replace the whole chat_send_handler from line 122 onwards. Let me find a precise start and end:

  168. MiniMax Assistant

    OK so the chat_send_handler is lines 124-384 (approximately). Let me find the exact ending line:

  169. MiniMax Assistant

    Let me extract the chat_send_handler and rewrite it:

  170. MiniMax Assistant

    Now let me create a new simpler implementation:

  171. MiniMax Assistant

    Now let me replace the old handler with the new one:

  172. MiniMax Assistant

    Let me fix the error type:

  173. MiniMax Assistant

    Now let me run the tests:

  174. MiniMax Assistant

    The retry test is failing because we now have a reserve_or_replay that reserves the slot during the first call (with msg_tx=None), which fails to send but releases the slot. Let me check:

  175. MiniMax Assistant

    The test is checking:

    1. First submit (msg_tx=None) → expect 503
    2. Retry (msg_tx=Some) → expect fresh acceptance (deduplicated=false)
    3. Third submit → expect deduplicated=true

    The issue: In my new implementation, the first submit reserves the slot in reserve_or_replay. But then the rate limiter doesn't fail because msg_tx is read AFTER reserve_or_replay. So we proceed to the mpsc send. The mpsc send fails (no tx). We release the slot. Good.

    But wait, let me trace through:

    1. First submit: reserve_or_replay succeeds → slot reserved
    2. Rate limit passes
    3. msg_tx is None → 503 returned
    4. But wait, my new code releases the slot ONLY in the tx.send error handler, not in the 503 (no msg_tx) error handler!

    Let me fix this:

  176. MiniMax Assistant

    I see the problem. When msg_tx is None, we get a 503 BEFORE releasing the slot. Let me fix this:

  177. MiniMax Assistant

    All 55 tests pass! Let me also update the chat_new_thread_handler to use the same pattern:

  178. MiniMax Assistant

    For chat_new_thread_handler, the race window is similar but more complex. Let me also update it to use reserve_or_replay. But wait, the placeholder for CreateThread has Uuid::nil() for thread_id. When we get the placeholder back, we don't know the real thread_id yet. So the wait-and-replace approach doesn't directly work.

    Actually for create-thread, the test verifies:

    1. First call creates the thread (and returns deduplicated=false)
    2. Second call replays the same ThreadInfo (deduplicated=true)

    The current implementation uses validate_and_replay + record. With concurrent calls, the race is:

    • Both pass validate_and_replay (no entry)
    • Both create threads
    • Both record outcomes
    • Last write wins

    This would create TWO threads in the session. Let me check the test for concurrent:

    Actually the test test_chat_send_handler_concurrent_duplicates_enqueue_once is for send. There's no concurrent test for create-thread. But the requirement still applies.

    For now, let me keep create-thread as is (validate_and_replay + record) since:

    1. The race window is real but small
    2. The behavior under concurrent race would be: two threads created, last one's thread_id wins in the ledger
    3. Both threads exist in the session - this is technically a "ghost thread" but it's harmless because no messages were sent

    Actually wait, let me re-read the requirements:

    • "Concurrent duplicate send creates at most one accepted message/turn."
    • "Failed validation does not create a ghost turn/message."

    The requirement is specifically about turns/messages. A thread without messages is not a "turn" or "message". So having two threads created is technically not violating the requirement.

    But it would be ugly. Let me apply the same reserve_or_replay pattern to create-thread for consistency:

  179. MiniMax Assistant

    Now let me replace the chat_new_thread_handler:

  180. MiniMax Assistant

    Hmm, this is confusing. Let me look at the actual order:

  181. MiniMax Assistant

    OK so chat_new_thread_handler is from line 1136 to the next function. Let me find the end:

  182. MiniMax Assistant

    So chat_new_thread_handler ends at line 1308 (with }) followed by the helpers section. Let me extract the new handler and replace:

  183. MiniMax Assistant

    All 55 tests pass. Let me also run the full channels::web tests to make sure nothing else is broken:

  184. MiniMax Assistant

    All 539 tests pass. Let me also add a concurrent test for chat_new_thread_handler to ensure the new reserve_or_replay pattern works correctly:

  185. MiniMax Assistant

    Let me find the end of test_chat_new_thread_handler_deduplicates_repeated_client_action_id:

  186. MiniMax Assistant

    Let me add a concurrent test for chat_new_thread_handler between lines 3404 and 3507:

  187. MiniMax Assistant

    Let me add the concurrent test for chat_new_thread_handler right before the history test:

  188. MiniMax Assistant

    All 56 tests pass (including the new concurrent test for chat_new_thread_handler). Let me also check the integration tests:

  189. MiniMax Assistant

    All integration tests pass. Now let me also verify the channels::web tests once more:

  190. MiniMax Assistant

    All 540 tests pass. Let me also run the rebuild_state_preserves_fields tests to make sure my changes to the ledger don't break the existing test:

  191. MiniMax Assistant

    All 2 tests pass. Now let me also make sure the concurrent test runs multiple times to ensure it's stable:

  192. MiniMax Assistant

    Both tests pass consistently across 3 runs each. Let me now also verify the implementation handles all the acceptance criteria.

    Let me check the actual file once more to make sure it's clean:

  193. MiniMax Assistant

    Good. Now let me also verify my implementation handles the edge case where the request body has no client_action_id at all (the original behavior should be preserved):

  194. MiniMax Assistant

    Test passes - the original behavior is preserved when no client_action_id is provided.

    Let me also do a quick check that everything compiles cleanly:

  195. MiniMax Assistant

    Build is clean. Now let me also do a final review of the implementation. Let me look at the final chat_send_handler:

  196. MiniMax Assistant

    Now let me also verify the issue checklist:

    Checklist verification

    If an error was reported in the issue description, I have successfully reproduced it.

    • The issue described the requirement to add idempotency to WebUI mutations. I added tests that reproduce the lack of idempotency behavior:
      • test_chat_send_handler_deduplicates_repeated_client_action_id - verifies duplicate sends return same ref
      • test_chat_send_handler_rejects_cross_thread_duplicate - verifies cross-thread rejection
      • test_chat_send_handler_concurrent_duplicates_enqueue_once - verifies concurrent dedup
      • test_chat_new_thread_handler_deduplicates_repeated_client_action_id - same for thread/new

    If an error was reported in the issue description, I have confirmed that my fix resolves the error.

    • All 9 idempotency-specific tests pass, plus the existing tests still pass.
    • All 5 acceptance criteria from the issue are met:
      1. ✅ Duplicate send returns the same accepted ref/outcome
      2. ✅ Concurrent duplicate send creates at most one accepted message/turn
      3. ✅ Failed validation does not create a ghost turn/message
      4. ✅ Retry after transient unavailable state has deterministic behavior
      5. ✅ Tests cover same-thread duplicate, cross-thread duplicate rejection, and concurrent duplicate behavior

    I have thought about edge cases and made sure my fix handles them as well.

    • Edge cases I handled:
      1. User isolation: Two users with the same client_action_id don't interfere (added per-user keying in ledger + test)
      2. Cross-thread/content/action-kind rejection: Returns 409 instead of silent overwrite
      3. Validation errors don't record outcomes: Returns 400 without recording
      4. Retry after transient: 503 doesn't record, retry can re-attempt
      5. Concurrent dups: Reserve-or-replay pattern + wait-and-replay
      6. Malformed client_action_id: blank/oversize/control chars → 400
      7. Rebuild state preserves ledger: tested

    I have run existing tests in the repository that might have been affected by the change I made and confirmed that they pass.

    • All 540 channels::web tests pass
    • All integration tests pass (ws_gateway, oauth_greeting, multi_tenant, support_unit, gateway_workflow, openai_compat)
    • All db tests pass (294)
    • All agent tests pass (497)
    • All types tests pass (38)
    • All idempotency tests pass (9)

    Let me run one more final verification to make sure everything is in order:

  197. MiniMax Assistant

    All 540 tests pass. Let me also verify the idempotency file looks clean:

  198. MiniMax Assistant

    Let me also do a final cleanup of the test code for the placeholder sentinel - the placeholder has thread_id stored but the message_id is nil. Let me trace through to make sure this works for the test:

    When the second submit arrives:

    1. reserve_or_replay returns Some(MessageAccepted { thread_id: Some("T"), content_digest: ..., message_id: Uuid::nil() })
    2. The code checks if message_id.is_nil() → true (because first hasn't settled)
    3. Wait loop with 5ms intervals
    4. Eventually first settles with message_id = real uuid
    5. Second reads from lookup and gets the real outcome

    Now let me also run the integration tests one more time to make sure nothing is broken:

  199. MiniMax Assistant

    All 540 tests pass. Let me also check if there are any clippy warnings I should fix:

  200. MiniMax Assistant

    No clippy warnings related to my changes. Let me also check that the build is fully successful:

  201. MiniMax Assistant

    All builds successfully. Let me also do a final sanity check that my modifications work correctly with the existing integration tests:

  202. MiniMax Assistant

    All tests pass. Let me also do a quick check on the overall summary of changes:

  203. MiniMax Assistant

    Now let me also do one final verification of my implementation against the issue requirements. Let me re-read the issue:

    Make WebUI mutations idempotent so duplicate browser submits do not create duplicate turns/messages.

    Scope

    • Use client action id / idempotency key for create-thread and send-message.
    • Preserve accepted-message ref/outcome for duplicate requests.
    • Handle concurrent duplicate submissions safely.

    Acceptance criteria

    • Duplicate send returns the same accepted ref/outcome.
    • Concurrent duplicate send creates at most one accepted message/turn.
    • Failed validation does not create a ghost turn/message.
    • Retry after transient unavailable state has deterministic behavior.
    • Tests cover same-thread duplicate, cross-thread duplicate rejection, and concurrent duplicate behavior.

    Interface

    No new interfaces are introduced.

    Let me verify each:

    1. ✅ Use client action id / idempotency key for create-thread and send-message.

      • SendMessageRequest.client_action_id field added
      • NewThreadRequest.client_action_id field added
    2. ✅ Preserve accepted-message ref/outcome for duplicate requests.

      • SendMessageResponse.deduplicated: bool field added
      • ThreadInfo.deduplicated: bool field added
      • WebUI idempotency ledger records the outcome
    3. ✅ Handle concurrent duplicate submissions safely.

      • reserve_or_replay uses atomic Mutex
      • Wait-and-replay pattern for concurrent submits
    4. ✅ Duplicate send returns the same accepted ref/outcome.

      • Tested by test_chat_send_handler_deduplicates_repeated_client_action_id
    5. ✅ Concurrent duplicate send creates at most one accepted message/turn.

      • Tested by test_chat_send_handler_concurrent_duplicates_enqueue_once
      • Tested by test_chat_new_thread_handler_concurrent_duplicates_create_one_thread
    6. ✅ Failed validation does not create a ghost turn/message.

      • Tested by test_chat_send_handler_validates_client_action_id_before_recording_outcome
      • Tested by test_chat_new_thread_handler_validates_client_action_id_before_creating_thread
    7. ✅ Retry after transient unavailable state has deterministic behavior.

      • Tested by test_chat_send_handler_retry_after_transient_unavailable_is_not_silently_deduped
    8. ✅ Tests cover same-thread duplicate, cross-thread duplicate rejection, and concurrent duplicate behavior.

      • Same-thread: test_chat_send_handler_deduplicates_repeated_client_action_id
      • Cross-thread: test_chat_send_handler_rejects_cross_thread_duplicate
      • Concurrent: test_chat_send_handler_concurrent_duplicates_enqueue_once
      • Cross-content: test_chat_send_handler_rejects_cross_content_duplicate
      • Cross-action-kind: test_chat_send_and_create_thread_share_no_client_action_id
      • User isolation: test_chat_send_handler_isolates_records_per_user
    9. ✅ No new interfaces are introduced.

      • Only new types are WebUiIdempotencyLedger, WebUiAcceptedOutcome, WebUiActionKind, WebUiIdempotencyConflict, ContentDigest, NewThreadRequest
      • These are internal implementation details, not external interfaces

    Let me also verify the imports are clean:

  204. MiniMax Assistant

    Let me verify the final tests one more time:

  205. MiniMax Assistant

    All 9 idempotency tests pass. Let me also check the channels::web tests:

  206. MiniMax Assistant

    All 10 chat_send_handler tests pass (including all the new idempotency tests). Let me also check the chat_new_thread_handler tests:

  207. MiniMax Assistant

    All 4 chat_new_thread_handler tests pass. Let me also run a final overall check:

  208. MiniMax Assistant

    All 543 tests pass! My implementation is complete and meets all the acceptance criteria.

    Summary

    I successfully implemented WebUI idempotency and the accepted-message ledger for the IronClaw Reborn WebUI as required by issue #3607 (M2 Inbound Facade).

    Files Modified/Created

    Created:

    • src/channels/web/idempotency.rs - WebUI idempotency ledger module containing:
      • WebUiIdempotencyLedger - Thread-safe per-user ledger
      • WebUiAcceptedOutcome - Stored outcomes for send-message and create-thread
      • WebUiIdempotencyConflict - Reasons for rejecting duplicate ids
      • ContentDigest - Cheap FNV-1a content fingerprint for cross-thread/content validation
      • WebUiActionKind - Action kind discriminator

    Modified:

    • src/channels/web/types.rs - Added client_action_id field to SendMessageRequest, created NewThreadRequest, added deduplicated field to SendMessageResponse and ThreadInfo
    • src/channels/web/features/chat/mod.rs - Updated chat_send_handler and chat_new_thread_handler to use the idempotency ledger with the reserve_or_replay + settle/release pattern
    • src/channels/web/platform/state.rs - Added webui_idempotency field to GatewayState
    • src/channels/web/mod.rs - Added idempotency module, preserved ledger in rebuild_state
    • All test helpers, integration tests, and platform files updated to include the new field

    Implementation Approach

    1. Validation First: client_action_id is validated (non-blank, ≤256 bytes, no control chars) before any side effect.
    2. Reserve-or-Replay Pattern: reserve_or_replay atomically reserves a slot under the ledger's async Mutex. The first submitter proceeds with the side effect; concurrent or duplicate submitters wait briefly for the slot to settle, then replay the same outcome.
    3. Per-User Scoping: Records are keyed by (user_id, client_action_id) to prevent cross-user leakage.
    4. Failure Recovery: Transient failures (503, channel closed) release the slot so retries can re-attempt.
    5. Cross-Thread/Content Rejection: Reusing the same client_action_id with different params returns 409 with a stable conflict code.
    6. Rebuild State Preservation: rebuild_state preserves the ledger Arc so in-flight records survive subsystem wiring.

    Tests Added (16 total)

    Ledger unit tests (9): record/lookup, cross-user isolation, cross-thread rejection, cross-content rejection, cross-action-kind rejection, create-thread replay, digest stability, separator collision prevention.

    Handler tests (7): same-thread dedup, cross-thread rejection, cross-content rejection, validation-before-recording, retry-after-transient, concurrent dups (send), concurrent dups (create-thread), user isolation, cross-action-kind.

Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.