sequenceDiagram participant CE as Cache Engine participant SM as StorageManager…
- User
sequenceDiagram participant CE as Cache Engine participant SM as StorageManager participant LCPU as LocalCPUBackend participant RB as Remote Backend (e.g., DiskBackend) participant LS as Lookup Server
%% Store Operation CE->>CE: store(tokens, mask) Note over CE: Process tokens and generate keys loop For each key CE->>SM: contains(key) SM->>RB: contains(key) RB-->>SM: Return boolean alt Key not already in storage CE->>SM: allocate(kv_shape, kv_dtype) SM->>LCPU: allocate(shape, dtype) LCPU-->>SM: Return MemoryObj SM-->>CE: Return MemoryObj CE->>CE: gpu_connector.from_gpu(memory_obj, start, end) Note over CE: Copy KV cache from GPU to CPU memory CE->>SM: put(key, memory_obj) SM->>RB: submit_put_task(key, memory_obj) Note over RB: Asynchronously store data alt P2P enabled CE->>LS: insert(key) Note over LS: Update lookup server with key end end end %% Retrieve Operation CE->>CE: retrieve(tokens, mask) Note over CE: Process tokens and generate keys loop For each key CE->>SM: get(key) alt Key in prefetch tasks SM->>SM: Wait for prefetch task to complete end loop For each backend SM->>RB: get_blocking(key) alt Key found in backend RB-->>SM: Return MemoryObj alt Backend is not LocalCPUBackend SM->>LCPU: write_back(key, memory_obj) Note over LCPU: Cache data in local CPU memory end SM-->>CE: Return MemoryObj CE->>CE: gpu_connector.to_gpu(memory_obj, start, end) Note over CE: Copy KV cache from CPU to GPU CE->>CE: memory_allocator.ref_count_down(memory_obj) break Exit backend loop end RB-->>SM: Return None Note over SM: Continue to next backend end alt P2P enabled and key not found locally CE->>CE: Run distributed_server.issue_get(key) Note over CE: Try to get from P2P network end end %% Prefetch Operation CE->>CE: prefetch(tokens, mask) Note over CE: Process tokens and generate keys loop For each key CE->>SM: prefetch(key) alt Key not in LocalCPUBackend and not already prefetching loop For each backend SM->>RB: submit_prefetch_task(key) alt Backend can prefetch RB-->>SM: Return Future SM->>SM: Add callback to Future Note over SM: Will process when prefetch completes break Exit backend loop end RB-->>SM: Return None Note over SM: Continue to next backend end end end %% Prefetch Callback (happens asynchronously) RB-->>SM: Prefetch task completes SM->>SM: prefetch_callback(future, key) SM->>LCPU: submit_put_task(key, memory_obj) Note over LCPU: Cache prefetched data in local CPU memory修复一下这个mermaid
Error: Parse error on line 56: ...in local CPU memory ----------------------^ Expecting 'SPACE', 'NEWLINE', 'create', 'box', 'end', 'autonumber', 'activate', 'deactivate', 'title', 'legacy_title', 'acc_title', 'acc_descr', 'acc_descr_multiline_value', 'loop', 'rect', 'opt', 'alt', 'par', 'par_over', 'critical', 'break', 'participant', 'participant_actor', 'destroy', 'note', 'links', 'link', 'properties', 'details', 'ACTOR', got '1'
Snippets
- Assistant
下面是修复后的 Mermaid
sequenceDiagram,主要改动是在多单词或包含特殊字符的参与者别名外面加上了双引号(否则空格和括号会导致解析错误):sequenceDiagram participant CE as "Cache Engine" participant SM as "StorageManager" participant LCPU as "LocalCPUBackend" participant RB as "Remote Backend (e.g., DiskBackend)" participant LS as "Lookup Server" %% Store Operation CE->>CE: store(tokens, mask) Note over CE: Process tokens and generate keys loop For each key CE->>SM: contains(key) SM->>RB: contains(key) RB-->>SM: Return boolean alt Key not already in storage CE->>SM: allocate(kv_shape, kv_dtype) SM->>LCPU: allocate(shape, dtype) LCPU-->>SM: Return MemoryObj SM-->>CE: Return MemoryObj CE->>CE: gpu_connector.from_gpu(memory_obj, start, end) Note over CE: Copy KV cache from GPU to CPU memory CE->>SM: put(key, memory_obj) SM->>RB: submit_put_task(key, memory_obj) Note over RB: Asynchronously store data alt P2P enabled CE->>LS: insert(key) Note over LS: Update lookup server with key end end end %% Retrieve Operation CE->>CE: retrieve(tokens, mask) Note over CE: Process tokens and generate keys loop For each key CE->>SM: get(key) alt Key in prefetch tasks SM->>SM: Wait for prefetch task to complete end loop For each backend SM->>RB: get_blocking(key) alt Key found in backend RB-->>SM: Return MemoryObj alt Backend is not LocalCPUBackend SM->>LCPU: write_back(key, memory_obj) Note over LCPU: Cache data in local CPU memory end SM-->>CE: Return MemoryObj CE->>CE: gpu_connector.to_gpu(memory_obj, start, end) Note over CE: Copy KV cache from CPU to GPU CE->>CE: memory_allocator.ref_count_down(memory_obj) break Exit backend loop end RB-->>SM: Return None Note over SM: Continue to next backend end alt P2P enabled and key not found locally CE->>CE: Run distributed_server.issue_get(key) Note over CE: Try to get from P2P network end end %% Prefetch Operation CE->>CE: prefetch(tokens, mask) Note over CE: Process tokens and generate keys loop For each key CE->>SM: prefetch(key) alt Key not in LocalCPUBackend and not already prefetching loop For each backend SM->>RB: submit_prefetch_task(key) alt Backend can prefetch RB-->>SM: Return Future SM->>SM: Add callback to Future Note over SM: Will process when prefetch completes break Exit backend loop end RB-->>SM: Return None Note over SM: Continue to next backend end end end %% Prefetch Callback (happens asynchronously) RB-->>SM: Prefetch task completes SM->>SM: prefetch_callback(future, key) SM->>LCPU: submit_put_task(key, memory_obj) Note over LCPU: Cache prefetched data in local CPU memory主要修复点:
- 参与者别名(
as后面)如果含有空格或括号,需要用双引号括起来。 - 保证所有
loop/alt块都有对应的end。
- 参与者别名(
Other models, same or similar prompt
Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.