Mission: Design and implement a scalable, extensible “D‑and‑D Cutscene Audio/Video…
- User
Mission: Design and implement a scalable, extensible “D‑and‑D Cutscene Audio/Video Manager”—a browser‑based timeline editor that lets Dungeon Masters assemble atmospheric cut‑scenes from any mix of audio tracks and video clips.
Core user goals Multi‑file import – drag‑and‑drop any number of audio and video files; each file appears as a draggable “clip.”
Dual‑lane timeline – one lane for audio, one for video, sharing a common time‑ruler.
Clip editing – split, trim, resize, slip‑edit, duplicate, group, multi‑select, undo/redo; edges snap to grid or neighbour clips.
Clip roles – any clip or clip‑group can be tagged Intro, Loop, Outro.
Playback engine – • space‑bar preview that honours the current timeline layout • seamless intro → loop → outro flow; Loop may repeat until the DM triggers Outro • cross‑fade at all segment boundaries
Export – • project JSON for later re‑load • WAV mix‑down of the audio lane • MP4 mux (mixed audio + arranged video lane) via WebAssembly/ffmpeg
Quality of life – zoom, snap toggle, keyboard shortcuts, role‑coloured clips, theming, accessibility, i18n hooks, performance on >30 tracks.
Engineering expectations Framework freedom – you may pick the front‑end stack (React/Redux, Vue, Svelte, vanilla, etc.) but justify the choice for scalability and maintainability.
Modular architecture – design with clear domain layers (UI / state / audio‑engine / export), dependency injection where sensible, plugin points for future effects or MIDI triggers.
Type safety – favour TypeScript or equivalent.
Worker off‑loading – long media operations (decode, mix‑down, ffmpeg) should not block the main thread.
Testing strategy – unit tests for utilities, integration tests for timeline ops, and e2e playback sanity checks (e.g. Playwright).
Tooling – lint, prettier, commit hooks, CI pipeline.
Documentation – high‑level architecture diagram, public API docs for plugin authors, and an onboarding README.
Deliverables Solution outline – chosen tech stack, core libraries (waveform‑playlist, Tone.js, ffmpeg.wasm, etc.), high‑level component diagram, data models, and build pipeline description.
Incremental roadmap – sprint plan from MVP (load/preview a single audio clip) to full feature‑set.
Source code – organised in a mono‑repo (e.g. PNPM workspaces) with lint/test scripts, ready to run via pnpm install && pnpm dev.
Sample project files – demo JSON scene, example cutscene assets, and a short ‘getting started’ guide.
(Optional stretch) – plug‑in SDK example: add a real‑time reverb effect or MIDI foot‑switch trigger.
Important: This prompt is spec‑only—do not output production code until asked; focus first on architecture and planning to maximise extensibility and developer velocity.
- Assistant
🎲 D‑and‑D Cutscene Audio/Video Manager
High‑level Architecture & Delivery Plan (spec‑only)
1. Solution Outline
1.1 Tech‑stack & Rationale
Concern Choice Why UI layer React 18 + Vite Mature ecosystem, Suspense for async decoding, huge 3rd‑party component pool, easy DOM+Canvas mixing State mgmt Zustand (or Redux‑Toolkit if team already fluent) Tiny API, no boilerplate, good middle‑ground between React‑state and Redux; works with immer for immutability & undo/redo Rendering React‑Konva Canvas for timeline, plain DOM for panels Canvas handles 30‑40 tracks at 60 FPS; React‑Konva gives React props/hydration Audio engine Tone.js atop Web Audio High‑level transport, precise scheduling, effects chain, easy cross‑fade envelopes Video playback HTMLVideo element grid (hidden) + <canvas>compositingKeeps sync with Tone.js via requestVideoFrameCallback & shared clock Export ffmpeg.wasm (web‑worker) Pure browser, no installs; mux WAV + video lane to MP4 Type‑safety TypeScript everywhere Compile‑time safety, self‑doc Testing Jest + React‑Testing‑Library (unit/integration) ; Playwright (e2e) Full pyramid, CI‑friendly Package mgmt PNPM monorepo workspace isolation, fast Lint/format ESLint, Prettier, Husky + lint‑staged Enforced quality Core libs:
• drag‑and‑drop: @dnd‑kit/core
• waveform drawer: wavesurfer.js (audio thumbnails)
• hotkeys: hotkeys‑js
• i18n: i18next
• theming: styled‑components + theme‑provider
1.2 Modular Architecture
Mermaid overview:
flowchart LR subgraph UI TimelineCanvas -->|events| UIStore InspectorPanel --> UIStore TransportBar --> UIStore AssetBin --> UIStore end UIStore -- actions --> DomainCore subgraph DomainCore ProjectState((Project State)) UndoRedo Selection SnapEngine end DomainCore -- schedule --> AudioEngine DomainCore -- schedule --> VideoEngine AudioEngine -. JSON .-> ExportWorker VideoEngine -. frames .-> ExportWorker ExportWorker -->|MP4| FileSaverLayer responsibilities:
- UI (React): pure visual & user input; no business logic.
- DomainCore (
@cutscene/core): pure TS models, timeline algorithms, undo/redo. Zero browser APIs -> testable & node‑runnable. - Engines (
@cutscene/audio,@cutscene/video): WebAudio + Media; subscribe to DomainCore commands/events. - Workers (
/workers/decode,/workers/export): heavy lifting (decoding, waveforms, ffmpeg). - Plugin API: events + DI container exposes hooks (onTransportStart, onRenderFrame…) + shared utils.
Dependency rule: UI 👉 DomainCore 👉 Engines 👉 Workers.
1.3 Key Data Models (simplified TS)
// clip.ts export type Role = 'Intro' | 'Loop' | 'Outro' | 'None'; export interface Clip { id: UUID; fileId: UUID; // points to Asset lane: 'audio' | 'video'; start: number; // timeline position (sec) duration: number; // visible length offset: number; // in‑file start role: Role; groupId?: UUID; } // project.ts export interface Project { id: UUID; name: string; bpm?: number; // grid snap option assets: Record<UUID, AssetMeta>; clips: Record<UUID, Clip>; groups: Record<UUID, ClipGroup>; settings: ProjectSettings; }Undo/redo is implemented via immer‑patches snapshot queue (time‑travel friendly).
1.4 Build & Tooling Pipeline
pnpm installpnpm dev→ Vite serves/apps/editor(HMR, worker bundling).- Pre‑commit: lint, type‑check, unit tests under 10 s.
- CI (GitHub Actions):
•pnpm ci:test(all unit/integration)
•pnpm ci:e2e(Playwright headed‑false)
•pnpm ci:build(production build, storybook static) - Storybook published via GitHub Pages for components.
2. Incremental Roadmap
Sprint (2 w) Goals & Acceptance criteria 0 – Setup Repo scaffolding, CI, TS strict, monorepo packages, Storybook. 1 – MVP‑Audio • Drag‑drop single audio file. • Clip appears on audio lane. • Space‑bar plays from cursor. 2 – Timeline Core • Multi‑file import. • Move/trim/resize. • Zoom & snap toggle. • Undo/redo 10 steps. 3 – Roles & Looping • Apply Intro/Loop/Outro tags. • Transport logic: Intro→Loop→Outro w/ cross‑fades. 4 – Video Lane • Add video lane & rendering in sync. • Basic trimming & resizing. 5 – Groups & Multi‑select • Shift‑click selection, group/ungroup, duplicate. 6 – Export‑Audio • WAV mix‑down via worker (offline Tone.js) + FileSaver. 7 – Export‑AV • ffmpeg.wasm mux to MP4. • Progress UI, cancel button. 8 – QoL polish • Keyboard shortcuts, coloured roles, theming support, waveforms. 9 – Accessibility & i18n • ARIA for timeline, preferred‑language JSON. 10 – Perf & Stress • 30 + clips benchmark, virtualised rendering, off‑screen canvas. 11 – Plugin SDK (stretch) • Public typings, example “Reverb” & “MIDI foot‑switch” plugins. 12 – Docs & Launch • Architecture diagram, API docs, quick‑start guide, sample project assets.
3. Sample File Formats
// project.cutscene.json { "schema": "dnd-cutscene/v1", "meta": { "name": "Goblin Ambush", "fps": 30 }, "assets": { "a01": { "type": "audio", "name": "forest_loop.wav", "size": 2342345 }, "v01": { "type": "video", "name": "campfire.mp4", "poster": "campfire.jpg" } }, "clips": { "c01": { "fileId": "a01", "lane": "audio", "start": 0, "duration": 6, "offset": 0, "role": "Intro" }, "c02": { "fileId": "a01", "lane": "audio", "start": 6, "duration": 30, "offset": 6, "role": "Loop" }, "c03": { "fileId": "v01", "lane": "video", "start": 0, "duration": 36, "offset": 0, "role": "None" } } }
4. Extensibility Hooks
Plugins get injected with:
export interface PluginCtx { project: Readonly<Project>; on(event: 'transport:start' | 'transport:stop', cb: Fn): VoidFn; registerEffect(nodeFactory: (ctx: Tone.Context) => Tone.AudioNode): string; // returns id }Plugins are ESM bundles dropped into
/pluginsand lazy‑loaded.
5. Next Steps
- Confirm stack & sprint cadence with stakeholders.
- Spin up repo (
pnpm init -w). - Begin Sprint 0 tasks – CI green within week 1.
🤘 Ready to roll initiative!
Other models, same or similar prompt
Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.
Identical first prompt
gemini-2.0-flash-001Compare side by side →
qwen-max-2025-01-25Compare side by side →
gemma-3-27b-itCompare side by side →
- gpt-4o-mini-2024-07-18Compare side by side →
gemini-2.5-pro-preview-03-25Compare side by side →
claude-3-5-haiku-20241022Compare side by side →
gemini-2.0-flash-001Compare side by side →
- gpt-4o-mini-2024-07-18Compare side by side →
- o4-mini-2025-04-16Compare side by side →
- o3-2025-04-16Compare side by side →
claude-3-7-sonnet-20250219Compare side by side →