built basic ui structure in /hierarchy

2026-02-14 18:03:13 -06:00
parent d645bda942
commit e4e0775472
24 changed files with 1174 additions and 106 deletions
@@ -0,0 +1,40 @@
+---
+trigger: model_decision
+description: When working on the voxer-like audio recording, chunking, streaming and playback setup
+---
+
+# Audio Recording & Streaming Architecture
+
+This project implements a "Voxer-like" audio messaging system using Phoenix Channels for binary streaming and Oban for background processing.
+
+## 1. Components & Data Flow
+- **Capture**: `AudioRecorder` JS Hook (`assets/js/hooks/audio_recorder.js`) uses `MediaRecorder` with `audio/webm;codecs=opus`.
+- **Transport**: Raw binary chunks are pushed via Phoenix Channels (`audio:lobby`).
+- **Persistence**: Server appends chunks to a `.webm` file in `priv/static/uploads/`.
+- **Processing**: `audio_end` event triggers an **Oban** job (`RiverConnect.Workers.ProcessRecording`) to finalize and update status.
+- **Serving**: Files are served from `/uploads/` via a dedicated `Plug.Static` entry in `RiverConnectWeb.Endpoint`.
+
+## 2. Channel Protocol (`AudioChannel`)
+- **`audio_start`**: `%{"id" => id}`. Initializes the recording session and creates a temp file.
+- **`audio_chunk`**: `{:binary, payload}`. Appends binary to the file and broadcasts to other listeners for live playback.
+- **`audio_end`**: `%{"duration_ms" => ms, "id" => id}`. Creates the `AudioMessage` DB record and enqueues the Oban worker.
+
+## 3. Real-time Playback
+- While a recording is in progress, the channel broadcasts `audio_chunk` events.
+- The receiving `AudioRecorder` hook uses `AudioContext.decodeAudioData` to play these chunks immediately, providing a "near-realtime" walkie-talkie experience.
+
+## 4. Database Schema (`AudioMessage`)
+- `user_id`: String (Guest ID or User ID).
+- `file_path`: Relative path (e.g., `/uploads/uuid.webm`).
+- `duration_ms`: Total recording duration.
+- `status`: `"processing"` | `"ready"`.
+- **Note**: The schema must `@derive {Jason.Encoder, ...}` to be sent over Phoenix Channels.
+
+## 5. Environment Constraints (Windows)
+- Commands like `mix` may need to be executed via `cmd /c` or with explicit PATH to `erl.exe` and `mix.bat` in backgrounds tasks.
+- Static files in `priv/static/uploads` must be exposed explicitly in `endpoint.ex` and may require a server restart to be served correctly after initial creation.
+
+## 6. Development Tips
+- **Restart Required**: Adding new sockets or `Plug.Static` rules in `endpoint.ex` requires a full `mix phx.server` restart.
+- **Permissions**: Ensure the browser has microphone permissions for the specific port (default 4000/4005).
+- **Streaming Jitter**: Basic playback uses `decodeAudioData`. For production-grade streaming without gaps, a jitter buffer/audio scheduler would be required.