Rivertalk/river_connect/.agent/rules/audio-recording-streaming-design.md

trigger, description

trigger	description
model_decision	When working on the voxer-like audio recording, chunking, streaming and playback setup

Audio Recording & Streaming Architecture

This project implements a "Voxer-like" audio messaging system using Phoenix Channels for binary streaming and Oban for background processing.

1. Components & Data Flow

• Capture: AudioRecorder JS Hook (assets/js/hooks/audio_recorder.js) uses MediaRecorder with audio/webm;codecs=opus.
• Transport: Raw binary chunks are pushed via Phoenix Channels (audio:lobby).
• Persistence: Server appends chunks to a .webm file in priv/static/uploads/.
• Processing: audio_end event triggers an Oban job (RiverConnect.Workers.ProcessRecording) to finalize and update status.
• Serving: Files are served from /uploads/ via a dedicated Plug.Static entry in RiverConnectWeb.Endpoint.

2. Channel Protocol (AudioChannel)

• audio_start: %{"id" => id}. Initializes the recording session and creates a temp file.
• audio_chunk: {:binary, payload}. Appends binary to the file and broadcasts to other listeners for live playback.
• audio_end: %{"duration_ms" => ms, "id" => id}. Creates the AudioMessage DB record and enqueues the Oban worker.

3. Real-time Playback

• While a recording is in progress, the channel broadcasts audio_chunk events.
• The receiving AudioRecorder hook uses AudioContext.decodeAudioData to play these chunks immediately, providing a "near-realtime" walkie-talkie experience.

4. Database Schema (AudioMessage)

• user_id: String (Guest ID or User ID).
• file_path: Relative path (e.g., /uploads/uuid.webm).
• duration_ms: Total recording duration.
• status: "processing" | "ready".
• Note: The schema must @derive {Jason.Encoder, ...} to be sent over Phoenix Channels.

5. Environment Constraints (Windows)

• Commands like mix may need to be executed via cmd /c or with explicit PATH to erl.exe and mix.bat in backgrounds tasks.
• Static files in priv/static/uploads must be exposed explicitly in endpoint.ex and may require a server restart to be served correctly after initial creation.

6. Development Tips

• Restart Required: Adding new sockets or Plug.Static rules in endpoint.ex requires a full mix phx.server restart.
• Permissions: Ensure the browser has microphone permissions for the specific port (default 4000/4005).
• Streaming Jitter: Basic playback uses decodeAudioData. For production-grade streaming without gaps, a jitter buffer/audio scheduler would be required.