Video streaming app
A clinical platform for recording, anonymizing, and reviewing therapy sessions.
A web application for psychologists to capture therapy sessions, review anonymized media, edit transcriptions, and tag clinical behavior. The frontend runs a full media pipeline, not just a data-entry screen: live capture, anonymization, synchronized review.
Session flow follows assignment status. An Active session opens the recording interface. Later statuses unlock the review workbench with streaming playback, editable transcripts, and behavioral annotations stored as structured data.
Recording starts only when both video and audio streams are live, so a half-connected session never leaves clinicians with incomplete capture.
Therapy video and audio reach anonymization workers in real time over two independent WebSocket channels, one for picture, one for sound. Each stream authenticates with a JWT on connect, then sends data through a structured protocol: a start signal, frame or audio chunks, and a stop signal when the session ends.
Video is grabbed from a getUserMedia feed via Canvas at ~15 FPS as JPEG and transmitted as base64. Recording begins only when both channels report ready. If either service fails to connect, the session never enters a half-recorded state.
Continuous microphone streaming would waste bandwidth and hurt transcription quality downstream. Voice activity detection runs in the browser via ONNX Runtime (`@ricky0123/vad-web`). Only speech segments are converted to PCM, queued, and sent serially, each tagged with a precise start timestamp.
That keeps audio aligned with the video timeline, reduces load on anonymization and speech-to-text services, and avoids dumping silence into the pipeline.
After processing, clinicians need playback synced with an editable transcript and behavioral tags. The review workbench uses Video.js with an HLS playlist and a remote WebVTT caption track. The active cue highlights as the video plays, so reading and watching stay in sync.
A dual timeline supports both point annotations and time-range tags, with seek, frame-step, and play-until-range-end controls. Edits to the transcript and tag boundaries persist as structured records with millisecond precision.
REST calls, live streams, and protected media URLs all need the same identity, but each channel accepts credentials differently. A Swagger-generated typed client handles API requests with a bearer token. WebSocket connections authenticate with a token as the first message. HLS playlists and VTT files load via token query parameters, because Video.js cannot set custom headers.
The frontend unifies these patterns behind a single session model, so clinicians move from recording to review without re-authenticating or hitting broken media URLs mid-playback.
Closer to a clinical media tool than a standard admin panel: WebSockets and getUserMedia for real-time capture, on-device VAD for speech gating, and a Video.js + HLS annotation workspace with synced WebVTT captions. The frontend is the control plane for a privacy-by-architecture loop from session to anonymized review.