Media Reference

This document is the complete working reference for Media: the video item's data model, the upload and transcoding lifecycle, renditions and captions, the library's organizing tools, the playback and stream-signing scheme, the AI layers, and the DVR and Restream data models.

It is the technical companion to Media, which explains the concepts and workflows. Read that first if you are new to the module; come here when you need the exact behavior of a field, a key, or a lifecycle. The AI transcription and dubbing tools that run on these same videos have their own reference: see Dubbing.

Table of Contents

1. Overview
2. The Video Item
3. Upload & Processing
4. Renditions
5. Captions & Transcripts
6. Organizing the Library
7. Playback & Streaming
8. AI on Media
9. DVR
10. Restream
11. Background Tasks
12. Storage & Key Map
13. Access & Permissions

1. Overview

Media is three surfaces over one backend, the HMS streaming platform at hms.pfs.gdn:

The console is a control surface. HMS ingests and transcodes uploads, records live channels into time-shiftable buffers, signs and delivers streams, and executes restream pushes. The dashboard reads and writes the records that describe this work and surfaces its status; it does not transcode, record, or push bytes itself.

Everything is scoped to the active project (project IN $CURRENT_USER.projects.project_id) and shared by that project's team.

2. The Video Item

A video is a row in the DUS media collection (numeric primary key). The operator-facing fields:

Rights-management fields (youtubeContentId, facebookCopyrightId, and their *Status companions) also exist for content-ID workflows but are outside the dashboard's editing surface.

The media collection has no flat width/height/fps/codec columns. Technical specs are read from sourceRawMetadata (the ffprobe video/audio streams), which is why a freshly uploaded video shows no resolution until probing completes.

3. Upload & Processing

Upload

Uploads are resumable and chunked (TUS, 50 MB chunks, auto-start), accepting video/*, audio/*, and .mxf. They run through a global upload panel that persists across navigation; a dropped connection resumes from the last chunk. Entry points: the library toolbar's upload button, and full-page drag-and-drop onto the list.

On a successful upload the dashboard creates the media row with the raw source only:

{ title: <filename>, sourceFileUrl, sourceFileName, sourceFileSize,
  sourceFileLastModified, mediaStatus: 'uploaded', project }

Everything else — the HMS key, duration, sourceRawMetadata, thumbnail, renditions, and captions — is filled in asynchronously by the HMS pipeline. There is no transcoding code in the console.

The mediaStatus lifecycle

mediaStatus tracks the pipeline, separately from the editorial status:

new (record exists, no source) → uploaded / uploading → transcoding / transferring → ready / completed, with error on failure and reprocess to re-run.

Two "processing" signals

The UI gates on processing in two independent ways — worth distinguishing when diagnosing:

4. Renditions

Each transcoded output is a row in the mediaFormats collection, a child of media:

5. Captions & Transcripts

Closed captions and transcript tracks are rows in the mediaVtt collection:

Caption tracks are produced by the HMS pipeline. They are also the input to AI chapter generation, which reads every cc-* track for a video, ordered by order (§8).

6. Organizing the Library

The list

The library renders as a responsive grid of cards. List state (search, status, sort, page, size) is reflected in the URL so a view is shareable; default page size is 50. A 30-second visibility-gated poll watches the item count and raises a new content banner (Refresh / Dismiss) when it changes.

• Search matches title, description, and key (case-insensitive), plus an exact numeric id match.
• Status filter defaults to "not archived" (status ≠ archived); can pin to an exact status or show all.

Tags vs programmable tags

An advancedTags entry carries { name, notes, config, status, project } (lowercase names recommended) and is reused across videos.

Chapters (segments)

Each chapter is { startTime, segmentTitle, segmentTitleAr }:

Chapters are added manually (a row seeds its start from the current player time) or generated by AI (§8); each row plays-from-here and locks/edits independently.

Intro markers

introStartTime / introEndTime (HH:MM:SS) bound the intro for skip-intro behavior; each has a "use current player time" control in the editor.

Bulk edit

A selection can be edited at once across status, season, episode, tags, and programmable tags. Array fields (tags) support add / remove / replace modes; the change is applied per selected video with a success/partial-failure summary.

7. Playback & Streaming

Signed stream URLs

On-demand playback is authorized per-play. The player asks the server to sign the video's key, and signStream returns a short-lived HLS URL:

https://hms.pfs.gdn/stream/{signReference}.m3u8?token={token}

The token is an AES-256-CBC encryption (random 16-byte IV; URL-safe base64) of a signature object:

The signing material — signKey and signReference — comes from the project's HMS feature configuration. Because the signature expires in five minutes, the player re-signs on demand rather than caching a URL.

A second mode, direct URL, plays a fully-formed URL (e.g. an authenticated source file) straight into the player, bypassing both signing and the transcode gate. This is what the picture-in-picture player uses for direct files.

Storyboard scrubbing

The player fetches a sprite-sheet manifest at:

https://hms.pfs.gdn/hms/v1/ondemand/{mediaKey}/storyboard.vtt

Each cue is a sprite with an #xywh= crop; hovering or seeking the progress bar renders the matching crop as a thumbnail. It is progressive enhancement — silent on any failure, and skipped in direct-URL mode.

Embeds and shortcuts

• Embed / open-in-new-tab: https://hms.pfs.gdn/embed/{key}.
• Keyboard: ,/. step ±1 s · Space/k play-pause · m mute · f fullscreen · ←/→ ±5 s · j/l ±10 s · Home/End start/end.
• Picture-in-picture: any video pops into a floating, draggable player that survives navigation. Volume is shared across every player.

8. AI on Media

Extractions

On ingest, the HMS pipeline analyzes each video into a structured summary stored in the project key/value store under description:{mediaId}. The dashboard reads it read-only (a missing record is simply absent):

Extractions are surfaced in a collapsible panel on the detail page and are a prerequisite for chapter generation (below).

Chapter generation

Generation turns a video's caption transcript into bilingual chapters. The flow:

1. Read the video's key and duration.
2. Gather every cc-* track from mediaVtt for that mediaKey, ordered by order, and join the body text.
3. Render the chapter prompt (parameterized by video duration) and call the language model with a strict JSON response contract.
4. Validate each returned chapter for startTime, segmentTitle, segmentTitleAr and write them to media.segments.

The model is instructed to start the first chapter at 00:00:00, target 2–8 minute chapters (~5–10 for a 30-minute video), write specific titles, and translate to Modern Standard Arabic (translation, not transliteration), chronological with no gaps.

Gating: the control is available only when the video is longer than 5 minutes and extractions exist. It runs as a background task keyed generateSegments:{mediaId} (§11); on completion the chapter editor is replaced with the result (after a confirmation when chapters already exist).

Deeper AI (Inference)

Editable transcription (speaker-diarized speech-to-text) and full dubbing (translation, synthetic voices, audio mixing, translated graphics, optional lip-sync) operate on these same media items but live in the Inference product, with their own data model and tasks. See Dubbing.

9. DVR

What a DVR channel is

A DVR channel is a live channel that HMS records into a time-shiftable buffer. It is read-only in the console — there is no create/edit/delete UI; channels are provisioned on HMS. Each channel is a key/value record (dvr:*) whose value an operator-side process populates:

DVR records live in a shared system project (77b81e67-9054-4d5e-b770-0d5e8fc07c5e) — the same store that holds per-user preferences — and are queried by the dvr: key prefix, not from the active workspace project.

The list

Channels render in a 1/2/3-column grid sorted by key. Thumbnails are kept current by refreshing one channel every 3 seconds in rotation (a cache-busting ?t= timestamp), and the rotation pauses entirely while the browser tab is hidden.

Card preview and playback

A card whose record carries playbackHls previews live on hover: after a 300 ms delay the card enlarges and plays the low-latency stream with sound (falling back to muted autoplay if the browser blocks audio). Clicking a card opens /media/dvr/{key}, whose detail page is the HMS embed player in an iframe. All seek, scrub, and time-shift controls over the recorded window come from that HMS player, not from the console.

10. Restream

Sources

A restream's input is a liveSources row — a live channel already ingested into HMS. Only sources with status: published are selectable.

The internalPlayback scheme is transformed for the preview player: m4s://host/path → https://host/path/index.m3u8 (thumbnail …/preview.jpg); hlss://host/path/index.m3u8 → the same https://…/index.m3u8 (thumbnail …/preview.jpg).

Restreams

A restream is a liveRestream row:

Creating a restream

A single-step dialog: choose a source (previewed live), name it, pick a platform, and supply the credentials. The console composes the platform-correct push URL:

On read, the platform is detected back from the destination URL (by hostname). A stream key that starts with rtmp is rejected as a likely pasted-URL mistake.

The list and lifecycle

Restreams split into Running (anything not archived) and Recent (archived). A running card shows the source → destination flow with a connection indicator, a live-ticking duration, and the worker's serverStatus. The broadcast itself is executed by the HMS worker — the console only writes the record and mirrors state. Stopping a restream sets status: archived (there is no delete and no in-place restart); it moves to Recent. One restream targets one destination, so fanning a source out to several platforms means several restreams.

11. Background Tasks

Long-running AI work runs through a shared NATS task system. The media-relevant jobs are chapter generation (generateSegments) and transcription (transcribeMedia, the ASR/Inference entry point that operates on a media item).

Lifecycle:

1. The client creates a task; the request is authorized, the per-type validator runs with the caller's token, a deterministic task key is derived, a pending progress record is written to tasks:{type}:{key}, and a message is published to NATS.
2. The client polls tasks:{type}:{key} every 2 seconds until completed/failed, skipping fetches while the tab is hidden, and re-attaching to an already-running task on mount.
3. The consumer routes the message to its handler, patches the record to processing, reports progress as it runs, and finally patches completed (with a result) or failed. A separate reaper marks tasks stuck without progress for ~5 minutes as failed.

Task keys are deterministic per item and stage — generateSegments:{mediaId} — so anyone viewing the same video sees the same in-flight task, and re-requesting a running stage attaches rather than starting a second one.

12. Storage & Key Map

13. Access & Permissions

Media access is governed by DUS policies, all project-scoped (project IN $CURRENT_USER.projects.project_id):

liveRestream allows create/read/update but not delete — which is why stopping is implemented as archiving. liveSources is read-only from the console: sources are ingested and managed on HMS.

For the concepts and workflows behind these models, see Media.