mirror of
https://github.com/streamyfin/streamyfin.git
synced 2026-06-02 12:08:37 +01:00
docs(casting): add device profiles & capability detection design spec
Design for sub-project A of the Chromecast refactor: per-device capability detection, profile builder replacing the two static profiles, unified cast load, and load-path bug fixes. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This commit is contained in:
Uruk
249
docs/superpowers/specs/2026-05-21-chromecast-profiles-design.md
Normal file
249
docs/superpowers/specs/2026-05-21-chromecast-profiles-design.md
Normal file
@@ -0,0 +1,249 @@
|
||||
# Chromecast Device Profiles & Capability Detection — Design
|
||||
|
||||
**Date:** 2026-05-21
|
||||
**Branch:** `refactor-chromecast` (PR #1402)
|
||||
**Sub-project:** A (cornerstone) of the Chromecast refactor
|
||||
**Status:** Approved design — pending implementation plan
|
||||
|
||||
---
|
||||
|
||||
## 1. Problem
|
||||
|
||||
Casting to Chromecast crashes for a large class of media:
|
||||
|
||||
- `Media control channel status code 2100` (LOAD_FAILED) on movies and high-bitrate media.
|
||||
- 5.1 / multichannel audio crashes the receiver (issue #1085).
|
||||
- Casting reportedly only works at video bitrate ≤ 2 Mb/s (issue #1423).
|
||||
- Movies fail to load entirely; some media loads forever.
|
||||
|
||||
**Root cause:** the sender ships a device profile that does not match the actual
|
||||
Chromecast. Two near-identical static profiles (`utils/profiles/chromecast.ts`,
|
||||
`utils/profiles/chromecasth265.ts`) are selected by a single global boolean
|
||||
`enableH265ForChromecast`. A 1st/2nd/3rd-gen Chromecast (H.264-only, 1080p, 2-channel
|
||||
audio, no HEVC) can receive an HEVC / 10-bit / high-bitrate stream it cannot decode,
|
||||
producing status 2100.
|
||||
|
||||
Secondary defects in the load path:
|
||||
|
||||
- `getStreamUrl` defaults `audioStreamIndex` to `0`, which is the video stream — wrong
|
||||
audio track selection.
|
||||
- `getStreamUrl` always takes `MediaSources[0]`, ignoring a requested `mediaSourceId`.
|
||||
- Progress reporting uses `mediaInfo.contentId` (the item id) as `PlaySessionId` — the
|
||||
Jellyfin dashboard cannot correlate the session.
|
||||
- The load sequence (`getStreamUrl` → `buildCastMediaInfo` → `client.loadMedia`) is
|
||||
duplicated in three places: `PlayButton.tsx`, and `casting-player.tsx`
|
||||
(`reloadWithSettings` and `loadEpisode`).
|
||||
- `loadMedia` failures are swallowed into `console.error` — the user sees nothing.
|
||||
|
||||
## 2. Scope
|
||||
|
||||
**In scope**
|
||||
|
||||
- Per-device Chromecast capability detection.
|
||||
- A profile builder replacing the two static profile files.
|
||||
- A single unified cast-load function.
|
||||
- The load-path bug fixes listed above.
|
||||
- Settings migration from `enableH265ForChromecast` to an explicit profile mode.
|
||||
- Failure resilience (downgrade-on-failure retry, user-visible errors).
|
||||
- A test matrix to empirically calibrate conservative defaults.
|
||||
|
||||
**Out of scope** (later sub-projects)
|
||||
|
||||
- Audio / subtitle / quality track switching — sub-project B.
|
||||
- Splitting the 52 KB `casting-player.tsx` — sub-project C.
|
||||
- Episode navigation, remote controller, UX features — sub-project D.
|
||||
- Custom Cast receiver (PR #1521) — separate later sub-project.
|
||||
|
||||
## 3. Approach
|
||||
|
||||
Approach 3 of the brainstorm: a **profile builder** driven by a **capability
|
||||
registry** with a **conservative default**, plus **advanced override settings**.
|
||||
|
||||
Detection produces smart defaults; power users can override. An unknown device always
|
||||
falls back to a safe baseline (H.264 / 1080p / 2 channels) that cannot crash the
|
||||
receiver. The user's test hardware (Chromecast 1/2/3 HD) maps exactly onto that
|
||||
baseline.
|
||||
|
||||
## 4. Architecture
|
||||
|
||||
### New modules
|
||||
|
||||
| File | Responsibility |
|
||||
|---|---|
|
||||
| `utils/casting/capabilities.ts` | `ChromecastCapabilities` type, model registry, `detectCapabilities(device, settings)` |
|
||||
| `utils/casting/buildProfile.ts` | `buildChromecastProfile(caps)` → Jellyfin `DeviceProfile` |
|
||||
| `utils/casting/castLoad.ts` | `loadCastMedia(...)` — unified load, used by all call sites |
|
||||
|
||||
### Removed
|
||||
|
||||
- `utils/profiles/chromecast.ts`
|
||||
- `utils/profiles/chromecasth265.ts`
|
||||
|
||||
### Modified
|
||||
|
||||
- `components/PlayButton.tsx` — use `loadCastMedia`.
|
||||
- `app/(auth)/casting-player.tsx` — `reloadWithSettings` and `loadEpisode` use `loadCastMedia`.
|
||||
- `utils/atoms/settings.ts` — replace `enableH265ForChromecast`.
|
||||
- Chromecast settings UI (`ChromecastSettings.tsx` / `ChromecastSettingsMenu.tsx`) — new controls.
|
||||
|
||||
## 5. Capability model
|
||||
|
||||
```ts
|
||||
export interface ChromecastCapabilities {
|
||||
/** HEVC 8-bit (Main profile) decode support. */
|
||||
hevc: boolean;
|
||||
/** HEVC 10-bit (Main10) decode support. */
|
||||
hevc10bit: boolean;
|
||||
/** Maximum video resolution height. */
|
||||
maxResolution: 1080 | 2160;
|
||||
/** Maximum video bitrate in bits/second. */
|
||||
maxVideoBitrate: number;
|
||||
/** Maximum audio channels the receiver can output. */
|
||||
maxAudioChannels: number;
|
||||
}
|
||||
```
|
||||
|
||||
### Conservative default
|
||||
|
||||
Used for any unrecognised device. Equals a 1st/2nd/3rd-gen Chromecast:
|
||||
|
||||
```ts
|
||||
const CONSERVATIVE: ChromecastCapabilities = {
|
||||
hevc: false,
|
||||
hevc10bit: false,
|
||||
maxResolution: 1080,
|
||||
maxVideoBitrate: 8_000_000, // initial guess — calibrated by the test matrix
|
||||
maxAudioChannels: 2,
|
||||
};
|
||||
```
|
||||
|
||||
> `maxVideoBitrate` starts at 8 Mb/s. Issue #1423 (≤ 2 Mb/s) is suspicious — 2 Mb/s is
|
||||
> below any real hardware limit and likely points to a transcode-readiness or HLS
|
||||
> issue rather than a bitrate ceiling. The test matrix (§9) confirms the true value
|
||||
> before this number is finalised.
|
||||
|
||||
### Registry
|
||||
|
||||
Keyed by `device.modelName`. Initial entries:
|
||||
|
||||
- `"Chromecast"` — gen 1/2/3 → conservative baseline.
|
||||
- `"Chromecast Ultra"` — HEVC, 2160p, higher bitrate, multichannel passthrough.
|
||||
- `"Chromecast with Google TV"` — HEVC, HEVC 10-bit.
|
||||
- `"Google TV Streamer"` — HEVC, HEVC 10-bit, 2160p.
|
||||
|
||||
`modelName` is ambiguous (gen 1/2/3 all report `"Chromecast"`, and Google TV devices
|
||||
may report a TV model string). This is acceptable: the registry only *upgrades* away
|
||||
from the conservative baseline for confidently-identified devices; everything else
|
||||
stays safe.
|
||||
|
||||
### `detectCapabilities(device, settings)`
|
||||
|
||||
1. Look up `device.modelName` in the registry; fall back to `CONSERVATIVE`.
|
||||
2. Apply advanced settings overrides:
|
||||
- `chromecastProfile: "force-hevc"` → `hevc: true`.
|
||||
- `chromecastProfile: "force-h264"` → `hevc: false, hevc10bit: false`.
|
||||
- `chromecastMaxBitrate` set → clamp `maxVideoBitrate`.
|
||||
3. Return the merged capabilities.
|
||||
|
||||
## 6. Profile builder
|
||||
|
||||
`buildChromecastProfile(caps)` returns a Jellyfin `DeviceProfile`:
|
||||
|
||||
- **Video codecs:** `h264` always; `hevc` only when `caps.hevc`. When `hevc` is
|
||||
enabled but `caps.hevc10bit` is false, add a codec condition rejecting 10-bit so the
|
||||
server transcodes 10-bit sources down.
|
||||
- **Audio:** both `DirectPlayProfiles` and `CodecProfiles` constrained to
|
||||
`caps.maxAudioChannels`. This forces multichannel audio to transcode to stereo —
|
||||
fixes the 5.1 crash (#1085). The current profiles only constrain `CodecProfiles`,
|
||||
leaving a direct-play path open.
|
||||
- **Bitrate:** `MaxStreamingBitrate` / `MaxStaticBitrate` = `caps.maxVideoBitrate`.
|
||||
- **Resolution:** codec condition capping height at `caps.maxResolution`.
|
||||
- Subtitle and container profiles preserved from the current working profile.
|
||||
|
||||
## 7. Unified cast load
|
||||
|
||||
```ts
|
||||
loadCastMedia({
|
||||
client, // RemoteMediaClient
|
||||
device, // Cast Device — for capability detection
|
||||
api, item, userId,
|
||||
settings,
|
||||
options: { audioStreamIndex?, subtitleStreamIndex?, maxBitrate?, mediaSourceId?, startPositionMs? },
|
||||
}): Promise<{ ok: true } | { ok: false; error }>
|
||||
```
|
||||
|
||||
Sequence:
|
||||
|
||||
1. `detectCapabilities(device, settings)` → `buildChromecastProfile`.
|
||||
2. `getStreamUrl` with the built profile.
|
||||
3. `buildCastMediaInfo`.
|
||||
4. `client.loadMedia`.
|
||||
|
||||
Replaces all three duplicated call sites. Bug fixes folded in:
|
||||
|
||||
- **Audio index:** when `options.audioStreamIndex` is absent, resolve the media
|
||||
source's `DefaultAudioStreamIndex` (or the first `Audio`-type stream) instead of
|
||||
defaulting to `0`.
|
||||
- **Media source:** pass and honour `mediaSourceId`; select the matching
|
||||
`MediaSource` rather than `[0]`.
|
||||
- **PlaySessionId:** use the real `PlaySessionId` returned by `getStreamUrl`'s
|
||||
`getPlaybackInfo` call, not `contentId`.
|
||||
|
||||
## 8. Settings
|
||||
|
||||
Replace `enableH265ForChromecast: boolean` with:
|
||||
|
||||
| Setting | Type | Default |
|
||||
|---|---|---|
|
||||
| `chromecastProfile` | `"auto" \| "force-hevc" \| "force-h264"` | `"auto"` |
|
||||
| `chromecastMaxBitrate` | `number \| undefined` | `undefined` |
|
||||
|
||||
**Migration:** existing `enableH265ForChromecast === true` → `chromecastProfile:
|
||||
"force-hevc"`; `false` or unset → `"auto"`.
|
||||
|
||||
UI: a profile-mode selector and an optional max-bitrate field in the Chromecast
|
||||
settings screen. `"auto"` is presented as the recommended default; the overrides are
|
||||
advanced controls.
|
||||
|
||||
## 9. Resilience
|
||||
|
||||
- **Downgrade-on-failure:** if `client.loadMedia` rejects with status 2100, retry
|
||||
exactly once with a forced conservative profile (`force-h264`, 2 channels, low
|
||||
bitrate). Handles unknown / mis-detected devices gracefully. A second failure
|
||||
surfaces to the user.
|
||||
- **User-visible errors:** a failed load shows an alert with an actionable message
|
||||
instead of a silent `console.error`.
|
||||
|
||||
## 10. Verification — test matrix
|
||||
|
||||
A document, `docs/chromecast-test-matrix.md`, lists sample-file casting tests run
|
||||
against the Chromecast HD test device:
|
||||
|
||||
| Dimension | Values |
|
||||
|---|---|
|
||||
| Video codec | H.264, HEVC 8-bit, HEVC 10-bit |
|
||||
| Audio | AAC stereo, AAC 5.1, AC3 5.1, DTS, TrueHD |
|
||||
| Container | MP4, MKV, TS |
|
||||
| Bitrate | low, ~8 Mb/s, ~16 Mb/s, source-max |
|
||||
|
||||
Each row records: direct-play vs transcode, load result (OK / 2100 / infinite), notes.
|
||||
Results calibrate `CONSERVATIVE.maxVideoBitrate` and confirm the true cause of #1423
|
||||
and the 5.1 crash before the conservative defaults are finalised.
|
||||
|
||||
## 11. Success criteria
|
||||
|
||||
- A movie that previously failed with 2100 plays on the Chromecast HD.
|
||||
- 5.1 / multichannel media plays (transcoded to stereo) without crashing.
|
||||
- High-bitrate media plays or transcodes cleanly within the calibrated limit.
|
||||
- Audio track defaults to the correct (non-video) stream.
|
||||
- The Jellyfin dashboard shows a correctly correlated cast session.
|
||||
- The load sequence exists in exactly one place.
|
||||
- `bun run typecheck` and `bun run check` pass.
|
||||
|
||||
## 12. Risks
|
||||
|
||||
- The true cause of #1423 / the 5.1 crash is not yet confirmed; the test matrix
|
||||
(§10) is the gate before defaults are locked. If the cause is HLS handling rather
|
||||
than a profile mismatch, the design extends with a targeted fix at that point.
|
||||
- `modelName` ambiguity limits registry precision — mitigated by the conservative
|
||||
default never producing an unplayable stream.
|
||||
Reference in New Issue
Block a user