[Feature] Add DASH / CMAF support for adaptive, smooth video streaming

[Thinkscape]

I have searched the existing feature requests to make sure this is not a duplicate request.

• Yes

The feature

Sister feature to: #1674

Intro

DASH, also known as MPEG-DASH, is a similar standard to HLS but is not proprietary. It allows high-quality streaming of media content over the internet delivered from conventional HTTP web servers. Like HLS, DASH works by breaking the content into a series of small segments.

• Codec Agnostic: Unlike HLS, DASH is codec agnostic, meaning it can use any codec.
• Adaptive Streaming: It also supports adaptive streaming by allowing the client to choose from several different quality levels depending on available bandwidth.
• Manifest Files: DASH uses an XML-based manifest file (Media Presentation Description, MPD) that describes segment information for adaptive playback.

Why

• Support for slower connections (by serving lower quality video)
• Better streaming performance and resilience (i.e. on unstable connections)
• Adaptive streaming - could serve one of several quality versions, downgrade on slower connections.
• CDN-friendly - would allow i.e. CloudFlare to cache much smaller chunks (otherwise videos hit the MB limit which all plans have)
• Well supported in browsers and foundational to all video streaming services (Netflix, Youtube etc.)

How

Some reference:

• https://ffmpeg.org/ffmpeg-formats.html#dash-2
• https://blog.zazu.berlin/internet-programmierung/mpeg-dash-and-hls-adaptive-bitrate-streaming-with-ffmpeg.html
• https://github.com/shaka-project/shaka-player?tab=readme-ov-file#dash-features
• https://github.com/video-dev/hls.js/

Platform

• Server
• Web
• Mobile

[bo0tzz]

DASH doesn't magically do adaptive quality, right? IE we would still need to do dynamic transcoding on the server, making this a (partial) duplicate of #7666?

[Thinkscape]

Ideally you leave this option to the administrator - config option: real-time vs pre-encoded.
If Immich is hosted on a reasonably powerful machine, then it might be able to live-transcode videos (like Plex does).
For everyone else, a pre-encoding (like it currently does) seems to be a good compromise.

In the latter option, you are correct that it will increase the encoded-video size, but keep in mind that lower qualities will yield much smaller files, while vastly improving the user experience.

The best solution is to do what Plex does - allow for remuxing - this might result in SMALLER encoded-video - i.e. on fast connections you could remux (real-time) and have the original bitrate of the source video (i.e. 4k 20-100Mbps from iPhone 13+, all modern browsers play HEVC just fine), on slower connections it falls back to 1080p, 720p, 480p versions.

In my personal case I'm sharing my photos with family overseas, and because of peering it's painfully slow. In current version of Immich, even behind a CDN, I am forced to over-compress everything so it fits the slowest user's connection, while local users (even on 10Gb LAN) will be get the same crappy low-bitrate videos.

[Thinkscape]

A low-hanging fruit here, short-term would be to experiment with:

1. Add quality switcher to the video player
2. Update the /api/file handler to accept video quality/version query param.
3. Allow qualities: Original (serves the original file) Transcoded (XXXXp) (depending on what the max resolution/bitrate admin set in Video settings)
4. Add an onError handler to the video player to fall back to Transcoded if browser is unable to play the source file directly (i.e. b/c of container, hevc/h264 level, bit depth etc.)

This is very low effort, but at least allows the end user to switch between 2 qualities, before we add multi-bitrate transcoding support.

Thoughts? 🤔

[bo0tzz]

We already have logic to serve the original file for images, so I think extending that to videos would be acceptable. @jrasm91 @mertalev thoughts?

[NicholasFlamy]

No one's asking me but I'll say that even without more than 2 video sources (original and transcoded) having it load the original on Wi-Fi and the transcoded not on Wi-Fi or when the connection is slow would be useful.

[mertalev]

I think it's conceptually fine to support displaying the original video. That being said, the logic for using the generated vs. original is pretty janky at the moment, so I think we'd want to refactor that area of the code before adding onto it.