We will support HEVC playback via Media Foundation transform (MFT).
HEVC playback will be supported via the Media Foundation Transform (MFT)
and WMF decoder module will check if there is any avaliable MFT which
can be used for HEVC then reports the support information.
HEVC playback can only be support on
(1) users have purchased paid HEVC extension on their computer (SW decoding)
(2) HEVC hardware decoding is available on users' computer
For now, I'd like to only enable HEVC for the media engine playback, but
keep the HEVC default off on the MFT. Because the media engine is an
experimental feature, which is off by default, it's fine to enable HEVC
for that.
HEVC playback needs hardware decoding, and it currently only support on
Windows. HEVC playback check would be run when the task is in the
mda-gpu, which has the ability for hardware decoding. On other
platforms, HEVC should not be supported.
This was a use case I was introduced to directly before I discovered Firefox was introducing support for HEVC decoding.
If it weren't for iOS, VP9 could take its place, or so the Mozilla developers thought. HEVC and newer codecs like VP9, AV1, VVC, EVC, etc. offer better compression but often at the cost of compute. I imagine hardware decoding evens the scales a lot; I haven't done any benchmarking myself. I don't know how much impact the complexity of H.265 vs H.264 has on battery life, if any. Of course, hardware encoding on VP9 is not really a thing (AV1 is ahead of it, even), so HEVC has the edge there.
In a few years, AV1 hardware implementations will hopefully be ubiquitous; that would solve the efficiency and software patent problems at the same time. It'll probably coincide with the last of H.264's patents expiring. So on the one hand, I can understand why Mozilla is in no rush to support HEVC.
So I imagine you use a Chromium-based browser for Jellyfin?
I use a variety of different Jellyfin clients.
The confusing thing is that although every device I use supports hardware decoding, I am not completely sure if it actually works for each Jellyfin client (I am even less sure after this thread). Regardless HEVC hardware transcoding works on my server (using Intel QuickSync), so my files can always play even if they aren't directly supported by the client.
As for AV1, I want to move to it in the future, but as of right now, most of my clients don't have hardware decoding for it. My server can't transcode AV1 either, I would need a newer GPU or iGPU. Even the NVIDIA SHIELD TV that I have (a powerful media device), does not have hardware decoding for AV1 (I am still looking forward to a SHIELD refresh).
Ah, okay. It sounds like you have a physical server, too...you would need to upgrade it yourself if you wanted to use AV1, right? Sounds expensive and annoying...
My understanding is the client needs to explicitly support hardware decoding with the relevant APIs, and Jellyfin probably accomplishes it with FFmpeg. There is no way Jellyfin would be implementing a software decoder for HEVC, but they should have no problem implementing hardware decoders for every platform.
iOS doesn't even have a software decoder for AV1 yet, but the iPhone 15 Pro hardware decoder is a start. Likewise, only expensive Android phones have hardware AV1 decoders right now. More desktop GPUs are implementing AV1 decoders. But this transition looks like it's going to take another 3 years (?) to hit every market segment (cheapest to most expensive)... sigh. I don't have an AV1 hardware decoder on any of my devices either. It's insane how long it takes for new hardware decoders to become mainstream. Many HEVC patents might be close to expiring by then, lol.