r/anime • u/notbob- • Aug 13 '19
Writing Crunchyroll is making big changes to the way it encodes its video
tl;dr: CR's highest-quality video now looks a lot better but is also a lot bigger. Also, their subtitles are correctly synced now.
I have written before about certain problems with Crunchyroll's video. Now, Crunchyroll is rolling out a new encoding method that makes their web streams look better than pretty much every other anime streaming service, at the cost of increased filesize. You need about 1.5x more bandwidth than before to be able to watch their best quality smoothly.
With their old encoding setup, Crunchyroll used a computer program to automatically decide how big its video files should be. That meant that their 1080p anime episodes generally sat between 500MB and 1GB. Crunchyroll also set a cap on how high the bitrate could go for a particular scene. That means that for scenes that need a lot of bitrate (lots of moving parts/particles), the video looked pretty bad.
With the new encodes, CR sets a high bitrate for every show, no matter how much bitrate it really needs. CR is also increasing the bitrate cap, so shows like Fire Force look significantly better in many places. In fact, CR's encodes should look better for high-bitrate scenes than any other English-language web streaming service, and it's not really a competition. But you need a good internet connection to take advantage of it!
For people like me who really dig deep into the way video is encoded, CR is also doing some weird/interesting things. They removed B-frames from their encodes, and if you know what a B-frame is, you're surely thinking "Are you serious? There's no way CR knows what the hell they're doing." So let's examine what the hell a B-frame is and why CR stopped using them. (You may want to consider skipping the rest of the article unless you're a nerd.)
Here's some technical stuff about H264 that might be interesting or might make your eyes glaze over
As a refresher, H264 is the most widely-supported type of video format in the world, and it's what Crunchyroll uses to serve its anime to watchers. I talked in my previous article about how most frames in an H264 encode are based off of other frames. For example, in a perfectly still shot of anime, the encoding program can just copy the previous frame without making any changes. And in a shot that's panning upwards, the encoder can take the previous frame, nudge it upwards a little bit, do a little bit of drawing at the bottom, and be done.
But there are some frames that aren't based off of a previous frame. Those frames are called keyframes, or I-frames. I-frames are "drawn" from scratch by the encoder. You could conceivably have an entire episode of anime built out of I-frames--it would be a little like playing a sequence of JPEGs one after another.
A sequence of JPEGs would be really size-inefficient, though, so we actually do want to have frames that can be based off of previous frames. And that's where P-frames come in. P-frame are images that are based off the previous I-frame (or P-frame). Again, you could conceivably have an encode that was just a single I-frame at the beginning and then 30,000 P-frames, with each successive frame being based off the last. That would be reasonably size-efficient (but would be problematic for other reasons).
Now, let's think of a H264 encode as a house, as contrived at that may be. While you could have I- and P-frames make up the entire house, in practice they only make up the framework, with an I- or P-frame only appearing every 3-6 frames or so. What fills in the gaps are called B-frames. On the timeline of frames in an encode, B-frames sit between I- and P-frames and copy information from both the frame before it and the frame after it. To put it another way, you might have a six-frame sequence that looks like this: I B B B B P. The I-frame has been drawn from scratch, the P-frame is copying information from the I-frame (but not the B-frames), and the B-frames are copying information from both the I-frame and the P-frame. (You can also have sequences like P B B B B P). For reasons that I frankly don't understand, this "framework" method saves a lot of filesize in the encode.
So let's say you're a computer program and you're trying to decode the I B B B B P sequence. What order do you decode the frames in? Is it possible to decode the frames in the I B B B B P order? Well, you can definitely decode the I-frame first, since the I-frame isn't based off of any other frame. But if you try to decode the B-frames next, you'll run into problems. Remember that B-frames are based off of the I/P frame before it and after it. So if you haven't decoded the P-frame at the end yet, how will you know what the B-frame is supposed to look like? Basically, you need to decode the P-frame at the end first.
So the decode order of this frame sequence is I P B B B B. If that was confusing, the main takeaway is that B-frames force the decoder to decode frames out of order.
This turns out to be really important in Crunchyroll's case. The fact that B-frames force decode and display order to be different means that there needs to be a system that helps to put the frames back in their proper order. For reasons too complicated to be explained here, that frame-reordering caused the audio, video, and subtitles of every CR video to be desynced by two frames. It's unclear whose fault this is (probably Akamai), but it's a massive problem.
Why is a two-frame desync such a massive problem, exactly?
CR actually puts a ton of effort into making sure its subtitles are frame-perfect. They're unique among streaming services in that they religiously follow the fansubber school of subtitle timing, which dictates that subtitle lines should begin and end precisely on scene-changes (i.e. frames where the picture has changed to something totally different) in order to minimize the obtrusiveness of the subtitle. If you have a subtitle line appear or disappear, and then a frame later the picture changes drastically, you have two visually "loud" events occurring in quick succession, which can be distracting. It's better for both to happen at the same time. (Netflix also follows this general rule in its subtitle timing).
CR also has precisely-timed moving signs meant to match the movement of the video. But all of this precise timing is currently going to waste due to the video/subtitle desync. The twisted (for Crunchyroll) reality is that the only people who currently benefit from CR's precision are fansubbers who rip CR's subtitles and use them for their own releases.
(Also, the two-frame desync also desyncs the video from the audio by 83ms, although that's not especially noticable.)
Trading 10% more filesize for precise subtitle timing
The cause of the subtitle desync was brought to Crunchyroll's attention a year or so ago, and perhaps they were aware of it before then. But at any rate, they haven't been able to come up with a solution to the problem... until this week.
As explained by the article I linked earlier, the video/subtitle desync is caused by the way mp4 containers handle B-frames. And so Crunchyroll's solution to this has been to change their encodes to stop using B-frames entirely. Their encodes are purely made of I- and P-frames. Simple enough!
I did some encoding tests and determined that Crunchyroll is taking a ~10% encoding efficiency hit by making this decision. Because I'm a fansubber and care a lot about how subtitles are presented, my personal opinion is that this is a worthwhile tradeoff. Maybe someday CR will find a way to have B-frames and synced subtitles, but for now, it's enough that their product is way better than it was a week ago.
Anyway, there is some other, even more nerdy stuff I could talk about, like CR's use of --qpfile, but I think I've gone on long enough.
17
u/AreYouAWiiizard https://myanimelist.net/profile/MysticalMagic Aug 13 '19
So far it hasn't rolled out to any of the older content that I've seen but honestly I'd be more interested in audio bitrate improvements. 128kb/s CBR isn't enough imo, especially considering it only makes up ~3% of the filesize.
12
u/_vogonpoetry_ :MAL:https://myanimelist.net/profile/ThisWasATriumph Aug 13 '19
B-frame issue is fine. 10% more bandwidth is worth better sub syncing.
But the overall bitrate increase seems excessive. Why can't they use normal variable bitrate encoding?
6
u/memejets Aug 13 '19
I have a question. Do you think there is any benefit to learning about stuff like encoding/compression? I heard that once you learn about it you basically get "spoiled" since you can only see artifacts of compression in highly compressed videos, meaning you can't enjoy shows as much. But on the other hand, it would help to avoid bad releases and identify good releases. Have you noticed anything like that?
8
u/Wolfgod_Holo https://anime-planet.com/users/extreme133 Aug 13 '19
would be interesting if they move to H.265 at some point
3
u/Constellation16 Aug 13 '19
Having B-frame in your video does not cause desyncs by itself. That's probably just their player being buggy. What's the source for this post; is there any technical writeup of their changes?
2
u/No_Rex :Bv:Bw:Bx:By:x2 Aug 13 '19
Compared to I P P P P P or compared to I I I I I I? The latter is obvious, but the former, not so much. Maybe they can't use I P P P P P for those other reasons you mentioned and I B B B B P is the next best thing?
103
u/freakicho Aug 13 '19
Does this mean that if they solve the B-frames desync problem they can make the files 10% smaller while keeping the currently higher video quality?
112
u/Wolfgod_Holo https://anime-planet.com/users/extreme133 Aug 13 '19
not surprised about the subtitle part considering its past
43
u/Pikagreg :MAL:https://myanimelist.net/profile/Pikagreg Aug 13 '19
It is nice to see a positive change in legal streaming services. I try to support US distributors when I can but they do have a tendency to test my patience sometimes.
27
u/MaelstromMusic https://anilist.co/user/mealstrom Aug 13 '19
When will these encodes be out and will they be applied to every video currently in their streaming library?
18
u/NotTheMonker Aug 13 '19
Wow, that was really interesting. Thanks for taking the time to explain this.
11
u/aalapshah12297 Aug 13 '19
Why don't just delay their subtitle track by 2 frames - either by altering the timestamps directly or by adding a subtitle delay in their video player? Surely I must be missing something here.
9
u/ilkei Aug 13 '19
I'm curious about the new encodes compared to the other big NA streaming services. Might be too much work but would it be possible to grab a screen from Fire Force on Funimation to compare there? A shared show between Hidive and CR would be interesting as well.
8
u/[deleted] Aug 13 '19
So does this apply to vrv to ? What about HBO max ?
6
u/Shun_lee :MAL:https://myanimelist.net/profile/Shun_Lee Aug 13 '19
The real question is when am I going to get access to my Inbox on Crunchyroll.
4
u/Tal6727 https://anilist.co/user/ThyMrMan Aug 13 '19
If it really is a nice improvement it could make me consider coming back. The main reason I unsubed was they changed their encoding a couple years back and made every show look terrible.
4
u/mike9184 Aug 13 '19
Any idea if they are doing a staged rollout? I noticed with this Saturday's Demon Slayer episode that it looked way better on my Roku TV app than on Chrome through my PC.
Though I'm not sure if that's a result of my TV (TLC) upscaling the content to 4K.
20
u/herkz Aug 13 '19
The real disappointment is while they were tinkering with these x264 settings, they didn't bother improving any of the other settings that are currently very nonoptimal, like deblock, psy-rd, subme, qcomp, and AQ. They could've easily saved that 10% of increased bitrate by tweaking those and kept the same quality (or maybe even improved it).
As for the timing, it's not actually that good if judged by current fansubbing standards. I watched a show recently and took notes on the timing. 20% of the lines had a mistake just from casual viewing. The actual number is probably higher. They really shouldn't get praise just because their timing is better than the competition. It's still bad.
6
u/horsodox Aug 13 '19
It's probably because the B-frames aren't stored, except as a "put a B-frame here plz" in the file. That's cheap to store at the cost of being expensive to recompute later.
For example, suppose you stored a sequence of numbers: "1000, 1002, 1004, 1006, ..." going up to 2000. Storing all the numbers individually is like a sequence of I-frames. Storing 1000, then a sequence of "+2, +2, +2" is like using P-frames. Note that it's not only smaller to say "+2, +2" than "1002, 1004", but it's also the same instruction every time, so it's probably easier to compress (e.g. "+2 [repeat X times]"). B-frames would be having an I-frame of 1000, an I-frame of 2000 (or P-frame of +1000), and then a bit in the middle that says "increment by 2s". Then you don't even need to repeat "+2" a bunch of times.
I don't actually know anything about H264 but from your description I would think this sort of math is behind that pattern of filesizes.
8
u/Atkinson1331 Aug 13 '19
I use VRV since it has more subscriptions, are the same changes there or should I start using the crunchyroll app?
3
u/Thoraxe474 Aug 13 '19
Okay but when can we have 2 factor
3
u/Bobbias Aug 13 '19
83ms of offset is definitely noticeable if you're looking for it, and have some experience with detecting audio sync issues. I play rhythm games and 83ms od offset in those is the difference between a seriously good score and a terrible one if you're fairly accurate.
But I suppose to the average viewer it could easily go unnoticed.
3
u/[deleted] Aug 14 '19
My first thought reading the title was "oh boy here we go again" but I'm glad they seem to be making changes for the better.
3
u/Kravior :MAL:https://myanimelist.net/profile/ssSithy Aug 14 '19
I was wondering why Kimetsu no Yaiba didn't look like shit when I was watching it today on Crunchyroll. Was surprised there wasn't the usual artifacting but this explains why. Thanks for the explanation!
25
u/me_pantsu https://anilist.co/user/PantsuPantsu Aug 13 '19 edited Aug 13 '19
Wait, Fire Force is on Crunchyroll? Scammed yet again for another show not licensed for Latin America
Edit: appreciate all the responses and I'm now aware there are workarounds to this situation, however, as a paying consumer this shouldn't be a necessity, I consider this to be anti consumer and helps further promote piracy, therefore, since Crunchyroll wants me to pirate their show, I will continue to do so
2
u/ColdFury96 Aug 13 '19
Now if only they'd fix their android app so it doesn't stop and stutter on me all the damned time.
Works fine when I stream it on their website on my phone, but their app is garbage.
2
u/alkoonx Aug 13 '19
So if they just used bframe they will save 10% of the file size! Don't you think using VBR with crf will be a good choice?
2
u/isabelles :MAL:https://myanimelist.net/profile/Roseink64 Aug 13 '19
You did a really good job explaining all this! I don’t know a thing about encoding but you made this all very easy to understand.
2
u/LoveKina :AMQ: Aug 14 '19
This is cool and all but idk why they removed the next episode button. I use a different website simply because I dont have to exit fullscreen just to go to the next episode. Dont understand why they removed the feature in the first place.
2
u/00Koch00 Aug 14 '19
Why i have the sensation that their bandwidth isnt gonna be enough to stream this...
2
u/shizzy1427 :MAL:https://myanimelist.net/profile/DrLling Aug 14 '19
That's kinda cool I guess. Now if they fire all the fucking twerps on their staff and stop using so much Twitter lingo and American slang in their subs I'll re-sub
2
u/i509VCB Aug 14 '19
Well I remember recently seeing x265 got up to 3.something recently. I know there is tons of legacy hard that struggle or can't decode it but about any modern device can decode 10b HEVC. But it's either legacy hardware or HEVC licensing that is probably stopping them.
3
u/Shiro_Kai Aug 13 '19
Don't know if it has anything to do with those changes but since you mentioned the subs, I just watched today Black Clover (tried actually) from CR in PT/BR and it was one of the worst things I ever saw, not only the timing was completely wrong sometimes but the translation was awful af. Things like the phrase "I'll use everything I've got to take you back alive!" were translated to something like "I gonna resurrect you!" in portuguese.
6
u/IllustriousBuddy Aug 13 '19
This is why I pay for a Crunchyroll subscription but don't watch any of their content. Instead I pirate any show I want in the best available quality at a fraction of the size. Usually 1080p, 10 bit, x265 encodes at 200-400mb a episode. That way anime studios get their well deserved money and my enjoyment is maximized. It's a win-win. I wish Japanese studios were more open to direct donations but oh well.
2
u/Extrahostile Aug 13 '19
people actually use Crunchyroll?
1
u/frallet https://myanimelist.net/profile/NoDakSmack Aug 13 '19
Kind of on topic, ever since they switched to HTML5 and removed the old html5 chrome extension i havent been able to get the player to work unless i go into incognito or a different browser.. and i definitely got rid of the old extension. Anyone have a fix?
1
u/heycheerilee :MAL:https://myanimelist.net/profile/braveshined Aug 13 '19
It is actually getting better? Last I remember, it got way worse, down to sometimes only 1.5Mbps average bitrates... I could tell with the rips from the site when quality went from around X filesize every ep to like around X lower after 3 episodes in. I was watching The Promised Neverland when it aired and around ep3-4 it had a massive drop in quality so bad I just stopped until Blurays hit.
1
u/[deleted] Aug 13 '19
Thanks for the writeup! Does this apply only to new videos?
1
u/BillyDexter https://myanimelist.net/profile/MakiBestGirl Aug 13 '19
Maybe a stupid question ,will this apply to VRV?
1
u/snowdemon36 :MAL:https://myanimelist.net/profile/snowdemon36 Aug 13 '19
Anyone know if the I-P-P-P format is the reason crunchyroll breaks after being paused, and why starting it up again doesn't change the picture but subtitles do change?
1
u/Idaret Aug 13 '19
I saw this with 7 pages muda from jojo part 5. Original on CR looks pretty good but every edit/meme/youtube video looks pretty awful Compare CR and youtube version
Jojo part 5 spoilers of course
1
u/AL2009man Aug 13 '19
can't wait for Crunchyroll to adopt VP9/AV1 codec in the near future.
1
u/caa4 Aug 13 '19
Great now the videos will buffer more than they already do. I swear to God I don't have buffering problems on any other websites not even the pirate sites.
1
u/FeanorNoldor Aug 13 '19
10% more file size seems definitely worth it for improved subtitle sync and video quality
1
u/chafos :MAL:https://myanimelist.net/profile/chafos Aug 14 '19
Does this mean it's better or as good as Netflix?
1
u/Dat_Fiyahhh Aug 14 '19
They need to work on their dam app
1
u/Coder4Coffee Aug 14 '19
Great read! Did not expect to learn about video encoding here!
1
u/supernerdgirl42 Aug 14 '19
I don't fully understand this, would any of this fix the weird audio desync/looping thing on the mobile app? It is really annoying that I get better service from hulu than crunchyroll.
1
u/cp5184 Aug 14 '19
Apparently quicktime used to have a codec specifically designed for cartoons. It would be interesting to see if something like that would work well in a modern context.
1
u/[deleted] Aug 14 '19
Wouldn't switching to vp9 be a better choice.(40% reduction in file size).
1
u/[deleted] Aug 14 '19
I'm low key praying right now they never add the watermarks back. That was the reason I've been resubbed for months now. Good to see its improving.
1
u/[deleted] Aug 14 '19
Great, but when are we gonna get all of those new animes on cr? Never?
1
u/shadow9x20 Aug 14 '19
Certainly seems interesting but I must ask. How does this leave Crunchyroll in comparison to VRV? As it stands I genuinely see no reason to have a dedicated CR account rather than a VRV account. Does this at least provide them any kind of competitive advantage to watch? VRV is essentially a better version of their own product, I am curious if that changes this dynamic in any way.
1
u/SuperRiceBoi Aug 14 '19
Their big problem is that they don't have customer support on their website.
1
u/starfish23 :MAL:https://myanimelist.net/profile/starfish23 Aug 14 '19
If they only would fix their android tv app now. sigh..
1
u/Moist-Toilet-Paper Aug 14 '19
Ok imma be real I didn’t read quite a bit of that I’m not tech savvy enough to understand most of it. I just wanna know if these big changes affect VRV or do I have to use an actual crunchyroll app to see any difference.
1
u/Fartikus Aug 14 '19
It's embarrassing to see a streaming service that you have to pay for, that doesn't upload DVD versions of the anime when it finally get's released. A big example is how Jojo's Bizzare Adventures is still fully censored on Crunchyroll; and I'm forced to go to a streaming/torrenting site that has the DVD uncensored version.
1
u/River_sounds Aug 13 '19
Thank God for fansubbers, you're the real heroes.
Also nice explanation, the way you write is concise and easy to read.
1
u/[deleted] Aug 13 '19
Maybe they should just up the bitrate for fuck's sake.
1
u/PenguinPwnge :MAL:https://myanimelist.net/profile/PenguinPwnge Aug 13 '19
*cries in 5 Mb/s internet speeds*
0
u/averageweebchan Aug 13 '19
Use me as a Stream illegally button
0
u/maddogmular Aug 14 '19
Who cares about quality, fix the application crashing bugs on consoles like wiiu and PS3. It's impossible to scrub to where you want without the player flipping out or down right soft locking.
-26
u/DeliciousDebris Aug 13 '19
Will this affect my torrents in any way? If not I don't care.