Re: no more hslv format ?
On 3/05/2018 9:24 AM, RS wrote: > On 02/05/18 23:21, Owen Smith wrote: > >> >> I've been mystified for a while why people talked about "dropping >> every other frame" as if it were trivial to do, and an email earlier >> in this chain looked like someone was trying to do that again. I was >> explaining why that simply is not possible in the general case. >> > > If I have caused confusion by talking about dropping alternate frames > I apologise. I had come across some posts in another forum which > suggested it could be done, but I now recognise I was wrong. What > makes it worse is that I have since come across a thread in this > listserver from two years ago where I was asking exactly the same > questions, and Vangelis pointed out I was wrong and directed me to a > Wikipedia article on H.264. > > It is not possible to change the frame rate using -c:v=copy in ffmpeg; > it is necessary to re-encode which is why it takes so long. > Inevitably there will be losses, added to which the codecs available > to us may be inferior to those used by the BBC. > > I think it was Nick Payne who said he had experimented with > re-encoding in HEVC (H.265) and found that the file size was the same > for 25fps as it was for 50fps, which led him to conclude that frames > were being duplicated to achieve 50fps. Yes, I had some of the coverage of the UK snooker championships where they provided HLS downloads @ 1280x720 25fps and other coverage was only available as HVF downloads @ 1280x720 50fps. When I ran these both through Handbrake with identical settings to convert to HEVC, retaining the frame rate of the downloaded files, the size reduction for the 50fps downloads was about twice that for the 25fps downloads, and the size of the files output by Handbrake was proportional to the length of the program and not the frame rate. e.g. 4h58m match: D/L size 5.03Gb @ 25fps, output from Handbrake was 1.55Gb 4h44m match: D/L size 10.2Gb @ 50fps, output from Handbrake was 1.48Gb Nick ___ get_iplayer mailing list get_iplayer@lists.infradead.org http://lists.infradead.org/mailman/listinfo/get_iplayer
Re: no more hslv format ?
On 02/05/18 23:21, Owen Smith wrote: I've been mystified for a while why people talked about "dropping every other frame" as if it were trivial to do, and an email earlier in this chain looked like someone was trying to do that again. I was explaining why that simply is not possible in the general case. If I have caused confusion by talking about dropping alternate frames I apologise. I had come across some posts in another forum which suggested it could be done, but I now recognise I was wrong. What makes it worse is that I have since come across a thread in this listserver from two years ago where I was asking exactly the same questions, and Vangelis pointed out I was wrong and directed me to a Wikipedia article on H.264. It is not possible to change the frame rate using -c:v=copy in ffmpeg; it is necessary to re-encode which is why it takes so long. Inevitably there will be losses, added to which the codecs available to us may be inferior to those used by the BBC. I think it was Nick Payne who said he had experimented with re-encoding in HEVC (H.265) and found that the file size was the same for 25fps as it was for 50fps, which led him to conclude that frames were being duplicated to achieve 50fps. The broadcast signal (at least on satellite) is 1920x1080i at 25fps usually with two audio streams, AC3 and NAR. If the broadcast signal is recorded, the resultant file size is about 3GByte/h. The BBC'S explanation of what it does with the broadcast signal is, "The Elemental encoders are used to convert the 1920x1080 interlaced content to 960x540 for progressive encoding at 50fps." It also says, "The 50fps, 1280x720 profile, however, will be available to those with 5Mbit/s broadband connections." but it does not explain where it comes from or why it cannot generate a 1280x720p 25fps profile. Best wishes Richard ___ get_iplayer mailing list get_iplayer@lists.infradead.org http://lists.infradead.org/mailman/listinfo/get_iplayer
Re: no more hslv format ?
I'm replying on an iPad, which makes large scale editing of email replies a huge amount of work. So I top post, because it takes only a tenth of the time. If Apple made it only cost me double the amount of time to reply properly I'd do it. I've been mystified for a while why people talked about "dropping every other frame" as if it were trivial to do, and an email earlier in this chain looked like someone was trying to do that again. I was explaining why that simply is not possible in the general case. -- Owen Smith Cambridge, UK > On 2 May 2018, at 22:33, Ralph Corderoy wrote: > > Hi Owen, > >> What do you mean this isn't a lossy transcoding? > > Is that aimed at me? > > Perhaps if you didn't top post, and instead wrote that under a quote of > mine I'd know to which bit of the two ffmpeg invocations you were > referring! :-) >> How can ffmpeg go from 50fps to 25fps without losing anything? > > I don't think it can, and didn't suggest it could. It is lossy. I said > the first, default, one wasn't, and that therefore it wasn't worth > combining this extra, 50->25, one with it, as you would want to if both > were lossy. > > -- > Cheers, Ralph. > https://plus.google.com/+RalphCorderoy > > ___ > get_iplayer mailing list > get_iplayer@lists.infradead.org > http://lists.infradead.org/mailman/listinfo/get_iplayer ___ get_iplayer mailing list get_iplayer@lists.infradead.org http://lists.infradead.org/mailman/listinfo/get_iplayer
Re: no more hslv format ?
Hi Owen, > What do you mean this isn't a lossy transcoding? Is that aimed at me? Perhaps if you didn't top post, and instead wrote that under a quote of mine I'd know to which bit of the two ffmpeg invocations you were referring! :-) > How can ffmpeg go from 50fps to 25fps without losing anything? I don't think it can, and didn't suggest it could. It is lossy. I said the first, default, one wasn't, and that therefore it wasn't worth combining this extra, 50->25, one with it, as you would want to if both were lossy. -- Cheers, Ralph. https://plus.google.com/+RalphCorderoy ___ get_iplayer mailing list get_iplayer@lists.infradead.org http://lists.infradead.org/mailman/listinfo/get_iplayer
Re: no more hslv format ?
That doesn't make any difference to the H.264 encoding. At best if you get lucky every odd frame will be marked as being interpolated from the preceding even frame and then no actual changes within the frame. But all it takes is for the encoder to slip by a frame somewhere or to encode an I frame (a complete standalone frame) in the even or odd half that you are trying to discard every one of and then you are screwed. And if the 50fps file genuinely does have every other frame saying "I'm identical to the previous frame" the stripping these won't save much on the file size. The fact that people complain the 50fps files are double the size of the same resolution 25fps files implies to me this isn't how it has been done. Even if the BBC have only created motion interpolated frames from a 25fps source, they are still encoded in the video stream and you still can't casually toss alternate frames without screwing up the I-P-B frame interpolation. -- Owen Smith Cambridge, UK > On 2 May 2018, at 19:43, Peter S Kirk wrote: > > Many posts back it was mentioned they are not true 50fps, instead each > frame from a 25fps is duplicated merely to allow BBC to boast about 50fps > streaming. > > On 2 May 2018 at 19:07, Owen Smith Owen Smith > wrote: > >> What do you mean this isn't a lossy transcoding? How can ffmpeg go from >> 50fps to 25fps without losing anything? The >> frames are not all complete frames, software can't just throw alternate >> frames away. Well it could, but the only way >> to do that is a full H.264 decode, then discard alternate frames, then a >> full H.264 encode again which is going to >> involve loss. >> >> Most frames are not fully present in the original stream, they are >> interpolated from previous and subsequent frames. >> You can't throw any of those away, because other frames are interpolated >> from them. It would need to be a very special >> original encode which had all even frames only interpolated from other even >> frames and ditto for odd frames to allow >> alternate frames to be discarded. And a special encode like that would bloat >> the file size substantially, almost >> doubling it I would expect. > > > > ___ > get_iplayer mailing list > get_iplayer@lists.infradead.org > http://lists.infradead.org/mailman/listinfo/get_iplayer ___ get_iplayer mailing list get_iplayer@lists.infradead.org http://lists.infradead.org/mailman/listinfo/get_iplayer
Re: no more hslv format ?
Many posts back it was mentioned they are not true 50fps, instead each frame from a 25fps is duplicated merely to allow BBC to boast about 50fps streaming. On 2 May 2018 at 19:07, Owen Smith Owen Smith wrote: > What do you mean this isn't a lossy transcoding? How can ffmpeg go from 50fps > to 25fps without losing anything? The > frames are not all complete frames, software can't just throw alternate > frames away. Well it could, but the only way > to do that is a full H.264 decode, then discard alternate frames, then a full > H.264 encode again which is going to > involve loss. > > Most frames are not fully present in the original stream, they are > interpolated from previous and subsequent frames. > You can't throw any of those away, because other frames are interpolated from > them. It would need to be a very special > original encode which had all even frames only interpolated from other even > frames and ditto for odd frames to allow > alternate frames to be discarded. And a special encode like that would bloat > the file size substantially, almost > doubling it I would expect. ___ get_iplayer mailing list get_iplayer@lists.infradead.org http://lists.infradead.org/mailman/listinfo/get_iplayer
Re: no more hslv format ?
What do you mean this isn't a lossy transcoding? How can ffmpeg go from 50fps to 25fps without losing anything? The frames are not all complete frames, software can't just throw alternate frames away. Well it could, but the only way to do that is a full H.264 decode, then discard alternate frames, then a full H.264 encode again which is going to involve loss. Most frames are not fully present in the original stream, they are interpolated from previous and subsequent frames. You can't throw any of those away, because other frames are interpolated from them. It would need to be a very special original encode which had all even frames only interpolated from other even frames and ditto for odd frames to allow alternate frames to be discarded. And a special encode like that would bloat the file size substantially, almost doubling it I would expect. -- Owen Smith Cambridge, UK > On 2 May 2018, at 16:20, Ralph Corderoy wrote: > > Hi Jim, > >>> You could use get_iplayer's --command option to run a command to >>> move each final file off tmpfs as the download is finished. Its >>> --output affects all the intermediate files too, AIUI. >> >> The challenge for me is to work out how to get the fetched file to go >> onto the tmpfs > > Well, `df -t tmpfs' will probably show /tmp is a tmpfs so you could > `--output /tmp' and you should see its intermediate files, and the final > file, only appear there. `--command' could then move that final file to > the SSD, or run a conversion command that writes to the SSD and then > removes the tmpfs input. > > As for altering the ffmpeg command that get_iplayer is using, I'm not > sure that's worthwhile? It isn't doing any transcoding, just changing > the container format, or splicing in better audio, that kind of thing. > So your `lossy' slow-down ffmpeg from 50 fps to 25 fps won't be a second > lossy one that you'd prefer to combine with the first. I could be > wrong, not knowing how to have ffmpeg do this conversion. When you find > out, let the list know. :-) > > -- > Cheers, Ralph. > https://plus.google.com/+RalphCorderoy > > ___ > get_iplayer mailing list > get_iplayer@lists.infradead.org > http://lists.infradead.org/mailman/listinfo/get_iplayer ___ get_iplayer mailing list get_iplayer@lists.infradead.org http://lists.infradead.org/mailman/listinfo/get_iplayer
Re: no more hslv format ?
Hi Jim, > > You could use get_iplayer's --command option to run a command to > > move each final file off tmpfs as the download is finished. Its > > --output affects all the intermediate files too, AIUI. > > The challenge for me is to work out how to get the fetched file to go > onto the tmpfs Well, `df -t tmpfs' will probably show /tmp is a tmpfs so you could `--output /tmp' and you should see its intermediate files, and the final file, only appear there. `--command' could then move that final file to the SSD, or run a conversion command that writes to the SSD and then removes the tmpfs input. As for altering the ffmpeg command that get_iplayer is using, I'm not sure that's worthwhile? It isn't doing any transcoding, just changing the container format, or splicing in better audio, that kind of thing. So your `lossy' slow-down ffmpeg from 50 fps to 25 fps won't be a second lossy one that you'd prefer to combine with the first. I could be wrong, not knowing how to have ffmpeg do this conversion. When you find out, let the list know. :-) -- Cheers, Ralph. https://plus.google.com/+RalphCorderoy ___ get_iplayer mailing list get_iplayer@lists.infradead.org http://lists.infradead.org/mailman/listinfo/get_iplayer
Re: no more hslv format ?
In article <60733938-d7ea-4db5-bd1d-2e6f07d25...@zoho.com>, RS wrote: > On 30/04/18 13:56, Jim web wrote: > > I've been discussing the 'loss' of the 1280x720 25fps version with > > someone at the BBC. > > > > IIUC this stemmed from 'Red Bee' days of yore, and until recently > > people at the BBC had thought they had stopped it long ago. Someone > > apparently noticed recently that it was still available. And then > > actually disabled it. Hence the mysterious recent ending of its > > availablity. > > > > I have made the point that the 1280x720 25fps version is useful for > > people with a content 'cap' problem, etc, so will be missed because > > the 50fps version means somewhat bigger files and/or stream rates. But > > I doubt this will cause a rethink. > > > HLS is a much more recent innovation than Flash. get_iplayer now has > its own built-in downloader which downloads HLS 2 or 3 times as fast as > Flash. HLS has been referred to as a legacy mode so it was always > likely that it would be removed eventually. The surprise was that HLS > was removed at the same time as Flash. > It is unrealistic to expect the BBC to restore HLS or Flash. Understood. I've probably been conflating different issues [snip BBC's comments] > For that reason I find it surprising that 960x540p at 50fps "delivers > significantly better pictures on TV screens across a wide range of > popular content (such as EastEnders and Top Gear) due to its higher > frame rate" than 1280x720p at 25fps but the BBC has done the viewing > tests and I have not. I'm also puzzled by this conclusion by the BBC. But I haven't viewed more than a few of the 960x540 examples, so my findings may not be typical. > More importantly BBC Four and to a lesser extent BBC 2 have a lot of > programmes about paintings, sculpture, architecture and nature where > there is a lot of fine detail and little motion. Intuitively such > programmes would not benefit from the higher frame rate but would > benefit from the higher resolution. There is no mention in the blog > that such programmes were included in the viewing tests, and they ought > to have been. I'd agree. I'll see if I can pass forward some of these points. I need to be able to contact someone further 'up the stack' to ask more 'policy' questions I guess. I know more about the radio side, so I don't know how I'll get on with the tv side. [edit] Just sent an email making some of the points. I also added a comment of my own that I'd *much* prefer "Sky at Night" as the 1280x720 25fps than a lower resolution. Time will tell if I manage to get someone 'higher up' to be willing to discuss this with me. Jim -- Electronics https://www.st-andrews.ac.uk/~www_pa/Scots_Guide/intro/electron.htm Armstrong Audio http://www.audiomisc.co.uk/Armstrong/armstrong.html biog http://jcgl.orpheusweb.co.uk/history/ups_and_downs.html Audio Misc http://www.audiomisc.co.uk/index.html ___ get_iplayer mailing list get_iplayer@lists.infradead.org http://lists.infradead.org/mailman/listinfo/get_iplayer
Re: no more hslv format ?
Please see below ... On 02/05/2018 11:50, RS wrote: On 30/04/18 13:56, Jim web wrote: I've been discussing the 'loss' of the 1280x720 25fps version with someone at the BBC. IIUC this stemmed from 'Red Bee' days of yore, and until recently people at the BBC had thought they had stopped it long ago. Someone apparently noticed recently that it was still available. And then actually disabled it. Hence the mysterious recent ending of its availablity. I have made the point that the 1280x720 25fps version is useful for people with a content 'cap' problem, etc, so will be missed because the 50fps version means somewhat bigger files and/or stream rates. But I doubt this will cause a rethink. We have known for some time that the BBC was going to stop using Flash at some stage. Dinky anticipated that change and get_iplayer no longer supports Flash. The last version with Flash capability was v2.99. HLS is a much more recent innovation than Flash. get_iplayer now has its own built-in downloader which downloads HLS 2 or 3 times as fast as Flash. HLS has been referred to as a legacy mode so it was always likely that it would be removed eventually. The surprise was that HLS was removed at the same time as Flash. It is unrealistic to expect the BBC to restore HLS or Flash. Yes, it' s unrealistic to expect the BBC to respond to any criticism whatsoever, because they never have in the past whenever changes they've introduced have broken other people's kit such as Network Media Players and Smart TVs. I have not seen any comments here which suggest that anyone is unhappy with HVF. get_iplayer uses the same built-in downloader as for HLS. The BBC's full list of HVF modes is set out in a table in this document. https://www.bbc.co.uk/rd/blog/2015-07-the-development-of-new-video-factory-profiles-for-bbc-iplayer What I think a number of people here would like to see is the addition of one more High H.264 profile mode to that list, namely 1280x720p 25fps. Not *necessarily* - I think the problem is more subtle than that. See below ... The BBC refers to viewing tests it has carried out, "using a range of content and included clips, from popular shows such as Strictly Come Dancing, East Enders and Top Gear." It concludes, "At around 3 Mbit/s a 960x540 profile at 50fps will be made available to Connected TVs and set top boxes. The Elemental encoders are used to convert the 1920x1080 interlaced content to 960x540 for progressive encoding at 50fps. Although the 960x540 profile has a reduced spatial resolution compared to the outgoing 1280x720 at 25fps, subjective assessments shows it delivers significantly better pictures on TV screens across a wide range of popular content (such as EastEnders and Top Gear) due to its higher frame rate. The 50fps, 1280x720 profile, however, will be available to those with 5Mbit/s broadband connections. That's what I see as the problem, not a single natural history or art or historical documentary mentioned in the testing mix. For such programmes, I would suggest that often an increase in spatial resolution would be preferable to an increase in temporal resolution. I don't care how popular the above programmes are, I've not watched one of them in more than 20 years, and the other two ever. The BBC is a public service broadcaster, and shouldn't only be thinking about what is best for populist programming. They should be doing what is best for the content of a particular programme, or if that is too complicated and difficult, then, yes, let us have the choice of a 1280x720x25 profile, and let us choose which to download. I accept that there is a trade off between resolution and frame rate. With differential encoding schemes like H.264, at a higher frame rate the changes between frames are smaller, so there is less to encode. Even so, for the uncompressed video the resolution at 50fps would need to be reduced to 905x510p to give the same bit rate as 1280x720p at 25fps. For that reason I find it surprising that 960x540p at 50fps "delivers significantly better pictures on TV screens across a wide range of popular content (such as EastEnders and Top Gear) due to its higher frame rate" than 1280x720p at 25fps but the BBC has done the viewing tests and I have not. More importantly BBC Four and to a lesser extent BBC 2 have a lot of programmes about paintings, sculpture, architecture and nature where there is a lot of fine detail and little motion. Intuitively such programmes would not benefit from the higher frame rate but would benefit from the higher resolution. There is no mention in the blog that such programmes were included in the viewing tests, and they ought to have been. Yes, agreed (I wrote the above before I'd read your post entirely, which was probably not best practice, for which apologies, but I'm in something of a hurry). But there is also another elephant in the room, which is the capacity
Re: no more hslv format ?
On 30/04/18 13:56, Jim web wrote: I've been discussing the 'loss' of the 1280x720 25fps version with someone at the BBC. IIUC this stemmed from 'Red Bee' days of yore, and until recently people at the BBC had thought they had stopped it long ago. Someone apparently noticed recently that it was still available. And then actually disabled it. Hence the mysterious recent ending of its availablity. I have made the point that the 1280x720 25fps version is useful for people with a content 'cap' problem, etc, so will be missed because the 50fps version means somewhat bigger files and/or stream rates. But I doubt this will cause a rethink. We have known for some time that the BBC was going to stop using Flash at some stage. Dinky anticipated that change and get_iplayer no longer supports Flash. The last version with Flash capability was v2.99. HLS is a much more recent innovation than Flash. get_iplayer now has its own built-in downloader which downloads HLS 2 or 3 times as fast as Flash. HLS has been referred to as a legacy mode so it was always likely that it would be removed eventually. The surprise was that HLS was removed at the same time as Flash. It is unrealistic to expect the BBC to restore HLS or Flash. I have not seen any comments here which suggest that anyone is unhappy with HVF. get_iplayer uses the same built-in downloader as for HLS. The BBC's full list of HVF modes is set out in a table in this document. https://www.bbc.co.uk/rd/blog/2015-07-the-development-of-new-video-factory-profiles-for-bbc-iplayer What I think a number of people here would like to see is the addition of one more High H.264 profile mode to that list, namely 1280x720p 25fps. The BBC refers to viewing tests it has carried out, "using a range of content and included clips, from popular shows such as Strictly Come Dancing, East Enders and Top Gear." It concludes, "At around 3 Mbit/s a 960x540 profile at 50fps will be made available to Connected TVs and set top boxes. The Elemental encoders are used to convert the 1920x1080 interlaced content to 960x540 for progressive encoding at 50fps. Although the 960x540 profile has a reduced spatial resolution compared to the outgoing 1280x720 at 25fps, subjective assessments shows it delivers significantly better pictures on TV screens across a wide range of popular content (such as EastEnders and Top Gear) due to its higher frame rate. The 50fps, 1280x720 profile, however, will be available to those with 5Mbit/s broadband connections. "Additional lower bit-rate profiles will be made available to computers on wifi. This enables video playback to continue, even when an individual's available bit-rate is reduced by users sharing a connection. These profiles will also cater for similar bit-rate restrictions on public wifi connections." I accept that there is a trade off between resolution and frame rate. With differential encoding schemes like H.264, at a higher frame rate the changes between frames are smaller, so there is less to encode. Even so, for the uncompressed video the resolution at 50fps would need to be reduced to 905x510p to give the same bit rate as 1280x720p at 25fps. For that reason I find it surprising that 960x540p at 50fps "delivers significantly better pictures on TV screens across a wide range of popular content (such as EastEnders and Top Gear) due to its higher frame rate" than 1280x720p at 25fps but the BBC has done the viewing tests and I have not. More importantly BBC Four and to a lesser extent BBC 2 have a lot of programmes about paintings, sculpture, architecture and nature where there is a lot of fine detail and little motion. Intuitively such programmes would not benefit from the higher frame rate but would benefit from the higher resolution. There is no mention in the blog that such programmes were included in the viewing tests, and they ought to have been. Best wishes Richard ___ get_iplayer mailing list get_iplayer@lists.infradead.org http://lists.infradead.org/mailman/listinfo/get_iplayer
