Re: Just noticed the download title includes [legal]
On Wed Jul 17 22:22:58 BST 2019, Budge wrote: Just noticed this after pid when getting Men of Rock... No idea what it means or why it is there. Hi - you don't specify a PID in your report, but selecting for example the latest 3rd episode with --pid=b00wvjnq (BTW this is a repeat from 2010, hence a PID starting with "b00"...), I can simply tell you that "legal" is just the label/name of the episode VERSION that GiP managed to fetch; other episode versions "supposedly" available are Original version Sign language Dubbed Audio Described Dubbed Audio Described + Legal all easily findable in https://www.bbc.co.uk/programmes/b00wvjnq.json Via GiP itself: perl get_iplayer-321w.pl --type=tv --pid=b00wvjnq -i | FindStr versions => versions:audiodescribed,legal Since the default --file-prefix includes the substitution parameter, it's no wonder the .mp4's name contains "legal" ! So, nothing ominous/sinister really :-) Best regards ___ get_iplayer mailing list get_iplayer@lists.infradead.org http://lists.infradead.org/mailman/listinfo/get_iplayer
Re: New distro.
On Mon Jun 24 16:19:00 BST 2019, MacFH - C E Macfarlane wrote: with someone on XP being unable to run newer versions of GiP. I haven't myself tried to run the latest Windows installer on either XP or Vista; my guess is it would run to completion, successfully installing the application under those OSes... See Wiki notes: https://github.com/get-iplayer/get_iplayer/wiki/windows#before-you-begin Latest installer bundles a 32-bit binary of FFmpeg 4.1.1, compiled by Zeranoe... That executable won't run in either XP or Vista and has to be replaced with a compatible compile... FFmpeg vanilla code has dropped XP support in branch 4.x.x, but remains Vista compatible; the reason 4.1.1 won't launch under Vista is mainly because third party library libx265 (video encoder) has been configured with NUMA (Non-Uniform Memory Access) support, a feature absent under Vista+XP: https://docs.microsoft.com/en-us/windows/desktop/procthread/numa-support Windows Server 2008, Windows Vista, Windows Server 2003 and Windows XP: Processor groups are not supported. Finding a recent XP and/or Vista compatible FFmpeg binary is not easy; XP die-hards do offer some builds, one has to know where to look for them; e.g. http://blog.k-tai-douga.com/category/359294-1.html https://sourceforge.net/projects/mplayer-win32/files/FFmpeg/ https://rwijnsma.home.xs4all.nl/files/ffmpeg/ Vista (but not XP) compatible, automated, daily builds: https://jenkins.maeyanie.com/job/ffmpeg/ However, in the context of GiP one doesn't need the most up-to-date code, because, as already told by others, the ffmpeg binary inside GiP doesn't access the web but is normally solely used for (off-line) remuxing purposes; that part of the code is mature enough in the last XP (v3.0) and Vista (v3.0.1) compatible binaries available in the VideoHelp archive: https://www.videohelp.com/software/ffmpeg/old-versions#downloadold Greetings ___ get_iplayer mailing list get_iplayer@lists.infradead.org http://lists.infradead.org/mailman/listinfo/get_iplayer
Re: New distro.
On Mon Jun 24 11:25:50 BST 2019, MacFH - C E Macfarlane wrote: I'm surprised it worked for you, because AFAICR I'd understood that the version of Strawberry Perl used was supposed not to support XP. ... The latest GiP installer (v3.20.0) bundles a self-contained (portable) mini-package of Strawberry Perl 5.26.2.1-win32, back from mid-April 2018: https://imgur.com/AZZC0PF http://strawberryperl.com/release-notes/5.26.2.1-32bit.html As you might see, the Release Notes do NOT specify a minimum winOS version as "system requirements"; I was able to launch perl 5.26.2.1 successfully under Vista SP2 x86, but, sadly, am nowhere near an XP box to test; my gut feeling is it should properly launch there, too (I'd expect one to have kept the XP sytem as updated as possible - MSVC++ redistributables installed, etc.). The GiP installer has been compiled in the InnoSetup format, one can decompile it - for testing purposes - without actually running it via the latest version of Inno Setup Unpacker: http://innounp.sourceforge.net/ Best regards ___ get_iplayer mailing list get_iplayer@lists.infradead.org http://lists.infradead.org/mailman/listinfo/get_iplayer
Re: Poodcast that isn't a podcast
On Wed Jan 16 18:56:20 GMT 2019, George Eycott wrote: and being audio it will be quicker to let it get on with it and delete them afterwards than it will be to work out a way around it From 3.18's Long Help file: --mark-downloaded : Mark programmes in search results or specified with --pid/--url as downloaded by inserting records in download history. So something like: perl get_iplayer-318w.pl --pid=p04x5pd7 --pid-recursive --mark-downloaded will insert "history" entries (but not actually fetch audio files) for all 80 available episodes; takes 15sec, max! Then, properly fetch the latest episode(s) you're after via --pid = --force; finally, setup a pid-recursive WebPVR recording for series pid=p04x5pd7 and you're good to go... Hopefully this will save others bandwidth, time, disk space etc... Best regards ___ get_iplayer mailing list get_iplayer@lists.infradead.org http://lists.infradead.org/mailman/listinfo/get_iplayer
Re: BBC-wants-shows-available-iPlayer-12-months-bid-compete-rivals
On Thu Jan 10 08:38:45 GMT 2019, David Lake wrote: they use the same CDN providers as all the others - Akamai, Limelight and Level 3. I don't think Level3 is anymore in the picture; as I recall, it was exclusively used for one of the FlashHD tvmodes and L3-hosted streams were the first to be axed when stream management was delegated to "Video Factory"; an Akamai-hosted FlashHD stream remained available for quite a while afterwards, until someone at the beeb belatedly pulled the plug on it, as well as on the Akamai-hosted "legacy" HlsHD streams (effectively killing all sources for the 720p25 TV mode :-( ). And apart from commercial Akamai+Limelight CDNs, BBC manages its own BIDI CDN, for clients within the UK: http://www.bbc.co.uk/blogs/internet/entries/8c6c2414-df7a-4ad7-bd2e-dbe481da3633 A quite interesting source of BBC iPlayer related tech info (aimed mostly at coders) is https://iplayer.engineering/ From there, certainly worth reading: https://iplayer.engineering/the-tech-stack-that-powers-bbc-iplayer-471575318e2a Best regards, V. ___ get_iplayer mailing list get_iplayer@lists.infradead.org http://lists.infradead.org/mailman/listinfo/get_iplayer
Re: BBC-wants-shows-available-iPlayer-12-months-bid-compete-rivals
On Tue Jan 8 16:28:02 GMT 2019, James Scholes wrote: It's perhaps worth noting that, in the case of BBC Store, downloads were encrypted but the actual streams that you could watch on-demand were not. Hi James, thanks for clarifying on that! Now that you've mentioned it, I do recollect faintly that the BBC Store "mediaselector API" URI was specially protected, dependent on a logged-in user/buyer cookie or something of the sort, I just couldn't remember whether the actual stream itself was DRM'd or not - so I gambled (and lost!) that it was... I also wrote a (unreleased) Kodi add-on I realise this is OT, but did you come up with something similar for the 4K UHD VOD streams for the "Dynasties" series? https://forum.kodi.tv/showthread.php?tid=239378&pid=2796801#pid2796801 (This is best continued off-list if you did...) So maybe that provides some hope that the iPlayer will remain DRM-free. ... Fingers crossed! Best regards ___ get_iplayer mailing list get_iplayer@lists.infradead.org http://lists.infradead.org/mailman/listinfo/get_iplayer
Re: BBC-wants-shows-available-iPlayer-12-months-bid-compete-rivals
On Tue Jan 8 08:52:54 GMT 2019, CJB wrote:
https://www.dailymail.co.uk/news/article-6566501/BBC-wants-shows-available-iPlayer-12-months-bid-compete-rivals.html
From that:
to compete with Netflix and Amazon Prime
... I am surprised it was not brought up in this discussion,
but these BBC "competitors" exclusively use (unbreakable)
DRM protected streams/downloads EVERYWHERE!
Channel 4 was the first commercial TV station to
switch to DRM protected streams ("Flash Access",
now renamed "Adobe Primetime"), to be soon followed
by Channel 5 (probably using MPEG-DASH with WV).
In the spring of last year, ITV switched to Widevine DRM
for their Live TV streams and to MPEG-DASH/ClearKey for
VOD (mobile streams still on HLSe "softer" DRM).
Towards the end of last year, another on-line UK TV
portal, UKTVPlay, switched to enhanced DRM schemes
(MPEG-DASH/WV, Fairlplay) and I am informed Ireland's
iPlayer equivalent,
https://beta.rte.ie/player/
has also switched to (uncrackable) DRM streams...
BBC had already imposed DRM on their (now closed)
BBC Store bought streams, and continue to do so in
their iPlayer Downloads (for PC) and BBC Sounds
Apps.
If the current industry trend is anything to go by,
I fear content providers/rights holders would ask for extended
protection to be implemented to their offerings, if they were
to OK the extended availability period...
Obviously I am not knowledgeable with regards
to rights negotiations, but is it only a matter for Ofcom
to agree upon the one-year-on-line availability proposal?
If affirmative, wouldn't the beeb have to renegotiate online
rights with content providers, for an increased price-tag
(eventually translated to an increased TV licence fee)?
Eventual DRM implementation on BBC online streams
is, for me, the only issue that might be get_iplayer related
(i.e. it would kill it :-( ).
Of course, if the proposal goes through without
such bad surprises, then GiP would've to be patched
to cater to the 365 days of available TV+radio programmes
Massive storage required for this
AFA GiP is concerned, only an increase would be observed
in the sizes of the cache files and some (slight) increase in the
time required to refresh them, especially when rebuilding
the one-year cache from scratch :-)
Regards
___
get_iplayer mailing list
get_iplayer@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/get_iplayer
[Announcement]GiP 3.18 released!
... by maintainer @dinkypumpkin/@notnac Changelog: https://github.com/get-iplayer/get_iplayer/compare/v3.17...v3.18 Release Notes: https://github.com/get-iplayer/get_iplayer/wiki/release310to319#release318 Packages: https://github.com/get-iplayer/get_iplayer_win32/releases/tag/3.18.0 https://github.com/get-iplayer/get_iplayer_macos/releases/tag/3.18.0 Source code: https://github.com/get-iplayer/get_iplayer/archive/v3.18.tar.gz Happy New Year to all members here :-) Vangelis ___ get_iplayer mailing list get_iplayer@lists.infradead.org http://lists.infradead.org/mailman/listinfo/get_iplayer
Re: Fake get_iplayer mail
On Mon Nov 12 15:26:36 GMT 2018, artisticforge Niemand wrote: I have seen about 12 dozen in the last month. Hi Terry :-) ; 144 spam e-mails in the course of a month is an auful lot, TBH ... :-( I surmise this is with your gmail account ? My e-mail account is not free webmail, but provided by my ISP, as part of their service; they do filter spam adequately before it even reaches my inbox... I am pleased I don't get spam from this list, given that my e-mail address (as much as any other member's) is human readable in both iterations of the list archives... Best wishes ___ get_iplayer mailing list get_iplayer@lists.infradead.org http://lists.infradead.org/mailman/listinfo/get_iplayer
Re: Fake get_iplayer mail
On Mon Nov 12 14:55:31 GMT 2018, Dave Widgery wrote: The strange this is that I am not getting these on my current registered get_iplayer email address ... Likewise here; except for the odd spam mail that somehow manages to get through the list's filters (last one was in Sept, http://lists.infradead.org/pipermail/get_iplayer/2018-September/011988.html NOTHING malicious there, just spam), I have not got anything suspicious recently pretending to have come from this list :-) but on a different address that I stopped using with get_iplayer a few years ago. It's possible THAT ONE got hacked/harvested by a spam bot; I fear e-mails and related software is not a field I am expert in, this list contains many other members highly learned in this sector who, no doubt, are able to offer you assistance towards addressing the issue with that compromised e-mail account... Kind regards ___ get_iplayer mailing list get_iplayer@lists.infradead.org http://lists.infradead.org/mailman/listinfo/get_iplayer
Re: Downloading Subtitles
On Mon, 8 Oct 2018 00:24:29 BST, Ray Penn wrote: I always used to make use of the --subsonly option. Does that still work? On Mon, Oct 8 00:41:00 BST 2018, J K.Eason wrote: but there's no mention of --subsonly on the page I linked to. I assume it's been superseded by --subtitles-only. According to https://github.com/get-iplayer/get_iplayer/blob/57ac0a5b92c926277d53d591e0d46659321583e2/get_iplayer#L144 --subsonly should still be valid! (As should be --subtitlesonly --subs-only ) Best regards, V. ___ get_iplayer mailing list get_iplayer@lists.infradead.org http://lists.infradead.org/mailman/listinfo/get_iplayer
Re: Downloading Subtitles
On Mon Oct 8 00:18:40 BST 2018, Peter S Kirk wrote: Until recently, subtitles were downloaded first thus get_i pvr could be stopped once done. Now they are downloaded after video. Why? ... Because the powers that be (i.e. the one and only person still maintaining the code of GiP) decreed so: https://github.com/get-iplayer/get_iplayer/commit/35d8e7b4d23815b85937727868bd9364ae829f0a The logic behind that change appears to be # Get subtitles if they exist and are required and media download succeeded i.e. the author ASSUMES you wouldn't want the subs (only) if the actual media download failed... The status quo ante was: # Get subtitles if they exist and are required # best to do this before streaming file so that the subtitles can be enjoyed while recording progresses In any case, even back when the subs were fetched before media download completion, they still had the ".partial.srt" suffix and that was only removed when the video itself was fully dumped to file... And it wasn't a "recent"change, 't was more than a year ago... (... I couldn't help noticing the comment to the aforelinked commit: https://github.com/get-iplayer/get_iplayer/commit/35d8e7b4d23815b85937727868bd9364ae829f0a#commitcomment-25039704 which, of course, remains unanswered to this day...) Best regards, Vangelis. ___ get_iplayer mailing list get_iplayer@lists.infradead.org http://lists.infradead.org/mailman/listinfo/get_iplayer
Re: dvf mode only downloads audio without video
On Thu Aug 23 09:16:31 BST 2018, Live Musicfs wrote: but the dvf mode never work for me because it only download audio without video. Hello there :-) "dvf " tvmodes use sequential downloading of separate audio+video raw streams (in iso-dash container) and then the standalone downloaded streams are muxed locally into a proper MP4 container, via FFmpeg. To aid troubleshooting, you should first supply some info about your OS, version of GiP, whether you use the GUI or CLI and, preferably, exact command used... As terry suggested, creating and sharing a verbose log (preferably in a "pastebin" type hoster, or as a .txt attachment to your e-mail) will, hopefully, shed some more light onto your predicament... Using a custom Windows installation on Vista SP2 32-bit, issuing perl get_iplayer-317w.pl --type=tv --pid=p06g9pmw --tvmode=dvflow2 as a test download, I get: get_iplayer 3.17.0-MSWin32, Copyright (C) 2008-2010 Phil Lewis This program comes with ABSOLUTELY NO WARRANTY; for details use --warranty. This is free software, and you are welcome to redistribute it under certain conditions; use --conditions for details. Episodes: European Championships - 2018 Extra, Diving: Men's Synchronised 10m Platform fin al, BBC iPlayer, p06g9pmw INFO: 1 total programmes WARNING: A UK TV licence is required to access BBC iPlayer TV content legally INFO: Downloading tv: 'European Championships: 2018 Extra - Diving: Men's Synchr onised 10m Platform final (p06g9pmw) [original]' 3.7% of ~28.66 MB @ 0.8 Mb/s ETA: 00:04:35 (dvflow2/ak) [audio] If I let the audio stream (28.66 MB) download finish to completion, GiP proceeds to downloading the video stream (130.44 MB): INFO: Downloaded: 28.56 MB (00:39:48) @ 1.15 Mb/s (dvflow2/ak) [audio] 1.8% of ~130.44 MB @ 1.5 Mb/s ETA: 00:11:23 (dvflow2/ak) [video] I CTRL+C'ed the video download afterwards, having no desire for a full programme fetch... Have you waited for the audio stream download to complete first? What happens in your case after that? Is there an "--audio-only" switch inside your preferences (from memory, I can't recall whether that switch also applies to dvf tvmodes, or only to hvf ones...)? but for some programme, iplayer only provides dvf mode, for example: p06g9pmw ... Actually, this is not true, at least for the PID referenced; while mediaset=pc only yields DASH streams https://open.live.bbc.co.uk/mediaselector/5/select/version/2.0/mediaset/pc/vpid/p06g9pn0 and, indeed, mediaset=iptv-all doesn't yield ANY streams at all: https://open.live.bbc.co.uk/mediaselector/5/select/version/2.0/mediaset/iptv-all/vpid/p06g9pn0 mediaset=apple-ipad-hls DOES yield HLS (of the hvf type) streams: https://open.live.bbc.co.uk/mediaselector/5/select/version/2.0/mediaset/apple-ipad-hls/vpid/p06g9pn0 Due to changes implemented in GiP 3.13+, these streams are NOT DETECTED by default, unless the "--stream-http" flag is used: = perl get_iplayer-312w.pl --type=tv --pid=p06g9pmw -i | FindStr modes => modes: original: dvfhd1,dvfhd2,dvfsd1,dvfsd2,dvfxsd1,dvfxsd2,dvfxhigh1, dvfxhigh2,dvflow1,dvflow2,hvfhd1,hvfhd2,hvfhd3,hvfsd1,hvfsd2,hvfsd3,hvfxsd1,hvfx sd2,hvfxsd3,hvfxhigh1,hvfxhigh2,hvfxhigh3,hvfstd1,hvfstd2,hvfstd3,hvflow1,hvflow 2,hvflow3 = whereas: = perl get_iplayer-317w.pl --type=tv --pid=p06g9pmw -i | FindStr modes => modes: original: dvfhd1,dvfhd2,dvfsd1,dvfsd2,dvfxsd1,dvfxsd2,dvfxhigh1 ,dvfxhigh2,dvflow1,dvflow2 = but: = perl get_iplayer-317w.pl --type=tv --pid=p06g9pmw --stream-http -i | FindStr modes => modes: original: hvfhd1,hvfhd2,hvfhd3,dvfhd1,dvfhd2,hvfsd1,hvfsd2,hvfs d3,dvfsd1,dvfsd2,hvfxsd1,hvfxsd2,hvfxsd3,dvfxsd1,dvfxsd2,hvfxhigh1,hvfxhigh2,hvf xhigh3,dvfxhigh1,dvfxhigh2,hvflow1,hvflow2,hvflow3,dvflow1,dvflow2 = Best regards, Vangelis. ___ get_iplayer mailing list get_iplayer@lists.infradead.org http://lists.infradead.org/mailman/listinfo/get_iplayer
Re: BBC Radio Scotland, Highlands and Islands schedule variation discontinued...
@Alan & @george, thanks for the feedback :-) It may well look as though, from Aug 1st 2018, the BBC Radio Highlands and Islands variation HAS BEEN DISCONTINUED! (snip) if you're on latest GiP (3.16) and want to prevent it from trying to scrape a non-existent schedule page (for BBC Radio Scotland, H & I variation), you can comment out (#) line 7303 of the vanilla 3.16 script (snip) I am certain a future release of GiP will take care of this in a proper fashion... This has now appeared in the "Known Issues" section of the wiki: https://github.com/get-iplayer/get_iplayer/wiki/issues#get_iplayer-known-issues 2018-08-06 (v3.16): From 2018-08-01, there is no longer a broadcast schedule published for BBC Radio Scotland Highland and Islands. This leads to error messages during cache updates: ERROR: Failed to download BBC Radio Scotland schedule page (3/3): https://www.bbc.co.uk/schedules/p00fzl8f/2018/w32 ERROR: Response: 404 Not Found The error messages can be ignored. The affected schedule page URLs will be removed in the next release of get_iplayer. V. ___ get_iplayer mailing list get_iplayer@lists.infradead.org http://lists.infradead.org/mailman/listinfo/get_iplayer
BBC Radio Scotland, Highlands and Islands schedule variation discontinued...
Hello all :-) Being physically located many thousands of kilometers away from Scotland, I am in no position to check this myself, but an internet search has shown that (regional) BBC Radio Scotland comes in five schedule variations: BBC Radio Orkney BBC Radio Shetland BBC Radio Highlands and Islands BBC Radio Scotland FM BBC Radio Scotland MW When GiP refreshes the radio cache and Regional Stations have not been excluded [--refresh-exclude-groups-radio=regional], all 5 schedule variations for BBC Radio Scotland are being web-scraped. It may well look as though, from Aug 1st 2018, the BBC Radio Highlands and Islands variation HAS BEEN DISCONTINUED! For populating the caches, I am currently on a heavily modded older version of GiP (3.01+personal patches); when I refreshed the radio cache manually, I noticed the following: WARNING: Failed to download schedule page: http://www.bbc.co.uk/radioscotland/programmes/schedules/highlandsandislands/this_week That URL redirects in a browser to https://www.bbc.co.uk/schedules/p00fzl8f/2018/w32 which, when visited, informs that: Highlands and Islands Schedule Broadcast schedule ended on Tuesday 31 July 2018 The redirection to PID-based schedule page URLs was implemented already in GiP 3.14: https://github.com/get-iplayer/get_iplayer/commit/aa8fd57 Recent versions of GiP have become very silent in their output, perhaps you don't get any warning with 3.16 when you manually refresh the radio cache and no schedule is returned for BBC Radio Highlands and Islands currently. Running get_iplayer --type=radio --fields=channel "BBC Radio Scotland" I get "354 Matching Programmes", but it's hard to tell how many of them are unique to the H & I variation; if you're on latest GiP (3.16) and want to prevent it from trying to scrape a non-existent schedule page (for BBC Radio Scotland, H & I variation), you can comment out (#) line 7303 of the vanilla 3.16 script: 'p00fzl8f' => 'BBC Radio Scotland', # radioscotland/programmes/schedules/highlandsandislands I am certain a future release of GiP will take care of this in a proper fashion... Best regards, Vangelis ___ get_iplayer mailing list get_iplayer@lists.infradead.org http://lists.infradead.org/mailman/listinfo/get_iplayer
Re: Re: Proms puzzles
On Tue Jul 24 15:13:06 BST 2018, Jim Lesurf wrote:
and they show a little 'clock' item and a text showing something
like '2d' which I take to mean 'available for another two days'.
... Wrong! "2d" designates the rough time (in calendar days)
that has passed since the audio | video clip ("excerpt" in your
terminology) was first made available (uploaded) on that
page (i.e. iPlayerRadio)
E.g. Prom 5, on Tue 17 Jul 2018
https://www.bbc.co.uk/events/ecc6gw =>
https://www.bbc.co.uk/events/ecc6gw/play/ahfxc8/p06f928w [5d]
Now, the correct pid is the final string, so --pid=p06f928w
Navigate to this URL
https://www.bbc.co.uk/programmes/p06f928w
and you'll find the audio clip was uploaded on 20 July 2018
(ca. 5 days ago). You can always --info a pid to query its expiry date:
get_iplayer --type=radio --pid=p06f928w -i | FindStr expires =>
expires:in 80y 159 days 0 hours (2099-01-01T00:00:00+00:00)
Another example:
Prom 9, on Sat 21 Jul 2018
https://www.bbc.co.uk/events/emrz3d =>
https://www.bbc.co.uk/events/emrz3d/play/aw2mn3/b0bbp9kx [4d]
https://www.bbc.co.uk/programmes/b0bbp9kx
This audio excerpt was first broadcast on 21/07/2018, 19:30 BST,
if you --info the pid and search for "firstbcastrel" you'll find
firstbcastrel: 4 days 5 hours ago
The integer approximation of this time interval is 4days,
hence "4d" next to the clock sign; as you can see yourself,
this clip expires in 25 days, not in 4...
*But* if I try their PIDs I find GIP fails to fetch them.
Instead it tells me that the HAF modes aren't available.
Yet this didn't happen for the first few proms.
Questions here are why, and am I doing something wrong?
Are you sure you are passing the correct PID string to GiP?
Please post, in detail, non-working clips, and I'm sure other
competent list members (if not me again) can test and
advise further...
Cool greetings (to ease the UK heat, that is...)
___
get_iplayer mailing list
get_iplayer@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/get_iplayer
Re: Tagging from History
On Sun Jul 15 15:52:31 BST 2018, RS wrote: He says the file needs to be in the original location, and gives a workaround if it is not. He also says the --tag-podcast-radio is needed. Hi Alastair, Richard et company :-) Yes, I am still alive (however numerous real life and additional health issues are currently plaguing me...) The reasons behind my vanishing act from the list will remain, for the moment at least, undisclosed; let me assure you they are not related in any sense with anyone among the list members, especially with those more "permanent" ones that I, dare say, became closer with over the course of the years frequenting this list; it sort of shakes me emotionally when I still see my name mentioned here in a positive manner, although I do realise I wasn't always pleasant to all, how could I have been? Extended apologies to my, undoubtedly, best friend here amomg you all, Alan, for failing to match his friendship and failing to respond to his off-list queries about my whereabouts/well-being... A, probably long, apologetic personal mail is scheduled sometime later, when I commit myself to it... Alan: THANKS! Besides Alan, I had /have a soft spot for Alastair (Budge), because he was /is always so polite, kind and thankful; I just broke my silence here because of him, to finish off what started in the referenced 2015 reply of mine... I won't offer a detailed analysis here (that I was infamous for), but, as you may be aware, huge changes were brought upon get_iplayer since 2015 (and not all of them for the better...), when my linked solution was relevant. Using a fairly recent version of GiP, you can tag an untagged (at the time of download) (tv|radio) BBC file if you are in possession of only one piece of data: the PID string. The file doesn't have to reside in the original downloads folder, the programme it pertains to doesn't have to be present inside your (tv|radio).cache, doesn't have to be present in your history file (should be there by default, though), so the history needs not be searched. relevant options: --tag-only Only update the programme metadata tag and not download the programme. Use with --history or --tag-only-filename. --tag-only-filename Add metadata tags to specified file (ignored unless used with --tag-only) : Full path to the current location of untagged file (doesn't have to be original location; just indicate to GiP the (current) location of the file to be tagged) --tag-podcast-radio Tag only downloaded radio programmes as iTunes podcasts (needed when you want to import the radio programme to iTunes; I find it doesn't hurt to have this switch on by default, hence I recommended it...). --tag-tracklist Add track list of music played in programme (if available) to lyrics field. --tag-credits Add programme credits (if available) to lyrics field. (useful mostly for TV, but radio progs may have credits, too). For audio downloads, I prefer square thumbnails, so of use also: --thumbnail-square Download square version of thumbnail image. Example: Say I downloaded --pid=w3cswrxf with the --no-tag flag; downloaded file is untagged; I then archived the file to a new location "D:\Vangelis\Chart Show\535 - TOC,06-07-18" with a modified file name: "BBCWS_TOTPs_Florence&TheMachine_20180707030600Z[w3cswrxf].m4a" If I want to tag (or re-tag) the file at its new location, I'll use get_iplayer --type=radio --pid=w3cswrxf --tag-only --tag-podcast-radio --tag-only-filename="D:\Vangelis\Chart Show\535 - TOC,06-07-18\BBCWS_TOTPs_Florence&TheMachine_20180707030600Z[w3cswrxf].m4a" --tag-tracklist --tag-credits --thumbnail-square --thumbnail-size 384 Recent versions of GiP have become really stingy with their verbosity, use --verbose to see more details... PS: The TOTPs World Service show, when initially uploaded on a Saturday morning, only has a minimal set of metadata available; it takes several days for someone to manually update the metadata to its full set, hence I find I have to retag the file at a second stage... One additional detail: when AP retags a file, it adds a second thumbnail if the file was originally tagged without the --no-artwork flag; all other metadata fields are overwritten with new values; as said in the past, your best choice to inspect the metadata is MediaInfo (other apps may be used...). Best regards ___ get_iplayer mailing list get_iplayer@lists.infradead.org http://lists.infradead.org/mailman/listinfo/get_iplayer
Re: parser error
On Fri Nov 3 12:19:35 GMT 2017, RS wrote: but is seems more likely that the BBC would respond to something causing an error in the iPlayer. The BBC does correct errors. Hi Richard :-) by opening http://www.bbc.co.uk/iplayer/subtitles/ng/modav/bUnknown-5df25dc8-d38f-43e5-93a2-38b6c778f852_b09c79wx_1509625417009.xml one can read: Created on 2/11/2017 at 12:23:23 so this was just fixed at noon yesterday (most recent Suspicion repeat aired on 22/10/2017@13:30; so it took them eleven days to identify and remedy the problem)... I have no doubt an iPlayer user did alert them; however, this is not mentioned as a "Recently Fixed Fault" at: https://www.bbc.co.uk/iplayer/help/programme-availability/programme-issues What is more interesting is that neither the file you refer to nor the captions file it links to (URI snipped) are XML files. ... Well, I'm a complete dunce with regards to XML structure, but when I see an .xml extension in the URI, I "assume" it points to an ".xml" file... FWIW, the "belisage" article you referenced says the existing XML standard for timed text, TTML This then got me to https://www.w3.org/TR/ttml1/#content-attribute-id https://www.w3.org/TR/2005/REC-xml-id-20050909/ https://www.w3.org/TR/ttml1/#content-attribute-lang https://www.w3.org/TR/ttml1/#content-attribute-space When viewed in Firefox, as you said The captions file begins - so xml:lang does imply it's an XML subset... Of course, Ralph came to the rescue for both of us, as fetching the file on disk and viewing it with an editor (BTW, I use PSPad), I do see first lines being http://www.w3.org/2006/10/ttaf1"; xmlns:ttp="http://www.w3.org/2006/10/ttaf1#parameter"; and it's those bits that Fx omits ;-( where - is a dash character I can't copy. Other tags are preceded by a similar dash character. This "dash" character you are referring to is not actually present inside the XML files, but it's added by Fx; by clicking it, you can collapse/expand content between matching tags; clicking turns it into a plus sign and the XML element's content gets hidden: - SUSPICION - BRD00 Ericsson 2017 turns into + As you can infer, default behaviour is the expanded state... Kindest regards, Vangelis. ___ get_iplayer mailing list get_iplayer@lists.infradead.org http://lists.infradead.org/mailman/listinfo/get_iplayer
Re: parser error
On Fri Oct 27 22:16:03 BST 2017, Ralph Corderoy wrote: If the BBC haven't already been informed that a particular URL serves broken XML then that's the first thing to change, including pointing out the NUL bytes that are causing the problem. I'm sure they'd like to work out what went wrong, and stop it happening again. It looks as though the problem has been fixed upstream! After navigating to (geo-filtered): http://open.live.bbc.co.uk/mediaselector/5/select/version/2.0/mediaset/iptv-all/vpid/b09c79wx all three "connection href"s for service="captions" load and render perfectly now in Firefox, without generating an XML Parsing Error... Someone from the BBC staff does browse this list or was it perhaps an in-house find? Regards, Vangelis. ___ get_iplayer mailing list get_iplayer@lists.infradead.org http://lists.infradead.org/mailman/listinfo/get_iplayer
Re: Proxy trouble with latest version
On Thu Oct 26 09:51:53 BST 2017, Charles Johnson wrote: Proxy trouble with latest version Hi Charles, there's some ambivalence there, are you implying things worked in the same setup but with a previous GiP version? What was that then, before updating to latest (3.06)? In the 3.05 -> 3.06 changelog nothing obvious (proxy related) stands out; going back to older versions, the most recent proxy-related commit is https://github.com/get-iplayer/get_iplayer/commit/3f8fb93 which took place in the transition 3.01 -> 3.02 ... On Thu Oct 26 12:24:21 BST 2017, Charles Johnson wrote The result is that the message now awaits moderator approval owing to its size This is a misleading message; if the size of the mail body or size of attachments exceeds a pre-configured max value, that message is printed to poster, but in reality the original infringing post will never make it to the list (speaking from experience here...). For a voluminous text output, better use a third party service to upload, then just use the generated link inside a list post to share. On Thu Oct 26 12:31:49 BST 2017, Charles Johnson wrote: I'm using an SSH tunnel too and originally wrote: get_iplayer --proxy=http://127.0.0.1:1080 Port 1080 already was a hint for me, but it was helpful to indicate the use of an SSH tunnel. An SSH tunnel, properly configured, creates a SOCKS4/5 proxy, by default listening on port 1080. I can see you're not using Windows, most sadly I have very little knowledge about other OSes, but I still think you'll find parts of the following exchange helpful: https://github.com/GetiPlayerAutomator/get-iplayer-automator/issues/483 I am hand-picking some relevant bits: As far as Socks proxy support in GiP is concerned..., don't get me started! NOWHERE in GiP's --longhelp, Man Page, wiki, documentation etc. exists even a mere mention of "Socks" :-( Only the --proxy option is briefly covered... As you know very well, the talk of Proxy (be it HTTP or SOCKS) or VPN is strictly off-limits on the mailing list, SquarePenguin's Support Forum and the GitHub issue tracker Furthermore, GiP lacks global support for SOCKS4 (that I am aware); --proxy (-p) is reserved only for HTTP(S) proxies - this isn't the case with, say, get-flash-videos, where an incantation like: --proxy socks://proxyhost:proxyport redirects both HTTP(S) & RTMP traffic through the SOCKS4 proxy... Through the SSH tunnel you can enable a SOCKS4/5 proxy connection to the remote server Only those apps on your machine that need to be proxied to the UK (browsers, downloaders etc.) and are compatible with SOCKS would have to be configured individually to use the proxy You can use proxy chaining to proxy apps that do not support SOCKS (like GiP). On Windows, I used Privoxy (Mac OS X version available), a local HTTP proxy with default port=8118, and edited its main configuration file by adding the line: forward-socks5 / 127.0.0.1:1080 . This actually forwards the HTTP traffic Privoxy receives to the SOCKS proxy, itself forwarding it to the SSH server So, you can use GiP get_iplayer -p http://localhost:8118 I am sure more knowledgeable list members with non-Windows OSes may offer additional advice... Regards, Vangelis. ___ get_iplayer mailing list get_iplayer@lists.infradead.org http://lists.infradead.org/mailman/listinfo/get_iplayer
Re: parser error
On Thu Oct 26 00:51:15 BST 2017, RS wrote: I have received an email from someone who told me the Suspicion subtitles download fine on his XP installation with v3.01. When I use v3.01 I still get the problem. He also mentioned that there was something in the v3.01 release notes about changes to subtitle handling. I used --subsfmt=default and the subtitles downloaded without problem. I can't do that in v3.05 because --subsfmt has been removed. Hi Richard This is consistent with what I previously posted: of note is the fact that every GiP version from 3.00 onwards does fail to convert this corrupted subtitles file, but, lo-and-behold, v2.99 does so successfully: 2.99 was the last to use (by default) the old XML parsing code; in 3.00 the new "coloured subtitles" feature was implemented, introducing new XML parsing code: https://github.com/get-iplayer/get_iplayer/wiki/release300to309#2-subtitles-now-in-colour The subtitles conversion in get_iplayer has been re-implemented using XML::LibXML Fall-back to the old code was still kept in both 3.00/3.01 via the --subsfmt=default option. However, that option is deprecated and will be removed in a future release That was done in 3.02+ https://github.com/get-iplayer/get_iplayer/wiki/release300to309#changes-in-302 Removed deprecated options: --subsfmt Noticed this in the 3.00 Release Notes: so if you find programmes whose subtitles can't be processed with the new implementation, report them in the forums. I suspect it's now too late, since the "new" implementation has been the only one since 3.02+, but I suspect it wouldn't hurt to let the maintainer know about this current occurrence... Best regards, Vangelis ___ get_iplayer mailing list get_iplayer@lists.infradead.org http://lists.infradead.org/mailman/listinfo/get_iplayer
Re: parser error
On Tue Oct 24 20:35:11 BST 2017, RS wrote: The resultant .mp4 file can be played in VLC, but MediaInfo shows no metadata. Hello Richard :-) If you ended up, for whatever reason, with an untagged file, you can always (re-)tag post download with the --tag-only switch: get_iplayer --type=video --pid=b00gmlrx --tag-only --tag-podcast-tv --tag-only-filename="path\to\Suspicion.mp4" (I assume you renamed the "Suspicion.partial.mp4" to just "Suspicion.mp4") The programme is the editorial version of the 1941 Hitchcock film Suspicion, b00gmlrx pid=b00gmlrx => vpid=b09c79wx (needed later...) Does anyone have any idea what causes a parser error? Answered by Colin; some further analysis below... On Tue Oct 24 21:41:54 BST 2017, RS wrote: I'm glad I asked because I hadn't realised that was where subtitles came from. I had assumed there was a ready-made .srt file to download. On-line media portals (like iPlayer) rarely use the .srt (subrip text) format, because it's usually incompatible with their embedded player (Flash based/HTML5 one); I'm certainly not an expert on this subject, but Flash based players usually require an XML caption file (referred to also as DFXP), while HTML5 ones may use the WebVTT (.vtt) format. DFXP is s a timed-text format that was developed by W3C (stands for "Distribution Format Exchange Profile"); it is currently referred to as TTML, read more at: https://en.wikipedia.org/wiki/Timed_Text_Markup_Language GiP will use mediaselector URLs (which contain the vpid string) to retrieve the URIs pointing to the iPlayer ttml files; PC/iptv-all/apple-ipad-hls mediasets are tried. The URI you included in your original post will be found, e.g., in http://open.live.bbc.co.uk/mediaselector/5/select/version/2.0/mediaset/iptv-all/vpid/b09c79wx (geo-filtered) in the I see from --info there are three subtitle modes. I used GiP 3.05 and the following command: perl get_iplayer-305w.pl --type=tv --pid=b00gmlrx -i --streaminfo > Streams.txt 2>&1 and yes, there are 3 captions modes identified, but, alas, I can sure tell there's a bug in the detection scheme somewhere; no sign of the legacy format, plus there's duplication, as subtitles3=subtitles1 == stream: subtitles1 bitrate: expires:2017-11-21T14:05:00Z ext:srt priority: 20 size: 118212 streamer: http streamurl: http://vod-sub-uk-live.bbcfmt.hs.llnwd.net/iplayer/subtitles/ng/moda v/bUnknown-591e0c64-779b-4f16-9582-bd3bc6c441bd_b09c79wx_1508034118207.xml?s=150 8878211&e=1508921411&h=c1d8bb45cd85f418d83103af0ef1979a type: (captions) http stream (CDN: mf_limelight_uk_plain/20) stream: subtitles2 bitrate: expires:2017-11-21T14:05:00Z ext:srt priority: 10 size: 118212 streamer: http streamurl: http://vod-sub-uk-live.akamaized.net/iplayer/subtitles/ng/modav/bUnk nown-591e0c64-779b-4f16-9582-bd3bc6c441bd_b09c79wx_1508034118207.xml?__gda__=150 8921411_8042e3b62cef7eb303c0b44d69225c99 type: (captions) http stream (CDN: mf_akamai_uk_plain/10) stream: subtitles3 bitrate: expires:2017-11-21T14:05:00Z ext:srt priority: 20 size: 118212 streamer: http streamurl: http://vod-sub-uk-live.bbcfmt.hs.llnwd.net/iplayer/subtitles/ng/moda v/bUnknown-591e0c64-779b-4f16-9582-bd3bc6c441bd_b09c79wx_1508034118207.xml?s=150 8878211&e=1508921411&h=c1d8bb45cd85f418d83103af0ef1979a type: (captions) http stream (CDN: mf_limelight_uk_plain/20) == but all three point to the same file! Now, if you load the legacy URL http://www.bbc.co.uk/iplayer/subtitles/ng/modav/bUnknown-591e0c64-779b-4f16-9582-bd3bc6c441bd_b09c79wx_1508034118207.xml in a Firefox tab, the browser will print: XML Parsing Error: not well-formed Location: (The URI) Line Number 7, Column 18: SUSPICION Right-click -> View Page Source and you'll be able to view the file contents and actually visualise the corruption: https://i.imgur.com/lsh8188.jpg With the aid of Fx's Page Source and a Text Editor, I managed to reconstitute a proper TTML file, then used SubtitleEdit to convert to (monochrome) .srt. If you're in need of it, contact me off-list... If I try --subtitles-only --tvmode=subtitles2 it tells me No media streams found. I don't think subtitle mode user selection is supported; legacy GiP code assumed only one captions mode, so this could be a new requested feature; I see no reason for it though; all modes point to the same file, negligible speed differences between CDNs for such small files of just a few KBs... get_iplayer ought when unable to download subtitles successfully to continue to call AtomicParsley to add metadata. While in this case it's not the actual downloading that failed, but rather the conversion to .srt (e.g. you can fetch the raw corrupt ttml with --subsraw), I too agree with that. After another series of tests made, of note is the fact that every GiP version from 3.00 onwards does fail to co
Re: BBC Radio 3 podcast quality
On Tue Oct 24 08:32:41 BST 2017, Simon Morgan wrote: but I would be interested to know if podcasts can be downloaded at a higher quality. I think that there has been an answer to a similar query in the past but I can't locate it. Fellow list members (Clive, d.lake & RS) have already covered this beautifully, but FTR there has been a relevant discussion about BBC Podcast bitrates last August: http://lists.infradead.org/pipermail/get_iplayer/2017-August/011013.html Happy listening! Cheers, Vangelis. ___ get_iplayer mailing list get_iplayer@lists.infradead.org http://lists.infradead.org/mailman/listinfo/get_iplayer
Re: Radio Programme Not in Cache
On Sun Oct 22 14:17:54 BST 2017, RS wrote: As for whether it's been explained before, the last time I remember was when Phil Lewis was still involved. ... And for those monitoring the Support Forums, there is a related issue/thread posted as recently as Oct 16th; plus there's me mentioning this subject somewhere inside http://lists.infradead.org/pipermail/get_iplayer/2017-September/011174.html which was last month... Cheers, Vangelis. ___ get_iplayer mailing list get_iplayer@lists.infradead.org http://lists.infradead.org/mailman/listinfo/get_iplayer
Re: get_iplayer 3.05 released
On Wed Oct 18 23:05:03 BST 2017, Budge wrote: there is now a much more prominent advisory on deleting older downloads. Hello Alastair; this is the result of https://github.com/get-iplayer/get_iplayer/commit/c0162b2 From https://github.com/get-iplayer/get_iplayer/wiki/release300to309#changes-in-303 get_iplayer no longer prompts you to delete downloaded files more than 30 days old. It displays a warning message instead. The warning is also now displayed with --pid and --url and with PVR searches (CLI and Web PVR Manager). As before, the warning can be suppressed with get_iplayer --prefs-add --no-purge What has prompted this additional reminder? Your guess is as good as mine; you'd have to register with the Support Forums and ask the developer there for a definitive answer... I can no longer see the download progrees expressed in Bytes, bits or minutes and seconds. (snip) Can I get it back? --verbose perhaps? Again, the result of https://github.com/get-iplayer/get_iplayer/commit/8cf8913 (which was dubbed a "revamp"... ). From the 3.03 RN: The default logging output has been reduced. Some unnecessary output has been removed, and some is now only visible with --verbose Thankfully, I am not afraid of using my text editor to revert locally some of the changes I am not entirely OK with; for all you know, "unnecessary" might be indispensable to another person... (But I shouldn't be publicly critisising the decisions made by the coder, I am already blocked on GitHub for being vocal in the past...) my grateful thanks to all who make this possible. ... That would boil down to actually three persons: 1. @dinkypumpkin, the main maintainer/developer and owner of the GitHub repositories 2. @squarepenguin, owner of the Support Forum website 3. Jon Hedgerows, responsible for various GiP PPAs. Though not linked immediately with development, I would also thank the maintainer of this list (David Woodhouse) and praise all volunteers helping in this list and forums... Besr regards, Vangelis. ___ get_iplayer mailing list get_iplayer@lists.infradead.org http://lists.infradead.org/mailman/listinfo/get_iplayer
Re: BBC Collections
On Mon Oct 9 19:25:21 BST 2017, I wrote: On Mon Oct 9 08:21:19 BST 2017, Mike Casswell wrote: at BBC Collections locations such as http://www.bbc.co.uk/iplayer/group/p056n6px This method has worked in the past for apparently similar collections - but note the use of 'group' in the url which I do not recall from previous examples. Hi Mike... I don't think "group PIDs" were ever supported in GiP, only "brand" and "series" ones... so it might have been a brand/series page that has worked for you in the past... ... Well, it turns out that you were right and, still, I am right, too! Just came across the following "group" listing: http://www.bbc.co.uk/iplayer/group/b096k7q7 Unlike your previous example (group being an assortment of different programmes), this "group PID" does work with --pid-recursive because it's actually a series PID (you can tell because all members of the group are individual episodes of the same series): get_iplayer --type=tv --pid=b096k7q7 --pid-recursive -i => INFO: Series or Brand PID detected INFO: BBC Four - The Vietnam War - Available now INFO: Page 1 of 1 INFO: Series 1, Things Fall Apart (January 1968-June 1968) (b097ts0d) INFO: Series 1, This Is What We Do (July 1967-December 1967) (b097ts0b) INFO: Series 1, Doubt (January 1966-June 1967) (b096v3f2) INFO: Series 1, Hell Come to Earth (January 1964-December 1965) (b096v3dw) INFO: Series 1, Riding the Tiger (1961-1963) (b096k948) INFO: Series 1, Deja Vu (1858-1961) (b096k8wz) I hope this makes it clearer now... Cheers, Vangelis. ___ get_iplayer mailing list get_iplayer@lists.infradead.org http://lists.infradead.org/mailman/listinfo/get_iplayer
Re: ffmpeg issue?
On Tue Oct 10 09:52:02 BST 2017, Hugh Reynolds wrote: Many thanks. You're welcome! No obvious way of finding the version. In the CLI: get_iplayer -V If using the GUI exclusively, scroll all the way down and you should see a footer with the GiP version, e.g. get_iplayer Web PVR Manager 3.01-windows.0 I inspected the get_iplayer.pl file, it was 3.03. Just as I thought... Yet again, during installation, perl.exe was locked and the installation of v3.04 had failed. Once that was unlocked, by using IObit Unlocker, (snip) Perl.exe is locked again after installing v3.04. Why does perl.exe get locked? When neither the CLI nor GUI of GiP are running, you shouldn't see any perl.exe processes running in TM and perl.exe shouldn't be locked! It appears your issue is particular to your setup... I know this sounds like a really dumb question, but are you sure you have properly exited the Web PVR command prompt window prior to starting the update? Before installation, close all running instances of get_iplayer, including any Web PVR Manager server window that is open. For troubleshooting, IObit unlocker gives info about all processes locking file perl.exe, you should start from there... My gut feeling is you have an overzealous AV suite that locks file perl.exe, but hard to tell - if so, try to whitelist it inside your AV's exceptions list... All I'm doing is running the installation .exe. Have you tried with your AV temporarily turned off? (No real risk there, the installation file is clean; could also scan it prior to running it; after you're comfortable with it, turn briefly the AV off and start installation...). My aging memory doesn't help, but thanks to my local e-mail archive I was able to spot this previous list exchange: http://lists.infradead.org/pipermail/get_iplayer/2017-August/011080.html where I simply speculated about pretty much the same things, being prophetic at the same time: Using Unlocker is just a workaround; if you do not discover the root cause of your issue and rectify it, then there's a chance you'll revisit the issue in the next GiP update (?) Only you can debug your own system, sorry :-( (But anyone else can chime in with some other ideas to aid you in that...). Regards, Vangelis. ___ get_iplayer mailing list get_iplayer@lists.infradead.org http://lists.infradead.org/mailman/listinfo/get_iplayer
Re: ffmpeg issue?
On Mon Oct 9 13:28:18 BST 2017, Hugh Reynolds wrote: Is there a simple way to resolve this? ... Actually yes! But: 1. First indicate the version of GiP used; nothing in your post identifies that, my bet is you're still on previous version 3.03, so the rest of my answer is based on that assumption; if it's not true, then please disregard... 2. Both I and Alan recently posted in this list about the advent of bug-fix version GiP 3.04: http://lists.infradead.org/pipermail/get_iplayer/2017-October/011183.html List members are invited to read linked release notes even when not initially planning an immediate update... Fixed a serious bug introduced in v3.03 that could cause MP4 conversions to be skipped when using the PVR function. If you are using v3.03, you are strongly urged to upgrade to v3.04 immediately. For the record, the bug was identified after the following Forum thread: https://squarepenguin.co.uk/forums/thread-1518-newpost.html Code commit that fixed it: https://github.com/get-iplayer/get_iplayer/commit/bdf3c7eb990e49953aea34f43ce2f378307c959b So, if you're on GiP 3.03, the "simple way to resolve" your issue is to upgrade to 3.04 (!). I have commented about it in the past, but the current workflow in GiP development (issues tracker removed, "contribute" branch only updated publicly after a major stable release) might (and has several times) make the released version more prone to undetected bugs, which rear their ugly heads until after the "master" branch script is let loose... So that's why you should also look at https://github.com/get-iplayer/get_iplayer/wiki/issues some time after a tag release... Regards, Vangelis. ___ get_iplayer mailing list get_iplayer@lists.infradead.org http://lists.infradead.org/mailman/listinfo/get_iplayer
Re: BBC Collections
On Mon Oct 9 08:21:19 BST 2017, Mike Casswell wrote: at BBC Collections locations such as http://www.bbc.co.uk/iplayer/group/p056n6px This method has worked in the past for apparently similar collections - but note the use of 'group' in the url which I do not recall from previous examples. Hi Mike... I don't think "group PIDs" were ever supported in GiP, only "brand" and "series" ones... so it might have been a brand/series page that has worked for you in the past... Since we can't test things now :-( , it's mostly my word against your memory, hahah...; but I found a Forum entry from March 2015 that supports my claim: https://squarepenguin.co.uk/forums/thread-377-post-2112.html#pid2112 You can't download from the PID for a group, like the one you posted. On Mon Oct 9 11:54:56 BST 2017, Alan Milewczyk wrote: there was a change a while back that made this method obsolete (you'd have to go through the release notes of previous versions to determine when the change took place) That change was the removal of the various XML feeds by the beeb at the end of April (2017). GiP 3.00 was the first version that tried to alleviate it as much as possible https://github.com/get-iplayer/get_iplayer/wiki/release300#1-restored-functionality-broken-by-the-bbc but do note the following limitation: https://github.com/get-iplayer/get_iplayer/wiki/release300#recursive-downloads Using the individual PIDs is the only way AFAIK Currently yes; but I'm sure there's a number of dextrous list members who could conjure up a (simple?) script that would parse (web scrape) "group" pages to harvest the individual PIDs; just examining page source of http://www.bbc.co.uk/iplayer/group/p056n6px I'm seeing href="*" URIs with "#group=p056n6px" appended to them... That's the ones containing the PIDs. Wouldn't hurt to request such a feature in the Forum, either (but, TBH, very unlikely to be even considered by the dev...; will also be, like all web scraping, very fragile, even to the slightest beeb change...). Best regards all, Vangelis. ___ get_iplayer mailing list get_iplayer@lists.infradead.org http://lists.infradead.org/mailman/listinfo/get_iplayer
Re: Radio 1 Vintage
On Sat Sep 30 19:14:04 BST 2017, I wrote:
... Yes; you'd have to (temporarily) patch your local
copy of the GiP perl script so as to include the
"Radio 1 Vintage" schedule feed found at:
http://www.bbc.co.uk/radio1vintage/programmes/schedules
(snip)
# channel ids be found on http://www.bbc.co.uk/radio/stations
sub channels_schedule {
return {
'national' => {
'radio1/programmes/schedules/england' => 'BBC Radio 1',
+ 'radio1vintage/programmes/schedules' => 'Radio 1 Vintage',
'1xtra/programmes/schedules' => 'BBC Radio 1Xtra',
(snip)
Save the patch and force a radio.cache refrech.
This is done by
get_iplayer --type=radio -f --force
get_iplayer --type=radio "Radio 1 Vintage" =>
== A needed follow-up, thanks to a person contacting me direct ==
If you went ahead and patched your script last week,
successive radio.cache refreshes will have got you a total of
56 "Radio 1 Vintage" entries.
However, if you decide(d) to patch the script anytime
during this week (starting Monday Oct 2nd),
forcing a radio.cache refresh will only populate it with
Monday's "Radio 1 Vintage" programmes; likewise,
if you delay patching until after the end of this week
(after Sunday Oct 8th), then you'll get 0 "R1V" entries
inside the radio.cache; this is because a default refresh
will only append to existing cache the new content of current week
(http://www.bbc.co.uk/radio1vintage/programmes/schedules/this_week)
in the schedule feed.
If you've patched your GiP script after 01/10/2017,
you'll have to rebuild the radio.cache as far as
"Radio 1 Vintage" is concerned, by issuing in a
GiP command prompt:
get_iplayer --type=radio -f --refresh-include="Radio 1
Vintage" --refresh-limit-radio=30 --force
This will make sure you get all 56 R1V entries inside the cache!
Best regards,
Vangelis.
PS/OT: Here in Greece, the first day of the week
is Sunday, in Greek it's Κυριακή, translated in
English as the Day of our Lord (Jesus); Monday is
called Δευτέρα, litterally meaning "Second".
While dealing with this follow-up, I realised that
in the UK Monday is considered the first day of the week;
I did find the following small article
http://www.anythinglefthanded.co.uk/what-day-starts-the-week.html
quite interesting as to what is the standard
in various other countries :-)
___
get_iplayer mailing list
get_iplayer@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/get_iplayer
Re: get_iplayer 3.04 released
(Just a quick note to say that since 3.03.0 release, bundled ffmpeg.exe 32bit has been upgraded from v3.2.4 to v3.3.0). ... Just a small correction there, FFmpeg.exe was in fact updated to v3.3.3. Apologies for the initial error on my part. Vangelis. ___ get_iplayer mailing list get_iplayer@lists.infradead.org http://lists.infradead.org/mailman/listinfo/get_iplayer
get_iplayer 3.04 released
This is a bug-fix release: https://github.com/get-iplayer/get_iplayer/wiki/release304 Changelog: https://github.com/get-iplayer/get_iplayer/compare/v3.03...v3.04 Windows 7+ users: https://github.com/get-iplayer/get_iplayer_win32/releases/tag/3.04.0 (Just a quick note to say that since 3.03.0 release, bundled ffmpeg.exe 32bit has been upgraded from v3.2.4 to 3.3.0). Windows XP/Vista users ARE NOT OFFICIALLY SUPPORTED by the maintainer (but you might find the following post http://lists.infradead.org/pipermail/get_iplayer/2017-May/010711.html of value...) Non-Windows users know best how to upgrade (either manually if in a hurry or wait for the appropriate PPAs to be updated by their maintainers). For more info/help, visit the GitHub/Support Forum documentation Regards ___ get_iplayer mailing list get_iplayer@lists.infradead.org http://lists.infradead.org/mailman/listinfo/get_iplayer
Re: Radio 1 Vintage
On Sat Sep 30 14:32:01 BST 2017, Don Grunbaum wrote:
The BBC are broadcasting programmes on a "pop-up" station called Radio 1
Vintage.
Hi Don :-)
This is part of the celebrations for the 50-year anniversary of BBC Radio 1
(but I'm sure you already knew that!); more at
http://www.bbc.co.uk/programmes/p059cwkn
They do not appear to be picked up in the radio cache.
This is expected; only regular BBC Radio 1 schedule feed is
parsed by GiP:
http://www.bbc.co.uk/radio1/programmes/schedules
but out of curiosity, is there a way to "force" GiP
to pick up the programmes?
... Yes; you'd have to (temporarily) patch your local
copy of the GiP perl script so as to include the
"Radio 1 Vintage" schedule feed found at:
http://www.bbc.co.uk/radio1vintage/programmes/schedules
DO THIS AT YOUR OWN RISK!
On my script (heavily pathed v3.01 on Windows):
===
# channel ids be found on http://www.bbc.co.uk/radio/stations
sub channels_schedule {
return {
'national' => {
'radio1/programmes/schedules/england' => 'BBC Radio 1',
+ 'radio1vintage/programmes/schedules' => 'Radio 1 Vintage',
'1xtra/programmes/schedules' => 'BBC Radio 1Xtra',
===
(unfortunately, my mailer may mangle the patch
formatting, still you get the drift...).
Save the patch and force a radio.cache refrech.
After that, on my system I get:
get_iplayer --type=radio "Radio 1 Vintage" =>
Matches:
21885: Radio 1 Vintage - Radio 1 Vintage, starts 30 September at 6am, Radio
1 V
intage, p059k4m3
21886: Radio 1 Vintage - 50 Year Countdown, Radio 1 Vintage, p059kfx4
21887: Radio 1 Vintage - Nick Grimshaw and Tony Blackburn, Radio 1 Vintage,
p05
9kfxc
21888: Radio 1 Vintage - Kenny Everett, Radio 1 Vintage, p059kfxn
21889: Radio 1 Vintage - David Jensen, Radio 1 Vintage, p059kfxy
21890: Radio 1 Vintage - Colin & Edith, Radio 1 Vintage, p059kfy6
21891: Radio 1 Vintage - Adrian Juste, Radio 1 Vintage, p059kfyh
21892: Radio 1 Vintage - Classic Albums with Roger Scott, Radio 1 Vintage,
p059
kfyr
21893: Radio 1 Vintage - Fearne & Reggie, Radio 1 Vintage, p059kfz0
21894: Radio 1 Vintage - Nicky Campbell, Radio 1 Vintage, p059kfz8
21895: Radio 1 Vintage - The Radio 1 Roadshow, Radio 1 Vintage, p059kfzk
21896: Radio 1 Vintage - Tony Blackburn, Radio 1 Vintage, p059n6gl
As you can see, all shows have the same
& , for anything more specific you'd
have to refine your search, e.g.
get_iplayer --type=radio --fields=episode "Jensen" =>
Matches:
21889: Radio 1 Vintage - David Jensen, Radio 1 Vintage, p059kfxy
INFO: 1 Matching Programmes
Thanks for you help.
Anytime...
Best regards from (finally)
autumn-y Greece!
Vangelis.
___
get_iplayer mailing list
get_iplayer@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/get_iplayer
Re: Missing PIDs
On Mon Sep 18 18:10:11 BST 2017, Shareef Jalloq wrote: so it's very possible I've done something stupid. (snip) I would have searched using 'get_iplayer --type=radio Annie'. This fails to show any entries. Hi Shareef :-) I have just refreshed the radio.cache get_iplayer --type=radio -f --force and issuing get_iplayer --type=radio "Annie Mac" I get 20 matches: Matches: 11070: Annie Mac - Benji B co-hosts with Annie Mac, , b091vggc 11071: Annie Mac - PVRIS perform live, , b091vghz 11072: Annie Mac - Toddla T co-hosts with Annie Mac, , b091vgkl 11073: Annie Mac - Jorja Smith delivers her new single, , b091vgm3 11074: Annie Mac - Dan sits in for Annie with Reading highlights, , b0920296 11075: Annie Mac - Power Down Playlist, , b09202bt 11076: Annie Mac - Queens of the Stone Age play live, , b09202df 11077: Annie Mac - Hottest Records Wrap Up, , b09202g2 11078: Annie Mac - MistaJam sits in for Annie, , b092k8pc 11079: Annie Mac - Maya Jane Coles Mini Mix and Tiga in Ibiza, , b092kjt3 11080: Annie Mac - Nick Mulvey live gig, , b092kqfl 11081: Annie Mac - New music from Beck, , b092kqh8 11082: Annie Mac - Live from Bestival, , b092kqjw 11083: Annie Mac - Snakehips Mini Mix, , b092kql0 11084: Annie Mac - Arcane Roots Hottest Record, , b093pp6g 11085: Annie Mac - Nothing But Thieves play live, , b093pp82 11086: Annie Mac - Mercury Prize Pre-Party Playlist, , b093pp9p 11087: Annie Mac - New music from Lorde, NAO and Isaac Gracie, , b093ppc6 11088: Annie Mac - 15/09/2017, , b093ppdd 11089: Annie Mac - Four Tet curates the Power Down Playlist, , b094dmrv This is with latest 3.02 script; be advised that after the demise of the BBC .xml feeds last spring, to populate the caches GiP has to resort to the schedule feeds; this means that only programmes broadcast and contained inside the linear beeb services can be indexed (and this excludes many other PIDs available in iPlayer service itself); the above is true for both TV+Radio... % get_iplayer "Annie Mac Live" nothing ... This is expected for the following reasons: 1. You haven't specified programme type; by default, only the tv.cache will be searched; there's nothing currently in TV for "Annie Mac Live", hence your search will return empty... 2. "Annie Mac Live" (or rather "Annie Mac Live Mix"), pid=p05fdry0, isn't a standalone radio programme contained in the BBC Radio 1 schedule feeds, thus won't be found inside radio.cache. Annie Mac Live Mix (part of Radio 1 Mixes, http://www.bbc.co.uk/programmes/p034pnmt ), can be best described as "audio" clip; AFAIK, "Annie Mac Live Mix" is just an excerpt from the last hour of Annie's Friday radio show, provided for convenience as a standalone PID; for previous Fri Sep 15th, full show is under http://www.bbc.co.uk/programmes/b093ppdd Do note that the actual programme runs for 3hr in the player, while the info just underneath it erroneously reports a 2hr duration... As you may see, I had no problem finding last Friday's episode in my cache (index=11088). To sum things up, you won't be able to find Annie Mac Live Mix in the radio cache, use http://www.bbc.co.uk/programmes/p036ggzv/episodes/player and hopefully --type=radio --pid=p036ggzv --pid-recursive will work... If you are still unable to find proper (broadcast) shows inside your radio cache, please post back and we'll take it from there... Best regards, Vangelis. ___ get_iplayer mailing list get_iplayer@lists.infradead.org http://lists.infradead.org/mailman/listinfo/get_iplayer
Re: A few ffmpeg queries
On Fri Sep 8 13:43:53 BST 2017, RS wrote:
I realise you did not write the code but are only defending it
Hi Richard; I wasn't "defending" the code,
I merely provided some additional info that
would explain your "for some reason" previous
comment:
get_iplayer for some reason uses a different format
for ffmpeg options depending on the value of
! $opt->{ffmpegobsolete}
As far as I am aware -loglevel is not a new ffmpeg option.
Agreed; but it wasn't there from the start and its arguments
have changed over time; just browse:
https://github.com/get-iplayer/get_iplayer/blob/6dab3c86801d3ee2ce1d1a072c3471e945f04bf5/get_iplayer#L781-L799
Also, I have an old patched version
of GiP 2.97 lying around (used for
flash modes), its help contains the entry:
--ffmpeg-obsolete
Indicates you are using an obsolete version of ffmpeg (<0.7)
that does not support the -loglevel or -stats options,
so --quiet, --verbose and --debug
will not be applied to ffmpeg.
To conclude ffmpegobsolete, that
setting also dictates the syntax used for
applying the bitstream audio filter during remux:
https://github.com/get-iplayer/get_iplayer/blob/6dab3c86801d3ee2ce1d1a072c3471e945f04bf5/get_iplayer#L7038-L7042
v2.5 or above is needed for HLS
and v3.0 or above is needed for DASH.
HLS should be interpreted to actually mean
Video Factory HLS modes ("hvf", not the
"legacy" "hls" ones)
(Does that leave any modes
obsolete versions will work with?)
"hls" will work OK, yielding tagged MP4 files;
dvf/hvf will produce (untagged) .TS files
(playable though by most SW/HW players
with support for h.264/aaclc codecs).
I gather also daf (audio) modes would fail to
yield proper .m4a files; unsure about haf
modes, too...
Best regards,
Vangelis.
___
get_iplayer mailing list
get_iplayer@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/get_iplayer
Re: A few ffmpeg queries
On Fri Sep 8 00:07:08 BST 2017, Alan Milewczyk wrote: Lifelong learning! Great fun! Yes... I don't know whether your mailer/browser supports Ancient Greek characters, but γηράσκω δ' αἰεὶ πολλά διδασκόμενος => http://lyricstranslate.com/en/%CE%B3%CE%B7%CF%81%CE%AC%CF%83%CE%BA%CF%89-%CE%B4-%CE%B1%E1%BC%B0%CE%B5%E1%BD%B6-%CF%80%CE%BF%CE%BB%CE%BB%CE%AC-%CE%B4%CE%B9%CE%B4%CE%B1%CF%83%CE%BA%CF%8C%CE%BC%CE%B5%CE%BD%CE%BF%CF%82 and a similar variant γηράσκω ἀεί διδασκόμενος => http://lyricstranslate.com/en/%CE%B3%CE%B7%CF%81%CE%AC%CF%83%CE%BA%CF%89-%E1%BC%80%CE%B5%CE%AF-%CE%B4%CE%B9%CE%B4%CE%B1%CF%83%CE%BA%CF%8C%CE%BC%CE%B5%CE%BD%CE%BF%CF%82 So, let the learning never cease! Cheers, Vangelis. ___ get_iplayer mailing list get_iplayer@lists.infradead.org http://lists.infradead.org/mailman/listinfo/get_iplayer
Re: A few ffmpeg queries
On Thu Sep 7 22:19:48 BST 2017, RS wrote:
As far as I am aware
all the variants Vangelis suggested work.
Yes they do, with fairly recent FFmpeg versions
that are also backward compatible...
However, the GiP perl script must cater for a variety
of OSes and in the case of Linux distros some (still)
come by default with older FFmpeg packages;
the more recent
-c:v copy -c:a copy
syntax/iteration isn't recognised by those older FFmpeg versions,
so issuing "--ffmpeg-obsolete" in your terminal ("ffmpegobsolete 1"
inside options) will revert to the legacy syntax; from help:
--ffmpeg-obsolete
Indicates you are using an obsolete version of ffmpeg (<1.0)
that may not support certain options.
Without this option, MP4 conversion may fail
with obsolete versions of ffmpeg.
As hinted, "--ffmpeg-obsolete" also takes care of some
other ffmpeg switches (e.g. verbosity level), browse the
actual code and ye shall find...
Many greetings,
Vangelis.
___
get_iplayer mailing list
get_iplayer@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/get_iplayer
Re: OT: A few ffmpeg queries
ffmpeg -i progname.ts -c:av copy progname.mp4 Very early in the morning here, so I'll be laconic for now :-) Assuming you've not mistyped it, -c:av copy is not valid and probably the reason your command is initiating video+audio transcoding... Try ffmpeg -i progname.ts -c copy progname.mp4 and see how that fares... -c copy = -vcodec copy -acodec copy or -c:v copy -c:a copy or -codec: copy or -codec copy https://ffmpeg.org/ffmpeg.html#toc-Stream-copy https://ffmpeg.org/ffmpeg.html#toc-Stream-specifiers-1 According to https://en.wikipedia.org/wiki/Comparison_of_video_container_formats the MP4 container should be able to support both https://en.wikipedia.org/wiki/H.262/MPEG-2_Part_2 (video) and https://en.wikipedia.org/wiki/MPEG-1_Audio_Layer_II (audio) raw streams, playing back such a file in a HW/SW player is another matter, though (most expect h264+aac). The question is, why must you use the MP4 container in your NAS? V. ___ get_iplayer mailing list get_iplayer@lists.infradead.org http://lists.infradead.org/mailman/listinfo/get_iplayer
Re: Tidying up Radio Downloads
On Tue Aug 29 18:27:24 BST 2017, RS wrote: but Windows (as of 5 years ago) only supports up to id3v2.3 Yes: https://answers.microsoft.com/en-us/windows/forum/windows_7-pictures/how-to-add-id3v24-support-for-windows-7-64bit/a9427521-eb6f-4fe4-affb-f61532846503?auth=1 But then again, there's a third party Windows Explorer shell extension that adds id3v2.4 support to Windows: http://www.softpointer.com/AudioShell.htm (Have this installed for more than a year here, very pleased with it !) Some discussion about it: https://getmusicbee.com/forum/index.php?topic=20536.0 An interesting read wrt id3v*: http://support.davidsystems.com/sis_forum/viewtopic.php?t=107 Regards, V. ___ get_iplayer mailing list get_iplayer@lists.infradead.org http://lists.infradead.org/mailman/listinfo/get_iplayer
Re: Tidying up Radio Downloads
On Thu Aug 24 13:12:07 BST 2017, RS wrote: It seems this tells me how to add metadata using ffmpeg. https://wiki.multimedia.cx/index.php/FFmpeg_Metadata What I still haven't worked out is whether there is an equivalent of -codec copy for metadata. ... bit late, but maybe: -map_metadata outfile[,metadata]:infile[,metadata] => "set metadata information of outfile from infile" (switch found inside "Per-file main options" section of ffmpeg -h full ) Best regards, Vangelis. ___ get_iplayer mailing list get_iplayer@lists.infradead.org http://lists.infradead.org/mailman/listinfo/get_iplayer
Re: recording quality and vpn
On Sun Aug 27 14:18:36 BST 2017, cc wrote: can the use of a vpn limit the number of recording qualities available? (snip) I managed to get1280.x720 def on 25 aug (snip) it is now proving impossible for the various downloads I have tried. ... Probably the same issue (unless cc=Gautier) : https://squarepenguin.co.uk/forums/thread-1488-newpost.html i.e. hlshd not appearing inside available tvmodes for an overseas user (and inferred use of a VPN...). FWIW, many UK inhabitants and licence fee payers do use a VPN service for privacy and other reasons; this is also not allowed by the BBC: https://www.bbc.co.uk/iplayer/help/in_the_uk_message If you are using a VPN and are in the UK, try disabling to see if that helps. If we detect you might be using a VPN, you'll be unable to play programmes. This is because we're unable to detect the end point of your private network; we need to be confident you're in the UK. For the past two years, after news broke that several millions of Chinese netizens were (ab)using iPlayer, the BBC is under a constant battle against commercial VPN/ VPS/Proxy/SSL Tunnel services - remember, it's quite easy to spot a VPN server (since in the UK the commercial ones are usually to be found in known Data Centres with known IP pools), especially since a lot of iPlayer traffic appears to be requested by a single IP address. If you're using a VPN service or like, expect it to be blocked at any time; it's unwise to post about it here in the list, am afraid no help could be (/is allowed to be) offered to you... And it's certainly most unwise to post about it in the Forums, where 1. it's against their rules 2. they do IP checks of registered members (if not in the UK). Take your issue directly with your VPN provider... FWIW, when the BBC blacklist VPN IPs, they usually start with their mediaselector API URLs, so you do get blocked at the door, resulting in no streams at all offered to you... Selectively blocking only hlshd but allowing other streams to appear sounds a bit iffy; that means that the Akamai CDN server (serving the hlshd stream) itself blocks the VPN IP, but why would the BBC do that, if other streams could still be accessed? Most probably it's an issue with the VPN itself and its configuration, so as Gautier surmised So clearly there must be something wrong on my end. On Sun Aug 27 16:43:21 BST 2017, d.lake wrote: Don't use a VPN - use an HTTP proxy. That way you'll get full download speed from a local CDN at the full range of bit-rates. Hi, you appear to be in the UK, so I suspect your recommendation to use an HTTP(S) proxy is for those in the UK with privacy concerns... In the very remote chance your advice is for non-UK users, then in that case there's not a big difference between a UK VPN and a UK proxy speedwise; all TV streams are blocked currently at CDN server level, so they have to be redirected fully through the VPN/HTTP server... The above info is disclosed for pure "academic" reasons... I suppose I (and other members) should stop this discussion here, so as to not upset even further the high powers that be... Regards ___ get_iplayer mailing list get_iplayer@lists.infradead.org http://lists.infradead.org/mailman/listinfo/get_iplayer
Re: Problems with --info
On Fri Aug 25 17:30:44 BST 2017, I wrote: the on-line "develop" GitHub repository ... And that repo has just been renamed to "contribute"... Very nice (???!!!???) ... Plus, the wiki is in the process of being heavily re-edited/rearranged (thus breaking previously bookmarked wiki topics... :-( ). Regards, V. ___ get_iplayer mailing list get_iplayer@lists.infradead.org http://lists.infradead.org/mailman/listinfo/get_iplayer
Re: TV Audio Bitrates
On Fri Aug 25 16:28:50 BST 2017, I wrote: OP did confuse bitrate with sample rate... Oops, I meant the author of the e-mail I replied to (Nick Payne), not the OP of this thread (tellyaddict); apologies... V. ___ get_iplayer mailing list get_iplayer@lists.infradead.org http://lists.infradead.org/mailman/listinfo/get_iplayer
Re: Problems with --info
On Thu Aug 24 18:38:59 BST 2017, Alan Milewczyk wrote: I've just checked with the PID you quote Vangelis and concur. Where the PID isn't in the cache (as is the case here), v3.02 fails to find anything! Many thanks Alan for checking/verrifying ! (OT: Yes, sun still shining over here, but as it's usually the case, one longs for what one doesn't have... After this one being the hottest summer in more than a decade, with 3 major heatwaves striking thus far (last one was Lucifer, started end-July and lasted till mid August - temps as high as 39C during the day, not below 31 during night) I can say without the faintest hesitation that I'm fed up with the sun shining, could well do with some Manchester weather for a relief... And do keep in mind that over here real winter starts towards the end of October... :-) ) On Thu Aug 24 18:35:17 BST 2017, tellyaddict wrote: Thanks Vangelis. It's appeared in the Known Issues section of the wiki so Dinky is now aware of it. You are welcome. I genuinely don't mean to sound unpleasant, but this current situation of bugs being discovered after a major public release is a direct consequence of the way that the on-line "develop" GitHub repository is being currently updated (i.e. only after a major "master" release takes place, and I recall this is something that you and I discussed in a previous exchange on this list..). When the online "develop" branch was being (publicly) gradually committed to towards the next major release, advanced users/testers had the chance to test "unstable" snapshots of the code, which increased the likelihood of discovering and reporting (e.g. in the issues tracker, now no longer with us...) bugs/breakages in those snapshots; the dev could then hopefully iron out those bugs before a major release... Currently, fixing any (major) bug would necessitate a new master release, but if the dev feels OK with such a workflow, I fear I have no business at all discussing this... Best regards all, Vangelis. ___ get_iplayer mailing list get_iplayer@lists.infradead.org http://lists.infradead.org/mailman/listinfo/get_iplayer
Re: TV Audio Bitrates
On Fri Aug 25 08:16:58 BST 2017, Nick Payne wrote: With 3.02 of GiP, all HVF modes now use 320 kbps audio. Not so ... Yes, OP did confuse bitrate with sample rate... But still the "Not so" part of his post stands true for various pids outside the main BBC TV channels... Just try (with 3.02) get_iplayer --type=tv --pid=p059b38x --tvmode=hvfsd (pid expires in 2 days) and you'll discover it fetches as a 128kbps hvf* mode... One other oddity for this pid is it's not available as legacy hls* (/flash*) tvmodes, only dvf*/hvf*. I guess a better choice of wording would be: "GiP 3.02 tries to download every hvf* tvmode with a 320kbps audio stream, whenever that one is available on the CDN server" Notice I didn't say "when it's available by the BBC", because the BBC do not officialy and publicly make such a stream available via their iPlayer interface. It was a code hack first implemented by youtube-dl devs (Python) and then ported to GiP (perl) and fine-tuned afterwards that allows for 320kbps audio streams inside hvf* tv files... Regards Vangelis. ___ get_iplayer mailing list get_iplayer@lists.infradead.org http://lists.infradead.org/mailman/listinfo/get_iplayer
Re: Problems with --info
On Thu Aug 24 14:23:59 BST 2017, tellyaddict wrote: Is anyone else finding that --info no longer works in 3.02? Running "get_iplayer --pid b08z0lfw --info" now only does this: (snip) I am still at GiP 3.01, but I did fetch 3.02 just now to try this myself... I can confirm I can dupe this with a radio pid NOT IN THE CACHE: perl get_iplayer-302w.pl --type=radio --pid=b08xz0md -i => --- get_iplayer 3.02-windows.0, Copyright (C) 2008-2010 Phil Lewis This program comes with ABSOLUTELY NO WARRANTY; for details use --warranty. This is free software, and you are welcome to redistribute it under certain conditions; use --conditions for details. -==--==--==--==--==--==--==--==--==--==--==--==--==--==--==- INFO: radio episode PID detected (b08xz0md) INFO: Trying to download PID: b08xz0md using type: radio INFO: PID not found in cache: b08xz0md INFO: 0 matching programmes --- GiP 3.01 works as expected: perl get_iplayer-301w.pl --type=radio --pid=b08xz0md -i => --- get_iplayer 3.01-windows.0, Copyright (C) 2008-2010 Phil Lewis This program comes with ABSOLUTELY NO WARRANTY; for details use --warranty. This is free software, and you are welcome to redistribute it under certain conditions; use --conditions for details. NOTE: A UK TV licence is required to legally access BBC iPlayer TV content INFO: Episode PID detected INFO: Trying pid: b08xz0md using type: radio INFO: Trying to download PID using type radio INFO: pid not found in radio cache Matches: INFO: File name prefix = Sounds_of_the_80s_-_Nick_Heyward_Red_Button_Special_b08 xz0md_original brand: Sounds of the 80s categories: Entertainment category: Entertainment channel:BBC Radio 2 (snip) --- And audio/video clips will never populate the respective caches, this looks like a serious regression, possibly introduced by https://github.com/get-iplayer/get_iplayer/commit/554bb64 But before blaming the developer, I must say I am running an unsupported custom Windows installation on an unsupported Windows OS, so first thing to do is for other members to check whether -i works with GiP 3.02 on non-indexed PIDs; if affirmative, then @tellyaddict should report this in the Forums, to be hopefully fixed... Cheers, Vangelis. ___ get_iplayer mailing list get_iplayer@lists.infradead.org http://lists.infradead.org/mailman/listinfo/get_iplayer
Re: Tidying up Radio Downloads
On Thu Aug 24 16:17:34 BST 2017, I wrote:
because the online sources stay there for good
Sadly not true for the .json flavours :-{
I forgot that the beeb intends to remove them
on May 2018; but the metadata for a specific pid
will continue to be there, albeit in a different format...
Cheers,
V.
___
get_iplayer mailing list
get_iplayer@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/get_iplayer
Re: Tidying up Radio Downloads
On Thu Aug 24 13:12:07 BST 2017, RS wrote: whether I am at risk of losing tags or metadata. There is such a risk, but whenever FFmpeg has tagged files as input, it does its best to preserve (most) of the metadata present in the input file and carry it over to the output file, taking, of course, into account what metadata the output container has support for... The reason for my concern is that there seems to be a lot of code in get_iplayer to process metadata, although I can't pretend to understand what it all does. Yes, there's a lot of code, but in a nutshell what it does is parse online BBC sources (currently e.g. http://www.bbc.co.uk/programmes/.json http://www.bbc.co.uk/programmes//playlist.json ) to harvest what metadata is needed, then use it to construct AtomicParsley commands to tag the downloaded->remuxed .M4A files. Using --verbose or --debug into your GiP commands will let you inspect in detail what's being actually done :-) What I still haven't worked out is whether there is an equivalent of -codec copy for metadata. If my details above wrt ffmpeg still leave you in doubt, then please have a read of http://www.ffmpeg.org/ffmpeg-formats.html#Metadata-1 This is a manual labour, metadata present in the input file is first extracted and backed-up, to be inserted to the output file at a second stage... But again, since it's GiP audio files we're discussing (or maybe not?), you can tag the edited .m4a file with GiP via: get_iplayer --type=radio --pid= --tag-only --tag-podcast-radio --tag-only-filename="" "" in this case is THE COMPLETE PATH to your edited audio file (whereas in normal GiP use ""="\.EXT", "" being your "iPlayer Recordings" directory... The tagging command works even after a radio programme has expired from iPlayer Radio, because the online sources stay there for good (tied to the unique pid string!). All the best, Vangelis ___ get_iplayer mailing list get_iplayer@lists.infradead.org http://lists.infradead.org/mailman/listinfo/get_iplayer
Re: Top of the Lake China Girl episode 5 download problems
On Wed Aug 23 22:49:12 BST 2017, Dave Widgery wrote: Each time it just comes back to the command prompt with no errors, Hi Dave Unfortunately, I can't assess much of your posted log, as after line INFO: Trying pid: p057mnc6 using type: tv everything's mangled by Avast... You could increase verbosity and perhaps try get_iplayer --type=tv --pid=p057mnc6 -i -v > GiP_log.txt 2>&1 and then try to see what's happening (?) But it's just better to upgrade anyway... 1. Back up your XP compatible ffmpeg.exe 2. Upgrade your GiP from 3.00 to 3.02; it's highly possible a bug in 3.00 is the cause of your issue... 3. Once the installer has finished (hopefully without issues), rename/delete the ffmpeg.exe (v3.2.4) put in place by the installer and then restore in that same folder your previous ffmpeg.exe (v3.0 ?). Then try again the fetch with GiP 3.02... Hope this fixes it... Vangelis. ___ get_iplayer mailing list get_iplayer@lists.infradead.org http://lists.infradead.org/mailman/listinfo/get_iplayer
Re: New radio PIDs, more than 8 characters - "solved"
On Wed Aug 23 16:44:30 BST 2017, Hugh Reynolds wrote: Reboot didn't help Uninstall and Reboot didn't help iobit-unlocker helped. Clean install is now working. Many, many thanks. You're welcome! I'm glad you're back up and running :-) For the sake of completeness/closure, were you even able to determine (via Unlocker) the process(es) locking those .exe files even after a system reboot? Doesn't make much sense to me... Perhaps an overzealous antimalware suite? Using Unlocker is just a workaround; if you do not discover the root cause of your issue and rectify it, then there's a chance you'll revisit the issue in the next GiP update (?) - are you able now to, e.g., send ffmpeg.exe (temporarily) to the Recycle Bin? Just my 2p, I am not jinxing your future GiP upgrades! Best wishes, Vangelis. ___ get_iplayer mailing list get_iplayer@lists.infradead.org http://lists.infradead.org/mailman/listinfo/get_iplayer
Re: TV Audio Bitrates
On Thu Aug 24 00:49:20 BST 2017, tellyaddict wrote: Have you thought about multiplexing the 320kbit/s audio with 1280x720 25fps video from HLShd? Sounds interesting. How would I do that? ... Somewhat older Support Forum post: https://squarepenguin.co.uk/forums/thread-1442-post-6434.html#pid6434 Involves downloading --tvmode=hlshd and a hvf* mode with 320kbps audio, which now, with GiP 3.02, could be any; to save on bandwidth, choose the smallest available hvf mode for your pid... To make it more transparent to you, "25fps.mp4" in the post means "720p25fps96k.mp4" and "50fps-audio.mp4" can just mean now, e.g., "320k.mp4" (does not have to be 50fps anymore... ). "out.mp4" would then be "720p25fps320k.mp4" There are other means to just download the 320kbps audio stream (without an accompanying video stream) and then multiplex it with hlshd, but this involves tools other than GiP and is left for the more adventurous... Regards, Vangelis. ___ get_iplayer mailing list get_iplayer@lists.infradead.org http://lists.infradead.org/mailman/listinfo/get_iplayer
Re: Tidying up Radio Downloads
On Tue Aug 22 22:47:09 BST 2017, Ralph Corderoy wrote: this is to avoid attempting to copy any "video" album sleeve, etc? Hi Ralph :-) FFmpeg treats the MP4 embedded thumbnail (artwork) present in both .mp4/.m4a GiP files as a VIDEO STREAM in the input file. During manipulation of such files, it'll attempt to copy that thumbnail into a video stream muxed to the end MPEG-4 container. If you assign an ".mp4" file extension to the output file, which is indeed allowed for an audio-only MP4 file, then you'll end up with an unwanted video stream! However, if you assign the ".m4a" file extension, then ffmpeg is smart enough to not allow a video stream inside an audio-only Apple file and thus the conversion fails altogether (if only -c copy or -c:a copy was used). Explicitly including the -vn flag will allow for an audio-only mp4 file to be produced, the same goes for an "ipod brand" .m4a file. If you plan to edit a GiP .m4a file with ffmpeg, then it is advised you fetch it with the --no-artwork switch (no thumbnail is embedded) or even --no-tag (no metadata written). You can always (re-)tag the edited file afterwards... always using -map 0:0 or similar to pluck just the audio in those cases. ... Afraid I only use the -map switch in cases with multiple video/audio/subs streams in input file; I mostly deal with simple AV files with just 1 video + 1 audio stream; when I want to manipulate only one or the other, I find it easier to just issue: -vn -c:a copy (disregard video) or -an -c:v copy (disregard audio) But then again, I don't consider myself really FFmpeg savvy! Best regards, Vangelis. ___ get_iplayer mailing list get_iplayer@lists.infradead.org http://lists.infradead.org/mailman/listinfo/get_iplayer
Re: Tidying up Radio Downloads
On Tue Aug 22 16:27:05 BST 2017, Ralph Corderoy wrote: But it works by decompressing the .m4a, letting you edit it, and then lossy re-encoding if you want a .m4a back. ... If m4a metadata is important to OP (or anyone else editing GiP m4a files), the use of Audacity and any other similar audio (waveform) editor usually results in the loss of the MP4 tag in the process. One needs to re-tag the edited file with GiP (see --tag-only & --tag-only-filename options) The advantage of using FFmpeg is that all m4a metadata except for m4a thumbnail (a 4-year-old bug that still won't be fixed, see: https://trac.ffmpeg.org/ticket/2798 ) will be transferred over to the edited file... Be sure to include -vn in the command if the output file extension is ".m4a" Another similar project is https://github.com/nu774/m4acut but do not ask me how to compile it for Linux :-) On Windows, if one wants very precise m4a editing, then I would, once more, suggest the closed source but freeware mp3DirectCut; it doesn't (in contrast to m4acut) support the MP4 container of .m4a files, but if you provide it with an FFmpeg path it auto-demuxes to an .aac adts file, which it does support! After the very precise and lossless editing, one must remux back to .m4a file via ffmpeg/mp4box etc. and retag with GiP (as posted above), if, that is, original file was a GiP audio fetch... Best regards all, Vangelis. ___ get_iplayer mailing list get_iplayer@lists.infradead.org http://lists.infradead.org/mailman/listinfo/get_iplayer
Re: New radio PIDs, more than 8 characters - "solved"
On Tue Aug 22 15:52:56 BST 2017, Hugh Reynolds wrote: The installer failed in trying to delete perl.exe, atomicparsley.exe and ffmpeg.exe and left me with the PVR manager reporting "Access is denied." Those three files seem to be undeletable. I am a supervisor but even that doesn't seem to give me permission to delete those files. Any ideas what I could do now? Hi Hugh First thing is you seem to have "hijacked" a previous thread, because the title of the thread you posted in (New radio PIDs, more than 8 characters - "solved") has nothing to do with your current predicament... You'd better start a new thread titled, e.g., "Windows installation/upgrade problems". That thing aside, from looking at https://github.com/get-iplayer/get_iplayer_win32/compare/3.01.0...3.02.0 I can't spot any major changes to the Win installer... You should begin by supplying some info on 1. The WinOS version/architecture you're using 2. The GiP version previously used (prior to updating) First thing I'd suggest is REBOOT your machine. After loggin back in, inspect in Task manager whether any GiP related processes are running (perl.exe, atomicparsley.exe, ffmpeg.exe) and kill them if they do... Try to first uninstall GiP from Control Panel (you may want to back up your GiP user profile found at %USERPROFILE%\.get_iplayer) followed by a reboot, then proceed with a clean re-install of GiP. Your problem looks as if it's permissions related, or involving locked processes/files. I would recommend the Unlocker utility http://www.iobit.com/en/iobit-unlocker.php which has saved the day for me on numerous occassions, but use it only if you really know what you're doing... Hopefully a reboot will clear your issue; GiP 3.02 was only released a few days ago, if it's some bug in the installer then I'd expect more reports to come forth... At first glance it only looks as something at your end, but little do I know... If all fails, then maybe report your issue over at the support forum, where, hopefully, it'll be addressed by the maintainer himself Regards, Vangelis. ___ get_iplayer mailing list get_iplayer@lists.infradead.org http://lists.infradead.org/mailman/listinfo/get_iplayer
Re: World Service podcast bit rates
On Tue Aug 15 12:40:54 BST 2017, RS wrote: were limited to dafmed, hafmed and hlsaacmed and a bit rate of 96kbit/s. As a general rule, BBC World Service Radio does not impose any form of geo-filtering; in fact, for all "broadcast versions" of its content (not podcast vPIDs), i.e. the ones available as A-O-D streams on iPlayer Radio, they're employing the same set of streams worldwide (as opposed to a UK and an overseas set...). In the not so distant past, the chosen bitrate for WS Radio (both for live and OD) was set at HE-AACv1@64kbps/44.1kHz (and until the end of 2015 there existed live streams in the form of WMA9@32kbps & ShoutcastMP3@48kbps). Then at some point WSR got the "Audio Factory" treament, to be more on a par with the rest of the BBC Radio stations, and thus two stream quality variants were created: HE-AACv1@96kbps/48kHz (*med modes in GiP) and HE-AACv1@48kbps /48kHz (*low modes in GiP); these are equally available to UK/non-UK audiences, i.e. if in the UK you can't get > 96kbps... Since 95% of BBC WS Radio content is "talk-radio", I find 96kbps to be more than adequate for the task... Considering it's a "World" service, meaning they have to cater for a global audience, 96kbps is a fine compromise between quality and bandwidth costs... And even HE-AACv1@48kbps sounds acceptable for those parts of the world with very expensive/slow Internet access... They also mostly had podcasts, and in the UK I was offered a choice of 128kbit/s and 64kbit/s (snip) I don't know either whether the 128kbit/s podcast option is offered outside the UK. As you said, much of the BBC WS Radio content is turning up as MP3 Podcasts: http://www.bbc.co.uk/podcasts/worldserviceradio Programmes with copyrighted music (or other) content are excluded from the podcast treatment, in the rare occasions they do make it to podcast, music tracks are truncated to just 10sec excerpts... Prior to "Audio Factory", only podcasts encoded as 64kbpsCBR/44.1kHz/mono had been available. More and more programmes do show up now with an additional 128kbpsCBR/44.1kHz/stereo option, which IS NOT location specific... (I am only joking here, but several of your recent posts seem to be inquisitive of overseas BBC Radio bitrates, are you planning a retirement to Majorca, Richard?) it seems a lot more attractive than 96kbit/s HE-AAC with the SBR extension (snip) if HE-AAC with SBR ... HE-AAC always comes with SBR, HE-AAC = AAC-LC + SBR is played on a player that does not support SBR half the bandwidth is lost. I have been wondering how best to deal with that. Yet another topic that you're recently concerned with... It does appear as though you're the owner of a hardware device that is incapable of fully rendering HE-AACv1... FWIW, in 2017, 99% of software players on all modern OSes can play back fully HE-AACv1. Even browsers like Firefox 52.3.0ESR does on this old Vista laptop... HE-AACv1 (previously known as aacp/aac+) is even natively supported on most cheap mobile phones, where you need good sound quality at reduced bandwidth (because BW is expensive there...). If your device does not support HE-AACv1, have you contacted its vendor by any chance? Converting to MP3 seems a possibility, (snip) It seems ffmpeg does not support SBR Not true; FFmpeg DOES SUPPORT native decoding of HE-AACv1, as does ffplay, which can be used to play back HE-AACv1 encodes... It is only ENCODING to HE-AACv1 that the native AAC encoder of FFmpeg can't perform (you'll have to build it with the non-free FDK-AAC encoder instead...) I often transcode HE-AACv1 m4a encodes to mp3 files with ffmpeg, here's an example: Batch file used: --- FOR %%N in (*.m4a) DO ffmpeg -v 32 -stats -i "%%N" -vn -c:a libmp3lame -b:a 128k -ar 44100 -ac 2 -joint_stereo 1 -af "volume=2.15" "%%~nN.mp3" pause --- Console excerpt: --- ffmpeg version 3.3.1 Copyright (c) 2000-2017 the FFmpeg developers built with gcc 6.3.0 (Rev3, Built by MSYS2 project) * [mov,mp4,m4a,3gp,3g2,mj2 @ 00a9bfc0] stream 0, timescale not set Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'test-heaac.m4a': * Duration: 00:17:30.94, start: 0.00, bitrate: 63 kb/s Stream #0:0(und): Audio: aac (HE-AAC) (mp4a / 0x6134706D), 44100 Hz, stereo, fltp, 62 kb/s (default) * Stream mapping: Stream #0:0 -> #0:0 (aac (native) -> mp3 (libmp3lame)) Press [q] to stop, [?] for help Output #0, mp3, to 'test-heaac.mp3': * size= 16424kB time=00:17:30.95 bitrate= 128.0kbits/s speed=9.42x video:0kB audio:16422kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.012851% --- I can assure you the MP3 transcode has full audio bandwidth preserved! the latest episode of Science in Action, p05bdb8p. => "Risk of Lethal Heat Waves" ... The story of my life this summer! :-) But, returning on topic, get_iplayer --type=radio --pi
Re: New radio PIDs, more than 8 characters - "solved"
On Mon Aug 14 13:19:14 BST 2017, M Clark wrote:
changing all 7 occurrences (sigh...) of
[bp]0[a-z0-9]{6}
to
(?:[bp]0[a-z0-9]{6}|w[a-z0-9]{7,14})
solves the w3*, w1* problem for Me.
Hi Martin; that new code still assumes that both Red Bee & PIPs
PIDs will have "0" as the second character in the string.
I am not saying this is something that will have to be dealt with soon,
but I've watched Red Bee PIDs move from "b08*" to "b09*"
and, recently, from "b0909***" to "b0910***", e.g. "b0910w0x".
If this is a pattern, then I expect strings like "b0999***" to appear in
the future;
and the next logical (?) step would be strings beginning with "b1**"
(in which case the amended code will break..).
Pure speculation on my part, though...
I haven't done the maths myself (number of permutations of 7 alphanumeric
strings),
this is supposed to be a huge integer; but, as PIDs are unique
(can't be recycled), linked to a specific audio-visual offering from the
beeb,
that huge number is bound to be exhausted sometime...
Regards,
Vangelis.
___
get_iplayer mailing list
get_iplayer@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/get_iplayer
Re: New radio PIDs, more than 8 characters
On Sun Aug 13 18:28:42 BST 2017, RS wrote:
The Al Gore one is now p05c2b9j
(snip)
The changes may be temporary.
... Yes and no... If you browse to:
http://www.bbc.co.uk/programmes/p016tl04/episodes/player
you'll see there exist two distinct entries for
"Former US Vice President Al Gore on Climate Change":
http://www.bbc.co.uk/programmes/w172vg029mkl852
(which looks more like the proper episode page) and
http://www.bbc.co.uk/programmes/p05c2b9j
I haven't listened to them fully to compare them,
both are listed as being 52min59sec in duration...
The next ("Trump... ") episode of the series is still
only available as a "w" PID. And on
http://www.bbc.co.uk/programmes/p029zl67/episodes/player
latest episode is still available as an 8-digit "w" PID :-(
The crux of the matter is that these "w"-PID progs
are not recordable with the current iteration of GiP,
not even when the progs are indexed inside the
radio.cache and one uses " --get" to fetch!
Haven't seen the figures for UK audiences, but
I'll let you know WSR is very popular overseas!
On Sun Aug 13 20:54:11 BST 2017, C E Macfarlane wrote:
> [bpw][a-z0-9]{14}
(snip)
Firstly it would have to be ... {8,14},
otherwise shorter ones would not be picked up.
Rather "{7,14}", since the minimum character length
of a PID is 8...
But there is a potential problem with [a-z0-9]
because it would pick out many normal English words,
for example, ironically, 'programmes' and 'programming'.
Again, forgive me for being obtuse, still learning things,
but isn't that scenario assuming one actually inputs
--pid="programming"
into a GiP command? (and in that case, even if the string
is validated as a PID, the playlist.json URL will not
return any vpid for that PID, hence no download...).
Best regards,
Vangelis.
___
get_iplayer mailing list
get_iplayer@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/get_iplayer
Re: New radio PIDs
On Sun Aug 13 10:05:39 BST 2017, Ralph Corderoy wrote:
^(?:[bp]0|w3)[a-z0-9]{6}$
Cheers Ralph, your coding skills are always welcome in this list! :-)
I was about to say that the above assumes that ordinary PIDs
will continue to begin with "(b|p)0" and that "new" WSR ones
will always begin with "w3", when M Clark posted:
http://lists.infradead.org/pipermail/get_iplayer/2017-August/010994.html
which, of course, brought new data to light...
FWIW, I revisited the definitive article on PIDs (WARNING: Long read!):
http://smethur.st/posts/176135860
The only character in a PID that is "meaningful"
is the first one which denotes the authority of the PID
(the people responsible for creating it).
For PIDs starting b the generating authority is Red Bee,
for PIDs starting p it's PIPs,
for w it's the old World Service scheduling system.
(snip)
Because World Service programme information doesn't travel
through Red Bee they self-provision programme data into PIPS
which lead to their early PIDs looking slightly different
to most, being 11 characters, not 8:
http://www.bbc.co.uk/programmes/wcr5dr3dnl3
I'm not sure when the changeover happened
but these days World Service PIDs are generated as p PIDs
with a PIPs authority and are 8 characters long.
So in the distant past (June 2008) there used to be 11-digit
PIDs beginning with "w" for WSR programmes;
what brought about the new PIDs system with either 8
or 15-digit "w" PIDs remains unclear (?) ...
Maybe "Jim web" could, once more, poke his BBC Radio contacts?
On Sun Aug 13 19:51:22 BST 2017, James Scholes wrote:
^[0-9b-df-hj-np-tv-z]{8,}$
Hi James, so nice you dropped by with such
an interesting find!
Did a bit of googling myself, "[0-9b-df-hj-np-tv-z]{8,15}"
comes up inside deprecated projects:
http://www.bbc.co.uk/frameworks/micro/guides/projectjson
http://www.bbc.co.uk/frameworks/micro/guides/projectjsonv2
both part of MICRO...
Also,
http://www.bbc.co.uk/frameworks/route/reference
has it in this line:
programme.reqs.pid = "[0-9b-df-hj-np-tv-z]{8,15}"
Of relevance:
https://www.npmjs.com/package/bbc-pid
BBC Programmes Identifiers 0.4 spec
defines a PID as following:
Characters: digits 0-9 and lower case letters, less vowels.
Length: Minimum 8 digits. No defined maximum
- they will grow as necessary.
For historical reasons there are some 15 character pids
existing for World Service content.
... Historical reasons?
Cheers,
Vangelis.
___
get_iplayer mailing list
get_iplayer@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/get_iplayer
New radio PIDs
On Sun, Aug 13th 2017, Support Forum member "cje768" wrote:
https://squarepenguin.co.uk/forums/thread-1476-post-6567.html#pid6567
As things stand, GiP expects every valid 8-character alphanumeric PID
string
to begin with either "b0" or "p0".
New radio PIDs like "w3csv1y9" or "w3csvnyc", beginning with "w3",
are discarded as invalid, hence GiP does not retrieve any streamdata
for those... :-(
I am not a regex/perl specialist (quite the opposite, major dunce!),
but in my local copy of get_iplayer 3.01-windows.0 I replaced
every instance of
[bp]0[a-z0-9]{6}
with
[bpw][a-z0-9]{7}
and was thus able to fetch --pid=w3csv1y9;
still unsure as to whether other aspects of GiP
are broken by the above patch...
A stopgap solution, I guess...
(You can't teach an old dog new tricks,
or so the saying goes... Am afraid too old
to pick up learning perl from scratch :-( )
Happy Sunday, all!
Vangelis.
___
get_iplayer mailing list
get_iplayer@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/get_iplayer
Re: Playing BBC R3 FLAC files recorded by nightly VLC
People, people... When I first mentioned VLC-3.0.0-git (Nightly) as a means of recording the R3 flac MPEG-DASH stream, it was never my intention to contribute to the creation of a protracted OT thread; my references to VLC were terse on purpose and assumed some user familiarity; even if that wasn't the case, VLC has online documentation http://www.videolan.org/support/ wiki https://wiki.videolan.org/Documentation:VLC_for_dummies https://wiki.videolan.org/Knowledge_Base forum https://forum.videolan.org/ and user mailing list: https://mailman.videolan.org/listinfo/vlc Any more niche queries can be usually answered by searching specialised fora like videohelp, doom9, hydrogenaudio etc. @Richard (and possibly others): Yes, you need to preconfigure your VLC recordings directory prior to starting a stream recording (any stream); never mind setting a recording filename, VLC can work its way creating meaningful unique filenames with timestamps (e.g. vlc-record-2017-08-06-22h38m21s-Radio 3 lossless-.ogg). VLC Nightly issues: Just a WORD OF CAUTION for those having an installed VLC 2.2.x copy on Windows: the default/user settings of VLC are stored in %APPDATA%\vlc ; you'd better make a backup of that dir prior to executing VLC 3.0.0-git; 2.2.x vs 3.0.0 settings are not fully interchangeable; 3.0.0-git version, even when run from an unzipped package (like Richard did) will still write on %APPDATA%\ and overwrite 2.2.x's settings. As I'm on a 32bit OS, I have always used the win32.7z packages of nightly and never had any issues opening the "Media -> Open Network Stream" popup and pasting a URL there... ogg recording playback issues: As Jim has said, the whole technique of streaming FLAC encoded audio data via MPEG-DASH (FLAC encapsulation inside the MP4 container and creation of a segmented mp4 stream) is still quite experimental; as such, recording of that stream by VLC is even more so... Playback of the recorded .ogg file is supported by VLC 3 itself, BUT NOT by previous VLC versions. A few other software players also do, like latest PotPlayer (released and beta versions) and the released (1.7.13) and beta (1.7.13.60) versions of MPC-HC (which uses the FFmpeg based LAV Filters); MPC-BE will start to play the file, but will crash towards the end of the recording. .ogg recording extracting issues: I had already mentioned these towards the end of previous post: http://lists.infradead.org/pipermail/get_iplayer/2017-August/010957.html Please understand that the RAW flac stream inside the OGG container is the product of concatenation of successive DASH segments, so not really a proper flac stream; the whole situation is similar to iso6 mp4 files created by dash stream dumping; these are not standard mp4 files, and in the case of GiP ffmpeg is needed to remux them properly into a standard MP4 variant palatable to most (soft|hard)ware MP4 players. Perhaps in the not so far future, ffmpeg will be patched to losslessly extract this dash-flac ES from a container, for the moment though it's necessary to losslessly recode to flac to make this possible (and reduce FFmpeg verbosity to -v 8 to remove all the DTS warnings...). BTW, my command ffmpeg -v 8 -stats -i "vlc-record-2017-08-03-01h36m18s-Radio 3 lossless-.ogg" -vn -c:a flac "Radio 3 lossless.flac" is in essence identical to RS's ffmpeg -i=.ogg -f=flac -sample_fmt=s16 .flac as "-f flac -sample_fmt=s16" forces ffmpeg to use the flac encoder @all: For the nth time, the BBC DO NOT PROVIDE R3 AOD as lossless flac encodes, certainly not in their browser embedded DASH player (Flash is unsuitable for the task); hence no FLAC is retrievable via GiP! It is pure speculation (at best, very premature... ) to discuss now what will happen in the future... Audacity has come up in the past in this list (I think't was CJB who suggested it for audio recordings on-the-fly). Audacity and many other so-called audio recorders (I occasionally use mp3DirectCut) are acting as man-in-the-middle and making use of the so-called "analog loop(hole)" https://en.wikipedia.org/wiki/Analog_hole as detailed by Richard in http://lists.infradead.org/pipermail/get_iplayer/2017-August/010979.html VLC-3.0.0 (and patched FFmpeg) will produce digital clones of the flac stream. @Paul Thornett You are, of course, entitled to your own opinion, but I can't find personally any justification for your overall aversion to VLC; I think it's a hugely versatile Audio-Video open-source application, that has saved the day for me on countless occasions... And please, can you refrain from discussing BBC geo-blocking circumvention methods in this list? The list maintainer has explicitly stated in the past that such topics are not tolerated here... (And as you've discovered yourself, the beeb are on a constant battle blacklisting many of these methods...). FWIW, the test R3 live flac stream is not geo-filtered, can be accessed DIRECT... Enjoy it while you (and I) can, because if it ever makes it into a standard R3
Re: Possible To Get BBC R3 In FLAC using GiP?
On Thu Aug 3 12:38:53 BST 2017, C E Macfarlane wrote: already has a recent version of VLC installed, I wonder if that might be able to download the FLAC stream? Latest released version of VLC is 2.2.6; unfortunately, the 2.2.x branch is uncapable of playing back/recording the live MPEG-DASH R3 flac stream (plus, it's plagued by a still unresolved bug that causes extensive CPU usage on Windows OSes during playback/recording of AppleHLS streams...). As outlined in my previous post, you'd have to definitely go for the nightly VLC-3.0.0-git branch, and from there select a fairly recent build (mid-July and newer). wtf are you wasting holiday time on the likes of me?! Enjoy it while you can! ... I can assure you you have not been treated in any special way just because it was "the likes of" you! As for holiday time, already had a week's escape in July to a friend's house in a village by the sea, no more holiday time is scheduled for me... We've been hit (northern Greece) several times already by severe heatwaves (the likes of which we had more than 10 years to sustain), all this heat (coupled with high humidity) has a toll on you; 30 C at night breaks your sleeping patterns, one can't always rely upon A/C, it's too bad for one's health, the environment and wallet! I very rarely discuss politics in this list, but thanks to an 8 year long recession (which is unlikely to end soon) and the wills of our international creditors, "holiday" has become a taboo word for native middle-class Greeks; the majority are on permanent "holiday", due to lengthy unemployment, inside their homes, unable to cope with their bills and debts; losing their houses to banks and the state is what they fight against, am afraid no money is available for summer holidays to the islands and to other "hot" locations chosen by tourists :-( Best wishes, Vangelis. ___ get_iplayer mailing list get_iplayer@lists.infradead.org http://lists.infradead.org/mailman/listinfo/get_iplayer
Re: Strange CodePage Error in GiP2.99 On XP
On Wed Aug 2 21:44:06 BST 2017, C E Macfarlane wrote:
--type tv --tvmode hafhd,hlshd
... Unless it's a typo, hafhd is a non-existent tvmode...
Did you mean hvfhd (720p@50FPS) ?
--pid b007c0ys ... -g
--pid always triggers a fetch (unless --info etc. are used),
so --get (-g) is redundant in this case :-)
Cannot find encoding "cp850" at
C:\Programs\GetIPlayer\get_iplayer.pl
line 323
That line reads:
binmode(STDOUT, ":encoding($opt->{encodingconsoleout})");
From GiP-2.99's longhelp:
--encoding-console-out Character encoding used to encode search
results and other output. Encoding name must be known to Perl Encode
module. Default (only if auto-detect fails): Linux/Unix/OSX = UTF-8,
Windows = cp850
Now, I'm not a perl expert but the above indicate
that your installed Encode perl module should
autodetect your "encodingconsoleout"; if autodetection
fails or encoding is unknown to the module,
then it should default to "cp850".
It appears that senario is now true in your setup;
but "cp850" is still not found (?), hence the error shown...
FWIW, on my setup (WinVistaSP2 x86, en-US locale),
when I generate a debug log with GiP 2.99 I get:
encodingconsolein = cp737
encodingconsoleout = cp737
encodinglocale = cp1253
encodinglocalefs = cp1253
This is with Strawberry Perl 5.24.0.1.
Ralph Corderoy wrote:
Try using
"--encoding-console-out cp1252" option
Of course Ralph is a code wizard, so
he could possibly enlighten us further,
but according to --longhelp, "cp1252"
is the value used as a default fallback
when autodetection fails in the cases of
--encoding-locale & --encoding-locale-fs
C E Macfarlane wrote:
Also, the new FFMPEG fails on XP with:
Please refer to my May post here:
http://lists.infradead.org/pipermail/get_iplayer/2017-May/010711.html
for possible workarounds :-)
Kindest regards,
Vangelis.
___
get_iplayer mailing list
get_iplayer@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/get_iplayer
Re: Possible To Get BBC R3 In FLAC using GiP?
On Wed Aug 2 18:43:49 BST 2017, C E Macfarlane wrote: if it is possible to obtain the Proms iPlayer streams in FLAC, Hello Charles, I do hope all's well with you! The BBC do not provide AOD in flac, hence it's impossible to acquire such files via GiP (which currently only supports on-demand streams). Recent Support Forum thread: https://squarepenguin.co.uk/forums/thread-1463-post-6518.html are only live broadcasts available as FLAC? Precisely :-) Do note that this feature is only a BBC Labs test; so it might not return after Proms season is over. There has been a relevant (albeit off-topic) list thread back in April http://lists.infradead.org/pipermail/get_iplayer/2017-April/010436.html when the experiment first went live, but TBH the choice of thread title isn't very indicative of the thread's subject (so as to spot it easily inside the list archives)... ;-( The live flac stream is using MPEG-DASH (unencrypted) type of delivery, the exact mpd (media presentation description) URI is: https://vs-dash-ww-rd-live.bbcfmt.hs.llnwd.net/al/lossless/client_manifest.mpd (the -ww- part of the hostname, which belongs to a Limelight CDN, indicates the stream is not geo-blocked, so audiophiles from all over the world can test it, provided their OS+Firefox version support it). Back during the GiP 2.95-dev era (Jan 2016), the GiP coder briefly experimented with supporting live radio dash streams: https://github.com/get-iplayer/get_iplayer/commit/8a90279 but this feature never made it to a released version: https://github.com/get-iplayer/get_iplayer/commit/9f6b8c6 Probably someone else with the required perl (or other) expertise can conjure up a perl script to dump that live DASH (but FLAC encoded) stream to file. === OT info (STOP HERE if bothered by OT content): FWIW, a special patch for FFmpeg that only applies to a special (previous) snapshot of FFmpeg (commit git-20f7872) has been posted here: https://patchwork.ffmpeg.org/patch/3346/ (or http://ffmpeg.org/pipermail/ffmpeg-devel/2017-April/210141.html) but this initial DASH stream recording feature hasn't made it to the official repo yet... I have been successful in compiling a win32 FFmpeg build (FFmpeg-n3.4-dev-301-git-20170409-N-85396-g20f7872+dash_demuxer_v14-libressl-win32) that is able to capture this flac LIVE stream, unfortunately it's a non-free build, as it uses libressl (openssl fork) as the crypto lib; notice the stream uses HTTPS... The RAW flac stream that is dumped to disk is not a proper flac file, rather a "fragmented" flac file, the product of dash streaming; so simply issuing ffmpeg -v 16 -stats -re -i "https://vs-dash-ww-rd-live.bbcfmt.hs.llnwd.net/al/lossless/client_manifest.mpd"; -t 00:30:00 -vn -c:a copy BBCR3LL.flac will produce a non-seekable file with many software players; better mux the stream to the OGG container: ffmpeg -v 16 -stats -re -i "https://vs-dash-ww-rd-live.bbcfmt.hs.llnwd.net/al/lossless/client_manifest.mpd"; -t 00:30:00 -vn -c:a copy BBCR3LL.oga or experimental mux to the MP4 container: ffmpeg -v 16 -stats -re -i "https://vs-dash-ww-rd-live.bbcfmt.hs.llnwd.net/al/lossless/client_manifest.mpd"; -t 00:30:00 -vn -c:a copy -strict -2 BBCR3LL_tmp.mp4 ffmpeg -v 32 -stats -i BBCR3LL_temp.mp4 -vn -c:a copy -strict -2 -movflags faststart BBCR3LL.mp4 && del BBCR3LL_tmp.mp4 for seekable files (needs recent versions of players); or losslessly recode on-the-fly to flac: ffmpeg -v 16 -stats -re -i "https://vs-dash-ww-rd-live.bbcfmt.hs.llnwd.net/al/lossless/client_manifest.mpd"; -t 00:30:00 -vn -c:a flac BBCR3LL-rc.flac BTW, if you plan to edit the recorded flac stream further with FFmpeg, you'll bump on this bug: https://trac.ffmpeg.org/ticket/4905 You'll have to DECODE TO WAV => PIPE => EDIT => ENCODE TO FLAC E.g. to cut out a segment from an hour's worth of live stream, I used: ffmpeg -v 8 -stats -i original.flac -vn -f wav - | ffmpeg -v 8 -stats -i - -ss 00:42:19 -to 00:58:11 -vn -af aformat=s16:48000 cut.flac If you can't compile for yourself that special, DASH enabled, build of FFmpeg, then your best bet is to try a recent nightly build of VLC 3.0.0-git (win builds link): http://nightlies.videolan.org/build/win32/last/ Often times, those nightly builds are unstable (might even cause BSODs), but nothing evil has happened to my system by running the latest compiles (e.g. I'm on VLC 3.0.0-git-20170731-0258). You'll have to tweak the GUI to make that red Record button appear (not there by default), then search on how to record a live stream with VLC; use the mpd URI posted previously... VLC 3.0.0-git muxes the segmented FLAC stream inside the OGG container by default, resulting in an .ogg file. MediaInfo log: Format : Ogg Format/Info : Free Lossless Audio Codec File size: 7.47 MiB Duration : 1 min 21 s Overall bit rate mode
Re: Podcast sample rate
On Mon Jul 17 17:32:03 BST 2017, RS wrote: At 48kbit/s Mediainfo shows the sampling rate as 48.0kHz / 24.0kHz. ... If you are referring to the original *low radiomodes as fetched by GiP, then, and someone correct me, please, if I'm wrong, I believe 48.0kHz SR applies to hardware/software media players which are capable of reproducing the SBR portion of the HE-AACv1 encode; for SBR incompatible players, only the LC portion is played back at half the SR, i.e. 24.0kHz (resulting in most higher frequencies of the spectum being muted...). If I have misunderstood and you're referring to something else, please disregard and accept my apologies :-) Vangelis. ___ get_iplayer mailing list get_iplayer@lists.infradead.org http://lists.infradead.org/mailman/listinfo/get_iplayer
Re: Version "shortened"
On Mon Jul 17 12:32:31 BST 2017, RS wrote: in the LW schedule I referred to Dead Ringers started at 1845 but it only ran until 1900. In other words it was very much shortened from 27min 54s to just under 15 min. The version now available is the full length but it is still being called version shortened. Hi Richard :-) You are correct; last night when I visited http://www.bbc.co.uk/programmes/b08xcwz4/broadcasts before posting my answer to Alastair's query, the first broadcast entry was: Fri 14 Jul 2017 18:45 BBC Radio 4 LW that's why I mentioned it in my post; today I see that Fri 14 Jul 2017 18:30 BBC Radio 4 FM has been (belatedly) added as first broadcast date! As you said, browsing http://www.bbc.co.uk/radio4/programmes/schedules/lw/2017/07/14 one can clearly see the time of broadcast (and expected duration) of the LW version; clicking the DR entry in that schedule takes you to http://www.bbc.co.uk/programmes/b08xcwz4 where its duration is advertised as 15 minutes (on the page). But currently the embedded player streams the full episode (and GiP does fetch likewise...). What is still odd is that if one browses http://www.bbc.co.uk/radio4/programmes/schedules/fm/2017/07/14 one can see that DR (as you said) was broadcast for ca. 30min starting 18:30 BST, but the link there again takes you to the same episode page http://www.bbc.co.uk/programmes/b08xcwz4 where it is (again) advertised as being only 15min long! Me thinks this is a c*ckup on the beeb's end! They should've created two different pids (or even two different vpids), one for the abridged (on LW) and another one for the full (on FM) version. But I think I (possibly even you) have entered OT territory; those comments should've been communicated to the BBC; GiP can only grab what's put online by the BBC; if they messed things up, it's not a GiP's fault... Another curiosity is that estimated mode sizes in --info show much larger sizes for DAF modes than the corresponding HAF and HLSAAC modes. I think the keyword here is "estimated"; I did get_iplayer --type=radio --pid=b08xcwz4 -i | findstr modesizes and, e.g. hafhigh*=36MiB, dafmed*=19MiB, hafstd*=15MiB, hafmed*=11MiB But both dafmed & hafmed will yield a 19.3 MiB m4a file (HE-AACv1@96kbps) - so dafmed estimate is the most accurate; hafstd will, instead, produce a 25.8 MiB m4a file (AAC LC@128kbps), a big diff from the initial estimate of just 15MiB. I don't think something could be done codewise to improve the size estimations beforehand (i.e. without actually starting a downlod); GiP already tries to make best use of whatever size info is advertised in the relevant mediaselector pages, when/if that is present; especially for HLS (haf, hlsaac, hvf) modes, size has to be calculated based on average stream bitrate and sometimes expected and actual bitrate do differ (especially in video files). Another reason we're seeing here larger relative diffs is because of the small size of the audio file to begin with; if it were a big video file of, say, 600+ MiB, then a few MiBs offset wouldn't strike that bad... Best regards, Vangelis. ___ get_iplayer mailing list get_iplayer@lists.infradead.org http://lists.infradead.org/mailman/listinfo/get_iplayer
Re: Version "shortened"
On Sun Jul 16 23:55:54 BST 2017, Budge wrote: Just tried to get latest version of Dead Ringers Hello - I am unfamiliar with that BBC offering, but a bit of searching suggests you must be talking about the following R4 show: http://www.bbc.co.uk/programmes/b007gd85 According to http://www.bbc.co.uk/programmes/b007gd85/episodes/guide "latest" version is: Series 18, Episode 14/07/2017, with an iPlayerRadio page as follows: http://www.bbc.co.uk/programmes/b08xcwz4 That episode was first broadcast, well, last Friday 14/07/2017 at 18:45 BST on R4LW. and more than half the programme appears to be a cricket commentary with Dead Ringers coming in towards the end of the 30 mins for the last 5-6 mins. The download is shown not original but "shortened." If you are indeed talking about --pid=b08xcwz4, then this is something I can't replicate... On the iPlayer page itself, the programme does seem to be streamed in its entirety. get_iplayer --type=radio --pid=b08xcwz4 --radiomode=dafmed2 does fetch an audio file with default filename "Dead_Ringers_Series_18_-_2017-07-14_b08xcwz4_shortened.m4a", so indeed a "shortened" version, but the audio content is identical to what's on line (I wouldn't expect differently...). Furthermore, last Friday's episode has been also made available as a "podcast" version (as part of Friday Night Comedy from BBC Radio 4), either as an online stream at http://www.bbc.co.uk/programmes/p058rmcz (downloadable with GiP with --type=radio --pid=p058rmcz) or as a direct MP3 download via http://open.live.bbc.co.uk/mediaselector/5/redir/version/2.0/mediaset/audio-nondrm-download/proto/http/vpid/p058rm6r.mp3 All the above avenues do provide a full episode. There's an off-chance you downloaded via a PVR search very soon after the scheduled end of broadcast of that episode; for whatever reason, that specific episode was delayed (the show preceding it was overrun) and the version of DR uploaded initially (automatically) for AOD (in fact as an edit of the live radio stream) was in the state you described. At a later stage, the iPlayerRadio team did replace that initial incomplete version with the proper full show... More info at: http://iplayerhelp.external.bbc.co.uk/radio/computer/cut_short If you're actually meaning a different pid, then please specify... Regards, Vangelis. ___ get_iplayer mailing list get_iplayer@lists.infradead.org http://lists.infradead.org/mailman/listinfo/get_iplayer
Bug with "hlsaacmed" radiomode
Hello one and all :-) As you might already know, The BBC World Service Radio does not impose geo-filtering in its radiomodes, in fact the very same audio streams are used for both UK and overseas consumption... Hence, no matter where in the globe you are, using GiP 3.01 and issuing: get_iplayer --type=radio --pid=p057fw7p -i | FindStr modes (on Windows) will get you: - modes: original: dafmed1,dafmed2,dafmed3,dafmed4,daflow1,daflow2,daflow 3,daflow4,hafmed1,hafmed2,haflow1,haflow2,hlsaacmed1,hlsaaclow1 modesizes: original: dafmed1=36MiB,dafmed2=36MiB,dafmed3=36MiB,dafmed4=36Mi B,daflow1=18MiB,daflow2=18MiB,daflow3=18MiB,daflow4=18MiB,hafmed1=36MiB,hafmed2= 36MiB,haflow1=18MiB,haflow2=18MiB,hlsaacmed1=34MiB,hlsaaclow1=17MiB - The *med* radiomodes (ca 36 MiB) are HE-AACv1@96kbps encodes, while the *low* ones (half that size) are HE-AACv1@48kbps encodes. Now, running get_iplayer --type=radio --pid=p057fw7p you'll get: - INFO: dafmed1,dafmed2,dafmed3,dafmed4,hafmed1,hafmed2,daflow1,daflow2,daflow3,da flow4,haflow1,haflow2,hlsaaclow1 modes will be tried for version original INFO: Trying dafmed1 mode to record radio: Top of the Pops - 07/07/2017 GMT (large snip) INFO: Recorded: 36.73 MiB (00:53:00) [497] in 00:04:36 at 1.06 Mibit/s - (the more observant among you may have already spotted the bug...) dafmed1 is fine, however a bit slow with regards to my location; hafmed is equivalent (or even slower) to dafmed, but the really quick mode for me is hlsaac*. HOWEVER, when I issued get_iplayer --type=radio --pid=p057fw7p --modes=hlsaac I got: - INFO: hlsaaclow1 modes will be tried for version original INFO: Trying hlsaaclow1 mode to record radio: Top of the Pops - 07/07/2017 GMT - i.e. the lower quality (48kbps) started fetching, not the medium (in this case highest) at 96kbps :-( The same thing happens when I set "hlsaacbest" (instead of simply "hls" or "hlsaac"). OF COURSE, explicitly asking for --radiomode=hlsaacmed works, but this is just a workaround for existing bug... I had a quick browse of the code, first I added "hlsaacmed" to the quality shortcuts by modifying lines $mlist = main::expand_list($mlist, 'hlsaacvgood', 'hlsaacstd,hlsaacgood'); -$mlist = main::expand_list($mlist, 'hlsaacgood', 'hlsaacworse'); +$mlist = main::expand_list($mlist, 'hlsaacgood', 'hlsaacmed,hlsaacworse'); and $mlist = main::expand_list($mlist, 'radiovgood', 'dafstd,hafstd,hlsaacstd,radiogood'); -$mlist = main::expand_list($mlist, 'radiogood', 'dafmed,hafmed,radioworse'); +$mlist = main::expand_list($mlist, 'radiogood', 'dafmed,hafmed,hlsaacmed,radioworse'); and then added "hlsaacmed" to the default radiomodes by changing: -radiomode => [ 1, "radiomode|amode=s", 'Recording', '--radiomode ,,...', "Radio recording modes (overrides --modes): dafhigh,dafstd,dafmed,daflow,hafhigh,hafstd,hafmed,haflow,hlsaacstd,hlsaaclow. Shortcuts: worst,worse,good,vgood,better,best,daf,haf,hlsaac (default=dafhigh,hafhigh,dafstd,hafstd,hlsaacstd,dafmed,hafmed,daflow,haflow,hlsaaclow)."], +radiomode => [ 1, "radiomode|amode=s", 'Recording', '--radiomode ,,...', "Radio recording modes (overrides --modes): dafhigh,dafstd,dafmed,daflow,hafhigh,hafstd,hafmed,haflow,hlsaacstd,hlsaacmed,hlsaaclow. Shortcuts: worst,worse,good,vgood,better,best,daf,haf,hlsaac (default=dafhigh,hafhigh,dafstd,hafstd,hlsaacstd,dafmed,hafmed,hlsaacmed,daflow,haflow,hlsaaclow)."], Other parts of the code may need patching, too, but now my local copy of 3.01 behaves as expected: get_iplayer --type=radio --pid=p057fw7p --modes=hlsaacbest - Matches: 24945: Top of the Pops - 07/07/2017 GMT, BBC World Service, p057fw7p INFO: 1 Matching Programmes INFO: Checking existence of original version INFO: hlsaacmed1,hlsaaclow1 modes will be tried for version original INFO: Trying hlsaacmed1 mode to record radio: Top of the Pops - 07/07/2017 GMT INFO: File name prefix = Top_of_the_Pops_-_2017-07-07_GMT_p057fw7p_original (large snip) INFO: Recorded: 39.22 MiB (00:52:59) [318] in 00:00:41 at 7.65 Mibit/s - (NB that hlsaacmed1 downloads 7x faster than dafmed1) Regards, Vangelis. ___ get_iplayer mailing list get_iplayer@lists.infradead.org http://lists.infradead.org/mailman/listinfo/get_iplayer
Re: A bug in get_iplayer-3.01?
On Thu Jul 6 23:33:49 BST 2017, michael norman wrote: So can I go to sleep in the knowledge that the sky might not fall in before I wake up ? https://www.youtube.com/watch?v=7HKoqNJtMTQ ... But, seriously, no list activity is in general a good thing; no major breakage has occured in the app, no one had any serious problem that needed to be attended to here in the list or (possibly currently) it's simply the case of summer holidays/vacations for many members... :-) If you didn't take it (though I'm sure most did), I was merely hinting to the eventual breakage of things when that "I'll do it later" option in both iPlayer TV+Radio soon vanishes into thin air... And speaking from past list experience, when such a "storm" does hit, the list itself is inundated by the flood waters... Take the best of care, V. ___ get_iplayer mailing list get_iplayer@lists.infradead.org http://lists.infradead.org/mailman/listinfo/get_iplayer
Re: A bug in get_iplayer-3.01?
On Thu Jul 6 22:58:08 BST 2017, Ralph Corderoy wrote: and there hasn't been anything on the mailing list since then. ... and I call this "calm before the storm" ... :-) Regards, Vangelis. ___ get_iplayer mailing list get_iplayer@lists.infradead.org http://lists.infradead.org/mailman/listinfo/get_iplayer
Re: A bug in get_iplayer-3.01?
On Sat Jun 17 16:32:51 BST 2017, RS wrote:
Not only do they use it, but there is a demand for multi-level
subdirectories.
Hello Richard, thank you for bringing the "--subdir-format"
issue to everyone's attention :-). I, for one, was oblivious
to it, since not a --subdir (hence --subdir-format) user...
Your change is fine for users who just use
--subdir without --subdir-format,
(snip)
but it would prevent use of --subdir-format
to create multi-level subdirectories.
... Not on Windows, if backslashes are used,
get_iplayer --type=tv --pid=b08r69t1 --subdir --subdir-format="\"
-i | FindStr dir =>
dir:D:\Vangelis\iPlayer Recordings\Vets_24_7\Series_4
but indeed true for Linux/Unix/OSX
when forward slashes are used ;-(
if ($opt->{subdirformat} != '') {
my $subdir = $prog->substitute($opt->{subdirformat}, 1)
};
else {
$subdir = $prog->substitute ('', 0)
};
Later the same day, RS wrote:
Anyway I have now found why --subdirformat is not working.
!= is a numeric comparison operator.
I should have used the string comparison operator ne.
The revised version becomes replace line 4043 with
my $subdir;
if ($opt->{subdirformat} ne "")
... As the "--subdir-format" option always accepts a value,
(unlike the plain "--subdir" switch), hence it would be illogical
for someone to just issue "--subdir-format" in the command
(or have in options), I would have used myself simply
if ($opt->{subdirformat})
for the "if" part...
And, again, later in the day, RS wrote:
my $subdir;
if ($opt->{subdirformat} ne "") {
$subdir = $prog->substitute( $opt->{subdirformat}, 1);
}
else {
$subdir = $prog->substitute ('', 0);
}
Trying to stay as close to the original code as possible,
I would write what you're trying to accomplish as:
my $subdir = $prog->substitute( $opt->{subdirformat}, 1 ) ||
$prog->substitute( '', 0 );
HOWEVER, both your code and mine again fall short
of dealing with the original issue that started
this thread: the existence of a forward slash inside
, hence also inside and .
If one of the three <*name*> substitution parameters
is used inside --subdir-format value (which is highly
probable) then, because $opt->{subdirformat} undergoes
sanitize_mode == 1, that "/" will be left intact and
continue to cause the original issue...
On Windows, with your code:
get_iplayer --type=tv --pid=b08r69t1 --subdir --subdir-format="\"
-i | FindStr dir
returns:
dir:D:\Vangelis\iPlayer Recordings\Vets_24\7\Series_4
The proper solution to this issue would be
to write code that replaces that "/" with "_"
in the <*name*> parameters when --subdir is used,
standalone or with --subdir-format...
(Nigel Taylor, where are you? :-) )
Best regards,
Vangelis.
___
get_iplayer mailing list
get_iplayer@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/get_iplayer
Re: A bug in get_iplayer-3.01?
On Mon Jun 5 23:55:58 BST 2017, RS wrote:
I was trying to work out how the procedures
substitute and sanitize_path were intended to interact.
They both seem to be doing the same job
in different ways which are hard to follow.
RS previously wrote:
substitute needs to be called before create_dir,
so that the sub-directory name is sanitised
by removal of the / before the sub-directory is created.
and
In another part of the code there is a variable sanitize_mode
which can have values between 0 and 4
to denote different treatment for
sanitising file and directory names
This is:
# sanitize_mode == 0 then sanitize final string and also sanitize '/' in
field values
# sanitize_mode == 1 then sanitize final string but don't sanitize '/' (and
'\' on Windows) in field values
# sanitize_mode == 2 then just substitute only
# sanitize_mode == 3 then substitute then use encode entities for fields
only
# sanitize_mode == 4 then substitute then escape characters in fields only
for use in double-quoted shell text.
As Ralph (thanks!) has pointed out earlier in the thread:
I think the call where it's passed as 1 is
(snip)
elsif ( $opt->{subdir} ) {
my $subdir = $prog->substitute( $opt->{subdirformat} || '',
1 );
So, the substitution parameter is
undergoing sanitize_mode == 1, hence "/" isn't replaced.
Instead of applying the patch I posted previously,
I changed
-my $subdir = $prog->substitute( $opt->{subdirformat} || '', 1 );
+my $subdir = $prog->substitute( $opt->{subdirformat} || '', 0 );
which seems to also have the desired effect:
get_iplayer --type=tv --pid=b08r69t1 --subdir -i | FindStr dir =>
dir:D:\Vangelis\iPlayer Recordings\Vets_24_7_Series_4
This has only been tested on Windows, I don't use --subdir
myself TBH, so unsure whether this breaks other stuff...
Regards,
Vangelis.
___
get_iplayer mailing list
get_iplayer@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/get_iplayer
Re: A bug in get_iplayer-3.01?
On Mon Jun 5 14:51:45 BST 2017, RS wrote:
I take it the answer is that substitute needs to be called
before create_dir, so that the sub-directory name
is sanitised by removal of the /
before the sub-directory is created.
Hello Richard and many thanks for your analysis :-)
I am otherwise occupied lately, apologies I can't
contribute to the list to the extent I'd wish for...
Remember, I DO NOT KNOW perl, but patching the code like:
# Create a subdir for programme sorting option
elsif ( $opt->{subdir} ) {
my $subdir = $prog->substitute( $opt->{subdirformat} || '', 1 );
+# Replace forwardslashes in $subdir with _
+$subdir =~ s/\//_/g;
$prog->{dir} = File::Spec->catdir($prog->{dir}, $subdir);
has allowed for:
get_iplayer --type=tv --pid=b08r69t1 --subdir -i | FindStr dir =>
dir:D:\Vangelis\iPlayer Recordings\Vets_24_7_Series_4
and
get_iplayer --type=tv --pid=b08r69t1 --subdir -i | FindStr filename =>
filename: D:\Vangelis\iPlayer
Recordings\Vets_24_7_Series_4\Vets_24_7_Seri
es_4_-_4._Episode_4_b08r69t1_default.EXT
Obviously, someone with proper perl-fu can write
less crude code to deal with this bug; tested on
Windows only; am confident the maintainer
will address this in next release...
Best regards,
Vangelis.
___
get_iplayer mailing list
get_iplayer@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/get_iplayer
Re: News Quiz Extra pvr Download Failed
On Wed May 24 19:17:13 BST 2017, Alan Milewczyk wrote: On 24/05/2017 18:29, Des Watson wrote: (snip) without the underscores... Well spotted! LOL, they don't call him "Watson" for nothing, ehh? Regards (and keep your spirit high, those heinous terrorists won't prevail, ever!) Vangelis. ___ get_iplayer mailing list get_iplayer@lists.infradead.org http://lists.infradead.org/mailman/listinfo/get_iplayer
Re: An Art Lovers' Guide 1 Amsterdam
On Fri May 19 16:06:18 BST 2017, RS wrote: HLShd1 and HLShd2 seemed to have the same errors Good afternoon, Richard This is hardly any surprise, since hlshd1 and hlshd2 are in essence the same exact stream from the same CDN (Akamai), only difference is the protocol used (1= secure HTTP, "https", 2= plain HTTP, "http") Regards ___ get_iplayer mailing list get_iplayer@lists.infradead.org http://lists.infradead.org/mailman/listinfo/get_iplayer
Re: News Quiz Extra pvr Download Failed
On Thu May 18 23:27:02 BST 2017, Budge wrote: Let's see how it works. You should be fine, but let us know if otherwise... If you have any comments they are welcome. Actually I do, but my comment below lacks real substance (i.e. you can live with how things are already): refreshinclude = Radio 3,Radio 4 will only populate radio.cache with R3/R4/R4X offerings (that switch has --type=radio as a prerequisite, but you are fine as you have set: --type = tv,radio) refreshexcludegroups = local is a more generic option that excludes both TV & Radio "Local" groups; however, "Local" Radio groups (as well as every other station but R3/R4/R4X) have already been excluded by refreshinclude = Radio 3,Radio 4 My pedantic comment was just to make you realise how exactly those options work; I would have used refreshexcludegroupstv = local myself, but, as stated already, you're good as it is... Goodnight, V. ___ get_iplayer mailing list get_iplayer@lists.infradead.org http://lists.infradead.org/mailman/listinfo/get_iplayer
Re: News Quiz Extra pvr Download Failed
On Thu May 18 17:27:43 BST 2017, I wrote: via the --refresh-exclude-tv A correction to my oversight, the proper switch is: --refresh-exclude Apologies :-( Regards. ___ get_iplayer mailing list get_iplayer@lists.infradead.org http://lists.infradead.org/mailman/listinfo/get_iplayer
Re: News Quiz Extra pvr Download Failed
On Thu May 18 15:29:02 BST 2017, Budge wrote: get_iplayer --prefs-add --refresh-include="Radio 4","Radio 3" Is that the right syntax No; all you need to know is detailed here: https://github.com/get-iplayer/get_iplayer/wiki/documentation#filtering-channels-for-indexing Do remember that all regex in a command must be in double quotes, https://github.com/get-iplayer/get_iplayer/wiki/documentation#search-strings-as-regular-expressions On the command line, always quote search strings containing regular expressions. So what you should've conjured up is get_iplayer --prefs-add --refresh-include="Radio 3,Radio 4" are tv channels included by default or should I add tv explicitly? They are; you can always exclude individual TV stations or TV groups via the --refresh-exclude-tv and --refresh-exclude-groups-tv options; consult the documentation I referenced... ___ get_iplayer mailing list get_iplayer@lists.infradead.org http://lists.infradead.org/mailman/listinfo/get_iplayer
Re: News Quiz Extra pvr Download Failed
On Wed May 17 23:30:42 BST 2017, Budge wrote: pvrsearch = News_Quiz_Extra radiomode = better search0 = News_Quiz_Extra type = radio. Is there actually a fullstop (.) after radio? Was this PVR search created manually via an editor? I don't use the PVR often (ad-hoc downloader), but AFAIR GiP does not include by default "=" when storing PVR searches... Did my pvr download fail because I have type=radio rather than type=podcast? I think you are mixing two different connotations of "podcast" here; Podcast as programme type (hence --type=podcast) and podcast as an audio version of a radio show (hence --versions=podcast). Support for --type=podcast has been removed from GiP in v2.98 (it was already broken in 2.97, due to BBC changes made to the podcast feeds). However, support for the podcast version of a radio show (when that exists) is ongoing in 3.01. As Richard said, your show is to be found on http://www.bbc.co.uk/programmes/b08qgx61 get_iplayer --type=radio --pid=b08qgx61 -i | grep versions only finds the "original" version, the one RS fetched. On the show's page there's a link to http://www.bbc.co.uk/programmes/b010m2mj/episodes/downloads and from there to http://www.bbc.co.uk/programmes/p0530vlq which is the pid you used. In this case, get_iplayer --type=radio --pid=p0530vlq -i | grep versions only finds "versions: podcast" and that's the VERSION you manually recorded; FWIW, that got you an M4A file (AAC LC encode) with the same content as the MP3 file available at the link RS suggested: http://open.live.bbc.co.uk/mediaselector/5/redir/version/2.0/mediaset/audio-nondrm-download/proto/http/vpid/p0530tq6.mp3 (128kbpsCBR) Regards, Vangelis. ___ get_iplayer mailing list get_iplayer@lists.infradead.org http://lists.infradead.org/mailman/listinfo/get_iplayer
Legacy Low-Res tvmodes removed
"--info"-ing yesterday's --pid=b08qlg9p perl get_iplayer-301.pl --type=tv --pid=b08qlg9p -i | findstr hls one can see that only hlshd1,hlshd2,hlsvhigh1,hlsvhigh2 are now offered from the group of the legacy (non Video Factory, aka hvf*) AppleHLS tvmodes. Actually, both "1" variants are identical to their "2" counterparts and served from Akamai CDN, they only differ in the protocol used (1=HTTPS, 2= HTTP). So that leaves us with only 2 quality variants from the legacy AppleHLS streams: hlshd => 1280x720p @25FPS hlsvhigh => 832x468p @25FPS If you're still (or can use) previous versions of GiP that come with RTMP stream support, you'll find the same to be true for the (legacy) flash tvmodes. To re-enable detection of flash modes you'll have to apply the patch I posted in the second part of http://lists.infradead.org/pipermail/get_iplayer/2017-May/010720.html In GiP 2.99 (last version with flash support), perl get_iplayer-299.pl --type=tv --pid=b08qlg9p -i will reveal that now only flashhd1,flashvhigh1,flashvhigh2 modes are offered. flashhd => 1280x720p @25FPS flashvhigh => 832x468p @25FPS (1= Akamai CDN, 2= Limelight CDN). If it's anything to go by (obviously I haven't tested many PIDs), this displays a trend... V. ___ get_iplayer mailing list get_iplayer@lists.infradead.org http://lists.infradead.org/mailman/listinfo/get_iplayer
Re: Cache no-refresh and regexes
On Fri May 12 22:22:13 BST 2017, Mark Carroll wrote: I wonder what's the cleanest way to tell get_iplayer not to refresh the cache. ... Use the -e (--expiry) option with a huge value, e.g. the number of seconds included in a week or more, and set it in your user options; you do plan on manually updating at some point, don't you? Vangelis. ___ get_iplayer mailing list get_iplayer@lists.infradead.org http://lists.infradead.org/mailman/listinfo/get_iplayer
Re: BBC has added password controls to iPlayer
On Fri May 12 08:56:09 BST 2017, Chris Marriott wrote: No mention of it on the iPlayer Radio site at all. Hi Chris :-) Trying to play AOD on the iPlayer Radio site from an overseas IP does not, yet, generate a "Sign in" prompt - AIUI, this will also change in the near future... However, as you click the "Listen Now" button behind a UK IP, you should get: http://imgur.com/a/osfDK Don't you? Regards. ___ get_iplayer mailing list get_iplayer@lists.infradead.org http://lists.infradead.org/mailman/listinfo/get_iplayer
Re: daf* radiomodes broken by the BBC!
On Fri May 12 01:02:11 BST 2017, tellyaddict wrote: (daf* radiomodes broken by the BBC!) The feeds are still there in the mediaselector. Any ideas what needs to be changed to fix it? Well, I couldn't really sleep until I debugged this for myself, so there you go: In my GiP 2.97 copy, I restored daf* radiomodes by changing line (possibly mangled by mailer) $xml = main::request_url_retry( $ua, $media_stream_pc_prefix.$verpid.'/transferformat/dash?cb='.( sprintf "%05.0f", 9*rand(0) ), 3, undef, undef, 1, 1 ); to (also possibly mangled by mailer): $xml = main::request_url_retry( $ua, $media_stream_pc_prefix.$verpid.'/transferformat/dash', 3, undef, undef, 1, 1 ); Downloaded a dashmed radiomode successfully; I haven't tried this patch on GiP 3.01; since it also supports DASH video streams (dvf tvmodes), it may/may not break them; try and tell us... Flash modes seem to be vanishing as well in 2.99 and still just about available in 2.96. Again what's changed and what needs to be changed? Are you referring to flashaac radiomodes? In a similar fashion, I was able to restore flashaac radiomodes in GiP 2.97 (without, AFAICT, affecting flash tvmodes) by changing line (possibly mangled by mailer) $xml = main::request_url_retry( $ua, $media_stream_pc_prefix.$verpid.'/proto/rtmp?cb='.( sprintf "%05.0f", 9*rand(0) ), 3, undef, undef, 1, 1 ); to (also possibly mangled by mailer): $xml = main::request_url_retry( $ua, $media_stream_pc_prefix.$verpid.'/proto/rtmp', 3, undef, undef, 1, 1 ); However, the Akamai CDN (flashaac*2 mode) is misbehaving for me currently; don't know what to make of this... To conclude, if I was in the end able to find a crude solution to this issue after a 70min experimentation, having 0 coding skills, then it is highly probable you or anybody else could... Regards, Vangelis. ___ get_iplayer mailing list get_iplayer@lists.infradead.org http://lists.infradead.org/mailman/listinfo/get_iplayer
Re: daf* radiomodes broken by the BBC!
On Fri May 12 01:02:11 BST 2017, tellyaddict wrote:
Flash modes seem to be vanishing as well in 2.99
and still just about available in 2.96.
Having been away all day from home,
I only recently became aware
of the breakage I posted about :-(
But I was equally pd to discover that
not only the quick daf modes were gone,
flashaac modes in my GiP 2.97 copy
weren't picked up either... ;-{
Flashaac modes were very quick for me,
alas I tried GiP 2.96 but still no joy:
perl get_iplayer-296.pl --type=radio --pid=b08njqcb -i | FindStr modes =>
modes: default:
hafhigh1,hafhigh2,hafstd1,hafstd2,hafmed1,hafmed2,haflo
w1,haflow2,hlsaacstd1,hlsaaclow1
Haf* modes are VERY slow compared to either daf*/flash*,
while whereas the hlsaac* ones are much quicker, they may
be corrupted for many programmes :-(
I guess it's a safe bet to say we're in no luck getting
the flash modes back, as the maintainer has already
removed them in GiP 3.00+ and does not support
that deprecated code...
I'm finding it very frustrating at the moment
since Dinky has stopped publishing fixes
in the development version on Github
until the final version is actually released.
... Gone are those days; I can echo your feelings
completely... And he's not willing to divulge any
details in the forums either, so lose-lose situation
any way you look at it...
but what has actually changed?
(snip)
Again what's changed
Well, as you say both MPEG-DASH and RTMP
streams are both present inside the mediaselector API
URL for mediaset=pc, e.g. in the case of my mentioned
radio pid=b08njqcb => vpid=b08njqby =>
http://open.live.bbc.co.uk/mediaselector/5/select/version/2.0/mediaset/pc/vpid/b08njqby
I am not the most appropriate person to answer
your "what" questions, especially since it's very
late at night here... Obviously something has been changed
in the XML blocks there or in the href URLs
themselves and thus GiP can no longer parse
this data successfully because it's not in a form
expected by GiP; sadly, as I'm sure you know,
I'm not a coder or regex expert, so
can't provide solutions for you/us...
If it comes to that, DASH manifest files can be
downloaded with youtube-dl, while for RTMP
streams one can always construct manual rtmpdump
commands (TBH though, haven't done that in a
big while...).
I'm sorry to say this again, but what this "low-traffic" list
needs is for some serious perl coders to come forth and
share their knowledge in a more "liberal" fashion - yes,
the sole maintainer is highly commendable for his
unwanning efforts over the years, but there's a but...
Goodnight (and Lord Knows what else'll broken
tomorrow ;-( ).
___
get_iplayer mailing list
get_iplayer@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/get_iplayer
daf* radiomodes broken by the BBC!
This has been documented in Known Issue #2 over at the github wiki: https://github.com/get-iplayer/get_iplayer/wiki/issues#2-2017-05-11-dash-streams-unavailable-for-radio-programmes Do as advised... Awaiting eventual major breakages imminently ;-( ___ get_iplayer mailing list get_iplayer@lists.infradead.org http://lists.infradead.org/mailman/listinfo/get_iplayer
Re: Further Radio Modes Experience
On Thu May 11 12:25:58 BST 2017, RS wrote: A month or two ago someone else had a problem with a Linn player. His problem was that it would not play long M4A/AAC files. IIRC, that someone else was again @Budge, same person: On Tue May 9 09:49:15 BST 2017, Budge wrote: even though I am using their older firmware which solved my "long file" problem. Best regards, V. ___ get_iplayer mailing list get_iplayer@lists.infradead.org http://lists.infradead.org/mailman/listinfo/get_iplayer
FFmpeg-3.3-win32-static binary for XP/Vista+ users
As you may already know (or have found out the hard way),
the maintainer has ceased supporting Windows XP & Vista OSes
since GiP v3.00+; this is declared (in bold) in Forum rule #2:
https://squarepenguin.co.uk/forums/thread-655.html
and also included (in bold, too) in the "Windows Installation" wiki entry:
https://github.com/get-iplayer/get_iplayer/wiki/windows#windows-installation
Of course, both these OSes have been left orphan
by their vendor, Microsoft; XP SP3 was EOL'd April 2014
and Vista SP2 only a month ago, so noone can really
blame the maintainer for the decision taken...
It may also be (apart from lining himself up behind MS)
that he lacks those systems for testing purposes...
As things stand, the latest supported Windows installer, available at
https://github.com/get-iplayer/get_iplayer_win32/releases/latest ,
does not impose a minOS version check
(I'm not putting ideas to the dev :-), so it can be
installed on both XP and Vista.
The mini Strawbery Perl distribution provided by the installer
has also undergone some changes: an upgrade from Perl 5.24.0.1
to 5.24.1.1, plus some deprecated perl modules were removed
while new needed ones were added...
You can inspect recent changes to the win installer at:
https://github.com/get-iplayer/get_iplayer_win32/compare/2.99.0...3.01.0
AFAICT, the perl code itself in latest versions of
the scripts along with the provided miniPerl package
continue to be fully compatible with XP/Vista.
The one change that broke things for these OSes
was the move from the Zeranoe compile ffmpeg-3.0-win32
to the Zeranoe compile ffmpeg-3.2.4-win32:
-!define FFMPEGDIR "${UTILSDIR}\ffmpeg-3.0-win32-static"
+!define FFMPEGDIR "${UTILSDIR}\ffmpeg-3.2.4-win32-static"
The FFmpeg code itself has not stopped being
XP/Vista compatible, but the inclusion by Zeranoe
of some external libs that are either themselves by default
not compatible or have become incompatible at some
stage of their development is what broke recent
Zeranoe builds on XP/Vista.
E.g., --enable-libmfx breaks builds on XP, while
--enable-libx265 without the -DWINXP_SUPPORT=ON
flag breaks builds on both XP/Vista (and it's the reason
behind errors like:
https://squarepenguin.co.uk/forums/thread-1307.html ).
FFmpeg on GiP is used, as you may already know,
for remuxing (repackaging) downloaded fragmented
(dash) .mp4(/.m4a) and .ts files to the standard
MP4 container. Min version requirement for hvf modes
is 2.5, while for dvf modes is 3.0.
So, v3.0 included with previous installer
"get_iplayer-2.99.0.exe" is fine for the job
- this is to be removed from the repo
on May 30th, so grab it while you can...
You can extract the above NSIS installer with a utility
like 7-zip (without actually running it);
"ffmpeg.exe" inside "utils" folder is v3.0.
That version 3.0 originally came from the Zeranoe repo at:
https://ffmpeg.zeranoe.com/builds/win32/static/
HOWEVER, in another perfect demonstration of
Sod's Law, the repo was recently purged from any
content older than 07-Nov-2016, hence all currently
offered compiles won't run on XP/Vista.
Fortunately, VideoHelp site has an archived link of 3.0 at:
https://www.videohelp.com/software/ffmpeg/old-versions
(ffmpeg-3.0-win32-static.7z) that should become handy
when the above installer is withdrawn...
The general idea to restore XP/Vista compatibility in GiP 3.00+
is to replace provided 3.2.4 binary with the 3.0 one (or newer,
see below...). On XP/Vista 32bit, default GiP installation dir is
%PROGRAMFILES%\get_iplayer (or C:\Program Files\get_iplayer);
head over to utils dir, rename existing ffmpeg.exe (e.g to
ffmpeg.BAK) and place within your v3.0 copy of ffmpeg.exe
(I won't be held responsible if you mess up your system...).
Alternatively, if you feel unconfortable accessing
your "Program Files" dir, you can use the --ffmpeg
GiP option (and set it permanently in options) to
point the script to the path of your v3.0 ffmpeg.exe binary.
Although something more up-to-date is not strictly needed
by GiP, I have recently compiled a FFmpeg-3.3-win32-static
binary that is XP/Vista compatible and includes most
popular external libs (audio/video codecs, filters etc.).
It is a GPLv3 redistributable binary and I decided to share
it here with fellow GiP Windows users...
Together with source and licences, it's been uploaded to
http://www.datafilehost.com/d/3ddcd138
(Press gray DOWNLOAD button - you need 7-zip
or similar to decompress...).
This binary runs fine on XP+ Windows OSes. For
your other uses, run provided "ff-prompt.bat" and
type your ffmpeg commands there; for GiP inclusion,
binary (ffmpeg.exe) resides inside "bin" folder.
You should not need to use this binary on Win7+
(it does run fine there), but if you do you won't be
eligible for support in the Forums, since the maintainer
only supports the specific v3.2.4 binary supplied by
the Windows installer.
I hope someone finds the (lengthy) above of value...
Regards,
Vangelis.
Re: Further Radio Modes Experience
On Tue May 9 09:49:15 BST 2017, Budge wrote: but my Linn player will not play the file. A sense of deja vu here as I had similar problems back in January. (snip) Format profile : LC (snip) Bit rate : 320 kb/s (snip) Format profile : HE-AAC / LC (snip) Bit rate : 96.0 kb/s (snip) so it looks like a repeat of the Linn problem Hi Alastair! Is it warming up in Scotland? A very pleasant 27C currently here :-) Your "Linn" issue has been indeed discussed previously here in the list, often times by me... Your "problematic" firmware has no support for the High Efficiency (HE-AAC) encoding profile. Radiomodes with a bitrate <= 96 kbps use that, while the ones >=128 use the Linn compatible standard Low Complexity (LC) profile; actually, HE-AAC is LC too, better described as HE-AAC LC, as indicated by the MI log. You can consult the recently re-linked in the list modes tables to accommodate your radiomode settings: https://github.com/get-iplayer/get_iplayer/wiki/modes https://github.com/get-iplayer/get_iplayer/wiki/modesref https://github.com/get-iplayer/get_iplayer/wiki/modesref#radio-modes Best regards, Vangelis. ___ get_iplayer mailing list get_iplayer@lists.infradead.org http://lists.infradead.org/mailman/listinfo/get_iplayer
Re: Can't download with GiP 3.00
On Mon May 8 17:44:09 BST 2017, Alan Milewczyk wrote: Must admit, I was amused by the reference to a video profile as "outgoing" - wonder what the introverted profiles are like? ;-) LOL - Cheers, Alan! Best laugh of the day for me! I dug out the v2.99 get_iplayer.pl file and tried that but got an ffmpeg error - I couldn't be bothered decoding what the error meant ... Are you sure it wasn't an rtmpdump error? In GiP 3.00+, RTMP stream support (ergo flash modes) was removed, hence the same happened to the utility (RTMPdump) used to fetch those streams... Either way, you could've always posted that error here and hopefully a sympathetic soul (!) could have given it a look... Take the best of care, Vangelis. ___ get_iplayer mailing list get_iplayer@lists.infradead.org http://lists.infradead.org/mailman/listinfo/get_iplayer
Re: Programme in Series has `firstbcast' But no `available'.
On Sun May 7 22:07:00 BST 2017, J K.Eason wrote: If you use Textpad (for Windows), you can compare two files that you're viewing via Tools > Compare files... (F9). That brings up a third window with the differences between the two files. Very handy! Thanks! I'm using PSPad (freeware): http://www.pspad.com/en/ It has a similar, handy, functionality that allows for comparing an opened file to another one on disk; the two are then showed side-by-side in another tab, with the diffs clearly standing out with different colours, a la GitHub "split-diff" style: https://github.com/get-iplayer/get_iplayer/commit/0355c64ac2e0d8a6ee55af1db58c96b6a412be4d?diff=split Regards. ___ get_iplayer mailing list get_iplayer@lists.infradead.org http://lists.infradead.org/mailman/listinfo/get_iplayer
Re: Can't download with GiP 3.00
On Sun May 7 22:13:42 BST 2017, tellyaddict wrote: Can Apple devices cope with HVF streams as they are HLS-based? Don't own one, but they should; the mediaselector API URLs for mediaset=apple-ipad-hls do include Video Factory streams: pid=b04w0fyz => vpid=b04w0fqv http://open.live.bbc.co.uk/mediaselector/5/select/version/2.0/mediaset/apple-ipad-hls/vpid/b04w0fqv and look for service="stream-uk-mobile_streaming_concrete_combined_sd" (CDNs: Akamai, Limelight, Bidi) Is there a 720p 25fps HDS stream created by USP? No Have a nice week.. Vangelis. ___ get_iplayer mailing list get_iplayer@lists.infradead.org http://lists.infradead.org/mailman/listinfo/get_iplayer
Re: Can't download with GiP 3.00
On Sun May 7 23:16:20 BST 2017, RS wrote: That mode does not appear in the list in the blog. Well spotted! It is meant for ipad, tablets and other mobile devices... And that same blog describes the "1280x720 at 25fps" video profile as "outgoing", so that pretty much answers the question posed by @tellyaddict as to why "there isn't a 720p 25fps hvf option"... V. ___ get_iplayer mailing list get_iplayer@lists.infradead.org http://lists.infradead.org/mailman/listinfo/get_iplayer
Re: V3.00
On Sun May 7 21:59:36 BST 2017, Lucy Walker wrote: There seems to be something odd going on - the channel in question (Radio Somerset) seems to have updated some programmes in cache, but the show that I'm after, isn't in the PVR cache despite downloading happily through the CLI (http://www.bbc.co.uk/programmes/p050hczf). Hello again :-) That particular pid you're after is already inside my radio.cache, as demonstrated by: = C:\Program Files\get_iplayer>get_iplayer --type=radio pid:p050hczf get_iplayer 3.01-windows.0, Copyright (C) 2008-2010 Phil Lewis This program comes with ABSOLUTELY NO WARRANTY; for details use --warranty. This is free software, and you are welcome to redistribute it under certain conditions; use --conditions for details. NOTE: A UK TV licence is required to legally access BBC iPlayer TV content Matches: 18351: Geoff Barker's Rock 'n' Roll Party - Sonny Curtis, BBC Somerset, p050hcz f INFO: 1 Matching Programmes = Are you sure you did perform a radio.cache rebuild, too? Won't hurt to retry: get_iplayer --type=radio -f --refresh-limit-radio=30 --force --index-maxconn=10 optionally followed by get_iplayer --type=radio -f --refresh-future --force Other than that, I fear I'm out of more ideas... Cheers ___ get_iplayer mailing list get_iplayer@lists.infradead.org http://lists.infradead.org/mailman/listinfo/get_iplayer
Re: V3.00
On Sun May 7 16:39:44 BST 2017, Lucy Walker wrote:
I've noticed that certain stations (R2 and Local Radio)
no longer seem to be generating cache updates
Hi Lucy :-)
GiP 3.01 has recently been released, so first start
by updating to that:
https://squarepenguin.co.uk/forums/thread-1370.html
https://squarepenguin.co.uk/forums/thread-1371.html
Then have a read of
https://github.com/get-iplayer/get_iplayer/wiki/documentation#filtering-channels-for-indexing
BBC Radio 2 is being happily indexed here,
but remember radio cache refresh must be
performed manually once a week (unless you
do what is described in
https://github.com/get-iplayer/get_iplayer/wiki/documentation#programme-types
)
As for Local Radio, I'm not quite sure yet, but
it shouldn't be excluded from radio.cache;
I know Local TV is because of
https://github.com/get-iplayer/get_iplayer/commit/3522611
but in that same commit
# $opt->{refreshexcludegroupsradio} = "local";
is commented-out.
Please check your options file in case
you have excluded R2 and/or Local Radio
from radio.cache refreshes...
(running get_iplayer --type=radio "Pick of the Pops"
finds here latest episode broadcast yesterday:
Matches:
23896: Pick of the Pops - 1971 and 1983, BBC Radio 2, b08lg5f6
23897: Pick of the Pops - 1961 & 1974, BBC Radio 2, b08lzmsc
23898: Pick of the Pops - 1967 & 1980, BBC Radio 2, b08n0x9y
23899: Pick of the Pops - 1970 & 1986, BBC Radio 2, b08nj365
23900: Pick of the Pops - 1960 & 1984, BBC Radio 2, b08p4m6j
INFO: 5 Matching Programmes)
Regards.
___
get_iplayer mailing list
get_iplayer@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/get_iplayer
Re: Can't download with GiP 3.00
On Sun May 7 12:56:29 BST 2017, tellyaddict wrote: I really don't know why there isn't a 720p 25fps hvf option though. Elsewhere, tellyaddict wrote: HVF modes are processed by the BBC Actually, all DVF/HVF/DAF/HAF modes (plus AdobeHDS streams for TV/Radio that GiP does not support) are delegated to "Unified Streaming Platform", a BBC partner: http://www.unified-streaming.com/news/bbc-extends-contract-unified-streaming USP do not produce any 720p25fps encodes, I guess because the BBC do not ask them to... When the RTMP streams are first terminated (already announced), to be, no doubt, followed by the "legacy" AppleHLS streams (HLS modes), there'll be no way of getting 720p25fps... V. ___ get_iplayer mailing list get_iplayer@lists.infradead.org http://lists.infradead.org/mailman/listinfo/get_iplayer
Re: Programme in Series has `firstbcast' But no `available'.
On Sun May 7 11:32:54 BST 2017, Ralph Corderoy wrote: I hope it didn't come across that I was complaining. Not in the slightest :-) (I fear we're now getting OT but since you asked: ) Are you aware of the patch(1) program? I am aware of both patch & diff programs, however these are not natively available on Windows (maybe Win10 bash shell?); forks do exist: http://gnuwin32.sourceforge.net/packages/patch.htm http://gnuwin32.sourceforge.net/packages/diffutils.htm When it comes to few custom changes made, I usually manually/visually compare B to A with a text editor and note down the patched code. Then I load A2 and try to manually apply patches there, too; line numbers may have changed, but "search" facility will be able to point to the code segment I need re-patch. For many custom changes made, I do use diff & patch utilities that come with my MSYS2/MinGW32 compiler already installed in an external HDD; this is a Unix emulation environment but on Windows, I occasionally use it to compile open-source apps like FFmpeg... Best regards, Vangelis. ___ get_iplayer mailing list get_iplayer@lists.infradead.org http://lists.infradead.org/mailman/listinfo/get_iplayer
Re: [ANN] get_iplayer v3.01 released
On Sun May 7 14:35:24 BST 2017, Jim web wrote: always deals with this question in terms of (snip) and speed of fetching? GiP has never taken (and never will) speed of fetching into account - it's something out of its control... Somewhat related is the recent addition #13 to the Support Forums Rules: https://squarepenguin.co.uk/forums/thread-655.html Please also read https://github.com/get-iplayer/get_iplayer/wiki/release300#bandwidth-throttling-by-cdns Download speeds is a subject that, if memory serves right, has been discussed quite extensively here in this list previously... It depends on quite a lot of random factors (including the distance between you and the closest CDN node to your actual physical location), so what you must do is experiment with various (tv|radio)modes and between the available CDN modes (1, 2, etc.) for the same mode, and in the same time of day your downloads usually take place, to hopefully establish a pattern for your setup. Then stick to those which are most reliable/quick for your setup... I'll speak for Radio here (as 95% of my fetches are radio), unlike what has been reported by many, flashaac radiomodes (GiP 2.99 and earlier) were quite speedy for me, so my speed of fetching list looks like: hlsaac > flashaac >= daf >> haf The general consensus is that haf modes are quite slow... And my experiments have shown that usually Akamai > Limelight, but not a big difference... You can use get_iplayer -i --pid= --verbose to see what is what... Regards ___ get_iplayer mailing list get_iplayer@lists.infradead.org http://lists.infradead.org/mailman/listinfo/get_iplayer
Re: GiP v3.01 --cache-rebuild failed at line 18
On Sat May 6 18:48:58 BST 2017, CJB wrote: I raised the same issue after installing v 3.00 - I got no response. Hello Chris - as you say, it's the same error as in http://lists.infradead.org/pipermail/get_iplayer/2017-May/010586.html The Strawberry Perl mini-distribution (probably the Mojo module that enables concurrent indexing) that is installed by the GiP 3.00+ windows installer needs to open a temporary file (mojo.tmp) inside your %TEMP% directory, but fails for permissions related reasons (???). For whomever can understand it, line 18 of "Mojo/Asset/Memory.pm" reads my $file = Mojo::Asset::File->new; Perhaps your %TEMP% dir is inaccessible by Perl; Have you tried 1. Exit all running applications 2. Clear the content of %TEMP% (CCleaner or manual deletion) 3. Reboot machine Also, re-reading https://github.com/get-iplayer/get_iplayer/wiki/windows#windows-installation might help. There's no need to install as Admin. Since noone else reported similar issues either on this list or the Forums, am afraid I have nothing concrete to go by and makes me think it's an issue limited to your setup ;-( I have now had to delete all old cache files, and start from scratch. Did that work? Concurrent or Sequential indexing? ___ get_iplayer mailing list get_iplayer@lists.infradead.org http://lists.infradead.org/mailman/listinfo/get_iplayer
Re: Can't download with GiP 3.00
On Fri May 4 02:12:05 BST 2017, I wrote:
FWIW, flashhd was the only way I could get
720p25 for either pid=p04w8y7v or p0510g83
...Most sadly, I have come across at least one PID
(p050r6v9) for which both hlshd & flashhd are corrupted!
flashhd (in GiP 2.97) will constantly only fetch
a 10.4 MB partial.flv file before RTMPdump
starts choking...
No 720p25 version for this can be downloaded,
apparently ;-{ (I had to settle for 540p25).
Regards.
___
get_iplayer mailing list
get_iplayer@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/get_iplayer
Re: Programme in Series has `firstbcast' But no `available'.
On Sat May 6 12:59:02 BST 2017, Ralph Corderoy wrote: This has been very helpful, thanks. ... By the looks of it, you are a true Perl wizard, Ralph, so I dare say you'd have ultimately found the exit of the labyrinth without any of my help :-) As I've stated numerous times in the past, sadly I can't code (in Perl or other language), so my MO when troubleshooting issues for myself (... and others) is to first consult online documentation (wiki etc.), then reread --longhelp (Windows here, so no manpages), generate and study --(verbose|debug) GiP logs and carry out various "experiments"... Sometimes I have to also make some "educated guesswork" (or rather uneducated, if it turns out to be wrong...) to justify some of my findings, but I feel I have never posted blatant untruths :-) ! I have never claimed to be infallible and have welcome anybody who have spotted something out of place in my posts to say so, for it to be fixed... This list is being archived and I sense many persons do seek advice through the archive, so it'd be best errors are corrected... It is only rarely that I dig deep inside the perl code, having Google (other SEs available) and online perl tutorials as help... So it's my turn to thank you for the heads-up on current behaviour of "--refresh --refresh-future" which had eluded me... Your perl knowledge is very needed here in the list, so do stay around... BTW, GiP 3.01 has just been released, with changes to some topics discussed in this thread (this means more reading for me and porting customisations to the new script from 3.00.0 :-( ). All the best, Vangelis. ___ get_iplayer mailing list get_iplayer@lists.infradead.org http://lists.infradead.org/mailman/listinfo/get_iplayer
Re: Programme in Series has `firstbcast' But no `available'.
On Fri May 5 18:16:01 BST 2017, Ralph Corderoy wrote: Maybe your tv.cache has gaps. It seems to have. I have done ./get_iplayer --quiet --refresh --refresh-future "--refresh --refresh-future" won't help you in this case; your tv.cache gap for pid=b08nyc9z is for a broadcast date more than a week in the past; "--refresh --refresh-future" will index tv content for current and next calendar weeks and since pid=b08nyc9z won't be repeated in the future (as already told in my previous message), it won't be added to the cache; do as told to regenerate your past 15 (or more) days worth of the tv.cache file... If you remove it from the cache by deleting that line, does --info then obtain the availabledate from what's now out on the Internet to retrieve? I still fail to grasp the importance it holds for you to have that "available" entry in --info for that particular episode of HUTH, but then again I'm getting older (and grumpier...). If it's purely academic curiosity you want to satisfy, do read on... When you -i a specific PID, most of the results are read locally from the (tv|radio).cache file, that is if the PID is already indexed in the cache. This is done for "economy" reasons, so as not to redownload cached content. Other -i results, (like the long description & modes) which do not appear in the cache, are fetched from online metadata feeds, currently http://www.bbc.co.uk/programmes/b08pm8cl.json "available" is actually the 4th column of the tv.cache file, so if a (TV) PID is already indexed, -i --pid= will include the "available" entry, otherwise not. The "available" field for an indexed PID was originally populated by fetching the value of first_broadcast_date for that PID (from the feeds). If you compare "available:" vs "firstbcast:" values for that PID, you'll see they are identical When a PID is not indexed, you'll only get the "firstbcast:" -i result, not the "available:" one. I have conducted, as requested, experiments which you are free to try yourself. pid=b08pm8cl was inside my tv.cache (you can try with a different one already in your cache); get_iplayer --pid=b08pm8cl -i => - INFO: pid found in cache Matches: 3194: Homes Under the Hammer: Series 21 - Episode 9, BBC One, b08pm8cl INFO: File name prefix = Homes_Under_the_Hammer_Series_21_-_9._Episode_9_b08pm8c l_original available: 2017-05-02T10:00:00+01:00 brand: Homes Under the Hammer (snip) firstbcast: 2017-05-02T10:00:00+01:00 - tv.cache was manually edited to remove b08pm8cl entry; get_iplayer --pid=b08pm8cl -i => - INFO: Trying pid: b08pm8cl using type: tv INFO: Trying to download PID using type tv INFO: pid not found in tv cache Matches: INFO: File name prefix = Homes_Under_the_Hammer_Series_21_-_9._Episode_9_b08pm8c l_original brand: Homes Under the Hammer (snip) firstbcast: 2017-05-02T10:00:00+01:00 - To conclude, pid=b08nyc9z did not come up with an "available:" enrty simply because it was missing from your tv.cache. Reconstruct your past cache and retry... If it's only that pid you care about, here's the relevant line from my own cache: (may get mangled by mailer ;-( ) ) 3193|tv|Homes Under the Hammer: Series 21|b08nyc9z|2017-04-27T10:00:00+01:00|1495875600|Episode 8|21|8|default|3600|Featuring properties in Langton Green in Kent, Stoke-on-Trent and Northwood in Middlesex.|BBC One|||1493933022||http://www.bbc.co.uk/programmes/b08nyc9z| (you should change index number...) Have a good night, Vangelis. ___ get_iplayer mailing list get_iplayer@lists.infradead.org http://lists.infradead.org/mailman/listinfo/get_iplayer
