Re: [mythtv-users] tv_grab_au script from this list. Anyone elsehadproblems today?
> "Michael" == Michael Cheshire (Mailing Lists) <[EMAIL PROTECTED]> writes: Michael> When I tried d1, it didnt give episode information for Michael> example, so if you record a tv servies then myth doesnt Michael> know enough about the episode to tell if it has recorded Michael> that particular episode before. For some shows d1 does have episodes. When I stopped using it though, mythtv was treating every new episode of a show as a completely different TV show. I suspect it was appending the episode to the title. This meant every recording for the shows effected had to be manually configured. Also around this time, all Dr Who episodes had the same title, despite the fact the indicated episode ended the week before. -- Brian May <[EMAIL PROTECTED]> ___ mythtv-users mailing list mythtv-users@mythtv.org http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-users
Re: [mythtv-users] tv_grab_au script from this list. Anyone elsehadproblems today?
Dont know if anyone else has answered this but: The msn grabber gets MUCH more data than the d1 does. When I tried d1, it didnt give episode information for example, so if you record a tv servies then myth doesnt know enough about the episode to tell if it has recorded that particular episode before. M. - Original Message - From: "Brad" <[EMAIL PROTECTED]> To: "Discussion about mythtv" Sent: Tuesday, July 26, 2005 3:13 PM Subject: Re: [mythtv-users] tv_grab_au script from this list. Anyone elsehadproblems today? Is the ninmsn grabber better than the d1.com.au ? If so do you know where I could download it from. Also a clue on how to install if time permits. Thanks Brad On 7/25/05, Matthew Carle <[EMAIL PROTECTED]> wrote: I havn't noticed ay error messages (but I havn't looked too hard). I am however missing detailed information for Saturday and Sunday (I just have the program titles). A visit to the ninemsn page shows that it is ninemsn that is missing the information - the grabber isn't the problem. Matthew. > -Original Message- > From: [EMAIL PROTECTED] > [mailto:[EMAIL PROTECTED] On Behalf Of David Whyte > Sent: Monday, 25 July 2005 8:55 PM > To: Discussion about mythtv > Subject: [mythtv-users] tv_grab_au script from this list. > Anyone else hadproblems today? > > Received the following error in mythweb... > > Last mythfilldatabase run started on 2005-07-25 18:54 and ended on > 2005-07-25 18:58. mythfilldatabase ran, but did not insert > any new data into the Guide. This can indicate a potential > grabber failure. > There's guide data until 2005-07-31 08:00 (6 days). > > When I run it from the console I get... > > /var/local/tv_grab_au/31072005/12396398.html is too short, > removing and trying again > /var/local/tv_grab_au/31072005/12409880.html is too short, > removing and trying again > /var/local/tv_grab_au/31072005/12394350.html is too short, > removing and trying again Giving up on > /var/local/tv_grab_au/31072005/12409886.html > /var/local/tv_grab_au/31072005/12396398.html is too short, > removing and trying again > /var/local/tv_grab_au/31072005/12409880.html is too short, > removing and trying again > /var/local/tv_grab_au/31072005/12394350.html is too short, > removing and trying again Giving up on > /var/local/tv_grab_au/31072005/12396398.html > /var/local/tv_grab_au/31072005/12409880.html is too short, > removing and trying again > /var/local/tv_grab_au/31072005/12394350.html is too short, > removing and trying again > /var/local/tv_grab_au/31072005/12409880.html is too short, > removing and trying again Giving up on > /var/local/tv_grab_au/31072005/12394350.html > /var/local/tv_grab_au/31072005/12409880.html is too short, > removing and trying again > /var/local/tv_grab_au/31072005/12409880.html is too short, > removing and trying again > /var/local/tv_grab_au/31072005/12409880.html is too short, > removing and trying again Giving up on > /var/local/tv_grab_au/31072005/12409880.html > writing file > done > > I wonder if the format of the HTML pages has changed. This > is from NineMSN. > > Cheers, > Whytey > -- > -- > GMAIL is 'da bomb babyYEAH > > I have GMail invites, if you want one, email me direct. > ___ > mythtv-users mailing list > mythtv-users@mythtv.org > http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-users > ___ mythtv-users mailing list mythtv-users@mythtv.org http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-users ___ mythtv-users mailing list mythtv-users@mythtv.org http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-users ___ mythtv-users mailing list mythtv-users@mythtv.org http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-users
Re: [mythtv-users] tv_grab_au script from this list. Anyone elsehadproblems today?
F'Nog wrote: Berry, David wrote: Same here (Canberra) How do we fix it? - change the script? I had a poke around and the changes seem to be they changed the url from /closeup/ to /cu/ and changed a couple of the attributes that are searched for the find the programs id to find its close up details. The url changes are pretty simple and only exist in a couple of places however I don't know anywhere enough, well no, perl to change the regular expressions to filter out the right program info. Thanks for the hints... I'll try and sort out a fix shortly... Michael. ___ mythtv-users mailing list mythtv-users@mythtv.org http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-users
Re: [mythtv-users] tv_grab_au script from this list. Anyone elsehadproblems today?
> Does anyone have an updated NineMSN script for the version at
> http://immir.com/tv_grab_au. This is the one written by Michael
> 'Immir' Smith which is a bit easier to config (has a config file) than
> the Damon Searle version. It would be great if someone did because the
> changes described so far in this thread all apply to a different
> NineMSN script to the one some of us have.
>
> In the meantime I'm going to my OzTiVo backup. I've now modified my
> grabber shell script so I just change one line and it calls different
> grabbers depending on which one is up or down.
I had a go at changing the "Michael 'Immir' Smith" version myself. I
only changed 1 line :
my $DETAILS= "http://tvguide.ninemsn.com.au/closeup/default.asp";;
TO
my $DETAILS= "http://tvguide.ninemsn.com.au/cu/default.asp";;
I've run it and it seems to be working. Seems to good to be true that
it only required a 1 line change! Am I missing something? I thought
the "pid" stuff would all need changing but it seems to work with no
changes to any of that.
It's attached if you want it.
Regards,
Phill
#!/usr/bin/perl -w
eval 'exec /usr/bin/perl -w -S $0 ${1+"$@"}'
if 0; # not running under some shell
# --- new NMSN Australian TV grabber by Michael 'Immir' Smith...
# --- $Id: tv_grab_au,v 1.28 2005/08/03 23:27:50 michael Exp $
#
# A current version of this script should usually be available here:
#
#
#
# pod documentation coming later...
#
# in brief: --configure configure the grabber
# --config-file use specified config file
# --list-channels show subscribed channels and xmltvids
# --slowdownload a details page for every show
# --fastopposite of --slow (to override config)
# --days days to grab
# --outputxml output file
# --ignore-cacheignore cached information
#
# When run with mythfilldatabase, the config file may end up as
# ~/.mythtv/tv_grab_au.xmltv and this is where xmltvids for channels can
# be modified.
#
# Downloading of details pages can be controlled by the --slow option
# (possibly set by default during configuration) and regular expressions
# contained in the following two files (one regexp per line):
#
# ~/.tv_grab_au-detailedcontains a list of regular expressions matching
# "chanid:titles" of shows for which we want details
# e.g.: Doctor Who
# Simpsons
# abc.gov.au:.*Chef
#
# ~/.tv_grab_au-undetailed regular expressions matching "chanid:titles" to
# skip details downloads
# e.g.: News
# news.sbs.com.au
#
# NB: a match against the detailed list overrides any undetailed matches.
# When changing these files, use the --ignore-cache option to ensure
# all new details are downloaded.
#
# TZ change added as suggested by Greg Boundy.
use strict;
use Getopt::Long;
use LWP::UserAgent;
use Date::Manip;
use File::Path;
use File::Basename;
use Data::Dumper;
use HTML::TreeBuilder;
use XMLTV;
use XMLTV::Ask;
use XMLTV::ProgressBar;
use XMLTV::Config_file;
# --- global parameters/constants
my $conf; # hashref for configuration parameters
my $channels; # ref to hash of subscribed channel names to xmltvids
my %chanid; # mapping from lowercase channel name to xmltv channel id
my $lang = "en";
my $spoofMSID = 1; # spoof random MSIDs to avoid redirects?
my $output_dir = "/var/local/tv_grab_au";
my $cache_file = "$output_dir/cached.pl";
my $want_details_file = "$ENV{HOME}/.tv_grab_au-detailed";
my $skip_details_file = "$ENV{HOME}/.tv_grab_au-undetailed";
my (@want_details, @skip_details);
my $count_guide = 0; # global counter for reporting
my $count_detail = 0; # global counter for reporting
# --- various NMSN site URLs
my $NMSN = "http://tvguide.ninemsn.com.au";;
my $GUIDE = "http://tvguide.ninemsn.com.au/guide";;
my $DETAILS= "http://tvguide.ninemsn.com.au/cu/default.asp";;
my $TVTODAY= "http://tvguide.ninemsn.com.au/todaytv/default.asp";;
my ($Revision) = '$Revision: 1.27 $' =~ /Revision:\s*(\S+)/;
my $debug = 0;
$Data::Dumper::Sortkeys = 1;
$Data::Dumper::Indent = 1;
# --- Command-line options
my ($opt_configfile, $opt_configure, $opt_list_channels);
my $opt_output = "$output_dir/guide.xml";
my $opt_gui = 1;
my $opt_slow = 0;
my $opt_fast = 0;
my $opt_days = 5;
my $opt_skipcache = 0;
GetOptions( 'days=i' => \$opt_days,
'list-channels' => \$opt_list_channels,
'output=s'=> \$opt_output,
'config-file=s' => \$opt_configfile,
'configure' => \$opt_configure,
'gui=s' => \$opt_gui,
'slow'=> \$opt_slow,
'fast'=> \$opt_fast,
Re: [mythtv-users] tv_grab_au script from this list. Anyone elsehadproblems today?
thanx for that I have mod'd the script so it gets abc2 and sbs news for sydney and attached it thanx so much guys :) regards Mary -Original Message- From: Ian Ward <[EMAIL PROTECTED]> To: David Collett <[EMAIL PROTECTED]> Cc: Discussion about mythtv Date: Wed, 03 Aug 2005 18:48:26 +1000 Subject: Re: [mythtv-users] tv_grab_au script from this list. Anyone elsehadproblems today? > David Collett wrote: > > >I have attached by tv_grab_au with fixes for this new problem. It > >seems to be just a few urls have changed very slightly. Since my > >version might be different everyone elses anyway (there were quite a > >few iterations floating around) here is a diff which shows the > >necessary changes: > > > > > > > Thanks David for attaching your version of the ninemsn grabber. > The script has been designed to be interchangeable with the d1.com.au > grabber ( it outputs the same XMLTV channel ID's ) > It looks the goods to me because it allows me to switch between > d1.com.au and ninemsn without modifying any myth XMLTV channel ID's. I > ran it and took a look at the output, the information definately looks > more comprehensive than the d1.com.au info. > > David, perhaps next time you change it you could give yourself a wrap > in > the changelog. It will help us all keep track of the different > verisons > of the thing. > > I was previously looking to change to the ninemsn grabber available > from > here: http://immir.com/tv_grab_au > The current incarnation though looks like it suffers the same URL > glitch > ( closeup needs to change to cu ), > > SNIP> > # --- various NMSN site URLs > my $NMSN = "http://tvguide.ninemsn.com.au";; > my $GUIDE = "http://tvguide.ninemsn.com.au/guide";; > my $DETAILS= "http://tvguide.ninemsn.com.au/closeup/default.asp";; > my $TVTODAY= "http://tvguide.ninemsn.com.au/todaytv/default.asp";; > > Thanks again, > I'll give it a try in the "play" mythtv I have installed on a partition > on my PC > (The family get's upset when I break the "production" system) > Ian > > ___ > mythtv-users mailing list > mythtv-users@mythtv.org > http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-users tv_grab_au.pl Description: Perl program ___ mythtv-users mailing list mythtv-users@mythtv.org http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-users
Re: [mythtv-users] tv_grab_au script from this list. Anyone elsehadproblems today?
David Collett wrote: I have attached by tv_grab_au with fixes for this new problem. It seems to be just a few urls have changed very slightly. Since my version might be different everyone elses anyway (there were quite a few iterations floating around) here is a diff which shows the necessary changes: Thanks David for attaching your version of the ninemsn grabber. The script has been designed to be interchangeable with the d1.com.au grabber ( it outputs the same XMLTV channel ID's ) It looks the goods to me because it allows me to switch between d1.com.au and ninemsn without modifying any myth XMLTV channel ID's. I ran it and took a look at the output, the information definately looks more comprehensive than the d1.com.au info. David, perhaps next time you change it you could give yourself a wrap in the changelog. It will help us all keep track of the different verisons of the thing. I was previously looking to change to the ninemsn grabber available from here: http://immir.com/tv_grab_au The current incarnation though looks like it suffers the same URL glitch ( closeup needs to change to cu ), SNIP> # --- various NMSN site URLs my $NMSN = "http://tvguide.ninemsn.com.au";; my $GUIDE = "http://tvguide.ninemsn.com.au/guide";; my $DETAILS= "http://tvguide.ninemsn.com.au/closeup/default.asp";; my $TVTODAY= "http://tvguide.ninemsn.com.au/todaytv/default.asp";; I'll give it a try in the "play" mythtv I have installed on a partition on my PC (The family get's upset when I break the "production" system) Ian ___ mythtv-users mailing list mythtv-users@mythtv.org http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-users
Re: [mythtv-users] tv_grab_au script from this list. Anyone elsehadproblems today?
On 02/08/2005, at 10:36 PM, Matthew Phillips wrote: On 02/08/2005, at 9:12 PM, David Collett wrote: Hi, To all of those who have never actually tried the MSN grabber. It really is great. Very complete, consistent and reliable. The data comes from HWW who also supply to most other online tv guides (yahoo, seven, yourtv, etc). It is therefore essentially the definitive source of oz guide data. It doesnt get any better than that. Seconded. The MSN data is top notch and worth the occasional pain. The grabber script is also fairly well-written, although a few more sanity checks to better detect failures due to site changes would be good. That said, if IceTV could guarantee this level of quality and reduce price just a little, I'd probably switch just to avoid the occasional evening perl debugging session and possibility of NineMSN declaring war on mythtv :/ Matt. I agree and I'd also like to add that I'd probably switch to IceTV also if they added foxtel/optus channels. Maybe if I have some time I might go over the script and add more comments (it took me a little while to figure out what it was trying to do) and hopefully make it more capable of handling slight changes to the html. It won't be very pretty though, I know just enough perl to get by (as you could see from my previous post) :) ___ mythtv-users mailing list mythtv-users@mythtv.org http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-users
Re: [mythtv-users] tv_grab_au script from this list. Anyone elsehadproblems today?
Does anyone have an updated NineMSN script for the version at http://immir.com/tv_grab_au. This is the one written by Michael 'Immir' Smith which is a bit easier to config (has a config file) than the Damon Searle version. It would be great if someone did because the changes described so far in this thread all apply to a different NineMSN script to the one some of us have. In the meantime I'm going to my OzTiVo backup. I've now modified my grabber shell script so I just change one line and it calls different grabbers depending on which one is up or down. Regards, Phill ___ mythtv-users mailing list mythtv-users@mythtv.org http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-users
Re: [mythtv-users] tv_grab_au script from this list. Anyone elsehadproblems today?
On 02/08/2005, at 9:12 PM, David Collett wrote: Hi, To all of those who have never actually tried the MSN grabber. It really is great. Very complete, consistent and reliable. The data comes from HWW who also supply to most other online tv guides (yahoo, seven, yourtv, etc). It is therefore essentially the definitive source of oz guide data. It doesnt get any better than that. Seconded. The MSN data is top notch and worth the occasional pain. The grabber script is also fairly well-written, although a few more sanity checks to better detect failures due to site changes would be good. That said, if IceTV could guarantee this level of quality and reduce price just a little, I'd probably switch just to avoid the occasional evening perl debugging session and possibility of NineMSN declaring war on mythtv :/ Matt. ___ mythtv-users mailing list mythtv-users@mythtv.org http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-users
Re: [mythtv-users] tv_grab_au script from this list. Anyone elsehadproblems today?
> My solution ended up being minus the "$link =~ s/&.+//g; #&" and the > last two "$pid =~ s/(\d*).*/$1/;" lines. Am I missing something? Nah, dont think so. The point was that the pid ends with either an '&' or a ' '. However you split it up it doesnt really matter as long as you end up with the raw number in the end. I'm not even sure all the lines in my patch are necessary, I wrote them one at a time until it worked, and didnt bother to check over it afterwards :) I think the script could be made much more resilient to small changes like this. I mean, the variable name didn't even change, so if the parser tokenised stuff in a smarter (html aware) way in the first place this would not have broken at all. (though the closeup => cu thing would still have killed it). It would have been easier to fix if it was better documented and/or written in a real language like python :) Dave ___ mythtv-users mailing list mythtv-users@mythtv.org http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-users
Re: [mythtv-users] tv_grab_au script from this list. Anyone elsehadproblems today?
Nick Tan wrote:
Berry, David wrote:
Same here (Canberra)
How do we fix it? - change the script?
I had a poke around and the changes seem to be they changed the url from
/closeup/ to /cu/ and changed a couple of the attributes that are
searched for the find the programs id to find its close up details. The
url changes are pretty simple and only exist in a couple of places
however I don't know anywhere enough, well no, perl to change the
regular expressions to filter out the right program info.
It seems it's either fixing this all the time or going back to d1 (I
only ever got program data for SBS News here in Canberra, never played
with it much) or paying $150 a year with IceTV.
I think I've fixed it (it works with my setup anyways)
As you've said they've changed /closeup/ to /cu/ but they've also changed
how the pids are displayed.
Changes at around line 350 of the script:
if ($link =~ /cu\/default.asp/)
{
my $rowspan = $link;
$rowspan =~ s/.+rowspan=//g;
$rowspan =~ s/ .+//g;
my $name = $link;
#$name =~ s/.+target=new>(|)//g;
$name =~ s/.+target=_new class=tv>//g;
$name =~ s/<\/a>.+//g;
if ($name =~ /\]/) {
my @nameline=split/\]/,$name;
$name = $nameline[1];
$name =~ s/^ //g;
}
my $linktemp = $link;
$link =~ s/.+pid=//g;
$link =~ s/".+//g; #"
my @linksplit=split/&/,$link;
#$link = $linksplit[0];
my @linksplit2=split/ /,$linksplit[0];
$link = $linksplit2[0];
chomp($link);
chomp($rowspan);
chomp($name);
also change the fetch_details subroutine to be:
sub fetch_details
{
my $datepid=$datepids->dequeue;
my @datepidl=split /-/, $datepid;
my $date = $datepidl[0];
my $pidtemp = $datepidl[1];
my @pidsplit=split /&/, $pidtemp;
my $pid = $pidsplit[0];
while (($date!=0) and ($pid!=0))
{
my $guide_dir = $cache_dir . "/" . $date;
mkpath ($guide_dir);
my $url = "" . $pid;
my $details_file = $guide_dir . "/" . $pid . ".html";
for (my $retry=0; is_error(getstore($url, $details_file))
&& ($retry<$retrys); $retry++)
{
sleep($seconds_before_retry);
}
$datepid=$datepids->dequeue;
@datepidl=split /-/, $datepid;
$date = $datepidl[0];
# $pid = $datepidl[1];
my $pidtemp = $datepidl[1];
my @pidsplit=split /&/, $pidtemp;
my $pid = $pidsplit[0];
}
}
Like I said, this works for me, YMMV. But it should be a good starting
point for anyone who wants to clean it up a bit :)
Change
my @linksplit=split/&/,$link;
#$link = $linksplit[0];
my @linksplit2=split/ /,$linksplit[0];
$link = $linksplit2[0];
to
my @linksplit=split/&/,$link;
if ($#linksplit <= 0) {
@linksplit=split/ /,$link;
}
if ($#linksplit > 0) {
my @linksplit2=split/ / ,$linksplit[0];
$link=$linksplit2[0];
}
Otherwise changed_guide will fail when running the script and it's
checking currently entered program info. It ends up with
of &x= for some reason I'm
sure will be clear in the morning. Have fun :D
___
mythtv-users mailing list
mythtv-users@mythtv.org
http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-users
Re: [mythtv-users] tv_grab_au script from this list. Anyone elsehadproblems today?
On 02/08/2005, at 8:50 PM, David Collett wrote: I have attached by tv_grab_au with fixes for this new problem. It seems to be just a few urls have changed very slightly. Since my version might be different everyone elses anyway (there were quite a few iterations floating around) here is a diff which shows the necessary changes: Beat me to it. Now my head hurts *and* I don't get to save the day ;) My solution ended up being minus the "$link =~ s/&.+//g; #&" and the last two "$pid =~ s/(\d*).*/$1/;" lines. Am I missing something? Matt. ___ mythtv-users mailing list mythtv-users@mythtv.org http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-users
Re: [mythtv-users] tv_grab_au script from this list. Anyone elsehadproblems today?
Hi,
To all of those who have never actually tried the MSN grabber. It
really is great. Very complete, consistent and reliable. The data
comes from HWW who also supply to most other online tv guides (yahoo,
seven, yourtv, etc). It is therefore essentially the definitive source
of oz guide data. It doesnt get any better than that.
The problem of course is that MSN could change the html any time they
like. But as far as I am concerned, if we have to fix it from time to
time, so be it, It's not that hard. Then there is the potential legal
issues (if any), but until someone starts threatening people, let's
not worry about that :)
Dave
On 8/2/05, Nick Tan <[EMAIL PROTECTED]> wrote:
>
>
> > Berry, David wrote:
> >
> >>Same here (Canberra)
> >>
> >>How do we fix it? - change the script?
> >>
> >>
> >>
> > I had a poke around and the changes seem to be they changed the url from
> > /closeup/ to /cu/ and changed a couple of the attributes that are
> > searched for the find the programs id to find its close up details. The
> > url changes are pretty simple and only exist in a couple of places
> > however I don't know anywhere enough, well no, perl to change the
> > regular expressions to filter out the right program info.
> >
> > It seems it's either fixing this all the time or going back to d1 (I
> > only ever got program data for SBS News here in Canberra, never played
> > with it much) or paying $150 a year with IceTV.
>
> I think I've fixed it (it works with my setup anyways)
>
> As you've said they've changed /closeup/ to /cu/ but they've also changed
> how the pids are displayed.
>
> Changes at around line 350 of the script:
>
> if ($link =~ /cu\/default.asp/)
> {
> my $rowspan = $link;
> $rowspan =~ s/.+rowspan=//g;
> $rowspan =~ s/ .+//g;
>
> my $name = $link;
> #$name =~ s/.+target=new>(|)//g;
> $name =~ s/.+target=_new class=tv>//g;
> $name =~ s/<\/a>.+//g;
> if ($name =~ /\]/) {
> my @nameline=split/\]/,$name;
> $name = $nameline[1];
> $name =~ s/^ //g;
> }
> my $linktemp = $link;
> $link =~ s/.+pid=//g;
> $link =~ s/".+//g; #"
> my @linksplit=split/&/,$link;
> #$link = $linksplit[0];
> my @linksplit2=split/ /,$linksplit[0];
> $link = $linksplit2[0];
> chomp($link);
> chomp($rowspan);
> chomp($name);
>
>
> also change the fetch_details subroutine to be:
>
> sub fetch_details
> {
> my $datepid=$datepids->dequeue;
> my @datepidl=split /-/, $datepid;
> my $date = $datepidl[0];
> my $pidtemp = $datepidl[1];
> my @pidsplit=split /&/, $pidtemp;
> my $pid = $pidsplit[0];
>
> while (($date!=0) and ($pid!=0))
> {
> my $guide_dir = $cache_dir . "/" . $date;
> mkpath ($guide_dir);
>
> my $url = $details_url . $pid;
> my $details_file = $guide_dir . "/" . $pid . ".html";
>
> for (my $retry=0; is_error(getstore($url, $details_file))
> && ($retry<$retrys); $retry++)
> {
> sleep($seconds_before_retry);
> }
>
> $datepid=$datepids->dequeue;
> @datepidl=split /-/, $datepid;
> $date = $datepidl[0];
> # $pid = $datepidl[1];
> my $pidtemp = $datepidl[1];
> my @pidsplit=split /&/, $pidtemp;
> my $pid = $pidsplit[0];
> }
> }
>
>
>
>
> Like I said, this works for me, YMMV. But it should be a good starting
> point for anyone who wants to clean it up a bit :)
>
> ___
> mythtv-users mailing list
> mythtv-users@mythtv.org
> http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-users
>
___
mythtv-users mailing list
mythtv-users@mythtv.org
http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-users
Re: [mythtv-users] tv_grab_au script from this list. Anyone elsehadproblems today?
> Berry, David wrote:
>
>>Same here (Canberra)
>>
>>How do we fix it? - change the script?
>>
>>
>>
> I had a poke around and the changes seem to be they changed the url from
> /closeup/ to /cu/ and changed a couple of the attributes that are
> searched for the find the programs id to find its close up details. The
> url changes are pretty simple and only exist in a couple of places
> however I don't know anywhere enough, well no, perl to change the
> regular expressions to filter out the right program info.
>
> It seems it's either fixing this all the time or going back to d1 (I
> only ever got program data for SBS News here in Canberra, never played
> with it much) or paying $150 a year with IceTV.
I think I've fixed it (it works with my setup anyways)
As you've said they've changed /closeup/ to /cu/ but they've also changed
how the pids are displayed.
Changes at around line 350 of the script:
if ($link =~ /cu\/default.asp/)
{
my $rowspan = $link;
$rowspan =~ s/.+rowspan=//g;
$rowspan =~ s/ .+//g;
my $name = $link;
#$name =~ s/.+target=new>(|)//g;
$name =~ s/.+target=_new class=tv>//g;
$name =~ s/<\/a>.+//g;
if ($name =~ /\]/) {
my @nameline=split/\]/,$name;
$name = $nameline[1];
$name =~ s/^ //g;
}
my $linktemp = $link;
$link =~ s/.+pid=//g;
$link =~ s/".+//g; #"
my @linksplit=split/&/,$link;
#$link = $linksplit[0];
my @linksplit2=split/ /,$linksplit[0];
$link = $linksplit2[0];
chomp($link);
chomp($rowspan);
chomp($name);
also change the fetch_details subroutine to be:
sub fetch_details
{
my $datepid=$datepids->dequeue;
my @datepidl=split /-/, $datepid;
my $date = $datepidl[0];
my $pidtemp = $datepidl[1];
my @pidsplit=split /&/, $pidtemp;
my $pid = $pidsplit[0];
while (($date!=0) and ($pid!=0))
{
my $guide_dir = $cache_dir . "/" . $date;
mkpath ($guide_dir);
my $url = $details_url . $pid;
my $details_file = $guide_dir . "/" . $pid . ".html";
for (my $retry=0; is_error(getstore($url, $details_file))
&& ($retry<$retrys); $retry++)
{
sleep($seconds_before_retry);
}
$datepid=$datepids->dequeue;
@datepidl=split /-/, $datepid;
$date = $datepidl[0];
# $pid = $datepidl[1];
my $pidtemp = $datepidl[1];
my @pidsplit=split /&/, $pidtemp;
my $pid = $pidsplit[0];
}
}
Like I said, this works for me, YMMV. But it should be a good starting
point for anyone who wants to clean it up a bit :)
___
mythtv-users mailing list
mythtv-users@mythtv.org
http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-users
Re: [mythtv-users] tv_grab_au script from this list. Anyone elsehadproblems today?
I have attached by tv_grab_au with fixes for this new problem. It
seems to be just a few urls have changed very slightly. Since my
version might be different everyone elses anyway (there were quite a
few iterations floating around) here is a diff which shows the
necessary changes:
201c201
< my $details_url = "http://tvguide.ninemsn.com.au/cu/default.asp?pid=";;
---
> my $details_url = "http://tvguide.ninemsn.com.au/closeup/default.asp?pid=";;
374c374
< if ($link =~ /cu\/default.asp/) {
---
> if ($link =~ /closeup\/default.asp/) {
380c380
< $name =~ s/.+target=_new class=tv>(|)//g;
---
> $name =~ s/.+target=new>(|)//g;
384,385c384
< $link =~ s/&.+//g; #&
< $link =~ s/ .+//g; #&
---
> $link =~ s/".+//g; #"
704d702
< $pid =~ s/(\d*).*/$1/;
721d718
< $pid =~ s/(\d*).*/$1/;
short summary is: "closeup" changed to "cu" and the urls which the pid
variables are pulled out of got re-arranged and some extra variables
stuck in them which caused the regexes which pull them out to either
fail.
WARNING:
I am in Canberra, I seem to recall messing with the channel names some
time ago when SBS did some stuff, I'm not sure if it is currently
correct, or whether I compensated in other ways (eg. in the database).
Be warned that running this could add a whole bunch of crappy channels
to your tables. Your best bet might be to port the fixes to your own
script.
Good luck,
Dave
On 8/2/05, ffrr <[EMAIL PROTECTED]> wrote:
> F'Nog wrote:
>
> >
> >
> > It seems it's either fixing this all the time or going back to d1 (I
> > only ever got program data for SBS News here in Canberra, never played
> > with it much) or paying $150 a year with IceTV.
>
>
>
> I was so close to switching to the ninemsn script a few days ago. Boy
> am I glad I didn't...
> ___
> mythtv-users mailing list
> mythtv-users@mythtv.org
> http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-users
>
tv_grab_au
Description: Binary data
___
mythtv-users mailing list
mythtv-users@mythtv.org
http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-users
Re: [mythtv-users] tv_grab_au script from this list. Anyone elsehadproblems today?
F'Nog wrote: It seems it's either fixing this all the time or going back to d1 (I only ever got program data for SBS News here in Canberra, never played with it much) or paying $150 a year with IceTV. I was so close to switching to the ninemsn script a few days ago. Boy am I glad I didn't... ___ mythtv-users mailing list mythtv-users@mythtv.org http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-users
Re: [mythtv-users] tv_grab_au script from this list. Anyone elsehadproblems today?
Berry, David wrote: Same here (Canberra) How do we fix it? - change the script? I had a poke around and the changes seem to be they changed the url from /closeup/ to /cu/ and changed a couple of the attributes that are searched for the find the programs id to find its close up details. The url changes are pretty simple and only exist in a couple of places however I don't know anywhere enough, well no, perl to change the regular expressions to filter out the right program info. It seems it's either fixing this all the time or going back to d1 (I only ever got program data for SBS News here in Canberra, never played with it much) or paying $150 a year with IceTV. ___ mythtv-users mailing list mythtv-users@mythtv.org http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-users
RE: [mythtv-users] tv_grab_au script from this list. Anyone, elsehadproblems today?
Same here (Canberra) How do we fix it? - change the script? Try the community based tv guide data at minnie.tuhs.org if you can. Or pick another grabber from http://www.cse.unsw.edu.au/~willu/xmltv/ (list of australian TV grabbers) Screen scrapers are always going to be a pain to maintain, and sadly D1 doesn't do imparja. Dave ___ mythtv-users mailing list mythtv-users@mythtv.org http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-users
Re: [mythtv-users] tv_grab_au script from this list. Anyone elsehadproblems today?
Hi Ian, I can pitch in on OzTiVo... On 2/8/05 4:42 PM, "Ian Ward" <[EMAIL PROTECTED]> wrote: > That would be great, where are they located? I had not seen the OzTivo > site before. > Is this http://minnie.tuhs.org/Programs/Wktivoguide/ what you are > talking about? http://www.tvguide.org.au is the best place to start (that's the backend to the system at that site). BTW press Cancel a couple of times to browse around, you won't be able to see guide data without registering. > Do you find this more reliable than d1.com.au? I changed over because d1's reliability was dropping off. If you put yourself on the tvguide mailing list you're warned if things go wrong. The quality is excellent, and includes all of the paid channels (which is my main reason for not moving onto IceGuide). Joshua Send instant messages to your online friends http://au.messenger.yahoo.com ___ mythtv-users mailing list mythtv-users@mythtv.org http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-users
Re: [mythtv-users] tv_grab_au script from this list. Anyone elsehadproblems today?
Phill Edwards wrote: The reason they restrict the EPG data is because they know that people want it to use in their PVRs. And why? Because people skip ads with their PVRs. Who me =-O I love the ads! The other option apart from NineMSN is the OzTivo scripts. As I've mentioned before on this forum these have no legal issues associated because volunteers manually enter the data. I can help you guys get up tp speed with it if you like. The change-over is not actually very hard. Some of the data is not as complete - eg program genres - some is mor ecomplete - eg actors etc. That would be great, where are they located? I had not seen the OzTivo site before. Is this http://minnie.tuhs.org/Programs/Wktivoguide/ what you are talking about? Do you find this more reliable than d1.com.au? there really is two issues, reliability and quality. d1.com.au has the odd hiccup or two, but it is in ninemsn's interest to change things around just to screw up your grabber. If you have a modified grabber I would like to try it ( I have a second mythtv box so that I can experiment with a few things ) Regards, Ian ___ mythtv-users mailing list mythtv-users@mythtv.org http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-users
Re: [mythtv-users] tv_grab_au script from this list. Anyone elsehadproblems today?
Any plans to add Foxtel/Optus channels to the IceTV EPG? On 02/08/2005, at 2:42 PM, Daniel Hall wrote: Also if you are looking for a reliable and legal XMLTV EPG service in Australia then you can also have a look at IceTV. This is a subscription based service. Details can be found at http://www.icetv.com.au/ Cheers, ___ mythtv-users mailing list mythtv-users@mythtv.org http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-users
RE: [mythtv-users] tv_grab_au script from this list. Anyone elsehadproblems today?
Also if you are looking for a reliable and legal XMLTV EPG service in Australia then you can also have a look at IceTV. This is a subscription based service. Details can be found at http://www.icetv.com.au/ Cheers, Daniel IceTV Support Team IceTV Pty Limited Level 2/34-36 Chandos Street, St Leonards, NSW 2065 PO Box 698, Crows Nest, NSW 1585, Sydney, Australia ph: 1300 654 803 fax: 1300 654 208 www.icetv.com.au CAUTION - This message may contain privileged and confidential information intended only for the use of the addressee named above. If you are not the intended recipient of this message you are hereby notified that any use, disclosure, dissemination, copying, distribution or any action taken or omitted to be taken in reliance on it is prohibited and may be unlawful. If you have received this message in error please notify IceTV immediately via email at [EMAIL PROTECTED] or by phone at 1300 654 803. -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Mary Wright Sent: Tuesday, 2 August 2005 2:35 PM To: mythtv-users@mythtv.org Subject: Re: [mythtv-users] tv_grab_au script from this list. Anyone elsehadproblems today? Hey My mythfilldatabase seems to be spewing about pid errors today is anyone getting the same error no pids in /var/local/tv_grab_au/02082005/guide.prn no pids in /var/local/tv_grab_au/03082005/guide.prn no pids in /var/local/tv_grab_au/0WorldClient.dll?View=Main4082005/guide. prn no pids in /var/local/tv_grab_au/05082005/guide.prn no pids in /var/local/tv_grab_au/06082005/guide.prn no pids in /var/local/tv_grab_au/07082005/guide.prn no pids in /var/local/tv_grab_au/08082005/guide.prn I think the MSN may have changed something unless I am the only one getting the errors Latz Mary -Original Message- From: Brian May <[EMAIL PROTECTED]> To: Brad <[EMAIL PROTECTED]> Cc: Discussion about mythtv Date: Sat, 30 Jul 2005 17:38:58 +1000 Subject: Re: [mythtv-users] tv_grab_au script from this list. Anyone elsehadproblems today? > >>>>> "Brad" == Brad <[EMAIL PROTECTED]> writes: > > Brad> Is the ninmsn grabber better than the d1.com.au ? If so do > Brad> you know where I could download it from. Also a clue on how > Brad> to install if time permits. > > It seems to be better. > > I frequently get reports of conflicting programs though (Melbourne), > even when I run it two times in a row manually. Presumably, the first > time I run it should remove any conflicting programs, and there > shouldn't be any conflicts the second time I run it... > > So I suspect it is not perfect. > > However, it seems to get everything correct that I care about. > -- > Brian May <[EMAIL PROTECTED]> > ___ mythtv-users mailing list mythtv-users@mythtv.org http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-users
Re: [mythtv-users] tv_grab_au script from this list. Anyone elsehadproblems today?
> Are you guys using the ninemsn grabber? > This is why I have stuck to the older grab-au that uses d1.com.au > Thanks d1, it's not perfect ( Myth hates the spelling mistakes ;) , but > at least your always there. > I cannot understand why someone would not want to freely distribute > information about their services > Are they saying they make more money from charging for the stupid > listings than for the actual shows or what? The reason they restrict the EPG data is because they know that people want it to use in their PVRs. And why? Because people skip ads with their PVRs. The other option apart from NineMSN is the OzTivo scripts. As I've mentioned before on this forum these have no legal issues associated because volunteers manually enter the data. I can help you guys get up tp speed with it if you like. The change-over is not actually very hard. Some of the data is not as complete - eg program genres - some is mor ecomplete - eg actors etc. Regards, Phill ___ mythtv-users mailing list mythtv-users@mythtv.org http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-users
Re: [mythtv-users] tv_grab_au script from this list. Anyone elsehadproblems today?
Berry, David wrote: Same here (Canberra) How do we fix it? - change the script? To: Discussion about mythtv Subject: Re: [mythtv-users] tv_grab_au script from this list. Anyone elsehadproblems today? Mary Wright wrote: Hey My mythfill database seems to be spewing about pid errors today is anyone getting the same error I'm getting the same thing.. Sighz... Why can't the networks just provide the EPG free without these stupid restrictions, I'm sure the HDTV uptake will improve with a proper freely available EPG that cost nothing to the consumer. Are you guys using the ninemsn grabber? This is why I have stuck to the older grab-au that uses d1.com.au Thanks d1, it's not perfect ( Myth hates the spelling mistakes ;) , but at least your always there. I cannot understand why someone would not want to freely distribute information about their services Are they saying they make more money from charging for the stupid listings than for the actual shows or what? ___ mythtv-users mailing list mythtv-users@mythtv.org http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-users
RE: [mythtv-users] tv_grab_au script from this list. Anyone elsehadproblems today?
Same here (Canberra) How do we fix it? - change the script? -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Tj Sent: Tuesday, 2 August 2005 1:32 PM To: Discussion about mythtv Subject: Re: [mythtv-users] tv_grab_au script from this list. Anyone elsehadproblems today? Mary Wright wrote: >Hey > > My mythfill database seems to be spewing about pid errors today is anyone >getting the same error > > I'm getting the same thing.. Sighz... Why can't the networks just provide the EPG free without these stupid restrictions, I'm sure the HDTV uptake will improve with a proper freely available EPG that cost nothing to the consumer. ___ mythtv-users mailing list mythtv-users@mythtv.org http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-users ___ The information transmitted is for the use of the intended recipient only and may contain confidential and/or legally privileged material. Any review, re-transmission, disclosure, dissemination or other use of, or taking of any action in reliance upon, this information by persons or entities other than the intended recipient is prohibited and may result in severe penalties. If you have received this e-mail in error please notify the Security Advisor of the Department of Communications, Information Technology and the Arts, telephone (02) 6271-1880 and delete all copies of this transmission together with any attachments. For all other enquiries please ring (02) 6271-1000. ___ ___ mythtv-users mailing list mythtv-users@mythtv.org http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-users
