Re: [Freevo-users] IMDB lookup crashes
harri wrote: On Tuesday 19 December 2006 01:05, Duncan Webb wrote: You sound that you know Python's Unicode handling pretty well, would you be able to take a look at the library.rpy code, it breaks all over the place because the file names are being passed around as unicode strings. If I roll my sleeves and do an unicode audit, would a patch against rev 8742 be easy to handle, or should I use another revision? That will be great, you can use this revision or r8743, which includes two extra key mappings '[' PREV and ']' NEXT for the detached player. Duncan - Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT business topics through brief surveys - and earn cash http://www.techsay.com/default.php?page=join.phpp=sourceforgeCID=DEVDEV ___ Freevo-users mailing list Freevo-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/freevo-users
Re: [Freevo-users] IMDB lookup crashes
A lot of times i immediately get 'Unknown error connecting to IMDB',
other times I get a 'even MENU-SELECT crashed' here is the error trace
for one:
http://us.imdb.com/find?q=ringer%20;s=tt
[('0267891', u'The Ringer', u'2005', ''), ('0117480', u'Ringer',
u'1996', ''), ('0455857', u'When a Stranger Calls', u'2006', ''),
('0057997', u'Dead Ringer', u'1964', ''), ('0058191', u'Hexer, Der',
u'1964', ''), ('0078765', u'The American Success Company', u'1980', ''),
('0101673', u'Dead Ringer', u'1981', ''), ('0123678', u'Chock 6 - Det
ringer', u'1997', u'TV'), ('0045092', u'The Ringer', u'1952', ''),
('0109918', u'Golubinyj zvonar', u'1994', ''), ('0495150', u'Marcco, der
Ringer des Mikado', u'1922', ''), ('0127041', u'Naar juleklokkerne
ringer', u'1917', ''), ('0851435', u'The Old Bell-Ringer', u'1914', ''),
('0772974', u'A Ringer for Max', u'1915', ''), ('0019326', u'The
Ringer', u'1928', ''), ('0023404', u'The Ringer', u'1931', ''),
('0261955', u'The Ringer', u'1938', u'TV'), ('0201458', u'Beatrice
Fairfax Episode 13: The Ringer', u'1916', ''), ('0297775', u'Bell Ringer
of Antigua', u'1946', ''), ('0152208', u'Robot Ringer', u'1962', ''),
('0324189', u'Telefonen ringer', u'1960', ''), ('0341966',
u'#34;Telefonen ringer!#34;', u'1964', ''), ('0141065', u'Felix the
Cat Rings the Ringer', u'1926', '')]
type=2 key=13
rating : 5.8/10 (4,788 votes)
plot : A young guy's only option to erase a really bad debt is to rig
the Special Olympics by posing as a contestant.
tagline : Only a jackass would fix the Special Olympics
image : http://ia.ec.imdb.com/media/imdb/01/I/97/66/49m.jpg
title : The Ringer
year : 2005
genre : Comedy / Sport
runtime : 94 min / Argentina:95 min
id: 0267891 dvd: 0
url: http://us.imdb.com/title/tt0267891/dvd
Traceback (most recent call last):
File /usr/lib/python2.4/site-packages/freevo/main.py, line 285, in
eventhandler
app.eventhandler(event)
File /usr/lib/python2.4/site-packages/freevo/menu.py, line 621, in
eventhandler
action( arg=arg, menuw=self )
File /usr/lib/python2.4/site-packages/freevo/menu.py, line 92, in select
self.function(arg=self.arg, menuw=menuw)
File /usr/lib/python2.4/site-packages/freevo/video/plugins/imdb.py,
line 204, in imdb_create_fxd
fxd.setImdbId(arg[0])
File /usr/lib/python2.4/site-packages/freevo/util/fxdimdb.py, line
177, in setImdbId
self.parsedata(idpage, id)
File /usr/lib/python2.4/site-packages/freevo/util/fxdimdb.py, line
623, in parsedata
soup.feed(r.read())
File /usr/lib/python2.4/sgmllib.py, line 95, in feed
self.goahead(0)
File /usr/lib/python2.4/sgmllib.py, line 134, in goahead
k = self.parse_endtag(i)
File /usr/lib/python2.4/sgmllib.py, line 296, in parse_endtag
self.finish_endtag(tag)
File /usr/lib/python2.4/sgmllib.py, line 336, in finish_endtag
self.unknown_endtag(tag)
File /usr/lib/python2.4/site-packages/BeautifulSoup.py, line 1184,
in unknown_endtag
self.endData()
File /usr/lib/python2.4/site-packages/BeautifulSoup.py, line 1070,
in endData
o = containerClass(currentData)
UnicodeDecodeError: 'ascii' codec can't decode byte 0x92 in position
156: ordinal not in range(128)
Currently I have to use my old box to lookup IMDB info. Thanks.
Duncan Webb wrote:
Ryan Roth wrote:
I am currently running the SVN 1.X version of Freevo, which uses
BeautifulSoap.py to process XML. 90% of IMDB lookups crash, dumping an
error caused by BeautifulSoap. Should Freevo be using BeautifulSoap if
it so unreliable? I am running version 3.0.3 of BeautifulSoap, should I
be using a different version?
Will you post the error trace?
I've tried it one a few movies, and no crash so far.
Duncan
-
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT business topics through brief surveys - and earn cash
http://www.techsay.com/default.php?page=join.phpp=sourceforgeCID=DEVDEV
___
Freevo-users mailing list
Freevo-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/freevo-users
-
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT business topics through brief surveys - and earn cash
http://www.techsay.com/default.php?page=join.phpp=sourceforgeCID=DEVDEV
___
Freevo-users mailing list
Freevo-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/freevo-users
Re: [Freevo-users] IMDB lookup crashes
Ryan Roth wrote:
A lot of times i immediately get 'Unknown error connecting to IMDB',
other times I get a 'even MENU-SELECT crashed' here is the error trace
for one:
http://us.imdb.com/find?q=ringer%20;s=tt
[('0267891', u'The Ringer', u'2005', ''), ('0117480', u'Ringer',
u'1996', ''), ('0455857', u'When a Stranger Calls', u'2006', ''),
('0057997', u'Dead Ringer', u'1964', ''), ('0058191', u'Hexer, Der',
u'1964', ''), ('0078765', u'The American Success Company', u'1980', ''),
('0101673', u'Dead Ringer', u'1981', ''), ('0123678', u'Chock 6 - Det
ringer', u'1997', u'TV'), ('0045092', u'The Ringer', u'1952', ''),
('0109918', u'Golubinyj zvonar', u'1994', ''), ('0495150', u'Marcco, der
Ringer des Mikado', u'1922', ''), ('0127041', u'Naar juleklokkerne
ringer', u'1917', ''), ('0851435', u'The Old Bell-Ringer', u'1914', ''),
('0772974', u'A Ringer for Max', u'1915', ''), ('0019326', u'The
Ringer', u'1928', ''), ('0023404', u'The Ringer', u'1931', ''),
('0261955', u'The Ringer', u'1938', u'TV'), ('0201458', u'Beatrice
Fairfax Episode 13: The Ringer', u'1916', ''), ('0297775', u'Bell Ringer
of Antigua', u'1946', ''), ('0152208', u'Robot Ringer', u'1962', ''),
('0324189', u'Telefonen ringer', u'1960', ''), ('0341966',
u'#34;Telefonen ringer!#34;', u'1964', ''), ('0141065', u'Felix the
Cat Rings the Ringer', u'1926', '')]
type=2 key=13
rating : 5.8/10 (4,788 votes)
plot : A young guy's only option to erase a really bad debt is to rig
the Special Olympics by posing as a contestant.
tagline : Only a jackass would fix the Special Olympics
image : http://ia.ec.imdb.com/media/imdb/01/I/97/66/49m.jpg
title : The Ringer
year : 2005
genre : Comedy / Sport
runtime : 94 min / Argentina:95 min
id: 0267891 dvd: 0
url: http://us.imdb.com/title/tt0267891/dvd
Traceback (most recent call last):
File /usr/lib/python2.4/site-packages/freevo/main.py, line 285, in
eventhandler
app.eventhandler(event)
File /usr/lib/python2.4/site-packages/freevo/menu.py, line 621, in
eventhandler
action( arg=arg, menuw=self )
File /usr/lib/python2.4/site-packages/freevo/menu.py, line 92, in select
self.function(arg=self.arg, menuw=menuw)
File /usr/lib/python2.4/site-packages/freevo/video/plugins/imdb.py,
line 204, in imdb_create_fxd
fxd.setImdbId(arg[0])
File /usr/lib/python2.4/site-packages/freevo/util/fxdimdb.py, line
177, in setImdbId
self.parsedata(idpage, id)
File /usr/lib/python2.4/site-packages/freevo/util/fxdimdb.py, line
623, in parsedata
soup.feed(r.read())
File /usr/lib/python2.4/sgmllib.py, line 95, in feed
self.goahead(0)
File /usr/lib/python2.4/sgmllib.py, line 134, in goahead
k = self.parse_endtag(i)
File /usr/lib/python2.4/sgmllib.py, line 296, in parse_endtag
self.finish_endtag(tag)
File /usr/lib/python2.4/sgmllib.py, line 336, in finish_endtag
self.unknown_endtag(tag)
File /usr/lib/python2.4/site-packages/BeautifulSoup.py, line 1184,
in unknown_endtag
self.endData()
File /usr/lib/python2.4/site-packages/BeautifulSoup.py, line 1070,
in endData
o = containerClass(currentData)
UnicodeDecodeError: 'ascii' codec can't decode byte 0x92 in position
156: ordinal not in range(128)
Currently I have to use my old box to lookup IMDB info. Thanks.
It does look like a bug in BeautifulSoup. :(
Having grabbed the page with wget, edited it with vim, it does have 92
bytes in there.
The real problem is Python and it's unicode handling, it's not very good
but the fix is simple. :)
Change line 352 in /usr/lib/python2.4/site.py from:
encoding = ascii # Default value set by _PyUnicode_Init()
to:
encoding = iso-8859-1 # Default value set by _PyUnicode_Init()
Duncan
Duncan Webb wrote:
Ryan Roth wrote:
I am currently running the SVN 1.X version of Freevo, which uses
BeautifulSoap.py to process XML. 90% of IMDB lookups crash, dumping an
error caused by BeautifulSoap. Should Freevo be using BeautifulSoap if
it so unreliable? I am running version 3.0.3 of BeautifulSoap, should I
be using a different version?
Will you post the error trace?
I've tried it one a few movies, and no crash so far.
Duncan
-
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT business topics through brief surveys - and earn cash
http://www.techsay.com/default.php?page=join.phpp=sourceforgeCID=DEVDEV
___
Freevo-users mailing list
Freevo-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/freevo-users
Re: [Freevo-users] IMDB lookup crashes
Is there not a better way to fix it, or do we just need to wait on the
BeautifulSoap Guys?
Duncan Webb wrote:
Ryan Roth wrote:
A lot of times i immediately get 'Unknown error connecting to IMDB',
other times I get a 'even MENU-SELECT crashed' here is the error trace
for one:
http://us.imdb.com/find?q=ringer%20;s=tt
[('0267891', u'The Ringer', u'2005', ''), ('0117480', u'Ringer',
u'1996', ''), ('0455857', u'When a Stranger Calls', u'2006', ''),
('0057997', u'Dead Ringer', u'1964', ''), ('0058191', u'Hexer, Der',
u'1964', ''), ('0078765', u'The American Success Company', u'1980', ''),
('0101673', u'Dead Ringer', u'1981', ''), ('0123678', u'Chock 6 - Det
ringer', u'1997', u'TV'), ('0045092', u'The Ringer', u'1952', ''),
('0109918', u'Golubinyj zvonar', u'1994', ''), ('0495150', u'Marcco, der
Ringer des Mikado', u'1922', ''), ('0127041', u'Naar juleklokkerne
ringer', u'1917', ''), ('0851435', u'The Old Bell-Ringer', u'1914', ''),
('0772974', u'A Ringer for Max', u'1915', ''), ('0019326', u'The
Ringer', u'1928', ''), ('0023404', u'The Ringer', u'1931', ''),
('0261955', u'The Ringer', u'1938', u'TV'), ('0201458', u'Beatrice
Fairfax Episode 13: The Ringer', u'1916', ''), ('0297775', u'Bell Ringer
of Antigua', u'1946', ''), ('0152208', u'Robot Ringer', u'1962', ''),
('0324189', u'Telefonen ringer', u'1960', ''), ('0341966',
u'#34;Telefonen ringer!#34;', u'1964', ''), ('0141065', u'Felix the
Cat Rings the Ringer', u'1926', '')]
type=2 key=13
rating : 5.8/10 (4,788 votes)
plot : A young guy's only option to erase a really bad debt is to rig
the Special Olympics by posing as a contestant.
tagline : Only a jackass would fix the Special Olympics
image : http://ia.ec.imdb.com/media/imdb/01/I/97/66/49m.jpg
title : The Ringer
year : 2005
genre : Comedy / Sport
runtime : 94 min / Argentina:95 min
id: 0267891 dvd: 0
url: http://us.imdb.com/title/tt0267891/dvd
Traceback (most recent call last):
File /usr/lib/python2.4/site-packages/freevo/main.py, line 285, in
eventhandler
app.eventhandler(event)
File /usr/lib/python2.4/site-packages/freevo/menu.py, line 621, in
eventhandler
action( arg=arg, menuw=self )
File /usr/lib/python2.4/site-packages/freevo/menu.py, line 92, in select
self.function(arg=self.arg, menuw=menuw)
File /usr/lib/python2.4/site-packages/freevo/video/plugins/imdb.py,
line 204, in imdb_create_fxd
fxd.setImdbId(arg[0])
File /usr/lib/python2.4/site-packages/freevo/util/fxdimdb.py, line
177, in setImdbId
self.parsedata(idpage, id)
File /usr/lib/python2.4/site-packages/freevo/util/fxdimdb.py, line
623, in parsedata
soup.feed(r.read())
File /usr/lib/python2.4/sgmllib.py, line 95, in feed
self.goahead(0)
File /usr/lib/python2.4/sgmllib.py, line 134, in goahead
k = self.parse_endtag(i)
File /usr/lib/python2.4/sgmllib.py, line 296, in parse_endtag
self.finish_endtag(tag)
File /usr/lib/python2.4/sgmllib.py, line 336, in finish_endtag
self.unknown_endtag(tag)
File /usr/lib/python2.4/site-packages/BeautifulSoup.py, line 1184,
in unknown_endtag
self.endData()
File /usr/lib/python2.4/site-packages/BeautifulSoup.py, line 1070,
in endData
o = containerClass(currentData)
UnicodeDecodeError: 'ascii' codec can't decode byte 0x92 in position
156: ordinal not in range(128)
Currently I have to use my old box to lookup IMDB info. Thanks.
It does look like a bug in BeautifulSoup. :(
Having grabbed the page with wget, edited it with vim, it does have 92
bytes in there.
The real problem is Python and it's unicode handling, it's not very good
but the fix is simple. :)
Change line 352 in /usr/lib/python2.4/site.py from:
encoding = ascii # Default value set by _PyUnicode_Init()
to:
encoding = iso-8859-1 # Default value set by _PyUnicode_Init()
Duncan
Duncan Webb wrote:
Ryan Roth wrote:
I am currently running the SVN 1.X version of Freevo, which uses
BeautifulSoap.py to process XML. 90% of IMDB lookups crash, dumping an
error caused by BeautifulSoap. Should Freevo be using BeautifulSoap if
it so unreliable? I am running version 3.0.3 of BeautifulSoap, should I
be using a different version?
Will you post the error trace?
I've tried it one a few movies, and no crash so far.
Duncan
-
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT business topics through brief surveys - and earn cash
http://www.techsay.com/default.php?page=join.phpp=sourceforgeCID=DEVDEV
___
Freevo-users mailing list
Freevo-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/freevo-users
Re: [Freevo-users] IMDB lookup crashes
Also I am still getting 'unknown error while connecting to IMDB' on a bunch of other movies. Al that gets logged for them is: http://us.imdb.com/title/tt0457510/?fr=c2l0ZT1kZnx0dD0xfGZiPXV8cG49MHxxPW5hY2hvIGxpYnJlIHxteD0yMHxsbT01MDB8aHRtbD0x;fc=1;ft=20 - Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT business topics through brief surveys - and earn cash http://www.techsay.com/default.php?page=join.phpp=sourceforgeCID=DEVDEV ___ Freevo-users mailing list Freevo-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/freevo-users
Re: [Freevo-users] IMDB lookup crashes
I just noticed that the files that return that error or files that do not return multiple pages from IMDB, but instead go directly to an exact match. Also sorry for 'spamming' with so many separate messages. - Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT business topics through brief surveys - and earn cash http://www.techsay.com/default.php?page=join.phpp=sourceforgeCID=DEVDEV ___ Freevo-users mailing list Freevo-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/freevo-users
Re: [Freevo-users] IMDB lookup crashes
Freevo 1.x SVN? Because that is what I am running. Duncan Webb wrote: Ryan Roth wrote: Also I am still getting 'unknown error while connecting to IMDB' on a bunch of other movies. Al that gets logged for them is: http://us.imdb.com/title/tt0457510/?fr=c2l0ZT1kZnx0dD0xfGZiPXV8cG49MHxxPW5hY2hvIGxpYnJlIHxteD0yMHxsbT01MDB8aHRtbD0x;fc=1;ft=20 That's an error in BeautifulSoup that I have worked around in svn, AFAIK BeautifulSoup should process the entities eg. #244; as it returns a unicode string and says that the encoding is utf-8. The entities are latin-1 encoded. Anyway the workaround is in svn. Duncan - Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT business topics through brief surveys - and earn cash http://www.techsay.com/default.php?page=join.phpp=sourceforgeCID=DEVDEV ___ Freevo-users mailing list Freevo-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/freevo-users - Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT business topics through brief surveys - and earn cash http://www.techsay.com/default.php?page=join.phpp=sourceforgeCID=DEVDEV ___ Freevo-users mailing list Freevo-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/freevo-users
Re: [Freevo-users] IMDB lookup crashes
Create a file called 'Nacho Libre (2006).avi' and try a IMDB lookup on it. - Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT business topics through brief surveys - and earn cash http://www.techsay.com/default.php?page=join.phpp=sourceforgeCID=DEVDEV ___ Freevo-users mailing list Freevo-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/freevo-users
Re: [Freevo-users] IMDB lookup crashes
On Monday 18 December 2006 18:16, Duncan Webb wrote: The real problem is Python and it's unicode handling, it's not very good but the fix is simple. :) Change line 352 in /usr/lib/python2.4/site.py from: encoding = ascii # Default value set by _PyUnicode_Init() to: encoding = iso-8859-1 # Default value set by _PyUnicode_Init() Actually I think Python's unicode handling is fairly good, but it is not automatic. It would be easiest for maintenance if everything internally would be kept in utf-8, but I realize the modular plugin architecture of freevo might make it difficult. I suggest that site.py be bypassed all-together. This could be done by changing freevo startup line from #!/usr/bin/python to #!/usr/bin/python -S And changing the default encoding by adding import sys sys.setdefaultencoding(iso-8859-15) or perhaps sys.setdefaultencoding(utf-8) because as soon as freevo starts getting Asian users the latin-1 type encodings don't cut it any more. This way it will be specific to Freevo, and not system wide. Btw. difference between iso-8859-1 and iso-8859-15 is that the latter does support the new Euro character: €. For filenames, perhaps os.path.supports_unicode_filenames should be examined. From the doc: True if arbitrary Unicode strings can be used as file names (within limitations imposed by the file system), and if os.listdir() returns Unicode strings for a Unicode argument. New in version 2.3. Some ideas if anyone has time to try. -Harri - Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT business topics through brief surveys - and earn cash http://www.techsay.com/default.php?page=join.phpp=sourceforgeCID=DEVDEV ___ Freevo-users mailing list Freevo-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/freevo-users
Re: [Freevo-users] IMDB lookup crashes
The current SVN version solved the issue. While we're on the subject of IMDB lookups, after completing and update the listing does not auto update to show the changes like it use to. Instead you have to back one menu and then re-enter to see the results of the IMDB lookup. I am assuming this is just a matter of calling a menuw refresh. harri wrote: On Monday 18 December 2006 18:16, Duncan Webb wrote: The real problem is Python and it's unicode handling, it's not very good but the fix is simple. :) Change line 352 in /usr/lib/python2.4/site.py from: encoding = ascii # Default value set by _PyUnicode_Init() to: encoding = iso-8859-1 # Default value set by _PyUnicode_Init() Actually I think Python's unicode handling is fairly good, but it is not automatic. It would be easiest for maintenance if everything internally would be kept in utf-8, but I realize the modular plugin architecture of freevo might make it difficult. I suggest that site.py be bypassed all-together. This could be done by changing freevo startup line from #!/usr/bin/python to #!/usr/bin/python -S And changing the default encoding by adding import sys sys.setdefaultencoding(iso-8859-15) or perhaps sys.setdefaultencoding(utf-8) because as soon as freevo starts getting Asian users the latin-1 type encodings don't cut it any more. This way it will be specific to Freevo, and not system wide. Btw. difference between iso-8859-1 and iso-8859-15 is that the latter does support the new Euro character: €. For filenames, perhaps os.path.supports_unicode_filenames should be examined. From the doc: True if arbitrary Unicode strings can be used as file names (within limitations imposed by the file system), and if os.listdir() returns Unicode strings for a Unicode argument. New in version 2.3. Some ideas if anyone has time to try. -Harri - Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT business topics through brief surveys - and earn cash http://www.techsay.com/default.php?page=join.phpp=sourceforgeCID=DEVDEV ___ Freevo-users mailing list Freevo-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/freevo-users - Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT business topics through brief surveys - and earn cash http://www.techsay.com/default.php?page=join.phpp=sourceforgeCID=DEVDEV ___ Freevo-users mailing list Freevo-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/freevo-users
Re: [Freevo-users] IMDB lookup crashes
Ryan Roth wrote:
The current SVN version solved the issue. While we're on the subject of
IMDB lookups, after completing and update the listing does not auto
update to show the changes like it use to. Instead you have to back one
menu and then re-enter to see the results of the IMDB lookup. I am
assuming this is just a matter of calling a menuw refresh.
The very latest, the direct imdb hit, is fixed and so the htmlenties2txt
problem.
It now processes: 0109422 from the results. eg:
http://www.imdb.com/find?s=tt;site=aka;q=one%20flew%20over%20cuckoos%20nest%20
[
('0073486', uOne Flew Over the Cuckoo's Nest, u'1975', ''),
('0121735', u'Sha da jie fan zhuan feng ren yuan', u'1993', ''),
('0109422', u'Ranma #189;: Ch#244;-musabetsu kessen! Ranma team VS
densetsu no h#244;#244;', u'1994', ''),
('0030223', u'Healthy, Wealthy and Dumb', u'1938', '')
]
harri wrote:
You sound that you know Python's Unicode handling pretty well, would you
be able to take a look at the library.rpy code, it breaks all over the
place because the file names are being passed around as unicode strings.
I just don't know enough about HTTP and how to encode file names so that
they don't get converted.
What I find a problem with python's unicode handling is when a decode
fails it reports an UnicodeEncodingError and visa versa.
decode/encode are not really intuitive as you have to know from which
perspective the decode/encode works. What I mean is string.decode()
returns a unicodestr and unicodestr.encode() return a string, you have
to know that it is from the string.
unicode('ü') crashes with the default ascii encoding.
On Monday 18 December 2006 18:16, Duncan Webb wrote:
The real problem is Python and it's unicode handling, it's not very good
but the fix is simple. :)
Change line 352 in /usr/lib/python2.4/site.py from:
encoding = ascii # Default value set by _PyUnicode_Init()
to:
encoding = iso-8859-1 # Default value set by _PyUnicode_Init()
Actually I think Python's unicode handling is fairly good, but it is not
automatic. It would be easiest for maintenance if everything internally
would be kept in utf-8, but I realize the modular plugin architecture of
freevo might make it difficult.
I suggest that site.py be bypassed all-together. This could be done by
changing freevo startup line from
#!/usr/bin/python
to
#!/usr/bin/python -S
And changing the default encoding by adding
import sys
sys.setdefaultencoding(iso-8859-15)
or perhaps
sys.setdefaultencoding(utf-8)
because as soon as freevo starts getting Asian users the latin-1 type
encodings don't cut it any more.
Sound like a good idea, I'll try it out. :)
This way it will be specific to Freevo, and not system wide.
Btw. difference between iso-8859-1 and iso-8859-15 is that the latter does
support the new Euro character: €.
For filenames, perhaps os.path.supports_unicode_filenames should be examined.
From the doc: True if arbitrary Unicode strings can be used as file names
(within limitations imposed by the file system), and if os.listdir() returns
Unicode strings for a Unicode argument. New in version 2.3.
I noticed that the same file name from a fat (usb drive) and an ext3
partition were differently encoded. But I didn't try
os.path.supports_unicode_filenames.
Some ideas if anyone has time to try.
-Harri
-
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT business topics through brief surveys - and earn cash
http://www.techsay.com/default.php?page=join.phpp=sourceforgeCID=DEVDEV
___
Freevo-users mailing list
Freevo-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/freevo-users
Re: [Freevo-users] IMDB lookup crashes
On Tuesday 19 December 2006 01:05, Duncan Webb wrote: You sound that you know Python's Unicode handling pretty well, would you be able to take a look at the library.rpy code, it breaks all over the place because the file names are being passed around as unicode strings. If I roll my sleeves and do an unicode audit, would a patch against rev 8742 be easy to handle, or should I use another revision? -Harri - Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT business topics through brief surveys - and earn cash http://www.techsay.com/default.php?page=join.phpp=sourceforgeCID=DEVDEV ___ Freevo-users mailing list Freevo-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/freevo-users
Re: [Freevo-users] IMDB lookup crashes
Ryan Roth wrote: I am currently running the SVN 1.X version of Freevo, which uses BeautifulSoap.py to process XML. 90% of IMDB lookups crash, dumping an error caused by BeautifulSoap. Should Freevo be using BeautifulSoap if it so unreliable? I am running version 3.0.3 of BeautifulSoap, should I be using a different version? Will you post the error trace? I've tried it one a few movies, and no crash so far. Duncan - Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT business topics through brief surveys - and earn cash http://www.techsay.com/default.php?page=join.phpp=sourceforgeCID=DEVDEV ___ Freevo-users mailing list Freevo-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/freevo-users
