[Imdbpy-help] imdbpy2sql.py, PostgreSQL, UTF8, invalid byte sequence

[Vitaly Pashkov]

Hi all! Got a problem with IMDbPY 4.2 and tested it with a trunk version (4.3) too. I'm used recent plain text files, updated at 02.10.2009 and previous version of it, updated as i remember at 25.09.2009. Everytime i try to import them to PostgreSQL database i'm getting the following error:

$ imdbpy2sql.py -d ./ -u postgres://login:passw...@localhost/imdb

 warnings.warn('Unable to import the cutils.ratcliff function.'

 warnings.warn('Unable to import the cutils.soundex function.'
IMPORTING psyco... FAILED (not a big deal, everything is alright...)

RUNNING imdbpy2sql.py
SAVING imdbID values for movies... DONE! (0 entries)
SAVING imdbID values for people... DONE! (0 entries)
SAVING imdbID values for characters... DONE! (0 entries)
SAVING imdbID values for companies... DONE! (0 entries)
DROPPING current database... DONE!
CREATING new tables... DONE!

SCANNING movies: #1 (2005) (movieID: 1)
SCANNING movies: A Fu (2000) (movieID: 10001)
SCANNING movies: Absconding (2005) (V) (movieID: 20001)
SCANNING movies: Alone and Inhuman (2005) (movieID: 30001)
SCANNING movies: Apartment 23 (1919) (movieID: 40001)
SCANNING movies: Babalon (1999) (movieID: 50001)
SCANNING movies: Belye nochi (1959) (movieID: 60001)
SCANNING movies: Blood on the Turntable (2004) (TV) (movieID: 70001)
SCANNING movies: Bullets for Bandits (1942) (movieID: 80001)
SCANNING movies: Celebration (2010) (movieID: 90001)
* FLUSHING MoviesCache...
Traceback (most recent call last):
 File "/usr/bin/imdbpy2sql.py", line 5, in <module>
   pkg_resources.run_script('IMDbPY==4.3', 'imdbpy2sql.py')

   self.require(requires)[0].run_script(script_name, ns)

   execfile(script_filename, namespace, namespace)

   run()

   readMovieList()

   mid = CACHE_MID.addUnique(title, yearData)

   else: return self.add(key, miscData)

   self[key] = c

   self.flush()

   self._toDB(quiet)

   self._runCommand(l)

   CURS.executemany(self.sqlstr, self.converter(dataList))






Thanks.

/usr/lib/python2.5/site-packages/IMDbPY-4.3-py2.5.egg/imdb/parser/sql/__init__.py:119: UserWarning: Unable to import the cutils.ratcliff function. Searching names and titles using the "sql" data access system will be slower. /usr/lib/python2.5/site-packages/IMDbPY-4.3-py2.5.egg/imdb/parser/sql/__init__.py:326: UserWarning: Unable to import the cutils.soundex function. Searches of movie titles and person names will be a bit slower. # TIME dropping and recreating the database : 0min, 2sec (wall) 0min, 0sec (user) 0min, 0sec (system) File "/usr/lib/python2.5/site-packages/pkg_resources.py", line 448, in run_script File "/usr/lib/python2.5/site-packages/pkg_resources.py", line 1166, in run_script File "/usr/lib/python2.5/site-packages/IMDbPY-4.3-py2.5.egg/EGG-INFO/scripts/imdbpy2sql.py", line 2788, in <module> File "/usr/lib/python2.5/site-packages/IMDbPY-4.3-py2.5.egg/EGG-INFO/scripts/imdbpy2sql.py", line 2632, in run File "/usr/lib/python2.5/site-packages/IMDbPY-4.3-py2.5.egg/EGG-INFO/scripts/imdbpy2sql.py", line 1426, in readMovieList File "/usr/lib/python2.5/site-packages/IMDbPY-4.3-py2.5.egg/EGG-INFO/scripts/imdbpy2sql.py", line 1040, in addUnique File "/usr/lib/python2.5/site-packages/IMDbPY-4.3-py2.5.egg/EGG-INFO/scripts/imdbpy2sql.py", line 919, in add File "/usr/lib/python2.5/site-packages/IMDbPY-4.3-py2.5.egg/EGG-INFO/scripts/imdbpy2sql.py", line 829, in __setitem__ File "/usr/lib/python2.5/site-packages/IMDbPY-4.3-py2.5.egg/EGG-INFO/scripts/imdbpy2sql.py", line 852, in flush File "/usr/lib/python2.5/site-packages/IMDbPY-4.3-py2.5.egg/EGG-INFO/scripts/imdbpy2sql.py", line 1024, in _toDB File "/usr/lib/python2.5/site-packages/IMDbPY-4.3-py2.5.egg/EGG-INFO/scripts/imdbpy2sql.py", line 1028, in _runCommand psycopg2.DataError: неверная последовательность байт имя кодировки "UTF8": 0xc333 HINT: This error can also happen if the byte sequence does not match the encoding expected by the server, which is controlled by "client_encoding". The text at the "psycopg2.DataError:" says "invalid byte sequence for encoding "/UTF8/"" (sorry, but even using LANG=C does not make this text in english). I'm used psycopg version 1 before with the same error. client_encoding has been set to UTF8. Can't reproduce it on MySQL as i don't have one. Any suggestions what can it be and what can i do with it? Debian Lenny, PostgreSQL 8.3 from debian reps, IMDbPY 4.2, IMDbPY 4.3 (svn), python-psycopg version 1 and 2 from debian repositories. IMDbPY built without cutils and lxml support.

smime.p7s
Description: S/MIME Cryptographic Signature

------------------------------------------------------------------------------
Come build with us! The BlackBerry® Developer Conference in SF, CA
is the only developer event you need to attend this year. Jumpstart your
developing skills, take BlackBerry mobile applications to market and stay 
ahead of the curve. Join us from November 9-12, 2009. Register now!
http://p.sf.net/sfu/devconf
_______________________________________________
Imdbpy-help mailing list
Imdbpy-help@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/imdbpy-help