On Mar 19, Michael Liu <mikel...@gmail.com> wrote:
> I used imdbpy2sql to populate a local database with the IMDB data, but
> have some questions about the data.
Hi!
I take this as an opportunity to write some FAQs to put in the
documentation, since these questions came up a lot, lately. :-)
> In the titles table, the column imdb_id is empty. Am I missing a file
> I needed to download to fill this in? How can I get imdb_ids?
Q2: why the movieID (and other IDs) used in the 'sql' database are not
the same used on the IMDb.com site?
A2: first, a bit of nomenclature: we'll call "movieID" (or things like
"personID", for instance of the Person class) a unique identifier used
by IMDbPY to manage a single movie (or other kinds of object).
We'll call "imdbID" a unique identifier used, for the same kind
of data, by the IMDb.com site (i.e.: the 7-digit number in tt0094226,
as seen in the URL for "The Untouchables").
Using IMDbPY to access the web ('http' and 'mobile' data access
systems), movieIDs and imdbIDs are the same thing - beware that
in this case a movieID is a string, with the leading zeroes.
Unfortunately, populating a sql database with data from the plain
text data files, we don't have access to imdbIDs - since they are
not distributed at all - and so we have to made them by ourselves
(they are the 'id' column in tables like 'title' or 'name').
This mean that these values are valid only for your current database:
if you update it with a newer set of plain text data files, these IDs
will surely change (and, by the way, they are integers).
It's also obvious, now, that you can't exchange IDs between the
'http' (or 'mobile') data access system and 'sql', and in the same
way you can't use imdbIDs with your local database or vice-versa.
Q3: using a sql database, what's the imdb_id (or something like that)
column in tables like 'title', 'name' and so on?
A3: it's internally used by IMDbPY to remember the imdbID (the one
used by the web site - accessing the database you'll use the numeric
value of the 'id' column, as movieID) of a movie, once it stumbled
upon. This way, if IMDbPY is asked again about the imdbID of
a movie (or person, or ...), it doesn't have to contact again to
the web site. Notice that you have to access the sql database using
a user with write permission, to update it.
As a bonus, when possible, the values of these imdbIDs are saved
between updates of the sql database (using the imdbpy2sql.py script).
Beware that it's tricky and not always possible, but the script does
its best to succeed.
Q4: but what if I really need the imdbIDs, to use my database?
A4: no, you don't. Search for a title, get its information. Be happy!
Q5: I have a great idea: write a script to fetch all the imdbID from the
web site! Can't you do it?
A5: yeah, I can. But I won't. :-)
It would be somewhat easy to map every title on the web to its
imdbID, but there are still a lot of problems.
First of all, every user will end up doing it for its own copy
of the plain text data files (and this will make the imdbpy2sql.py
script painfully slow and prone to all sort of problems).
Moreover, the imdbIDs are unique and never reused, true, but movie
title _do_ change: to fix typos, override working titles, to cope
with a new movie with the same title release in the same year (not
to mention cancelled or postponed movies).
Besides that, we'd have to do the same for persons, characters and
companies. Believe me: it doesn't make sense.
Work on your local database using your movieIDs (or even better:
don't mind about movieIDs and think in terms of searches and Movie
instances!) and retrieve the imdbID only in the rare circumstances
when you really need them (see the next FAQ).
Repeat with me: I DON'T NEED ALL THE imdbIDs. :-)
> Without the imdb_id, is it possible for me to generate a link to a
> given movie on IMDB?
Q6: using a sql database, how can I convert a movieID (whose value
is valid only locally) to an imdbID (the ID used by the imdb.com site)?
A6: various functions can be used to convert a movieID (or personID or
other IDs) to the imdbID used by the seb site.
Example of code:
from imdb import IMDb
ia = IMDb('sql', uri=URI_TO_YOUR_SQL_DATABASE)
movie = ia.search_movie('The Untouchables')[0] # a Movie instance.
print 'The movieID for The Untouchables:', movie.movieID
print 'The imdbID used by the site:', ia.get_imdbMovieID(movie.movieID)
print 'Same ID, smarter function:', ia.get_imdbID(movie)
It goes without saying that get_imdbMovieID has some sibling
methods: get_imdbPersonID, get_imdbCompanyID and get_imdbCharacterID.
Also notice that the get_imdbID method is smater, and takes any kind
of instance (the other functions need a movieID, personID, ...)
Another method that will try to retrieve the imdbID is get_imdbURL,
which works like get_imdbID but returns an URL.
In case of problems, these methods will return None.
> Also, the online IMDB is aware of which titles are adult movies, but I
> don't see any similar column in my local database. How can I determine
> whether a movie is adult or not?
Read README.adult and see imdb/parser/sql/__init__.py: searching for
a title, it tries to guess if it's an adult title.
It can't be perfect and I don't assume any kind of responsibilities
on this matter. ;-)
> Lastly, online IMDB seems to know which movies are and aren't
> available on Amazon and Blockbuster. Is that in the database
> somewhere?
No.
Accessing the web ('http' and 'mobile'), there are parsers for
the 'amazon reviews' page, but these information are not published
in the plain text data files.
HTH,
--
Davide Alberani <davide.alber...@gmail.com> [GPG KeyID: 0x465BFD47]
http://www.mimante.net/
------------------------------------------------------------------------------
Download Intel® Parallel Studio Eval
Try the new software tools for yourself. Speed compiling, find bugs
proactively, and fine-tune applications for parallel performance.
See why Intel Parallel Studio got high marks during beta.
http://p.sf.net/sfu/intel-sw-dev
_______________________________________________
Imdbpy-help mailing list
Imdbpy-help@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/imdbpy-help
