I was trying to load imdb data to mysql database using IMDBPY 5.1.
But I always encounter the following issues in the end. I need to have the
complete primary key + foreign key constraints in the schema. Could you
please give me some hints about what I might have done wrong?
building database indexes (this may take a while)
# TIME createIndexes() : 38min, 6sec (wall) 0min, 0sec (user) 0min,
0sec (system)
adding foreign keys (this may take a while)
ERROR caught exception creating a foreign key: Cannot add or update a
child row: a foreign key constraint fails (`imdb`.`#sql-65bf_d`, CONSTRAINT
`title_episode_of_id_exists` FOREIGN KEY (`episode_of_id`) REFERENCES
`title` (`id`))
ERROR caught exception creating a foreign key: Cannot add or update a
child row: a foreign key constraint fails (`imdb`.`#sql-65bf_e`, CONSTRAINT
`aka_title_movie_id_exists` FOREIGN KEY (`movie_id`) REFERENCES `title`
(`id`))
ERROR caught exception creating a foreign key: Cannot add or update a
child row: a foreign key constraint fails (`imdb`.`#sql-65bf_e`, CONSTRAINT
`cast_info_movie_id_exists` FOREIGN KEY (`movie_id`) REFERENCES `title`
(`id`))
ERROR caught exception creating a foreign key: Cannot add or update a
child row: a foreign key constraint fails (`imdb`.`#sql-65bf_e`, CONSTRAINT
`complete_cast_movie_id_exists` FOREIGN KEY (`movie_id`) REFERENCES `title`
(`id`))
ERROR caught exception creating a foreign key: Cannot add or update a
child row: a foreign key constraint fails (`imdb`.`#sql-65bf_e`, CONSTRAINT
`movie_keyword_movie_id_exists` FOREIGN KEY (`movie_id`) REFERENCES `title`
(`id`))
ERROR caught exception creating a foreign key: Cannot add or update a
child row: a foreign key constraint fails (`imdb`.`#sql-65bf_e`, CONSTRAINT
`movie_link_movie_id_exists` FOREIGN KEY (`movie_id`) REFERENCES `title`
(`id`))
ERROR caught exception creating a foreign key: Cannot add or update a
child row: a foreign key constraint fails (`imdb`.`#sql-65bf_e`, CONSTRAINT
`movie_info_movie_id_exists` FOREIGN KEY (`movie_id`) REFERENCES `title`
(`id`))
ERROR caught exception creating a foreign key: Cannot add or update a
child row: a foreign key constraint fails (`imdb`.`#sql-65bf_e`, CONSTRAINT
`movie_info_idx_movie_id_exists` FOREIGN KEY (`movie_id`) REFERENCES
`title` (`id`))
ERROR caught exception creating a foreign key: Cannot add or update a
child row: a foreign key constraint fails (`imdb`.`#sql-65bf_e`, CONSTRAINT
`movie_companies_movie_id_exists` FOREIGN KEY (`movie_id`) REFERENCES
`title` (`id`))
# TIME createForeignKeys() : 655min, 16sec (wall) 0min, 0sec (user)
0min, 0sec (system)
RESTORING imdbIDs values for movies... WARNING: unable to restore
imdbIDs using the temporary table (falling back to dbm): missing
"title_extract" table (ok if this is the first run)
WARNING: unable to restore imdbIDs (ok if this is the first run)
RESTORING imdbIDs values for people... WARNING: unable to restore
imdbIDs using the temporary table (falling back to dbm): missing
"name_extract" table (ok if this is the first run)
WARNING: unable to restore imdbIDs (ok if this is the first run)
RESTORING imdbIDs values for characters... WARNING: unable to restore
imdbIDs using the temporary table (falling back to dbm): missing
"char_name_extract" table (ok if this is the first run)
WARNING: unable to restore imdbIDs (ok if this is the first run)
RESTORING imdbIDs values for companies... WARNING: unable to restore
imdbIDs using the temporary table (falling back to dbm): missing
"company_name_extract" table (ok if this is the first run)
WARNING: unable to restore imdbIDs (ok if this is the first run)
The commands I used are as follows
1) Install all the required packages.
sudo apt-get install -y gcc python python-dev libssl-dev libxml2-dev
libxslt1-dev zlib1g-dev python-setuptools python-pip
easy_install -U SQLObject
pip install MySQL-python
2) Install IMDBPY.
cd [IMDBPY_parent_directory]
wget http://prdownloads.sourceforge.net/imdbpy/IMDbPY-5.1.tar.gz
tar -xzf IMDbPY-5.1.tar.gz
cd IMDbPY-5.1
python setup.py install
3) In mysql, create a database "imdb", and grant all privileges to "user"
with password "password".
CREATE DATABASE imdb;
GRANT ALL PRIVILEGES ON imdb.* TO 'user'@'localhost' IDENTIFIED BY
'password';
FLUSH PRIVILEGES;
4) Download all IMDB data.
mkdir [imdb_data_directory]
cd [imdb_data_directory]
wget -r --accept="*.gz" --no-directories --no-host-directories --level
1 ftp://ftp.fu-berlin.de/pub/misc/movies/database/
5) Load IMDB data to mysql.
cd [IMDBPY_parent_directory]/IMDbPY-5.1/bin
python imdbpy2sql.py -d [imdb_data_directory]
'mysql://user:password@localhost/imdb'
My settings are:
- python: 2.7
- Mysql: 5.7
- Ubuntu 16.04
I also tried on macOS 10.12 + mysql 5.7 + python 2.7 and had the same issue.
Thanks.
Best,
Mark
--
Zhongjun(Mark) Jin
Computer Science and Engineering Dept
University of Michigan, Ann Arbor
Email: markjin@umich,edu*,* markjin1...@gmail.com
Personal Website: https://markjin1990.github.io/
------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Imdbpy-help mailing list
Imdbpy-help@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/imdbpy-help
