On Thu, Oct 16, 2008 at 09:21:54AM +1100, Brett Henderson wrote:
Just a quick stab in the dark as I do know almost nothing about osmosis and MySQL: If you create the indices before loading data, they will be constantly rebuilt as you add each row => would explain the exponential slow-down. The PostgreSQL manual has a section about optimizing database imports giving some useful tips & tricks; I'd guess the same is true for the MySQL one.The biggest problem I found wasn't the actual processing of INSERT statements, it was MySQL scaling non-linearly with the number of rows.MyISAM tables are very fast to import regardless of number of rows, but InnoDB seems to slow down as the number of rows increase. I'm surprisedloading with LOAD DATA INFILE fixes that.
If osmosis already defers index creation (including referential constraints which usually build indices implicitly), just ignore this mail. :)
CU Sascha -- http://sascha.silbe.org/ http://www.infra-silbe.de/
signature.asc
Description: Digital signature
_______________________________________________ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
