Most modern databases have some kind of "block-load" API for efficiently
importing large chunks of data into their tables.
It's really only the most simple stuff that __doesn't__ slow down when
importing using INSERT.
Although it should plateau to a constant time/INSERT pretty quickly.
Brett Hend
Sascha Silbe wrote:
> On Thu, Oct 16, 2008 at 09:21:54AM +1100, Brett Henderson wrote:
>
>> The biggest problem I found wasn't the actual processing of INSERT
>> statements, it was MySQL scaling non-linearly with the number of rows.
>> MyISAM tables are very fast to import regardless of number of r
Joachim Zobel wrote:
> Am Donnerstag, den 16.10.2008, 09:21 +1100 schrieb Brett Henderson:
>
>> Do you have any numbers to show what sort of performance increase can
>> be expected.
>>
>
> I can load europe.osm into the current_* tables in less than 12 hours
> (on my laptop - overpowered
On Thu, Oct 16, 2008 at 09:21:54AM +1100, Brett Henderson wrote:
The biggest problem I found wasn't the actual processing of INSERT
statements, it was MySQL scaling non-linearly with the number of rows.
MyISAM tables are very fast to import regardless of number of rows,
but
InnoDB seems to slow
Am Donnerstag, den 16.10.2008, 09:21 +1100 schrieb Brett Henderson:
> Do you have any numbers to show what sort of performance increase can
> be expected.
I can load europe.osm into the current_* tables in less than 12 hours
(on my laptop - overpowered CPU, 5400 disk).
> If it's significant I
Hi Joachim,
Nice work, any info on how to improve load speed is much appreciated.
On Thu, Oct 16, 2008 at 6:15 AM, Joachim Zobel <[EMAIL PROTECTED]>wrote:
> Hi.
>
> I have written a utility named scabies to load data from osm files into
> a mysql database. See
>
> http://www.heute-morgen.de/scab
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA512
Joachim Zobel schreef:
> Hi.
>
> I have written a utility named scabies to load data from osm files into
> a mysql database. See
>
> http://www.heute-morgen.de/scabies/
>
> Usage is rather straightforward, but you need lots of disk space.
> scabie
Hi.
I have written a utility named scabies to load data from osm files into
a mysql database. See
http://www.heute-morgen.de/scabies/
Usage is rather straightforward, but you need lots of disk space.
scabies_parse.pl parses the osm file and creates files that are then
loaded by scabies_load.sql
8 matches
Mail list logo