Re: [GENERAL] insert only unique values in to a table, ignore rest?

2007-01-08 Thread George Nychis
Jeremy Haile wrote: >> Note that things will go faster if you do your initial data load using >> "copy from stdin" for the initial bulk data load. individual inserts in >> postgresql are quite costly compared to mysql. It's the transactional >> overhead. by grouping them together you can make

Re: [GENERAL] insert only unique values in to a table, ignore rest?

2007-01-08 Thread Jeremy Haile
> Note that things will go faster if you do your initial data load using > "copy from stdin" for the initial bulk data load. individual inserts in > postgresql are quite costly compared to mysql. It's the transactional > overhead. by grouping them together you can make things much faster. > cop

Re: [GENERAL] insert only unique values in to a table, ignore rest?

2007-01-08 Thread Scott Marlowe
On Mon, 2007-01-08 at 15:59, George Nychis wrote: > Scott Marlowe wrote: > > On Mon, 2007-01-08 at 15:52, George Nychis wrote: > >> Scott Marlowe wrote: > >>> On Mon, 2007-01-08 at 14:58, George Nychis wrote: > Hi, > > I have approximately 2 billion data entries that I would like to

Re: [GENERAL] insert only unique values in to a table, ignore rest?

2007-01-08 Thread George Nychis
Scott Marlowe wrote: > On Mon, 2007-01-08 at 15:52, George Nychis wrote: >> Scott Marlowe wrote: >>> On Mon, 2007-01-08 at 14:58, George Nychis wrote: Hi, I have approximately 2 billion data entries that I would like to insert into a database. Each entry consists of: >>>

Re: [GENERAL] insert only unique values in to a table, ignore rest?

2007-01-08 Thread Scott Marlowe
On Mon, 2007-01-08 at 15:52, George Nychis wrote: > Scott Marlowe wrote: > > On Mon, 2007-01-08 at 14:58, George Nychis wrote: > >> Hi, > >> > >> I have approximately 2 billion data entries that I would like to insert > >> into a database. > >> Each entry consists of: > >> INT BOOLEAN INT BOOLEAN

Re: [GENERAL] insert only unique values in to a table, ignore rest?

2007-01-08 Thread George Nychis
Scott Marlowe wrote: > On Mon, 2007-01-08 at 14:58, George Nychis wrote: >> Hi, >> >> I have approximately 2 billion data entries that I would like to insert into >> a database. >> Each entry consists of: >> INT BOOLEAN INT BOOLEAN >> >> I want to populate a table such that it only contains the

Re: [GENERAL] insert only unique values in to a table, ignore rest?

2007-01-08 Thread Scott Marlowe
On Mon, 2007-01-08 at 14:58, George Nychis wrote: > Hi, > > I have approximately 2 billion data entries that I would like to insert into > a database. > Each entry consists of: > INT BOOLEAN INT BOOLEAN > > I want to populate a table such that it only contains the unique rows, all > other data

[GENERAL] insert only unique values in to a table, ignore rest?

2007-01-08 Thread George Nychis
Hi, I have approximately 2 billion data entries that I would like to insert into a database. Each entry consists of: INT BOOLEAN INT BOOLEAN I want to populate a table such that it only contains the unique rows, all other data should be thrown out. I would say a significant amount of the inser