[PERFORM] 答复: [PERFORM] Postgresql update op is very very slow

2008-06-26 Thread jay
within function yet? -邮件原件- 发件人: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] 代表 Heikki Linnakangas 发送时间: 2008年6月25日 18:11 收件人: jay 抄送: pgsql-performance@postgresql.org 主题: Re: [PERFORM] Postgresql update op is very very slow jay wrote: I've a table with about 34601755 rows ,when I

[PERFORM] Re: [PERFORM] 答复: [PERFORM] Postgresql update op is very very slow

2008-06-26 Thread Pavan Deolasee
2008/6/26 jay [EMAIL PROTECTED]: If we can do commit very 1000 row per round, it may resolve the problem. But PG not support transaction within function yet? Yeah, transaction control is not supported inside functions. There are some hacks using dblink to do transactions inside

Re: ??: [PERFORM] Postgresql update op is very very slow

2008-06-26 Thread Heikki Linnakangas
jay wrote: I know the problem, because there are about 35 million rows , which cost about 12G disk space and checkpoint segments use 64, but update operation is in one transaction which lead fast fill up the checkpoint segments and lead do checkpoints frequently, but checkpoints will

Re: [PERFORM] ??: Postgresql update op is very very slow

2008-06-26 Thread Holger Hoffstaette
Hi - I have been following this thread and find some of the recommendations really surprising. I understand that MVCC necessarily creates overhead, in-place updates would not be safe against crashes etc. but have a hard time believing that this is such a huge problem for RDBMS in 2008. How do

Re: [PERFORM] ??: Postgresql update op is very very slow

2008-06-26 Thread Craig Ringer
Holger Hoffstaette wrote: Hi - I have been following this thread and find some of the recommendations really surprising. I understand that MVCC necessarily creates overhead, in-place updates would not be safe against crashes etc. but have a hard time believing that this is such a huge problem

Re: [PERFORM] ??: Postgresql update op is very very slow

2008-06-26 Thread Andrew Sullivan
On Thu, Jun 26, 2008 at 02:40:59PM +0200, Holger Hoffstaette wrote: large databases treat mass updates? AFAIK both DB2 and Oracle use MVCC (maybe a different kind?) as well, but I cannot believe that large updates still pose such big problems. DB2 does not use MVCC. This is why lock

Re: [PERFORM] ??: Postgresql update op is very very slow

2008-06-26 Thread Andrew Sullivan
On Thu, Jun 26, 2008 at 09:16:25PM +0800, Craig Ringer wrote: I think Pg already does in place updates, or close, if the tuples being replaced aren't referenced by any in-flight transaction. I noticed a while ago that if I'm doing bulk load/update work, if there aren't any other

Re: [PERFORM] ??: Postgresql update op is very very slow

2008-06-26 Thread Mark Mielke
Holger Hoffstaette wrote: Hi - I have been following this thread and find some of the recommendations really surprising. I understand that MVCC necessarily creates overhead, in-place updates would not be safe against crashes etc. but have a hard time believing that this is such a huge problem

Re: [PERFORM] 答复: [PERFORM] Postgresql update op is very very slow

2008-06-26 Thread Tom Lane
jay [EMAIL PROTECTED] writes: I know the problem, because there are about 35 million rows , which cost about 12G disk space and checkpoint segments use 64, but update operation is in one transaction which lead fast fill up the checkpoint segments and lead do checkpoints frequently, but

[PERFORM] Re: [PERFORM] Re: [PERFORM] 答复: [PERFORM] Postgresql update op is very very slow

2008-06-26 Thread Scott Marlowe
2008/6/26 Pavan Deolasee [EMAIL PROTECTED]: 2008/6/26 jay [EMAIL PROTECTED]: If we can do commit very 1000 row per round, it may resolve the problem. But PG not support transaction within function yet? Yeah, transaction control is not supported inside functions. There are some

[PERFORM] Re: [PERFORM] 答复: [PERFORM] Postgresql update op is very very slow

2008-06-26 Thread Scott Marlowe
2008/6/26 Tom Lane [EMAIL PROTECTED]: jay [EMAIL PROTECTED] writes: I know the problem, because there are about 35 million rows , which cost about 12G disk space and checkpoint segments use 64, but update operation is in one transaction which lead fast fill up the checkpoint segments

Re: [PERFORM] ??: Postgresql update op is very very slow

2008-06-26 Thread Greg Smith
On Thu, 26 Jun 2008, Holger Hoffstaette wrote: How do large databases treat mass updates? AFAIK both DB2 and Oracle use MVCC (maybe a different kind?) as well An intro to the other approaches used by Oracle and DB2 (not MVCC) is at

Re: [PERFORM] Postgresql update op is very very slow

2008-06-25 Thread Rusty Conover
On Jun 24, 2008, at 9:12 PM, jay wrote: I've a table with about 34601755 rows ,when I execute 'update msg_table set type=0;' is very very slow, cost several hours, but still not complete? Why postgresql is so slowly? Is the PG MVCC problem? But I try it on Mysql, the same table and

Re: [PERFORM] Postgresql update op is very very slow

2008-06-25 Thread Heikki Linnakangas
jay wrote: I've a table with about 34601755 rows ,when I execute 'update msg_table set type=0;' is very very slow, cost several hours, but still not complete? Why postgresql is so slowly? Is the PG MVCC problem? Possibly. Because of MVCC, a full-table update will actually create a new

Re: [PERFORM] Postgresql update op is very very slow

2008-06-25 Thread Greg Smith
On Wed, 25 Jun 2008, jay wrote: Why postgresql is so slowly? Is the PG MVCC problem? Update is extremely intensive not just because of MVCC, but because a new version of all the rows are being written out. This forces both lots of database commits and lots of complicated disk I/O to

[PERFORM] Postgresql update op is very very slow

2008-06-24 Thread jay
I've a table with about 34601755 rows ,when I execute 'update msg_table set type=0;' is very very slow, cost several hours, but still not complete? Why postgresql is so slowly? Is the PG MVCC problem? But I try it on Mysql, the same table and rows, it only cost about 340 seconds. Any idea for