Re: [HACKERS] RFC: Very large scale postgres support

2004-02-28 Thread ElPeddy
: [HACKERS] RFC: Very large scale postgres support Recently I was tasked with creating a distribution system for postgres nodes here at work. This would allow us to simply bring up a new box, push postgres to it, and have a new database. At the same time, we have started to approach the limits

Re: [HACKERS] RFC: Very large scale postgres support

2004-02-09 Thread Josh Berkus
Alex, I find myself wondering what other people are doing with postgres that this doesn't seem to have come up. When one searches for postgres clustering on google, they will find lots of HA products. However, nobody seems to be attempting to create very high throughput clusters. Have you

Re: [HACKERS] RFC: Very large scale postgres support

2004-02-09 Thread Chris
That's what I said, and what I meant. Ten billion transactions equates to 115,740 transactions per second. Have you tried to look at the scientific comunity? CERN has setups that produce such large amounts of data - try searching google for

Re: [HACKERS] RFC: Very large scale postgres support

2004-02-09 Thread Keith Bottner
-Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Alex J. Avriette Sent: Saturday, February 07, 2004 12:29 PM To: [EMAIL PROTECTED] Subject: [HACKERS] RFC: Very large scale postgres support Recently I was tasked with creating a distribution system for postgres

Re: [HACKERS] RFC: Very large scale postgres support

2004-02-09 Thread Andreas Pflug
Keith Bottner wrote: Alex, I agree that this is something that is worth spending time on. This resembles the Oracle RAC (Real Application Cluster). While other people may feel that the amount of data is unreasonable I have a similar problem that will only be solved using such a solution. In

Re: [HACKERS] RFC: Very large scale postgres support

2004-02-09 Thread Keith Bottner
. Avriette'; [EMAIL PROTECTED] Subject: Re: [HACKERS] RFC: Very large scale postgres support Keith Bottner wrote: Alex, I agree that this is something that is worth spending time on. This resembles the Oracle RAC (Real Application Cluster). While other people may feel that the amount of data

Re: [HACKERS] RFC: Very large scale postgres support

2004-02-09 Thread Andreas Pflug
Keith Bottner wrote: I understand your position Andreas and respect your opinion; maybe what I have identified as requirements is what you are specifying as *real* issues. I hope so, because I to would like to avoid unnecessary dbms efforts. You got me very right. I didn't mean to declare high

Re: [HACKERS] RFC: Very large scale postgres support

2004-02-08 Thread Andreas Pflug
Alex J. Avriette wrote: I feel that it would be a very good thing if some thinking on this subject was done. In the future, people will hopefully begin using postgres for more intense applications. We are looking at perhaps many tens of billions of transactions per day within the next year or

Re: [HACKERS] RFC: Very large scale postgres support

2004-02-08 Thread Alex J. Avriette
On Sun, Feb 08, 2004 at 08:07:14PM +0100, Andreas Pflug wrote: I feel that it would be a very good thing if some thinking on this subject was done. In the future, people will hopefully begin using postgres for more intense applications. We are looking at perhaps many tens of billions of

Re: [HACKERS] RFC: Very large scale postgres support

2004-02-08 Thread Rod Taylor
The fact is, there are situations in which such extreme traffic is warranted. My concern is that I am not able to use postgres in such situations because it cannot scale to that level. I feel that it would be possible to reach that level with support in the postmaster for replication.

Re: [HACKERS] RFC: Very large scale postgres support

2004-02-08 Thread Alex J. Avriette
On Sun, Feb 08, 2004 at 08:01:38PM -0500, Rod Taylor wrote: Replication won't help if those are all mostly write transactions. If a small percentage, even 1% would be challenging, is INSERTS, UPDATES or DELETES, master / slave replication might get you somewhere. There is no way on earth we

Re: [HACKERS] RFC: Very large scale postgres support

2004-02-08 Thread Rod Taylor
On Sun, 2004-02-08 at 21:01, Alex J. Avriette wrote: On Sun, Feb 08, 2004 at 08:01:38PM -0500, Rod Taylor wrote: Replication won't help if those are all mostly write transactions. If a small percentage, even 1% would be challenging, is INSERTS, UPDATES or DELETES, master / slave

Re: [HACKERS] RFC: Very large scale postgres support

2004-02-08 Thread Rod Taylor
replication. Am I hearing that nobody believes scalability is a concern? I think many of us would like to see features that would allow large scale installations to be more practical. I also think most of us would agree that the current graft-on replication methods are sub-ideal. You really

Re: [HACKERS] RFC: Very large scale postgres support

2004-02-08 Thread Alex J. Avriette
On Sun, Feb 08, 2004 at 09:20:07PM -0500, Rod Taylor wrote: On Sun, 2004-02-08 at 21:01, Alex J. Avriette wrote: On Sun, Feb 08, 2004 at 08:01:38PM -0500, Rod Taylor wrote: Replication won't help if those are all mostly write transactions. If a small percentage, even 1% would be

[HACKERS] RFC: Very large scale postgres support

2004-02-07 Thread Alex J. Avriette
Recently I was tasked with creating a distribution system for postgres nodes here at work. This would allow us to simply bring up a new box, push postgres to it, and have a new database. At the same time, we have started to approach the limits of what we can do with postgres on one machine. Our