Re: [HACKERS] Constant WAL replay

2005-04-23 Thread Neil Conway
Hans-Jürgen Schönig wrote: The idea: We are looking for a way to implement a synchronous single-master / multiple slaves systems. Meanwhile we are able to serialize / deserialize WAL records and send them to a group communication system which transports those records to the slave database. BTW,

Re: [HACKERS] Constant WAL replay

2005-04-23 Thread Tom Lane
=?ISO-8859-1?Q?Hans-J=FCrgen_Sch=F6nig?= <[EMAIL PROTECTED]> writes: > We have toyed around with PostgreSQL's WAL structure and we are > wondering whether it is (theoretically) possible to replay WAL records > coming from a remote host while a system is working in read only mode. There's been so

[HACKERS] Constant WAL replay

2005-04-23 Thread =?ISO-8859-1?Q?Hans-J=FCrgen_Sch=F6nig?=
We have toyed around with PostgreSQL's WAL structure and we are wondering whether it is (theoretically) possible to replay WAL records coming from a remote host while a system is working in read only mode. The idea: We are looking for a way to implement a synchronous single-master / multiple sl

Re: [HACKERS] Wierd performance issue with 8.1cvs

2005-04-23 Thread Tom Lane
Josh Berkus writes: > At OSDL we're seeing a wierd performance crash on 8.1cvs. Permit me a digression for a pet peeve... "Weird" is spelled "weird". Not "wierd". Yes, I know the nursery rhyme as well as you do --- i before e except after c, etc etc. But weird is spelled weirdly. Appropriate

Re: [HACKERS] [PERFORM] Bad n_distinct estimation; hacks suggested?

2005-04-23 Thread Tom Lane
Josh Berkus writes: > Overall, our formula is inherently conservative of n_distinct. That is, I > believe that it is actually computing the *smallest* number of distinct > values which would reasonably produce the given sample, rather than the > *median* one. This is contrary to the notes in

Re: [HACKERS] possible TODO: read-only tables, select from indexes

2005-04-23 Thread Paul Tillotson
2. Simple tables can be much smaller since you don't need most of the HeapTupleHeaderData. What you are talking about is not a "read only" table, it is a "non-MVCC" table. This is a much greater assault on the fundamental semantics of Postgres than it's being painted to be in this thr

Re: [HACKERS] [PERFORM] Bad n_distinct estimation; hacks suggested?

2005-04-23 Thread Andrew Dunstan
Josh Berkus said: > > > Well, unusual distributions are certainly tough. But I think the > problem exists even for relatively well-distributed populations. > Part of it is, I believe, the formula we are using: > > n*d / (n - f1 + f1*n/N) > [snip] > > This is so broken, in fact, that I'm wonderin

Re: [HACKERS] [PERFORM] Bad n_distinct estimation; hacks suggested?

2005-04-23 Thread Josh Berkus
People, > Can someone whose math is more recent than calculus in 1989 take a look at > that paper, and look at the formula toward the bottom of page 10, and see > if we are correctly interpreting it?    I'm particularly confused as to > what "q" and "d-sub-n" represent.  Thanks! Actually, I manag

Re: [HACKERS] [PERFORM] Bad n_distinct estimation; hacks suggested?

2005-04-23 Thread Josh Berkus
Greg, > I looked into this a while back when we were talking about changing the > sampling method. The conclusions were discouraging. Fundamentally, using > constant sized samples of data for n_distinct is bogus. Constant sized > samples only work for things like the histograms that can be analyze

Re: [HACKERS] possible TODO: read-only tables, select from indexes only.

2005-04-23 Thread Tom Lane
Alvaro Herrera <[EMAIL PROTECTED]> writes: > One idea that may be closer to reality might be implementing heaps that > behave as indexes (or indexes that behave as heaps), or clustered > indexes as some other database call them; the main idea being that at > the leaf nodes of the index, the whole h

Re: [HACKERS] possible TODO: read-only tables, select from indexes only.

2005-04-23 Thread Alvaro Herrera
On Sat, Apr 23, 2005 at 06:27:38PM -0400, Tom Lane wrote: > While I don't say it's impossible to do, I do think that the work and > semantic ugliness involved would outweigh the possible benefits. In > particular, there are other, more transparent ways of doing #1. One idea that may be closer to

Re: [HACKERS] possible TODO: read-only tables, select from indexes only.

2005-04-23 Thread Tom Lane
Ron Mayer <[EMAIL PROTECTED]> writes: > Is this a fair summary of the potential benefits of READ-ONLY > tables? (from both this thread and the archives): > 1. Index-only scans are made possible fairly easily because > you wouldn't need to check the heap for visibility. > 2. Simple tab

Re: [HACKERS] possible TODO: read-only tables, select from indexes

2005-04-23 Thread Ron Mayer
Simon Riggs wrote: As Ron says, until we have partitioning, it isn't as useful for DW. Well, it's a bit stronger than what I said. The last big DW I dealt with did have previous quarter's data archived into different tables, and those could be marked read-only. Also, quite a few of our ID<->Valu

Re: [HACKERS] possible TODO: read-only tables, select from indexes only.

2005-04-23 Thread Ron Mayer
Bruce Momjian wrote: See this TODO: * Allow data to be pulled directly from indexes I think this is the direction we should be heading because it has more general usefulness. I think read-only tables would have a few different types of general usefulness in addition to enabling index scans. Is thi

Re: [HACKERS] Postgres: pg_hba.conf, md5, pg_shadow, encrypted passwords

2005-04-23 Thread Stephen Frost
* Kevin Brown ([EMAIL PROTECTED]) wrote: > Stephen Frost wrote: > > Unfortunately, Postgres doesn't currently encrypt pg_shadow and even if > > it did you'd have to have the password stored on disk somewhere in the > > clear if you wanted the database to start automatically, which is more > > impo

Re: [HACKERS] Postgres: pg_hba.conf, md5, pg_shadow, encrypted

2005-04-23 Thread Stephen Frost
* Antoine Martin ([EMAIL PROTECTED]) wrote: > Basically, multiple input data that have the same output hash, which is > of no use when what you are trying to find is the input. > Finding collisions quicker for a known input is one thing, but that is > not going to reduce the search space, not even

Re: [HACKERS] Postgres: pg_hba.conf, md5, pg_shadow, encrypted passwords

2005-04-23 Thread Kevin Brown
Stephen Frost wrote: > Unfortunately, Postgres doesn't currently encrypt pg_shadow and even if > it did you'd have to have the password stored on disk somewhere in the > clear if you wanted the database to start automatically, which is more > important if you havn't got backup databases and whatno