Re: [HACKERS] WAL Bypass for indexes

2006-04-02 Thread Tom Lane
"Martin Scholes" <[EMAIL PROTECTED]> writes: > Ok Tom, I stand corrected. > I downloaded the latest snapshot and both scenarios (normal and WAL bypass = > for indexes) produced between 185 and 230 tps on my machine. > The lesson here is that whatever WAL magic has been performed on the latest = >

Re: [HACKERS] WAL Bypass for indexes

2006-04-02 Thread Jonah H. Harris
On 4/2/06, Martin Scholes <[EMAIL PROTECTED]> wrote: > The lesson here is that whatever WAL magic has been performed on the latest > release gives over 100% speedup, and the speedup is so good that skipping > WAL for indexes does basically nothing. No need for me to reply to earlier messages :) -

Re: [HACKERS] semaphore usage "port based"?

2006-04-02 Thread Marc G. Fournier
Thanks all ... have moved this to just the freebsd-stable list, since I don't imagine most here are interested in FreeBSD :( On Mon, 3 Apr 2006, Andrew Thompson wrote: On Sun, Apr 02, 2006 at 11:41:01PM -0400, Kris Kennaway wrote: On Mon, Apr 03, 2006 at 12:30:58AM -0300, Marc G. Fournier w

[HACKERS] Back, heading to Boston

2006-04-02 Thread Bruce Momjian
FYI, I returned my vacation on Thursday and have read a little email. I am heading to Boston for Linuxworld for Tuesday-Thursday, so I might not be caught up with email requests until the week after. -- Bruce Momjian http://candle.pha.pa.us + If your life is a hard drive, Christ can be yo

Re: [HACKERS] semaphore usage "port based"?

2006-04-02 Thread Marc G. Fournier
On Sun, 2 Apr 2006, Kris Kennaway wrote: No-one is taking a position of being "uninterested", so please don't be hasty to reciprocate. I just posted it off the -hackers list, but there is an ancient patch in the FreeBSD queue for implementing Private IPCs for Jails ... not sure why it was ne

Re: [HACKERS] semaphore usage "port based"?

2006-04-02 Thread Marc G. Fournier
On Sun, 2 Apr 2006, Kris Kennaway wrote: On Sun, Apr 02, 2006 at 11:17:49PM -0400, Tom Lane wrote: Kris Kennaway <[EMAIL PROTECTED]> writes: On Sun, Apr 02, 2006 at 11:08:11PM -0400, Tom Lane wrote: If this is the story, then FBSD have broken their system and must revert their change. They d

Re: [HACKERS] semaphore usage "port based"?

2006-04-02 Thread Tom Lane
Kris Kennaway <[EMAIL PROTECTED]> writes: > On Sun, Apr 02, 2006 at 11:17:49PM -0400, Tom Lane wrote: >> I have no objection to doing that, so long as you are actually doing it >> correctly. This example shows that each jail must have its own SysV >> semaphore key space, else information leaks any

Re: [HACKERS] semaphore usage "port based"?

2006-04-02 Thread Tom Lane
Kris Kennaway <[EMAIL PROTECTED]> writes: > On Sun, Apr 02, 2006 at 11:08:11PM -0400, Tom Lane wrote: >> If this is the story, then FBSD have broken their system and must revert >> their change. They do not have kernel behavior that totally hides the >> existence of the other process, and therefor

Re: [HACKERS] semaphore usage "port based"?

2006-04-02 Thread Tom Lane
"Marc G. Fournier" <[EMAIL PROTECTED]> writes: > 'k, try this one ... looks better, actually has semget() calls in it :) OK, here's our problem: 84250: semget(0x52e2c1,0x11,0x780) ERR#17 'File exists' This is InternalIpcSemaphoreCreate failing because of key collision. As it should

Re: [HACKERS] WAL Bypass for indexes

2006-04-02 Thread Martin Scholes
Title: Converted from Rich Text Ok Tom, I stand corrected.  I downloaded the latest snapshot and both scenarios (normal and WAL bypass for indexes) produced between 185 and 230 tps on my machine.  The lesson here is that whatever WAL magic has been performed on the latest release give

Re: [HACKERS] semaphore usage "port based"?

2006-04-02 Thread Marc G. Fournier
Sent offlist ... On Sun, 2 Apr 2006, Tom Lane wrote: "Marc G. Fournier" <[EMAIL PROTECTED]> writes: On Sun, 2 Apr 2006, Tom Lane wrote: OK, could we see strace (or whatever BSD calls it) output for the second postmaster? I'd like to see exactly what results it's getting for the kernel calls

Re: [HACKERS] WAL Bypass for indexes

2006-04-02 Thread Christopher Kings-Lynne
Martin's proposal at least looks sensible; he just hasn't quite made the case that it's worth doing. If you're running a system that hardly ever crashes, you might be willing to accept index rebuilds during crash recovery, especially for indexes on relatively small, but frequently updated, tables

Re: [HACKERS] semaphore usage "port based"?

2006-04-02 Thread Tom Lane
"Marc G. Fournier" <[EMAIL PROTECTED]> writes: > On Sun, 2 Apr 2006, Tom Lane wrote: >> OK, could we see strace (or whatever BSD calls it) output for the second >> postmaster? I'd like to see exactly what results it's getting for the >> kernel calls it makes during IpcSemaphoreCreate. > 'k, dont'

Re: [HACKERS] WAL Bypass for indexes

2006-04-02 Thread Tom Lane
"Jonah H. Harris" <[EMAIL PROTECTED]> writes: > I guess I can think of a few instances, but none that I would've > chosen to use it in. IIRC, it's also more likely to increase the cost > of checkpointing and/or require a good amount of bgwriter tuning. How so? AFAICS it'd just eliminate WAL outp

Re: [HACKERS] semaphore usage "port based"?

2006-04-02 Thread Marc G. Fournier
On Sun, 2 Apr 2006, Tom Lane wrote: "Marc G. Fournier" <[EMAIL PROTECTED]> writes: On Sun, 2 Apr 2006, Tom Lane wrote: BTW, even before doing that, you should look at "ipcs -s" output to try to get a clue what's going on. The EINVAL failures may be because the second postmaster to start delet

Re: [HACKERS] Suggestion: Which Binary?

2006-04-02 Thread David Wheeler
On Apr 2, 2006, at 17:47, Robert Treat wrote: ISTM that by any measure of the general population, David Wheeler is a hard-core geek. :-) Actually by most measures of the "programming/ oss community" he is a hard core geek. But he still got tripped up by this. A lot of people never get pas

Re: [HACKERS] Suggestion: Which Binary?

2006-04-02 Thread Tom Lane
Robert Treat <[EMAIL PROTECTED]> writes: > ISTM that by any measure of the general population, David Wheeler is a > hard-core geek. :-) Actually by most measures of the "programming/oss > community" he is a hard core geek. But he still got tripped up by this. A > lot of people never get pass

Re: [HACKERS] semaphore usage "port based"?

2006-04-02 Thread Tom Lane
"Marc G. Fournier" <[EMAIL PROTECTED]> writes: > On Sun, 2 Apr 2006, Tom Lane wrote: >> BTW, even before doing that, you should look at "ipcs -s" output to try >> to get a clue what's going on. The EINVAL failures may be because the >> second postmaster to start deletes the semaphores created by

Re: [HACKERS] WAL Bypass for indexes

2006-04-02 Thread Jonah H. Harris
> AFAICS there are no circumstances, ever, in which update-in-place is > "safe". (No transaction can guarantee that it will commit.) In our case, it is totally safe. I'd certainly like to discuss it with you sometime at the anniversary. > Martin's proposal at least looks sensible; he just hasn'

Re: [HACKERS] semaphore usage "port based"?

2006-04-02 Thread Marc G. Fournier
On Sun, 2 Apr 2006, Tom Lane wrote: I wrote: Look at IpcSemaphoreCreate and InternalIpcSemaphoreCreate in src/backend/port/sysv_sema.c. It may be worth stepping through them with gdb to see what the semget calls are returning. BTW, even before doing that, you should look at "ipcs -s" output

Re: [HACKERS] uh-oh, buildfarm all red

2006-04-02 Thread Tom Lane
Andrew Dunstan <[EMAIL PROTECTED]> writes: > Tom Lane wrote: >> Looks like someone has re-introduced a test case that fails on the day >> of a US daylight-savings transition. > Hmm ... this test has actually been in there for ages. > SELECT CAST(CAST(date 'today' + time with time zone '01:30' >

Re: [HACKERS] Suggestion: Which Binary?

2006-04-02 Thread Robert Treat
On Saturday 01 April 2006 10:47, Andrew Dunstan wrote: > Jim C. Nasby wrote: > >On Fri, Mar 31, 2006 at 07:01:18PM -0500, Tom Lane wrote: > >>David Wheeler <[EMAIL PROTECTED]> writes: > >>>Yes, but even the environment variables get me what I want. I > >>>therefore respectfully submit the attached

Re: [HACKERS] WAL Bypass for indexes

2006-04-02 Thread Tom Lane
"Jonah H. Harris" <[EMAIL PROTECTED]> writes: > we're working on a prototype to reduce WAL I/O and index updates in a > large percentage of OLTP situations by employing an update-in-place > under *safe* conditions. AFAICS there are no circumstances, ever, in which update-in-place is "safe". (No t

Re: [HACKERS] uh-oh, buildfarm all red

2006-04-02 Thread Andrew Dunstan
Tom Lane wrote: Looks like someone has re-introduced a test case that fails on the day of a US daylight-savings transition. I vote we take that test case out again. Any objections? Hmm ... this test has actually been in there for ages. SELECT CAST(CAST(date 'today' + time wi

Re: [HACKERS] WAL Bypass for indexes

2006-04-02 Thread Jonah H. Harris
On 4/2/06, Martin Scholes <[EMAIL PROTECTED]> wrote: > I have long believed that the bottleneck in transaction-oriented systems is > the writing of the indexes, complete with splits and merges. A single update > to one field of a heavily-indexed table could cause dozens of index writes > to cascade

Re: [HACKERS] WAL Bypass for indexes

2006-04-02 Thread Tom Lane
"Martin Scholes" <[EMAIL PROTECTED]> writes: > I did some informal testing using pgbench on v8.07. First, I ran pgbench = > normally with 75 users doing 100 transactions, full vacuuming between runs. = > My machine consistently gave me 92 tps. > As an experiment, I commented out of the btree ind

[HACKERS] WAL Bypass for indexes

2006-04-02 Thread Martin Scholes
Title: Converted from Rich Text I have followed the discussion from 3 months ago on WAL bypass and wanted to offer some more information.  I have long believed that the bottleneck in transaction-oriented systems is the writing of the indexes, complete with splits and merges. A single u

Re: [HACKERS] pg_class catalog question...

2006-04-02 Thread Jonah H. Harris
On 4/2/06, Tom Lane <[EMAIL PROTECTED]> wrote: > Changing that is not "impossible", but the level of pain vastly exceeds > what this feature would be worth. I really like the wording, "the level of pain"... so true :) > you could create one or more fixed-size datatypes (ie, with various > positiv

Re: [HACKERS] semaphore usage "port based"?

2006-04-02 Thread Tom Lane
I wrote: > Look at IpcSemaphoreCreate and InternalIpcSemaphoreCreate in > src/backend/port/sysv_sema.c. It may be worth stepping through them > with gdb to see what the semget calls are returning. BTW, even before doing that, you should look at "ipcs -s" output to try to get a clue what's going o

Re: [HACKERS] semaphore usage "port based"?

2006-04-02 Thread Tom Lane
"Marc G. Fournier" <[EMAIL PROTECTED]> writes: > Or, more simply, I think ... is there somewhere in the Semaphore code that > is using the port # as a 'seed'? We use the port number as a basis for selecting the semaphore key (see semget(2)). There is code in there to pick a different key value i

Re: [HACKERS] pg_class catalog question...

2006-04-02 Thread Tom Lane
"Jonah H. Harris" <[EMAIL PROTECTED]> writes: > On 4/2/06, Tom Lane <[EMAIL PROTECTED]> wrote: >> If you're expecting that you'll be able to write BYTEA(n) and avoid >> storing a length word, you'll find that it's not a trivial matter. > It may not be trivial, but it's certainly not impossible. A

Re: [HACKERS] pg_class catalog question...

2006-04-02 Thread Jonah H. Harris
On 4/2/06, Tom Lane <[EMAIL PROTECTED]> wrote: > If you're expecting that you'll be able to write BYTEA(n) and avoid > storing a length word, you'll find that it's not a trivial matter. It may not be trivial, but it's certainly not impossible. -- Jonah H. Harris, Database Internals Architect Ent

Re: [HACKERS] semaphore usage "port based"?

2006-04-02 Thread Marc G. Fournier
'k, an excerpt from a thread on the freebsd lists ... I'm not sure how to answer: On Sun, Apr 02, 2006 at 05:24:10PM -0300, Marc G. Fournier wrote: On Sun, 2 Apr 2006, Kris Kennaway wrote: >>Right, but why are they doing it *consistently* in FreeBSD 6.x, when they >>never did it in Fr

Re: [HACKERS] psql \c error

2006-04-02 Thread Neil Conway
On Thu, 2006-03-30 at 11:20 +1000, Philip Yarra wrote: > Hi folks, I've found that CVS HEAD psql's \c doesn't quite behave as expected > when postmaster is listening on non-default port. I've committed a patch to HEAD that should improve this behavior. Let me know if the current behavior is still

Re: [HACKERS] Slony-I for circular replication

2006-04-02 Thread Gustavo Tonini
The isbn is 0-13-659707-6. I'm very interesting in study (and implement) a generic framework for replication in a project of the university. Gustavo. On 4/1/06, Guido Barosio <[EMAIL PROTECTED]> wrote: > Heya, > >Do you have the isbn of the book? Would be great. > > Cheers, > G.- > > On 4/1/0

[HACKERS] semaphore usage "port based"?

2006-04-02 Thread Marc G. Fournier
I've got an odd issue that I'm not sure how to fix ... or, if fixing is even possible ... I just put into place a FreeBSD 6.x server ... it has 2 jails running on it, and inside of each, I'm trying to run a PostgreSQL 7.4.12 server (OpenACS requirement, no choice there) ... Now, on my olde

Re: [HACKERS] pg_class catalog question...

2006-04-02 Thread Tom Lane
"Jonah H. Harris" <[EMAIL PROTECTED]> writes: > Yep. However, I've wanted to add a constrained, fixed-length version > of bytea for some time now; it's just not high on my priority list. If you're expecting that you'll be able to write BYTEA(n) and avoid storing a length word, you'll find that i

Re: [HACKERS] pg_class catalog question...

2006-04-02 Thread Jonah H. Harris
On 4/2/06, Tom Lane <[EMAIL PROTECTED]> wrote: > bytea does that. Yep. However, I've wanted to add a constrained, fixed-length version of bytea for some time now; it's just not high on my priority list. At EnterpriseDB, we've actually had a lot of customers who would prefer a constrained bytea (

Re: [HACKERS] pg_class catalog question...

2006-04-02 Thread Tom Lane
Thomas Hallgren <[EMAIL PROTECTED]> writes: > Jim C. Nasby wrote: >> Well, hex is much easier to deal with in many regards than raw bytes, >> though. But yes, the idea is that you'd just store raw bytes on disk. >> byte or octet would work fine if they existed. >> > IIRC, Oracle actually uses the

Re: [HACKERS] pg_class catalog question...

2006-04-02 Thread Thomas Hallgren
Jim C. Nasby wrote: On Sat, Apr 01, 2006 at 05:42:34PM +0200, Thomas Hallgren wrote: Why not simply a fixed number of bytes, i.e. byte(16) or octet(16)? Hexadecimal is just a convenient human-readable representation. Well, hex is much easier to deal with in many regards than raw bytes,

Re: [HACKERS] pg_class catalog question...

2006-04-02 Thread Jim C. Nasby
On Sat, Apr 01, 2006 at 05:42:34PM +0200, Thomas Hallgren wrote: > Jim C. Nasby wrote: > >On Fri, Mar 31, 2006 at 11:29:15AM -0500, Tom Lane wrote: > >>This argument falls flat when you consider that the width of a CHAR > >>entry is measured in characters, not bytes, and therefore its physical > >>

[HACKERS] uh-oh, buildfarm all red

2006-04-02 Thread Tom Lane
Looks like someone has re-introduced a test case that fails on the day of a US daylight-savings transition. I vote we take that test case out again. Any objections? regards, tom lane ---(end of broadcast)--- TIP 4: Have you

Re: [HACKERS] Problem compiling with mingw

2006-04-02 Thread Qingqing Zhou
"Juan Manuel Diaz Lara" <[EMAIL PROTECTED]> wrote > > bootparse.y:101:10: "b4_file_name" is not a valid filename > > bison-2.1.exe --> d:\gnuwin32 > Maybe it is bison's problem. I always use 1.875 since it has been there long enough && powerful enough ... Regards, Qingqing --

Re: [HACKERS] [COMMITTERS] pgsql: Correct some errors and do some SGML

2006-04-02 Thread Alvaro Herrera
Alvaro Herrera wrote: > Neil Conway wrote: > > On Sun, 2006-04-02 at 00:02 -0400, Neil Conway wrote: > > > Correct some errors and do some SGML police work on the reference pages > > > for REASSIGN OWNED and DROP OWNED. > > > > BTW, I notice that the patch adding these commands also neglected to >