Re: [HACKERS] Archiver behavior at shutdown

2008-01-04 Thread Fujii Masao
Simon Riggs wrote: My original one line change described on bug 3843 seems like the best solution for 8.3. +1 Is this change in time for RC1? -- Fujii Masao NIPPON TELEGRAPH AND TELEPHONE CORPORATION NTT Open Source Software Center TEL (03)5860-5115 FAX (03)5463-5490

Re: [HACKERS] Dynamic Partitioning using Segment Visibility Maps

2008-01-04 Thread Richard Huxton
Simon Riggs wrote: We would keep a dynamic visibility map at *segment* level, showing which segments have all rows as 100% visible. No freespace map data would be held at this level. Small dumb-user question. I take it you've considered some more flexible consecutive-run-of-blocks unit of

[HACKERS] Problem with PgTcl auditing function on trigger

2008-01-04 Thread Glyn Astill
Hi people, I've tried posting on the general list about this, but I never get a reply, so I'm trying here. I have a function that is run each time an INSERT, DELETE or UPDATE happens on a row and log into an audit table. It is based on the info here:

Re: [HACKERS] Dynamic Partitioning using Segment Visibility Maps

2008-01-04 Thread Simon Riggs
On Fri, 2008-01-04 at 10:22 +, Richard Huxton wrote: Simon Riggs wrote: We would keep a dynamic visibility map at *segment* level, showing which segments have all rows as 100% visible. No freespace map data would be held at this level. Small dumb-user question. I take it you've

Re: [HACKERS] Dynamic Partitioning using Segment Visibility Maps

2008-01-04 Thread Simon Riggs
On Fri, 2008-01-04 at 13:06 +0530, Gokulakannan Somasundaram wrote: a) This proposal would work for the kind of table organizations which are currently partitioned and maintained based on some kind of timestamp. Consider one of the use-case. A large Retail firm has a lot of stores. DBA

Re: [HACKERS] Dynamic Partitioning using Segment Visibility Maps

2008-01-04 Thread Richard Huxton
Simon Riggs wrote: On Fri, 2008-01-04 at 10:22 +, Richard Huxton wrote: Simon Riggs wrote: We would keep a dynamic visibility map at *segment* level, showing which segments have all rows as 100% visible. No freespace map data would be held at this level. Small dumb-user question. I take

[HACKERS] Problem with PgTcl auditing function on trigger

2008-01-04 Thread Glyn Astill
Hi people, I've tried posting on the general list about this, but I never get a reply, so I'm trying here. I have a function that is run each time an INSERT, DELETE or UPDATE happens on a row and log into an audit table. It is based on the info here:

Re: [HACKERS] Dynamic Partitioning using Segment Visibility Maps

2008-01-04 Thread Simon Riggs
On Fri, 2008-01-04 at 13:29 +0100, Markus Schiltknecht wrote: Given that we are operating on segments here, to which the DBA has very limited information and access, I prefer the term Segment Exclusion. I think of that as an optimization of sequential scans on tables with the above

Re: [HACKERS] timestamp typedefs

2008-01-04 Thread Alvaro Herrera
Warren Turkal escribió: On Jan 3, 2008 8:54 PM, Tom Lane [EMAIL PROTECTED] wrote: I wrote: Do we really need fhour_t and fminute_t on top of fsec_t? This seems like a bad factorization ... After some more thought: I think that what's bugging me is that fsec_t is intended to denote

Re: [HACKERS] Dynamic Partitioning using Segment Visibility Maps

2008-01-04 Thread Markus Schiltknecht
Hello Simon, Simon Riggs wrote: I've come up with an alternative concept to allow us to discuss the particular merits of each. ISTM that this new proposal has considerable potential. Hm.. interesting idea. If we were able to keep track of which sections of a table are now read-only then we

Re: [HACKERS] Dynamic Partitioning using Segment Visibility Maps

2008-01-04 Thread Markus Schiltknecht
Hi, Simon Riggs wrote: - any Fact table where measurements/observations/events are accumulated e.g. Web Hits (any Internet events) Call Detail Records Sales Security Events Scientific Measurements Process Control - any Major Entity where new entities are created from a sequence e.g. Orders,

Re: [HACKERS] Dynamic Partitioning using Segment Visibility Maps

2008-01-04 Thread Markus Schiltknecht
Hi, Simon Riggs wrote: The smaller the partition size the greater the overhead of managing it. Also I've been looking at read-only tables and compression, as you may know. My idea was that in the future we could mark segments as either - read-only - compressed - able to be shipped off to

Re: [HACKERS] SSL over Unix-domain sockets

2008-01-04 Thread Martijn van Oosterhout
On Fri, Jan 04, 2008 at 05:13:21PM +0100, Peter Eisentraut wrote: Using the attached patch, SSL will act over Unix-domain sockets. AFAICT, this just works. I didn't find a way to sniff a Unix-domain socket, however. Looks clear enough. You should be able to test if it works by using strace

[HACKERS] SSL over Unix-domain sockets

2008-01-04 Thread Peter Eisentraut
Using the attached patch, SSL will act over Unix-domain sockets. AFAICT, this just works. I didn't find a way to sniff a Unix-domain socket, however. How should we proceed with this? -- Peter Eisentraut http://developer.postgresql.org/~petere/ diff -ur

Re: [HACKERS] SSL over Unix-domain sockets

2008-01-04 Thread Bruce Momjian
Peter Eisentraut wrote: Using the attached patch, SSL will act over Unix-domain sockets. AFAICT, this just works. I didn't find a way to sniff a Unix-domain socket, however. How should we proceed with this? I am confused by the shortness of this patch. Right now pg_hba.conf has:

Re: [HACKERS] SSL over Unix-domain sockets

2008-01-04 Thread Magnus Hagander
Bruce Momjian wrote: Peter Eisentraut wrote: Am Freitag, 4. Januar 2008 schrieb Bruce Momjian: Peter Eisentraut wrote: Using the attached patch, SSL will act over Unix-domain sockets. AFAICT, this just works. I didn't find a way to sniff a Unix-domain socket, however. How should we

Re: [HACKERS] SSL over Unix-domain sockets

2008-01-04 Thread Bruce Momjian
Magnus Hagander wrote: Bruce Momjian wrote: Peter Eisentraut wrote: Am Freitag, 4. Januar 2008 schrieb Bruce Momjian: Peter Eisentraut wrote: Using the attached patch, SSL will act over Unix-domain sockets. AFAICT, this just works. I didn't find a way to sniff a Unix-domain socket,

Re: [HACKERS] SSL over Unix-domain sockets

2008-01-04 Thread Peter Eisentraut
Am Freitag, 4. Januar 2008 schrieb Bruce Momjian: Peter Eisentraut wrote: Using the attached patch, SSL will act over Unix-domain sockets. AFAICT, this just works. I didn't find a way to sniff a Unix-domain socket, however. How should we proceed with this? I am confused by the

Re: [HACKERS] SSL over Unix-domain sockets

2008-01-04 Thread Bruce Momjian
Peter Eisentraut wrote: Am Freitag, 4. Januar 2008 schrieb Bruce Momjian: Peter Eisentraut wrote: Using the attached patch, SSL will act over Unix-domain sockets. AFAICT, this just works. I didn't find a way to sniff a Unix-domain socket, however. How should we proceed with

Re: [HACKERS] Dynamic Partitioning using Segment Visibility Maps

2008-01-04 Thread Andrew Sullivan
On Fri, Jan 04, 2008 at 01:29:55PM +0100, Markus Schiltknecht wrote: Agreed. Just a minor note: I find marked read-only too strong, as it implies an impossibility to write. I propose speaking about mostly-read segments, or optimized for reading or similar. I do want some segments to be

Re: [HACKERS] Index performance

2008-01-04 Thread Andrew Sullivan
On Thu, Jan 03, 2008 at 07:11:07AM +0200, Brian Modra wrote: Thanks, I think you have me on the right track. I'm testing a vacuum analyse now to see how long it takes, and then I'll set it up to automatically run every night (so that it has a chance to complete before about 6am.) Note that

[HACKERS] OUTER JOIN performance regression remains in 8.3beta4

2008-01-04 Thread Kevin Grittner
There was a serious performance regression in OUTER JOIN planning going from 8.2.4 to 8.2.5. I know Tom came up with some patches to mitigate the issues in 8.2.5, but my testing shows that problems remain in 8.3beta4. The query: SELECT CH.caseNo, CH.countyNo, CH.chargeNo,

Re: [HACKERS] Problem with PgTcl auditing function on trigger

2008-01-04 Thread Richard Huxton
Glyn Astill wrote: Hi people, I've tried posting on the general list about this, but I never get a reply, so I'm trying here. I think you'll probably have more luck with a TCL list than the PG hackers list. However, I've attached some pltcl functions I put together ages ago to do this

Re: [HACKERS] Dynamic Partitioning using Segment Visibility Maps

2008-01-04 Thread Simon Riggs
On Fri, 2008-01-04 at 13:06 -0500, Andrew Sullivan wrote: On Fri, Jan 04, 2008 at 01:29:55PM +0100, Markus Schiltknecht wrote: Agreed. Just a minor note: I find marked read-only too strong, as it implies an impossibility to write. I propose speaking about mostly-read segments, or

Re: [HACKERS] SSL over Unix-domain sockets

2008-01-04 Thread Aidan Van Dyk
* Bruce Momjian [EMAIL PROTECTED] [080104 13:00]: Actually, if you just commit that patch *without* pg_hba modifications, it still solves the problem stated, no? Because the client can be configured to require ssl and to require server certificate validation, and that's the hole we're

Re: [HACKERS] SSL over Unix-domain sockets

2008-01-04 Thread Tom Lane
Bruce Momjian [EMAIL PROTECTED] writes: Yes, it would plug the hole without fully implementing SSL control on local sockets. However, the hole is already plugged by using directory permissions so I question the need for a partial solution at this point in 8.3. As already noted, fix /tmp's

Re: [HACKERS] SSL over Unix-domain sockets

2008-01-04 Thread Andrew Sullivan
On Fri, Jan 04, 2008 at 02:37:03PM -0500, Bruce Momjian wrote: The problem with adding SSL to local sockets is this slippery slope where we only do part of the job, but it isn't clear where to draw the line. I don't think part of the job for a patch is a slippery slope. It's what you do with

Re: [HACKERS] Dynamic Partitioning using Segment Visibility Maps

2008-01-04 Thread Andrew Sullivan
On Fri, Jan 04, 2008 at 10:26:54PM +0100, Markus Schiltknecht wrote: I'm still puzzled about how a DBA is expected to figure out which segments to mark. I think that part might be hand-wavy still. But once this facility is there, what's to prevent the current active segment (and the rest)

Re: [HACKERS] Dynamic Partitioning using Segment Visibility Maps

2008-01-04 Thread Markus Schiltknecht
Hi, Simon Riggs wrote: On Fri, 2008-01-04 at 13:06 -0500, Andrew Sullivan wrote: On Fri, Jan 04, 2008 at 01:29:55PM +0100, Markus Schiltknecht wrote: Agreed. Just a minor note: I find marked read-only too strong, as it implies an impossibility to write. I propose speaking about mostly-read

Re: [HACKERS] OUTER JOIN performance regression remains in 8.3beta4

2008-01-04 Thread Kevin Grittner
On Fri, Jan 4, 2008 at 12:16 PM, in message [EMAIL PROTECTED], Kevin Grittner [EMAIL PROTECTED] wrote: problems remain in 8.3beta4. As I poked around at this, it started to seem familiar. I had previously posted about this query's performance under 8.2.4.

Re: [HACKERS] Dynamic Partitioning using Segment Visibility Maps

2008-01-04 Thread Simon Riggs
On Fri, 2008-01-04 at 22:26 +0100, Markus Schiltknecht wrote: I'm still puzzled about how a DBA is expected to figure out which segments to mark. Simon, are you assuming we are going to pass on segment numbers to the DBA one day? No Way! That would stop Richard's idea to make the segment

Re: [HACKERS] Problem with PgTcl auditing function on trigger

2008-01-04 Thread Brett Schwarz
- Original Message From: Glyn Astill [EMAIL PROTECTED] To: pgsql-hackers@postgresql.org Sent: Friday, January 4, 2008 5:23:18 AM Subject: [HACKERS] Problem with PgTcl auditing function on trigger Hi people, I've tried posting on the general list about this, but I never get

Re: [HACKERS] OUTER JOIN performance regression remains in 8.3beta4

2008-01-04 Thread Tom Lane
Kevin Grittner [EMAIL PROTECTED] writes: As I poked around at this, it started to seem familiar. I had previously posted about this query's performance under 8.2.4. http://archives.postgresql.org/pgsql-performance/2007-10/msg00087.php Well, that thread gave some of the missing details, such

Re: [HACKERS] OUTER JOIN performance regression remains in 8.3beta4

2008-01-04 Thread Kevin Grittner
On Fri, Jan 4, 2008 at 4:40 PM, in message [EMAIL PROTECTED], Tom Lane [EMAIL PROTECTED] wrote: what the heck is CountyNoT? bigbird=# \dD CountyNoT List of domains Schema | Name| Type | Modifier | Check +---+--+--+--- public

Re: [HACKERS] OUTER JOIN performance regression remains in 8.3beta4

2008-01-04 Thread Kevin Grittner
On Fri, Jan 4, 2008 at 4:29 PM, in message [EMAIL PROTECTED], Tom Lane [EMAIL PROTECTED] wrote: Can't do much with this without seeing the table and view definitions involved. Understood. It was while I was putting that together that it struck me as familiar. They are the same as in this

Re: [HACKERS] OUTER JOIN performance regression remains in 8.3beta4

2008-01-04 Thread Tom Lane
Kevin Grittner [EMAIL PROTECTED] writes: There was a serious performance regression in OUTER JOIN planning going from 8.2.4 to 8.2.5. I know Tom came up with some patches to mitigate the issues in 8.2.5, but my testing shows that problems remain in 8.3beta4. Can't do much with this without

Re: [HACKERS] OUTER JOIN performance regression remains in 8.3beta4

2008-01-04 Thread Kevin Grittner
On Fri, Jan 4, 2008 at 4:46 PM, in message [EMAIL PROTECTED], Tom Lane [EMAIL PROTECTED] wrote: nor mentions the data types involved. Schema | Name | Type| Modifier | Check +---+-+--+--- public |

Re: [HACKERS] OUTER JOIN performance regression remains in 8.3beta4

2008-01-04 Thread Tom Lane
Kevin Grittner [EMAIL PROTECTED] writes: On Fri, Jan 4, 2008 at 4:29 PM, in message [EMAIL PROTECTED], Tom Lane [EMAIL PROTECTED] wrote: Can't do much with this without seeing the table and view definitions involved. Understood. It was while I was putting that together that it struck me

Re: [HACKERS] OUTER JOIN performance regression remains in 8.3beta4

2008-01-04 Thread Kevin Grittner
On Fri, Jan 4, 2008 at 4:51 PM, in message [EMAIL PROTECTED], Kevin Grittner [EMAIL PROTECTED] wrote: keyEventSeqNo | integer | COALESCE( CASE WHEN d.eventType IS NOT NULL THEN d.keyEventSeqNo::smallint ELSE b.keyEventSeqNo::smallint

Re: [HACKERS] OUTER JOIN performance regression remains in 8.3beta4

2008-01-04 Thread Tom Lane
Hmm ... I think I've managed to invent a test case, and unfortunately for you, what it shows is that 8.2 is optimizing the query incorrectly. create table t1 (f1 int primary key); create table t2 (f2 int primary key); create table t3 (f3 int primary key); insert into t1 values(53); insert into t2

Re: [HACKERS] OUTER JOIN performance regression remains in 8.3beta4

2008-01-04 Thread Kevin Grittner
On Fri, Jan 4, 2008 at 5:45 PM, in message [EMAIL PROTECTED], Kevin Grittner [EMAIL PROTECTED] wrote: On Fri, Jan 4, 2008 at 4:51 PM, in message [EMAIL PROTECTED], Kevin Grittner [EMAIL PROTECTED] wrote: keyEventSeqNo | integer | COALESCE( CASE

Re: [HACKERS] OUTER JOIN performance regression remains in 8.3beta4

2008-01-04 Thread Tom Lane
I wrote: [ Pokes at older branches... ] Oh, that's interesting, 8.1 seems to do the right thing already! Seems that 8.1 does the right thing for the wrong reason :-(. Just like 8.2, it falsely concludes that the f3 = f1 clause can be deleted, but it fails to get rid of every copy of it. The

Re: [HACKERS] Dynamic Partitioning using Segment Visibility Maps

2008-01-04 Thread Robert Treat
On Friday 04 January 2008 17:01, Simon Riggs wrote: On Fri, 2008-01-04 at 22:26 +0100, Markus Schiltknecht wrote: I'm still puzzled about how a DBA is expected to figure out which segments to mark. Simon, are you assuming we are going to pass on segment numbers to the DBA one day? No Way!

Re: [HACKERS] timestamp typedefs

2008-01-04 Thread Warren Turkal
On Jan 4, 2008 4:20 AM, Alvaro Herrera [EMAIL PROTECTED] wrote: Perhaps what you want here is to define a type for calculation results (double/int64). Whether it is used in the code for minutes or hours is irrelevant to the typedef. Okay...that sounds good. Do you have a good name for it?