Re: [HACKERS] PG 9.0 release timetable

2010-05-31 Thread Simon Riggs
On Sat, 2010-05-29 at 16:19 -0400, Bruce Momjian wrote:
 Assuming we want a release Postgres 9.0 by mid-August, here is how the
 timetable would look:
 
   Need RC release to be stable for 1-2 weeks before final
   RC must be released by August 1
   Beta must be stable for 2-3 weeks before RC
   Stable beta must be released by early July
 
 So, we have 5-6 weeks to get a stable beta.  Looking at the open issues:
 
   
 http://wiki.postgresql.org/wiki/PostgreSQL_9.0_Open_Items#Resolved_Issues
 
 it looks like we are doing OK, but we must continue progressing.

We've fixed most of the beta1 issues some time ago and beta testers are
waiting for next beta before doing further testing, so absence of new
bugs means very little.

We're currently at 4 weeks since last beta, with no new beta in sight.
If we want to stick to the timetable we should be releasing new beta
releases every 2-3 weeks, not every 4-5 weeks. Our objective (or
realisation of necessity) should be 4-5 betas each release. 

Waiting for stable just introduces delay during beta, though makes
sense for RC. Delay means hackers take their eyes off the release and do
other things, which further slows down the release. Let's accept that
its OK to release another beta while the open items list isn't empty and
reap the next crop of bugs from betas.

If we're going enforce code windows we should be enforcing things
throughout the whole release cycle. We must keep a sensible pace if we
want to keep people involved in the process.

-- 
 Simon Riggs   www.2ndQuadrant.com


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Streaming Replication: Checkpoint_segment and wal_keep_segments on standby

2010-05-31 Thread Heikki Linnakangas

On 30/05/10 06:04, Fujii Masao wrote:

On Fri, May 28, 2010 at 11:12 AM, Fujii Masaomasao.fu...@gmail.com  wrote:

On Thu, May 27, 2010 at 11:13 PM, Robert Haasrobertmh...@gmail.com  wrote:

I guess this happens because the frequency of checkpoint on the standby is
too lower than that on the master. In the master, checkpoint occurs for every
consumption of three segments because of checkpoint_segments = 3. On the
other hand, in the standby, only checkpoint_timeout has effect, so checkpoint
occurs for every 30 minutes because of checkpoint_timeout = 30min.

The walreceiver should signal the bgwriter to start checkpoint if it has
received more than checkpoint_segments WAL files, like normal processing?


Is this also an issue when using log shipping, or just with SR?


When using log shipping, checkpoint_segments always doesn't trigger a
checkpoint. So recovery after the standby crashes might take unexpectedly
long since redo starting point might be old.

But in file-based log shipping, since WAL files don't accumulate in
pg_xlog directory on the standby, even if the frequency of checkpoint
is very low, pg_xlog will not be filled with many WAL files. That
accumulation occurs only when using SR.

If we should avoid low frequency of checkpoint itself rather than
accumulation of WAL files, the bgwriter instead of the walreceiver
should check if we've consumed too much WAL, I think. Thought?


I attached the patch, which changes the startup process so that it signals
bgwriter to perform a restartpoint if we've already replayed too much WAL
files. This leads checkpoint_segments to trigger a restartpoint.


The central question is whether checkpoint_segments should trigger 
restartpoints or not. When PITR and restartpoints were introduced, the 
answer was no, on the grounds that when you're doing recovery you're 
presumably replaying the logs much faster than they were generated, and 
you don't want to slow down the recovery by checkpointing too often.


Now that we have bgwriter active during recovery, and streaming 
replication which retains the streamed WALs so that we now risk running 
out of disk space with long checkpoint_timeout, it's time to reconsider 
that.


I think we have three options:

1) Leave it as it is, checkpoint_segments doesn't do anything during 
recovery/standby mode


2) Change it so that checkpoint_segments does take effect during 
recover/standby


3) Change it so that checkpoint_segments takes effect during streaming 
replication, but not during recovery otherwise


I'm leaning towards 3), it still seems reasonable to not slow down 
recovery when recovering from archive, but the potential for out of disk 
space warrants doing 3.


--
  Heikki Linnakangas
  EnterpriseDB   http://www.enterprisedb.com

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


[HACKERS] Re: [COMMITTERS] pgsql: In walsender, don't sleep if there's outstanding WAL waiting to

2010-05-31 Thread Fujii Masao
Hi,

On Thu, May 27, 2010 at 7:21 AM, Heikki Linnakangas
hei...@postgresql.org wrote:
 Log Message:
 ---
 In walsender, don't sleep if there's outstanding WAL waiting to be sent,
 otherwise we effectively rate-limit the streaming as pointed out by
 Simon Riggs. Also, send the WAL in smaller chunks, to respond to signals
 more promptly.

 Modified Files:
 --
    pgsql/src/backend/replication:
        walsender.c (r1.20 - r1.21)
        
 (http://anoncvs.postgresql.org/cvsweb.cgi/pgsql/src/backend/replication/walsender.c?r1=1.20r2=1.21)

This commit seems to have introduced the bug that walsender exits
before sending all the WAL up to shutdown checkpoint record when
smart shutdown is requested.

The attached patch fixes the bug and a tiny typo.

Regards,

-- 
Fujii Masao
NIPPON TELEGRAPH AND TELEPHONE CORPORATION
NTT Open Source Software Center

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Streaming Replication: Checkpoint_segment and wal_keep_segments on standby

2010-05-31 Thread Fujii Masao
On Mon, May 31, 2010 at 6:37 PM, Heikki Linnakangas
heikki.linnakan...@enterprisedb.com wrote:
 The central question is whether checkpoint_segments should trigger
 restartpoints or not. When PITR and restartpoints were introduced, the
 answer was no, on the grounds that when you're doing recovery you're
 presumably replaying the logs much faster than they were generated, and you
 don't want to slow down the recovery by checkpointing too often.

Right.

 Now that we have bgwriter active during recovery, and streaming replication
 which retains the streamed WALs so that we now risk running out of disk
 space with long checkpoint_timeout, it's time to reconsider that.

 I think we have three options:

 1) Leave it as it is, checkpoint_segments doesn't do anything during
 recovery/standby mode

 2) Change it so that checkpoint_segments does take effect during
 recover/standby

 3) Change it so that checkpoint_segments takes effect during streaming
 replication, but not during recovery otherwise

 I'm leaning towards 3), it still seems reasonable to not slow down recovery
 when recovering from archive, but the potential for out of disk space
 warrants doing 3.

3) makes sense. But how about 4)?

4) Change it so that checkpoint_segments takes effect in standby mode,
but not during recovery otherwise

This would lessen the time required to restart the standby also in
file-based log shipping case. Of course, there is the tradeoff
between the speed of recovery and the recovery time.

Regards,

-- 
Fujii Masao
NIPPON TELEGRAPH AND TELEPHONE CORPORATION
NTT Open Source Software Center

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


[HACKERS] Re: [COMMITTERS] pgsql: In walsender, don't sleep if there's outstanding WAL waiting to

2010-05-31 Thread Fujii Masao
On Mon, May 31, 2010 at 7:03 PM, Fujii Masao masao.fu...@gmail.com wrote:
 Hi,

 On Thu, May 27, 2010 at 7:21 AM, Heikki Linnakangas
 hei...@postgresql.org wrote:
 Log Message:
 ---
 In walsender, don't sleep if there's outstanding WAL waiting to be sent,
 otherwise we effectively rate-limit the streaming as pointed out by
 Simon Riggs. Also, send the WAL in smaller chunks, to respond to signals
 more promptly.

 Modified Files:
 --
    pgsql/src/backend/replication:
        walsender.c (r1.20 - r1.21)
        
 (http://anoncvs.postgresql.org/cvsweb.cgi/pgsql/src/backend/replication/walsender.c?r1=1.20r2=1.21)

 This commit seems to have introduced the bug that walsender exits
 before sending all the WAL up to shutdown checkpoint record when
 smart shutdown is requested.

 The attached patch fixes the bug and a tiny typo.

Sorry. I forgot to attach the patch..

Regards,

-- 
Fujii Masao
NIPPON TELEGRAPH AND TELEPHONE CORPORATION
NTT Open Source Software Center


send_xlog_upto_shutdown_ckpt_v1.patch
Description: Binary data

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


[HACKERS] Re: [COMMITTERS] pgsql: In walsender, don't sleep if there's outstanding WAL waiting to

2010-05-31 Thread Heikki Linnakangas

On 31/05/10 13:20, Fujii Masao wrote:

On Mon, May 31, 2010 at 7:03 PM, Fujii Masaomasao.fu...@gmail.com  wrote:

This commit seems to have introduced the bug that walsender exits
before sending all the WAL up to shutdown checkpoint record when
smart shutdown is requested.

The attached patch fixes the bug and a tiny typo.


Sorry. I forgot to attach the patch..


Thanks, committed.

--
  Heikki Linnakangas
  EnterpriseDB   http://www.enterprisedb.com

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


[HACKERS] Re: [COMMITTERS] pgsql: Ensure that top level aborts call XLogSetAsyncCommit().

2010-05-31 Thread Fujii Masao
On Thu, May 13, 2010 at 8:39 PM, Simon Riggs sri...@postgresql.org wrote:
 Log Message:
 ---
 Ensure that top level aborts call XLogSetAsyncCommit(). Not doing
 so simply leads to data waiting in wal_buffers which then causes
 later commits to potentially do emergency writes and for all forms
 of replication to be potentially delayed without need or benefit.
 Issue pointed out exactly by Fujii Masao, following bug report
 by Robert Haas on a separate though related topic.

 Modified Files:
 --
    pgsql/src/backend/access/transam:
        xact.c (r1.290 - r1.291)
        
 (http://anoncvs.postgresql.org/cvsweb.cgi/pgsql/src/backend/access/transam/xact.c?r1=1.290r2=1.291)

This commit changed XLogSetAsyncCommitLSN() so that it's called
for abort case. So we need to change the comment of the function
as follows:

diff --git a/src/backend/access/transam/xlog.c
b/src/backend/access/transam/xlog.c
index c886571..ae10108 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -1860,7 +1860,7 @@ XLogWrite(XLogwrtRqst WriteRqst, bool flexible,
bool xlog_switch)

 /*
  * Record the LSN for an asynchronous transaction commit.
- * (This should not be called for aborts, nor for synchronous commits.)
+ * (This should not be called for synchronous commits.)
  */
 void
 XLogSetAsyncCommitLSN(XLogRecPtr asyncCommitLSN)


Regards,

-- 
Fujii Masao
NIPPON TELEGRAPH AND TELEPHONE CORPORATION
NTT Open Source Software Center

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


[HACKERS] Re: [COMMITTERS] pgsql: Ensure that top level aborts call XLogSetAsyncCommit().

2010-05-31 Thread Simon Riggs
On Mon, 2010-05-31 at 20:11 +0900, Fujii Masao wrote:
 On Thu, May 13, 2010 at 8:39 PM, Simon Riggs sri...@postgresql.org wrote:
  Log Message:
  ---
  Ensure that top level aborts call XLogSetAsyncCommit(). Not doing
  so simply leads to data waiting in wal_buffers which then causes
  later commits to potentially do emergency writes and for all forms
  of replication to be potentially delayed without need or benefit.
  Issue pointed out exactly by Fujii Masao, following bug report
  by Robert Haas on a separate though related topic.
 
  Modified Files:
  --
 pgsql/src/backend/access/transam:
 xact.c (r1.290 - r1.291)
 
  (http://anoncvs.postgresql.org/cvsweb.cgi/pgsql/src/backend/access/transam/xact.c?r1=1.290r2=1.291)
 
 This commit changed XLogSetAsyncCommitLSN() so that it's called
 for abort case. So we need to change the comment of the function
 as follows:

Agreed, will fix.

Will also rename function to better document its new role.

-- 
 Simon Riggs   www.2ndQuadrant.com


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] PG 9.0 release timetable

2010-05-31 Thread Thom Brown
On 31 May 2010 09:33, Simon Riggs si...@2ndquadrant.com wrote:

 We're currently at 4 weeks since last beta, with no new beta in sight.

My understanding was beta 2 would be out on 7th June.  Is that changing?

Thom

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] PG 9.0 release timetable

2010-05-31 Thread Marc G. Fournier

On Mon, 31 May 2010, Thom Brown wrote:


On 31 May 2010 09:33, Simon Riggs si...@2ndquadrant.com wrote:


We're currently at 4 weeks since last beta, with no new beta in sight.


My understanding was beta 2 would be out on 7th June.  Is that changing?


Yes, but Simon is correct in that 4-5 weeks between betas is a long time, 
when most bugs will be reported (and hopefully fixed) relatively quickly 
after a beta is released ... RC should be held to a more 'release 
standard', but beta's should be closer to a snapshot standard, with a more 
short/fixed timeframe so that debuggers aren't hitting the same bugs that 
were reported (and fixed), weeks earlier ...



Marc G. FournierHub.Org Hosting Solutions S.A.
scra...@hub.org http://www.hub.org

Yahoo:yscrappySkype: hub.orgICQ:7615664MSN:scra...@hub.org

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] PG 9.0 release timetable

2010-05-31 Thread Dave Page
On Mon, May 31, 2010 at 2:22 PM, Thom Brown thombr...@gmail.com wrote:
 On 31 May 2010 09:33, Simon Riggs si...@2ndquadrant.com wrote:

 We're currently at 4 weeks since last beta, with no new beta in sight.

 My understanding was beta 2 would be out on 7th June.  Is that changing?

No. It's very much in sight on my calendar :-)

-- 
Dave Page
EnterpriseDB UK: http://www.enterprisedb.com
The Enterprise Postgres Company

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] functional call named notation clashes with SQL feature

2010-05-31 Thread Bruce Momjian
Tom Lane wrote:
 So as far as I can tell, no one is opposed to replacing expr AS name
 with name := expr in the named-parameter syntax.  Obviously we had
 better get this done before beta2.  Is anyone actually working on the
 code/docs changes?  If not, I'll pick it up.

If we eventually are going to want to support the ANSI standard =
syntax, I am thinking we should just do it now.  The larger question is
what justification do we have of not supporting =.

-- 
  Bruce Momjian  br...@momjian.ushttp://momjian.us
  EnterpriseDB http://enterprisedb.com

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] functional call named notation clashes with SQL feature

2010-05-31 Thread Pavel Stehule
2010/5/31 Bruce Momjian br...@momjian.us:
 Tom Lane wrote:
 So as far as I can tell, no one is opposed to replacing expr AS name
 with name := expr in the named-parameter syntax.  Obviously we had
 better get this done before beta2.  Is anyone actually working on the
 code/docs changes?  If not, I'll pick it up.

 If we eventually are going to want to support the ANSI standard =
 syntax, I am thinking we should just do it now.  The larger question is
 what justification do we have of not supporting =.

I am for ANSI stanadard. I afraid so we can do nothing now. First we
have to implement substitution of = operator in hstore module.
Second we have to mark this operator as deprecated. Maybe we can do it
in 9.1 with integration of hstore to core. I would to see any hash
table support in core. It can be significant help for PLpgSQL coders.
What more - it can work nice with proposed JSON support.

Regards
Pavel

 --
  Bruce Momjian  br...@momjian.us        http://momjian.us
  EnterpriseDB                             http://enterprisedb.com

 --
 Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
 To make changes to your subscription:
 http://www.postgresql.org/mailpref/pgsql-hackers


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] PG 9.0 release timetable

2010-05-31 Thread Simon Riggs
On Mon, 2010-05-31 at 15:14 +0100, Dave Page wrote:
 On Mon, May 31, 2010 at 2:22 PM, Thom Brown thombr...@gmail.com wrote:
  On 31 May 2010 09:33, Simon Riggs si...@2ndquadrant.com wrote:
 
  We're currently at 4 weeks since last beta, with no new beta in sight.
 
  My understanding was beta 2 would be out on 7th June.  Is that changing?
 
 No. It's very much in sight on my calendar :-)

Can we make it 2 weeks per beta from now on?

That will allow us to get to Beta5 by 19 Jul, which will hopefully
become RC1 on 2 Aug and then release on 16 Aug. 

At the current pace we will be on BETA3 on 12 July, increasing the
probablity of delay, or reducing the number of bugs discovered prior to
release.

If that's a problem, do we need to release BetaN on all platforms? Where
do the majority of bug reports originate? Let's focus there.

-- 
 Simon Riggs   www.2ndQuadrant.com


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] functional call named notation clashes with SQL feature

2010-05-31 Thread Tom Lane
Bruce Momjian br...@momjian.us writes:
 Tom Lane wrote:
 So as far as I can tell, no one is opposed to replacing expr AS name
 with name := expr in the named-parameter syntax.  Obviously we had
 better get this done before beta2.  Is anyone actually working on the
 code/docs changes?  If not, I'll pick it up.

 If we eventually are going to want to support the ANSI standard =
 syntax, I am thinking we should just do it now.  The larger question is
 what justification do we have of not supporting =.

Not breaking hstore, as well as any third-party modules that might be
using that operator name.  Did you not absorb any of the discussion
so far?

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] functional call named notation clashes with SQL feature

2010-05-31 Thread Pavel Stehule
2010/5/31 Tom Lane t...@sss.pgh.pa.us:
 Bruce Momjian br...@momjian.us writes:
 Tom Lane wrote:
 So as far as I can tell, no one is opposed to replacing expr AS name
 with name := expr in the named-parameter syntax.  Obviously we had
 better get this done before beta2.  Is anyone actually working on the
 code/docs changes?  If not, I'll pick it up.

 If we eventually are going to want to support the ANSI standard =
 syntax, I am thinking we should just do it now.  The larger question is
 what justification do we have of not supporting =.

 Not breaking hstore, as well as any third-party modules that might be
 using that operator name.  Did you not absorb any of the discussion
 so far?


can we search thise applications? I know only about hstore module. We
can ask people who use it in own applications. But every major version
of PostgreSQL can breaks compatibility - like 9.0 with variable names
in plpgsql.

Regards
Pavel

                        regards, tom lane

 --
 Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
 To make changes to your subscription:
 http://www.postgresql.org/mailpref/pgsql-hackers


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] PG 9.0 release timetable

2010-05-31 Thread Tom Lane
Marc G. Fournier scra...@hub.org writes:
 On Mon, 31 May 2010, Thom Brown wrote:
 On 31 May 2010 09:33, Simon Riggs si...@2ndquadrant.com wrote:
 We're currently at 4 weeks since last beta, with no new beta in sight.
 
 My understanding was beta 2 would be out on 7th June.  Is that changing?

 Yes, but Simon is correct in that 4-5 weeks between betas is a long time, 

The reason it went like that is that (a) we had PGCon in there, and (b)
we had a set of security releases in there.  Asking for another beta
to have happened is pointless.  You might recall that I already *did*
ask that --- I wanted beta2 to happen concurrently with the security
releases --- and was turned down.

I find myself entirely unimpressed by proposals to make releases
according to some rigid schedule that takes no account of whether
packaging manpower is actually available.

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] PG 9.0 release timetable

2010-05-31 Thread Tom Lane
Simon Riggs si...@2ndquadrant.com writes:
 We're currently at 4 weeks since last beta, with no new beta in sight.

Eh?
http://archives.postgresql.org/pgsql-hackers/2010-05/msg01649.php
You can hardly claim to have not seen it.

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Streaming Replication: Checkpoint_segment and wal_keep_segments on standby

2010-05-31 Thread Tom Lane
Heikki Linnakangas heikki.linnakan...@enterprisedb.com writes:
 The central question is whether checkpoint_segments should trigger 
 restartpoints or not. When PITR and restartpoints were introduced, the 
 answer was no, on the grounds that when you're doing recovery you're 
 presumably replaying the logs much faster than they were generated, and 
 you don't want to slow down the recovery by checkpointing too often.

 Now that we have bgwriter active during recovery, and streaming 
 replication which retains the streamed WALs so that we now risk running 
 out of disk space with long checkpoint_timeout, it's time to reconsider 
 that.

 I think we have three options:

What about

(4) pay some attention to the actual elapsed time since the last
restart point?

All the others seem like kluges that are relying on hard-wired rules
that are hoped to achieve something like a time-based checkpoint.

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] functional call named notation clashes with SQL feature

2010-05-31 Thread Bruce Momjian
Tom Lane wrote:
 Bruce Momjian br...@momjian.us writes:
  Tom Lane wrote:
  So as far as I can tell, no one is opposed to replacing expr AS name
  with name := expr in the named-parameter syntax.  Obviously we had
  better get this done before beta2.  Is anyone actually working on the
  code/docs changes?  If not, I'll pick it up.
 
  If we eventually are going to want to support the ANSI standard =
  syntax, I am thinking we should just do it now.  The larger question is
  what justification do we have of not supporting =.
 
 Not breaking hstore, as well as any third-party modules that might be
 using that operator name.  Did you not absorb any of the discussion
 so far?

Yes, but if we are going to have to honor = eventually, shouldn't we
just do it now?  Supporting := and = seems confusing.

-- 
  Bruce Momjian  br...@momjian.ushttp://momjian.us
  EnterpriseDB http://enterprisedb.com

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] functional call named notation clashes with SQL feature

2010-05-31 Thread Tom Lane
Bruce Momjian br...@momjian.us writes:
 Yes, but if we are going to have to honor = eventually, shouldn't we
 just do it now?  Supporting := and = seems confusing.

Personally, I haven't accepted the if part of that, therefore I
feel no need to argue over the then.

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


[HACKERS] Index only scans

2010-05-31 Thread Shrish Purohit
Hi Heikki, Pgsql-Hackers,

Sometime back you have started with Separate Heap Fetch from Index Scan which 
was planned to support partial index only scans. Are you still working on it or 
do you know someone still working on it?

We did some development with Gokul's Index Only Patch and have got good 
performance numbers which are as follows:

Test table constitutes 0.5 billion records with thick index on (id,aid) on 
three machines {pg_normal , pg_enhanced( PGSQL with thick index feature ), 
Oracle} each having 16 Gb Ram. Disk I/O obtained using sar.

testdb=# \d test
   Table public.test
  Column   |   Type   | Modifiers
---+--+---
 id| integer  |
 startdate | date |
 enddate   | date |
 charge| double precision |
 firstname | text |
 lastname  | text |
 aid   | double precision |
 bid   | double precision |
Indexes:
taid THICK btree (id, aid) CLUSTER

Index size
On oracle   15.20 Gb
On Pg-normal14.73 Gb
Pg_enhanced 23.17 Gb (16bytes*0.5billion = ~7.6 GB)

PFA excel sheet for details. In general we saw fair amount of performance 
improvement, but one thing that surprises us is that after around 20% tuples 
updated we found oracle taking more time.

Regards,
Shrish Purohit |Senior Software Engineer|Persistent Systems
shrish_puro...@persistent.co.in |Cell:+91-9850-959-940|Tel:+91(20)302-34493
Innovation in software product design, development and delivery- 
www.persistentsys.com


DISCLAIMER
==
This e-mail may contain privileged and confidential information which is the 
property of Persistent Systems Ltd. It is intended only for the use of the 
individual or entity to which it is addressed. If you are not the intended 
recipient, you are not authorized to read, retain, copy, print, distribute or 
use this message. If you have received this communication in error, please 
notify the sender and delete all copies of this message. Persistent Systems 
Ltd. does not accept any liability for virus infected mails.


Thick_index_perf_numbers.xlsx
Description: Thick_index_perf_numbers.xlsx

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] PG 9.0 release timetable

2010-05-31 Thread Marc G. Fournier

On Mon, 31 May 2010, Tom Lane wrote:

I find myself entirely unimpressed by proposals to make releases 
according to some rigid schedule that takes no account of whether 
packaging manpower is actually available.


How many beta testers out there *rely* on a package to do their testing? 
I'm not saying don't try and get packages in place, I'm just saying it 
shouldn't be a requirement to stamp code BETA and create a tar ball ...



Marc G. FournierHub.Org Hosting Solutions S.A.
scra...@hub.org http://www.hub.org

Yahoo:yscrappySkype: hub.orgICQ:7615664MSN:scra...@hub.org

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] functional call named notation clashes with SQL feature

2010-05-31 Thread Bruce Momjian
Tom Lane wrote:
 Bruce Momjian br...@momjian.us writes:
  Yes, but if we are going to have to honor = eventually, shouldn't we
  just do it now?  Supporting := and = seems confusing.
 
 Personally, I haven't accepted the if part of that, therefore I
 feel no need to argue over the then.

Right, I am asking about the if part.

-- 
  Bruce Momjian  br...@momjian.ushttp://momjian.us
  EnterpriseDB http://enterprisedb.com

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] PG 9.0 release timetable

2010-05-31 Thread Bruce Momjian
Marc G. Fournier wrote:
 On Mon, 31 May 2010, Tom Lane wrote:
 
  I find myself entirely unimpressed by proposals to make releases 
  according to some rigid schedule that takes no account of whether 
  packaging manpower is actually available.
 
 How many beta testers out there *rely* on a package to do their testing? 
 I'm not saying don't try and get packages in place, I'm just saying it 
 shouldn't be a requirement to stamp code BETA and create a tar ball ...

Well, they can just grab nightly snapshots and test, right?  I don't
think a beta is fundamentally different from a nightly snapshot,
source-code wise.

-- 
  Bruce Momjian  br...@momjian.ushttp://momjian.us
  EnterpriseDB http://enterprisedb.com

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] PG 9.0 release timetable

2010-05-31 Thread Marc G. Fournier

On Mon, 31 May 2010, Bruce Momjian wrote:


Marc G. Fournier wrote:

On Mon, 31 May 2010, Tom Lane wrote:


I find myself entirely unimpressed by proposals to make releases
according to some rigid schedule that takes no account of whether
packaging manpower is actually available.


How many beta testers out there *rely* on a package to do their testing?
I'm not saying don't try and get packages in place, I'm just saying it
shouldn't be a requirement to stamp code BETA and create a tar ball ...


Well, they can just grab nightly snapshots and test, right?  I don't
think a beta is fundamentally different from a nightly snapshot,
source-code wise.


doesn't really give a good reference point for testing purposes ... if 
everyone downloads BETA2 and tests, they are all testing the exact same 
code ...



Marc G. FournierHub.Org Hosting Solutions S.A.
scra...@hub.org http://www.hub.org

Yahoo:yscrappySkype: hub.orgICQ:7615664MSN:scra...@hub.org

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] PG 9.0 release timetable

2010-05-31 Thread Marc G. Fournier

On Mon, 31 May 2010, Magnus Hagander wrote:


My guess would be most of them.


Do we not have any stats on # of beta downloads per package type?  I use 
FreeBSD ports when installing production, but when testing non-released 
code, I generally use the source code itself and build ...



Marc G. FournierHub.Org Hosting Solutions S.A.
scra...@hub.org http://www.hub.org

Yahoo:yscrappySkype: hub.orgICQ:7615664MSN:scra...@hub.org

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] PG 9.0 release timetable

2010-05-31 Thread Tom Lane
Marc G. Fournier scra...@hub.org writes:
 On Mon, 31 May 2010, Bruce Momjian wrote:
 Well, they can just grab nightly snapshots and test, right?  I don't
 think a beta is fundamentally different from a nightly snapshot,
 source-code wise.

 doesn't really give a good reference point for testing purposes ...

It's also inferior from a documentation standpoint --- we don't update
the release notes nightly.

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] PG 9.0 release timetable

2010-05-31 Thread Magnus Hagander
On Mon, May 31, 2010 at 5:35 PM, Marc G. Fournier scra...@hub.org wrote:
 On Mon, 31 May 2010, Magnus Hagander wrote:

 My guess would be most of them.

 Do we not have any stats on # of beta downloads per package type?  I use 
 FreeBSD ports when installing production, but when testing non-released code, 
 I generally use the source code itself and build ...

No. Most packages don't come off the postgresql.org servers - they
come out of the yum repositories, the deb repositories or the edb
download servers (windows).


-- 
 Magnus Hagander
 Me: http://www.hagander.net/
 Work: http://www.redpill-linpro.com/

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] PG 9.0 release timetable

2010-05-31 Thread Magnus Hagander
On Mon, May 31, 2010 at 5:30 PM, Bruce Momjian br...@momjian.us wrote:
 Marc G. Fournier wrote:
 On Mon, 31 May 2010, Tom Lane wrote:

  I find myself entirely unimpressed by proposals to make releases
  according to some rigid schedule that takes no account of whether
  packaging manpower is actually available.

 How many beta testers out there *rely* on a package to do their testing?
 I'm not saying don't try and get packages in place, I'm just saying it
 shouldn't be a requirement to stamp code BETA and create a tar ball ...

My guess would be most of them.

Unlike alphas where most probably just work off a tree - or so it seems.

That's obviously going to be very platform dependent.


 Well, they can just grab nightly snapshots and test, right?  I don't
 think a beta is fundamentally different from a nightly snapshot,
 source-code wise.

Source-code wise, no, it's not - except that it's a well defined point
in time, so it's easier to report bugs against, and to search for
known bugs against.

-- 
 Magnus Hagander
 Me: http://www.hagander.net/
 Work: http://www.redpill-linpro.com/

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] functional call named notation clashes with SQL feature

2010-05-31 Thread Greg Stark
On Mon, May 31, 2010 at 3:59 PM, Tom Lane t...@sss.pgh.pa.us wrote:
 Not breaking hstore, as well as any third-party modules that might be
 using that operator name.  Did you not absorb any of the discussion
 so far?


In fairness most of the discussion about breaking hstore was prior to
our learning that the sql committee had gone so far into the weeds.

If = is sql standard syntax then perhaps that changes the calculus.
It's no longer a matter of supporting some oracle-specific syntax that
diverges from sqlish syntax and conflicts with our syntax. Instead
it's a question of our operator syntax conflicting with the sql
standard.

Part of the earlier discussion was about how = was a tempting
operator name and other users may well have chosen it precisely
because it's so evocative. But we don't actually have any evidence of
that. Does anyone have any experience seeing = operators in the wild?

-- 
greg

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Index only scans

2010-05-31 Thread Sharmila Jothirajah
Hi Shrish,
I saw the excel that you sent to Heikki. Here are my comments.

1. Don't worry about comparing the results with Oracle. Just take pg-normal and 
pg-enhanced. That's what the community cares about. Later we can also add 
Oracle's performance. 
2. In the excel sheet you have 'simple queries' and 'simple queries 
repeated'...you have to compile them together.
3. Also for pg-normal, the queries are run with seq scan and not forcing 'index 
scan' . If you compare that run with pg-enhanced it will definitely look bad. 
You have to compare pg-enh with pg-normal's indexscan (and also seq scan). That 
will give a complete picture.

These performance numbers that we sent them is very important. Make sure the 
excel is formatted and is very clear before your sent the numbers. Its hard to 
get the community to respond if our reports are not clear

Thanks
Sharmila



-Original Message-
From: Shrish Purohit [mailto:shrish_puro...@persistent.co.in] 
Sent: Monday, May 31, 2010 11:09 AM
To: Heikki Linnakangas
Cc: pgsql-hackers@postgresql.org; Sharmila Jothirajah; Mahesh Nalkande; Arvind 
Hulgeri; Sameer Pendharkar
Subject: Index only scans 

Hi Heikki, Pgsql-Hackers,  

Sometime back you have started with Separate Heap Fetch from Index Scan which 
was planned to support partial index only scans. Are you still working on it or 
do you know someone still working on it? 

We did some development with Gokul's Index Only Patch and have got good 
performance numbers which are as follows:

Test table constitutes 0.5 billion records with thick index on (id,aid) on 
three machines {pg_normal , pg_enhanced( PGSQL with thick index feature ), 
Oracle} each having 16 Gb Ram. Disk I/O obtained using sar. 

testdb=# \d test
   Table public.test
  Column   |   Type   | Modifiers
---+--+---
 id| integer  |
 startdate | date |
 enddate   | date |
 charge| double precision |
 firstname | text |
 lastname  | text |
 aid   | double precision |
 bid   | double precision |
Indexes:
taid THICK btree (id, aid) CLUSTER

Index size 
On oracle   15.20 Gb
On Pg-normal14.73 Gb 
Pg_enhanced 23.17 Gb (16bytes*0.5billion = ~7.6 GB)

PFA excel sheet for details. In general we saw fair amount of performance 
improvement, but one thing that surprises us is that after around 20% tuples 
updated we found oracle taking more time. 

Regards,
Shrish Purohit |Senior Software Engineer|Persistent Systems 
shrish_puro...@persistent.co.in |Cell:+91-9850-959-940|Tel:+91(20)302-34493
Innovation in software product design, development and delivery- 
www.persistentsys.com
 

DISCLAIMER
==
This e-mail may contain privileged and confidential information which is the 
property of Persistent Systems Ltd. It is intended only for the use of the 
individual or entity to which it is addressed. If you are not the intended 
recipient, you are not authorized to read, retain, copy, print, distribute or 
use this message. If you have received this communication in error, please 
notify the sender and delete all copies of this message. Persistent Systems 
Ltd. does not accept any liability for virus infected mails.

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] PG 9.0 release timetable

2010-05-31 Thread Marc G. Fournier

On Mon, 31 May 2010, Tom Lane wrote:


Marc G. Fournier scra...@hub.org writes:

On Mon, 31 May 2010, Bruce Momjian wrote:

Well, they can just grab nightly snapshots and test, right?  I don't
think a beta is fundamentally different from a nightly snapshot,
source-code wise.



doesn't really give a good reference point for testing purposes ...


It's also inferior from a documentation standpoint --- we don't update
the release notes nightly.


There are three things that *have* to be involved in doing a Beta:

translation updated
release notes
tar ball

There doesn't need to be any web site announce or anything, only a note 
out to -hackers that we have a new beta ready for testing ...


If we were to do that every 2 weeks, on a Friday, then any packagers that 
are able to can get their package ready and up for testing ... but, for 
those that are able to build from sources (I would hope any/everyone on 
-hackers can handle that?), they would have a firm release to build / run 
tests on that includes all bugs fixed in the previous 2 weeks ...



Marc G. FournierHub.Org Hosting Solutions S.A.
scra...@hub.org http://www.hub.org

Yahoo:yscrappySkype: hub.orgICQ:7615664MSN:scra...@hub.org

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] functional call named notation clashes with SQL feature

2010-05-31 Thread Bruce Momjian
Greg Stark wrote:
 On Mon, May 31, 2010 at 3:59 PM, Tom Lane t...@sss.pgh.pa.us wrote:
  Not breaking hstore, as well as any third-party modules that might be
  using that operator name. ?Did you not absorb any of the discussion
  so far?
 
 
 In fairness most of the discussion about breaking hstore was prior to
 our learning that the sql committee had gone so far into the weeds.
 
 If = is sql standard syntax then perhaps that changes the calculus.
 It's no longer a matter of supporting some oracle-specific syntax that
 diverges from sqlish syntax and conflicts with our syntax. Instead
 it's a question of our operator syntax conflicting with the sql
 standard.
 
 Part of the earlier discussion was about how = was a tempting
 operator name and other users may well have chosen it precisely
 because it's so evocative. But we don't actually have any evidence of
 that. Does anyone have any experience seeing = operators in the wild?

Tangentially, I think the SQL committee chose = because the value, then
variable, ordering is so unintuitive, and I think they wanted that
ordering because most function calls use values so they wanted the
variable at the end.

-- 
  Bruce Momjian  br...@momjian.ushttp://momjian.us
  EnterpriseDB http://enterprisedb.com

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] PG 9.0 release timetable

2010-05-31 Thread Tom Lane
Marc G. Fournier scra...@hub.org writes:
 On Mon, 31 May 2010, Tom Lane wrote:
 I find myself entirely unimpressed by proposals to make releases 
 according to some rigid schedule that takes no account of whether 
 packaging manpower is actually available.

 How many beta testers out there *rely* on a package to do their testing? 

A lot of them --- probably approximately 100% of the Windows population,
for example.  People who are capable of working from source are likely
not waiting for beta packages anyway, just using CVS or nightly
snapshots.

 I'm not saying don't try and get packages in place, I'm just saying it 
 shouldn't be a requirement to stamp code BETA and create a tar ball ...

There's more work that goes into a beta release than just stamping,
as you should know as well as anyone.  Otherwise we might as well call
the nightly snapshots beta releases.

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] functional call named notation clashes with SQL feature

2010-05-31 Thread Tom Lane
Greg Stark gsst...@mit.edu writes:
 If = is sql standard syntax then perhaps that changes the calculus.

Well, it *isn't* standard, yet at least.  All we have is a report of the
current wording of a draft that's at least a year from release.

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] no universally correct setting for fsync

2010-05-31 Thread Bruce Momjian
Josh Berkus wrote:
 All,
 
 Updated docs based on tracking this discussion.  fsync through full page
 writes recorded below.

I have applied this doc update with the attached patch.

I added the change from every night to frequently, and reworded it
slightly so it was clear it affects the entire cluster, not just a
single database.

-- 
  Bruce Momjian  br...@momjian.ushttp://momjian.us
  EnterpriseDB http://enterprisedb.com
Index: doc/src/sgml/config.sgml
===
RCS file: /cvsroot/pgsql/doc/src/sgml/config.sgml,v
retrieving revision 1.279
diff -c -c -r1.279 config.sgml
*** doc/src/sgml/config.sgml	26 May 2010 23:49:18 -	1.279
--- doc/src/sgml/config.sgml	31 May 2010 15:44:36 -
***
*** 1413,1446 
 /para
  
 para
! However, using varnamefsync/varname results in a
! performance penalty: when a transaction is committed,
! productnamePostgreSQL/productname must wait for the
! operating system to flush the write-ahead log to disk.  When
! varnamefsync/varname is disabled, the operating system is
! allowed to do its best in buffering, ordering, and delaying
! writes. This can result in significantly improved performance.
! However, if the system crashes, the results of the last few
! committed transactions might be completely lost, or worse,
! might appear partially committed, leaving the database in an
! inconsistent state. In the
! worst case, unrecoverable data corruption might occur.
! (Crashes of the database software itself are emphasisnot/
! a risk factor here.  Only an operating-system-level crash
! creates a risk of corruption.)
 /para
  
 para
! Due to the risks involved, there is no universally correct
! setting for varnamefsync/varname. Some administrators
! always disable varnamefsync/varname, while others only
! turn it off during initial bulk data loads, where there is a clear
! restart point if something goes wrong.  Others
! always leave varnamefsync/varname enabled. The default is
! to enable varnamefsync/varname, for maximum reliability.
! If you trust your operating system, your hardware, and your
! utility company (or your battery backup), you can consider
! disabling varnamefsync/varname.
 /para
  
 para
--- 1413,1435 
 /para
  
 para
! While turning off varnamefsync/varname is often a performance
! benefit, this can result in unrecoverable data corruption in
! the event of an unexpected system shutdown or crash.  Thus it
! is only advisable to turn off  varnamefsync/varname if
! you can easily recreate your entire database from external
! data.
 /para
  
 para
! Examples of safe circumstances for turning off
! varnamefsync/varname include the initial loading a new
! database cluster from a backup file, using a database cluster
! for processing statistics on an hourly basis which is then
! recreated, or for a reporting read-only database clone which
! gets recreated frequently and is not used for failover.  High
! quality hardware alone is not a sufficient justification for
! turning off varnamefsync/varname.
 /para
  
 para
***
*** 1572,1583 
  
 para
  Turning this parameter off speeds normal operation, but
! might lead to a corrupt database after an operating system crash
! or power failure. The risks are similar to turning off
! varnamefsync/, though smaller.  It might be safe to turn off
! this parameter if you have hardware (such as a battery-backed disk
! controller) or file-system software that reduces
! the risk of partial page writes to an acceptably low level (e.g., ZFS).
 /para
  
 para
--- 1561,1570 
  
 para
  Turning this parameter off speeds normal operation, but
! might lead to either unrecoverable data corruption, or silent
! data corruption, after a system failure. The risks are similar to turning off
! varnamefsync/varname, though smaller, and it should be turned off
! only based on the same circumstances recommended for that parameter.
 /para
  
 para

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] functional call named notation clashes with SQL feature

2010-05-31 Thread Pavel Stehule
2010/5/31 Bruce Momjian br...@momjian.us:
 Greg Stark wrote:
 On Mon, May 31, 2010 at 3:59 PM, Tom Lane t...@sss.pgh.pa.us wrote:
  Not breaking hstore, as well as any third-party modules that might be
  using that operator name. ?Did you not absorb any of the discussion
  so far?
 

 In fairness most of the discussion about breaking hstore was prior to
 our learning that the sql committee had gone so far into the weeds.

 If = is sql standard syntax then perhaps that changes the calculus.
 It's no longer a matter of supporting some oracle-specific syntax that
 diverges from sqlish syntax and conflicts with our syntax. Instead
 it's a question of our operator syntax conflicting with the sql
 standard.

 Part of the earlier discussion was about how = was a tempting
 operator name and other users may well have chosen it precisely
 because it's so evocative. But we don't actually have any evidence of
 that. Does anyone have any experience seeing = operators in the wild?

 Tangentially, I think the SQL committee chose = because the value, then
 variable, ordering is so unintuitive, and I think they wanted that
 ordering because most function calls use values so they wanted the
 variable at the end.

maybe, maybe not. Maybe just adopt Oracle's syntax - nothing more,
nothing less - like like some others.

Regards
Pavel

 --
  Bruce Momjian  br...@momjian.us        http://momjian.us
  EnterpriseDB                             http://enterprisedb.com

 --
 Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
 To make changes to your subscription:
 http://www.postgresql.org/mailpref/pgsql-hackers


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] functional call named notation clashes with SQL feature

2010-05-31 Thread Andrew Dunstan



Tom Lane wrote:

Bruce Momjian br...@momjian.us writes:
  

Yes, but if we are going to have to honor = eventually, shouldn't we
just do it now?  Supporting := and = seems confusing.



Personally, I haven't accepted the if part of that, therefore I
feel no need to argue over the then.


  


OK, but if that's going to influence the decision, let's debate it.

I think we should aim to comply with the spec, and incidentally be 
compatible with Oracle too. = is used by a number of other languages, 
for this or  a similar purpose, so it would feel a bit more intuitive 
and familiar to some people.


I don't have strong feelings about the timing - I'd be very surprised if 
:= were to be used in this context for any other purpose, so I don't 
think we'd be biting ourselves too much by just using that now. But if 
we do that, we should deprecate use of = as an operator now, and 
definitely remove its use in hstore either now or in 9.1.



cheers

andrew

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] functional call named notation clashes with SQL feature

2010-05-31 Thread Bruce Momjian
Pavel Stehule wrote:
  Part of the earlier discussion was about how = was a tempting
  operator name and other users may well have chosen it precisely
  because it's so evocative. But we don't actually have any evidence of
  that. Does anyone have any experience seeing = operators in the wild?
 
  Tangentially, I think the SQL committee chose = because the value, then
  variable, ordering is so unintuitive, and I think they wanted that
  ordering because most function calls use values so they wanted the
  variable at the end.
 
 maybe, maybe not. Maybe just adopt Oracle's syntax - nothing more,
 nothing less - like like some others.

Yea, definitely they were copying Oracle.  My point is that the odd
ordering does make sense, and the use of an arrow-like operator also
makes sense because of the odd ordering.

-- 
  Bruce Momjian  br...@momjian.ushttp://momjian.us
  EnterpriseDB http://enterprisedb.com

  + None of us is going to be here forever. +


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] functional call named notation clashes with SQL feature

2010-05-31 Thread Pavel Stehule
2010/5/31 Bruce Momjian br...@momjian.us:
 Pavel Stehule wrote:
  Part of the earlier discussion was about how = was a tempting
  operator name and other users may well have chosen it precisely
  because it's so evocative. But we don't actually have any evidence of
  that. Does anyone have any experience seeing = operators in the wild?
 
  Tangentially, I think the SQL committee chose = because the value, then
  variable, ordering is so unintuitive, and I think they wanted that
  ordering because most function calls use values so they wanted the
  variable at the end.

 maybe, maybe not. Maybe just adopt Oracle's syntax - nothing more,
 nothing less - like like some others.

 Yea, definitely they were copying Oracle.  My point is that the odd
 ordering does make sense, and the use of an arrow-like operator also
 makes sense because of the odd ordering.


What I know - this feature is supported only by Oracle and MSSQL now.
MSSQL syntax isn't available, because expected @ before variables. So
there is available only Oracle's syntax. It is some like industrial
standard.

Pavel


 --
  Bruce Momjian  br...@momjian.us        http://momjian.us
  EnterpriseDB                             http://enterprisedb.com

  + None of us is going to be here forever. +



-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] functional call named notation clashes with SQL feature

2010-05-31 Thread Bruce Momjian
Pavel Stehule wrote:
 2010/5/31 Bruce Momjian br...@momjian.us:
  Pavel Stehule wrote:
   Part of the earlier discussion was about how = was a tempting
   operator name and other users may well have chosen it precisely
   because it's so evocative. But we don't actually have any evidence of
   that. Does anyone have any experience seeing = operators in the wild?
  
   Tangentially, I think the SQL committee chose = because the value, then
   variable, ordering is so unintuitive, and I think they wanted that
   ordering because most function calls use values so they wanted the
   variable at the end.
 
  maybe, maybe not. Maybe just adopt Oracle's syntax - nothing more,
  nothing less - like like some others.
 
  Yea, definitely they were copying Oracle. ?My point is that the odd
  ordering does make sense, and the use of an arrow-like operator also
  makes sense because of the odd ordering.
 
 
 What I know - this feature is supported only by Oracle and MSSQL now.
 MSSQL syntax isn't available, because expected @ before variables. So
 there is available only Oracle's syntax. It is some like industrial
 standard.

MSSQL?  Are you sure?  This is the example posted in this thread:

EXEC dbo.GetItemPrice @ItemCode = 'GXKP', @PriceLevel = 5

and it more matches our := syntax than = in its argument ordering.

-- 
  Bruce Momjian  br...@momjian.ushttp://momjian.us
  EnterpriseDB http://enterprisedb.com

  + None of us is going to be here forever. +


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] PG 9.0 release timetable

2010-05-31 Thread Simon Riggs
On Mon, 2010-05-31 at 11:10 -0400, Tom Lane wrote:
 Simon Riggs si...@2ndquadrant.com writes:
  We're currently at 4 weeks since last beta, with no new beta in sight.
 
 Eh?
 http://archives.postgresql.org/pgsql-hackers/2010-05/msg01649.php
 You can hardly claim to have not seen it.

Yes, completely wrong. A sunny weekend away wiped my mind. Hopefully
that doesn't detract from the other points I made.

-- 
 Simon Riggs   www.2ndQuadrant.com


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] functional call named notation clashes with SQL feature

2010-05-31 Thread Pavel Stehule
2010/5/31 Bruce Momjian br...@momjian.us:
 Pavel Stehule wrote:
 2010/5/31 Bruce Momjian br...@momjian.us:
  Pavel Stehule wrote:
   Part of the earlier discussion was about how = was a tempting
   operator name and other users may well have chosen it precisely
   because it's so evocative. But we don't actually have any evidence of
   that. Does anyone have any experience seeing = operators in the wild?
  
   Tangentially, I think the SQL committee chose = because the value, then
   variable, ordering is so unintuitive, and I think they wanted that
   ordering because most function calls use values so they wanted the
   variable at the end.
 
  maybe, maybe not. Maybe just adopt Oracle's syntax - nothing more,
  nothing less - like like some others.
 
  Yea, definitely they were copying Oracle. ?My point is that the odd
  ordering does make sense, and the use of an arrow-like operator also
  makes sense because of the odd ordering.
 

 What I know - this feature is supported only by Oracle and MSSQL now.
 MSSQL syntax isn't available, because expected @ before variables. So
 there is available only Oracle's syntax. It is some like industrial
 standard.

 MSSQL?  Are you sure?  This is the example posted in this thread:

        EXEC dbo.GetItemPrice @ItemCode = 'GXKP', @PriceLevel = 5

 and it more matches our := syntax than = in its argument ordering.

it's not important in this discussion. Important is using some usual
symbol '=' or special symbol '='. Our syntax is probably only one
possible solution in this moment (there are minimum controversy), bud
semantic isn't best. Using same operator as assign statement uses can
be messy. I don't know what is a true - you have to ask of ADA
designers.

Regards
Pavel


 --
  Bruce Momjian  br...@momjian.us        http://momjian.us
  EnterpriseDB                             http://enterprisedb.com

  + None of us is going to be here forever. +



-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] PG 9.0 release timetable

2010-05-31 Thread Simon Riggs
On Mon, 2010-05-31 at 11:30 -0400, Bruce Momjian wrote:

 Well, they can just grab nightly snapshots and test, right?  I don't
 think a beta is fundamentally different from a nightly snapshot,
 source-code wise.

There is only one difference: the signal to re-test.

Most people read new beta as meaning we fixed the bugs, try again
now. The delivery mechanism is unimportant.

IMHO the amount of testing we get is directly proportional to number of
announced beta releases, since most tests get run in first week.

If packaging is the issue, lets announce packaged releases every 4 weeks
and non-packaged snapshots every 2 weeks.

-- 
 Simon Riggs   www.2ndQuadrant.com


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] functional call named notation clashes with SQL feature

2010-05-31 Thread Andrew Dunstan



Bruce Momjian wrote:

MSSQL?  Are you sure?  This is the example posted in this thread:

EXEC dbo.GetItemPrice @ItemCode = 'GXKP', @PriceLevel = 5

and it more matches our := syntax than = in its argument ordering.
  


I think you are seriously confused, or else you are seriously confusing 
me. The = proposal is to have the ordering param_name = 
passed_value, just as Oracle has, just as MSSQL  has @param_name = 
passed_value, and just as the := proposal would have param_name := 
passed_value.


cheers

andrew

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] functional call named notation clashes with SQL feature

2010-05-31 Thread Bruce Momjian
Andrew Dunstan wrote:
 
 
 Bruce Momjian wrote:
  MSSQL?  Are you sure?  This is the example posted in this thread:
 
  EXEC dbo.GetItemPrice @ItemCode = 'GXKP', @PriceLevel = 5
 
  and it more matches our := syntax than = in its argument ordering.

 
 I think you are seriously confused, or else you are seriously confusing 
 me. The = proposal is to have the ordering param_name = 
 passed_value, just as Oracle has, just as MSSQL  has @param_name = 
 passed_value, and just as the := proposal would have param_name := 
 passed_value.

You are right;  I am seriously confused.  I thought it was value =
variable.  I was wrong.

I now see the Oracle syntax matches the Perl hash assignment syntax.

   The = operator is helpful in documenting the
   correspondence between keys and values in hashes, and
   other paired elements in lists.

   %hash = ( $key = $value );
   login( $username = $password );

-- 
  Bruce Momjian  br...@momjian.ushttp://momjian.us
  EnterpriseDB http://enterprisedb.com

  + None of us is going to be here forever. +


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Adding xpath_exists function

2010-05-31 Thread Bruce Momjian
Mike Fowler wrote:
 Robert Haas wrote:
  Please email your patch to the list (replying to this email is fine)
  and add it here:
  https://commitfest.postgresql.org/action/commitfest_view/open

 Here's my patch, developed against HEAD, that adds the function 
 'xpath_exists'. The function is a lot simpler than originally thought, 
 so none of the string manipulation previously discussed was required. 
 I've also included some regression tests that test the function with and 
 without xml namespaces. I should  note that before I added my tests all 
 existing tests passed.
 
 One observation that can be made is that I've largely copied the 
 existing xpath function and altered it to use a different method from 
 the libxml API. I've done it to save me redoing all the namespace 
 handling, however it's apparent to me that if we wanted to expose more 
 of the libxml api we will quickly start having a lot of duplicate code. 
 I notice that refactoring existing code whilst adding new code is 
 generally frowned upon, so once this patch is accepted I will look to 
 refactor the xpath and xpath_exists function. I could even add an 
 xpath_count method at the same time ;) .
 
 Thanks in advance for any and all feedback,

I have added this to the next commit-fest:

https://commitfest.postgresql.org/action/commitfest_view?id=6

-- 
  Bruce Momjian  br...@momjian.ushttp://momjian.us
  EnterpriseDB http://enterprisedb.com

  + None of us is going to be here forever. +


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] 9.0b1: ERROR: btree index keys must be ordered by attribute

2010-05-31 Thread Bruce Momjian
Tom Lane wrote:
 Ian Barwick barw...@gmail.com writes:
  Apologies, slight cp error; correct version of query:
 
  SELECT ov.object_id
 FROM object_version ov
WHERE ov.object_id = 1
  AND ov.version =0
  AND ov.object_status_id = (
  SELECT MAX(ov1.object_status_id)
FROM object_version ov1
   WHERE ov1.object_id=ov.object_id
 AND ov1.version = ov.version
 AND ov1.lang = ov.lang
)
  AND ov.lang = 'en';
 
 Ah, I see it:
 
-  Index Scan Backward using 
 object_version_object_id_version_object_status_id_lang_key on object_version 
 ov1  (cost=0.00..8.27 rows=1 width=4)
  Index Cond: ((object_id = $0) AND (version = $1) AND 
 (lang = $2) AND (object_status_id IS NOT NULL))
 
 where
 
 regression=# \d object_version_object_id_version_object_status_id_lang_key
 Index public.object_version_object_id_version_object_status_id_lang_key
   Column  | Type |Definition
 --+--+--
  object_id| integer  | object_id
  version  | integer  | version
  object_status_id | integer  | object_status_id
  lang | character(2) | lang
 unique, btree, for table public.object_version
 
 The index-based-max code is throwing in the IS NOT NULL condition
 without thought for where it has to go in the index condition order.
 Will look into fixing this tomorrow.

FYI, this no longer throws an error in current CVS so was fixed by Tom.

-- 
  Bruce Momjian  br...@momjian.ushttp://momjian.us
  EnterpriseDB http://enterprisedb.com

  + None of us is going to be here forever. +


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] tsvector pg_stats seems quite a bit off.

2010-05-31 Thread Jesper Krogh

On 2010-05-30 20:02, Jan Urbański wrote:

Here's a patch against recent git, but should apply to 8.4 sources as
well. It would be interesting to measure the memory and time needed to
analyse the table after applying it, because we will be now using a lot
bigger bucket size and I haven't done any performance impact testing on
it. I updated the initial comment block in compute_tsvector_stats, but
the prose could probably be improved.
   

Just a small follow up. I tried out the patch (or actually a fresh git
checkout) and it now gives very accurate results for both upper and
lower end of the MCE-histogram with a lower cutoff that doesn't
approach 2.

Thanks alot.

--
Jesper

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] tsvector pg_stats seems quite a bit off.

2010-05-31 Thread Tom Lane
Jesper Krogh jes...@krogh.cc writes:
 Just a small follow up. I tried out the patch (or actually a fresh git
 checkout) and it now gives very accurate results for both upper and
 lower end of the MCE-histogram with a lower cutoff that doesn't
 approach 2.

Good.  How much did the ANALYZE time change for your table?

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Keepalive for max_standby_delay

2010-05-31 Thread Bruce Momjian

Uh, we have three days before we package 9.0beta2.  It would be good if
we could decide on the max_standby_delay issue soon.

---

Simon Riggs wrote:
 On Wed, 2010-05-26 at 16:22 -0700, Josh Berkus wrote:
   Just this second posted about that, as it turns out.
   
   I have a v3 *almost* ready of the keepalive patch. It still makes sense
   to me after a few days reflection, so is worth discussion and review. In
   or out, I want this settled within a week. Definitely need some RR
   here.
  
  Does the keepalive fix all the issues with max_standby_delay?  Tom?
 
 OK, here's v4.
 
 Summary
 
 * WALSender adds a timestamp onto the header of every WAL chunk sent.
 
 * Each WAL record now has a conceptual send timestamp that remains
 constant while that record is replayed. This is used as the basis from
 which max_standby_delay is calculated when required during replay.
 
 * Send timestamp is calculated as the later of the timestamp of chunk in
 which WAL record was sent and the latest XLog time.
 
 * WALSender sends an empty message as a keepalive when nothing else to
 send. (No longer a special message type for the keepalive).
 
 I think its close, but if there's a gaping hole here somewhere then I'll
 punt for this release.
 
 -- 
  Simon Riggs   www.2ndQuadrant.com

[ Attachment, skipping... ]

 
 -- 
 Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
 To make changes to your subscription:
 http://www.postgresql.org/mailpref/pgsql-hackers

-- 
  Bruce Momjian  br...@momjian.ushttp://momjian.us
  EnterpriseDB http://enterprisedb.com

  + None of us is going to be here forever. +


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] fillfactor gets set to zero for toast tables

2010-05-31 Thread Alvaro Herrera
Excerpts from Takahiro Itagaki's message of mié may 26 03:32:56 -0400 2010:
 
 Alvaro Herrera alvhe...@commandprompt.com wrote:
 
  Excerpts from Tom Lane's message of vie may 14 15:03:57 -0400 2010:
  
   Maybe a better solution is to have some kind of notion of a default-only
   entry, which is sufficient to insert the default into the struct but
   isn't accepted as a user-settable item.
  
  This patch (for 8.4, but applies fuzzily to 9.0) implements this idea.
  Note that there's no explicit check that every heap option has a
  corresponding toast option; that's left to the developer's judgement to
  add.  I added the new member to relopt_gen struct so that existing
  entries did not require changes in initializers.
 
 The new default_only field can be initialized only from the internal codes
 and is not exported to user definded reloptions. We could add an additional
 argument to add_xxx_reloption() functions, but it breaks ABI.

Do we really need default_only entries in user-defined reloptions?

We have yet to see any indication that anybody is using user-defined
reloptions at all ...  It'd be good to have an use case at least (if
only to ensure that the API we're providing is sufficient).

If in the future we determine that we need to offer user-defined
default_only reloptions, perhaps we can add new entry points in the
reloptions API.

-- 
Álvaro Herrera alvhe...@commandprompt.com
The PostgreSQL Company - Command Prompt, Inc.
PostgreSQL Replication, Consulting, Custom Development, 24x7 support

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: pg_start_backup and pg_stop_backup Re: [HACKERS] Re: [COMMITTERS] pgsql: Make CheckRequiredParameterValues() depend upon correct

2010-05-31 Thread Bruce Momjian
Robert Haas wrote:
 On Sat, May 8, 2010 at 10:40 PM, Tom Lane t...@sss.pgh.pa.us wrote:
  Bruce Momjian br...@momjian.us writes:
  Uh, did we decide that 'wal_keep_segments' was the best name for this
  GUC setting? ?I know we shipped beta1 using that name.
 
  I thought min_wal_segments was a reasonable proposal, but it wasn't
  clear if there was consensus or not.
 
 I think most people thought it was another reasonable choice, but I
 think the consensus position is probably something like it's about
 the same rather than it's definitely better.  We had one or two
 people with stronger opinions than that on either side, I believe.

Agreed the current name seems OK.  However, was there agreement that
wal_keep_segments = -1 should keep all WAL segements?  I can see that as
useful for cases where you are doing a dump to be transfered to the
slave, and not using archive_command.  This avoids the need for the set
a huge value solution.

-- 
  Bruce Momjian  br...@momjian.ushttp://momjian.us
  EnterpriseDB http://enterprisedb.com

  + None of us is going to be here forever. +


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] fillfactor gets set to zero for toast tables

2010-05-31 Thread Tom Lane
Alvaro Herrera alvhe...@commandprompt.com writes:
 Excerpts from Takahiro Itagaki's message of mié may 26 03:32:56 -0400 2010:
 The new default_only field can be initialized only from the internal codes
 and is not exported to user definded reloptions. We could add an additional
 argument to add_xxx_reloption() functions, but it breaks ABI.

 Do we really need default_only entries in user-defined reloptions?

 We have yet to see any indication that anybody is using user-defined
 reloptions at all ...  It'd be good to have an use case at least (if
 only to ensure that the API we're providing is sufficient).

There probably isn't anyone using them, yet, which seems to me to be
a good argument to fix any obvious deficiencies in the API *now*
before there actually is anyone who'll be affected.  In particular,
I suggest that 9.0 would be a good time to add an int flags parameter
to the add_xxx_reloption functions.  The first flag could be
default_only and we'd have room to add more later without another API
break.

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] functional call named notation clashes with SQL feature

2010-05-31 Thread Dimitri Fontaine
Pavel Stehule pavel.steh...@gmail.com writes:
 it's not important in this discussion. Important is using some usual
 symbol '=' or special symbol '='. Our syntax is probably only one
 possible solution in this moment (there are minimum controversy), bud
 semantic isn't best. Using same operator as assign statement uses can
 be messy. I don't know what is a true - you have to ask of ADA
 designers.

Well you assign a value to a named parameter, so I don't see the point.

Now SELECT myfunc(a := 1, b = 2); is about fine, the only point is that
the = operator looks good for associative things such as hstore, so
chances that it has been used are not so low.

I guess we could choose to go with := for 9.1 and revisit the =
situation after the SQL standard has settled on the new version. Maybe
this move would even have some impact now that we have a voice over
there.

Regards,
-- 
dim

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


[HACKERS] bitmap-index-scan faster than seq-scan on full-table-scan (gin index)

2010-05-31 Thread Jesper Krogh

Hi.

The test data a set of generated terms using this perl-script
http://shrek.krogh.cc/~jesper/build-test.pl
and http://shrek.krogh.cc/~jesper/words.txt

I have generated a test dataset with an average tsvector length of
around 250 and 200.000 tuples in the dataset.

Conceptually searching for the full dataset would always be fastest
solved by a seq-scan. The query planner enforces this so much, so not
even enable_seqscan=off can convince it to to something else. So in
the next two explain analyze I compare a query searching 99% of the
table up with a seqscan. The 98% case is enforced to be a 
bitmap-index-scan
I would expect the runtime of the seqscan to be shortest and the 
bitmap-index-scan
to be quite a lot larger, due to random access and the fact that the 
index-data

also needs to be read in from disk.

Bot runs are run with a freshly started postgresql backend and
echo 3  /proc/sys/vm/drop_caches so the os caching should not come 
into play.


ftstest=# EXPLAIN ANALYZE select id from ftstest where body_fts @@ 
to_tsquery('commonterm98');
 QUERY 
PLAN


 Bitmap Heap Scan on ftstest  (cost=6579.81..992733.57 rows=195976 
width=4) (actual time=4813.258..7081.277 rows=195976 loops=1)

   Recheck Cond: (body_fts @@ to_tsquery('commonterm98'::text))
   -  Bitmap Index Scan on ftstest_gist_idx  (cost=0.00..6530.82 
rows=195976 width=0) (actual time=4787.513..4787.513 rows=195976 loops=1)

 Index Cond: (body_fts @@ to_tsquery('commonterm98'::text))
 Total runtime: 7389.346 ms
(5 rows)

ftstest=# set enable_bitmapscan = off;
WARNING:  terminating connection because of crash of another server process
DETAIL:  The postmaster has commanded this server process to roll back 
the current transaction and exit, because another server process exited 
abnormally and possibly corrupted shared memory.
HINT:  In a moment you should be able to reconnect to the database and 
repeat your command.

server closed the connection unexpectedly
This probably means the server terminated abnormally
before or while processing the request.
The connection to the server was lost. Attempting reset: Succeeded.
ftstest=# set enable_bitmapscan = off;
SET
ftstest=# EXPLAIN ANALYZE select id from ftstest where body_fts @@ 
to_tsquery('commonterm98');

  QUERY PLAN
--
 Seq Scan on ftstest  (cost=0.00..1006314.00 rows=195976 width=4) 
(actual time=96.077..60092.080 rows=195976 loops=1)

   Filter: (body_fts @@ to_tsquery('commonterm98'::text))
 Total runtime: 60436.556 ms
(3 rows)

So searching the full table via a bitmap-index-scan is actually 9 times
cheaper than a seq-scan.  (same on 9.0b1 and 8.4).

Digging more into it reveals that the body_fts tsvector is indeed needed for
the filter in the SeqScan. The tsvector data is stored in a TOAST 
table and

the in the bitmap-index-scan case it only needs to read in the main table
for checking visibillity. In the end it translates to reading in 1.4GB of
TOAST-data vs. reading in 34MB of table data.

Thinking a bit, then I dont think this is a particular rare case, 
allthough the
ratio between the tables may be a real cornercase. The ratio is not 1:33 
in the
dataset that looks like the production dataset, but more 1:10, but in 
all cases
in production there would be a much higher cache-hit ratio on the 
gin-index
and the main table pages than on the TOAST table, so even with a ratio 
of 1:1

there most likely would be a real-world benefit.

Would it be possible to implement the Filtering using the gin-index and
a subsequent visibillity-check as on the index-scan?

The same end up being the case for queries ordered by btree indexes and
filtered by gin-indexes.

Jesper
--
Jesper

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] INSERT and parentheses

2010-05-31 Thread Bruce Momjian

I have added this to the next commit-fest:

https://commitfest.postgresql.org/action/commitfest_view?id=6


---

Marko Tiikkaja wrote:
 Hi,
 
 This came up on IRC today and I recall several instances of this during
 the last two months or so, so I decided to send a patch.  The problem in
 question occurs when you have extra parentheses in an INSERT list:
 
 INSERT INTO foo(a,b,c) SELECT (a,b,c) FROM ..; or
 INSERT INTO foo(a,b,c) VALUES((0,1,2));
 
 Both of these give you the same error:
 ERROR:  INSERT has more target columns than expressions
 
 The first version is a lot more common and as it turns out, is sometimes
 very hard to spot.  This patch attaches a HINT message to these two
 cases.  The message itself could probably be a lot better, but I can't
 think of anything.
 
 Thoughts?
 
 
 Regards,
 Marko Tiikkaja

[ Attachment, skipping... ]

 
 -- 
 Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
 To make changes to your subscription:
 http://www.postgresql.org/mailpref/pgsql-hackers

-- 
  Bruce Momjian  br...@momjian.ushttp://momjian.us
  EnterpriseDB http://enterprisedb.com

  + None of us is going to be here forever. +


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] improve plpgsql's EXECUTE 'select into' message with a hint

2010-05-31 Thread Bruce Momjian
Jaime Casanova wrote:
 Hi,
 
 while TFM says that we can use EXECUTE 'select ' INTO instead of the
 non implemented EXECUTE 'select ... into ', the message in plpgsql
 fails to say the same thing... seems like a HINT to me

Applied with attached patch.  Thanks.

-- 
  Bruce Momjian  br...@momjian.ushttp://momjian.us
  EnterpriseDB http://enterprisedb.com

  + None of us is going to be here forever. +

Index: src/pl/plpgsql/src/pl_exec.c
===
RCS file: /cvsroot/pgsql/src/pl/plpgsql/src/pl_exec.c,v
retrieving revision 1.257
diff -c -c -r1.257 pl_exec.c
*** src/pl/plpgsql/src/pl_exec.c	14 Apr 2010 23:52:10 -	1.257
--- src/pl/plpgsql/src/pl_exec.c	31 May 2010 19:59:45 -
***
*** 3033,3039 
  if (*ptr == 'S' || *ptr == 's')
  	ereport(ERROR,
  			(errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
! 	errmsg(EXECUTE of SELECT ... INTO is not implemented)));
  break;
  			}
  
--- 3033,3040 
  if (*ptr == 'S' || *ptr == 's')
  	ereport(ERROR,
  			(errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
! 	errmsg(EXECUTE of SELECT ... INTO is not implemented),
! 	errhint(You might want to use EXECUTE ... INTO instead.)));
  break;
  			}
  

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] why do we have rd_istemp?

2010-05-31 Thread Bruce Momjian
Tom Lane wrote:
 Robert Haas robertmh...@gmail.com writes:
  Given Relation rel, it looks to me like rel-rd_rel-relistemp will
  always give the same answer as rel-rd_istemp.  So why have both?
 
 Might be historical --- relistemp is pretty new.

Is this a TODO or something we want to clean up?

-- 
  Bruce Momjian  br...@momjian.ushttp://momjian.us
  EnterpriseDB http://enterprisedb.com

  + None of us is going to be here forever. +


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] bitmap-index-scan faster than seq-scan on full-table-scan (gin index)

2010-05-31 Thread Tom Lane
Jesper Krogh jes...@krogh.cc writes:
 Conceptually searching for the full dataset would always be fastest
 solved by a seq-scan. The query planner enforces this so much, so not
 even enable_seqscan=off can convince it to to something else.
 ...
 Would it be possible to implement the Filtering using the gin-index and
 a subsequent visibillity-check as on the index-scan?

You're failing to make any sense whatsoever.  If you're reading the full
dataset, there is no filter condition.  If there is a potentially
indexable filter condition, the planner will certainly consider that.

Personally I think the issue here has got more to do with the
non-immutability of the single-argument form of to_tsquery, which means
it gets re-evaluated at every row during a seqscan.  Do your results
change if you work with to_tsquery('english', ...)  (or whatever your
default TS config is)?

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] why do we have rd_istemp?

2010-05-31 Thread Tom Lane
Bruce Momjian br...@momjian.us writes:
 Tom Lane wrote:
 Robert Haas robertmh...@gmail.com writes:
 Given Relation rel, it looks to me like rel-rd_rel-relistemp will
 always give the same answer as rel-rd_istemp.  So why have both?
 
 Might be historical --- relistemp is pretty new.

 Is this a TODO or something we want to clean up?

Doesn't strike me that it's worth the amount of code that would have to
change.  rd_istemp is known in a lot of places.  Replacing it with a
double indirection doesn't seem attractive anyway.

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Unexpected page allocation behavior on insert-only tables

2010-05-31 Thread Tom Lane
I wrote:
 In particular, now that there's a distinction between smgr flush
 and relcache flush, maybe we could associate targblock reset with
 smgr flush (only) and arrange to not flush the smgr level during
 ANALYZE --- basically, smgr flush would only be needed when truncating
 or reassigning the relfilenode.  I think this might work out nicely but
 haven't chased the details.

I looked into that a bit more and decided that it'd be a ticklish
change: the coupling between relcache and smgr cache is pretty tight,
and there just isn't any provision for having an smgr cache entry live
longer than its owning relcache entry.  Even if we could fix it to
work reliably, this approach does nothing for the case where a backend
actually exits after filling just part of a new page, as noted by
Takahiro-san.

The next most promising fix is to have RelationGetBufferForTuple tell
the FSM about the new page immediately on creation.  I made a draft
patch for that (attached).  It fixes Michael's scenario nicely ---
all pages get filled completely --- and a simple test with pgbench
didn't reveal any obvious change in performance.  However there is
clear *potential* for performance loss, due to both the extra FSM
access and the potential for increased contention because of multiple
backends piling into the same new page.  So it would be good to do
some real performance testing on insert-heavy scenarios before we
consider applying this.  Any volunteers?

Note: patch is against HEAD but should work in 8.4, if you reverse out
the use of the rd_targblock access macros.

regards, tom lane

Index: src/backend/access/heap/hio.c
===
RCS file: /cvsroot/pgsql/src/backend/access/heap/hio.c,v
retrieving revision 1.78
diff -c -r1.78 hio.c
*** src/backend/access/heap/hio.c	9 Feb 2010 21:43:29 -	1.78
--- src/backend/access/heap/hio.c	31 May 2010 20:44:29 -
***
*** 354,384 
  	 * is empty (this should never happen, but if it does we don't want to
  	 * risk wiping out valid data).
  	 */
  	page = BufferGetPage(buffer);
  
  	if (!PageIsNew(page))
  		elog(ERROR, page %u of relation \%s\ should be empty but is not,
! 			 BufferGetBlockNumber(buffer),
! 			 RelationGetRelationName(relation));
  
  	PageInit(page, BufferGetPageSize(buffer), 0);
  
! 	if (len  PageGetHeapFreeSpace(page))
  	{
  		/* We should not get here given the test at the top */
  		elog(PANIC, tuple is too big: size %lu, (unsigned long) len);
  	}
  
  	/*
  	 * Remember the new page as our target for future insertions.
- 	 *
- 	 * XXX should we enter the new page into the free space map immediately,
- 	 * or just keep it for this backend's exclusive use in the short run
- 	 * (until VACUUM sees it)?	Seems to depend on whether you expect the
- 	 * current backend to make more insertions or not, which is probably a
- 	 * good bet most of the time.  So for now, don't add it to FSM yet.
  	 */
! 	RelationSetTargetBlock(relation, BufferGetBlockNumber(buffer));
  
  	return buffer;
  }
--- 354,388 
  	 * is empty (this should never happen, but if it does we don't want to
  	 * risk wiping out valid data).
  	 */
+ 	targetBlock = BufferGetBlockNumber(buffer);
  	page = BufferGetPage(buffer);
  
  	if (!PageIsNew(page))
  		elog(ERROR, page %u of relation \%s\ should be empty but is not,
! 			 targetBlock, RelationGetRelationName(relation));
  
  	PageInit(page, BufferGetPageSize(buffer), 0);
  
! 	pageFreeSpace = PageGetHeapFreeSpace(page);
! 	if (len  pageFreeSpace)
  	{
  		/* We should not get here given the test at the top */
  		elog(PANIC, tuple is too big: size %lu, (unsigned long) len);
  	}
  
  	/*
+ 	 * If using FSM, mark the page in FSM as having whatever amount of
+ 	 * free space will be left after our insertion.  This is needed so that
+ 	 * the free space won't be forgotten about if this backend doesn't use
+ 	 * it up before exiting or flushing the rel's relcache entry.
+ 	 */
+ 	if (use_fsm)
+ 		RecordPageWithFreeSpace(relation, targetBlock, pageFreeSpace - len);
+ 
+ 	/*
  	 * Remember the new page as our target for future insertions.
  	 */
! 	RelationSetTargetBlock(relation, targetBlock);
  
  	return buffer;
  }

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] bitmap-index-scan faster than seq-scan on full-table-scan (gin index)

2010-05-31 Thread Jesper Krogh

On 2010-05-31 22:09, Tom Lane wrote:

Jesper Kroghjes...@krogh.cc  writes:
   

Conceptually searching for the full dataset would always be fastest
solved by a seq-scan. The query planner enforces this so much, so not
even enable_seqscan=off can convince it to to something else.
...
Would it be possible to implement the Filtering using the gin-index and
a subsequent visibillity-check as on the index-scan?
 

You're failing to make any sense whatsoever.  If you're reading the full
dataset, there is no filter condition.  If there is a potentially
indexable filter condition, the planner will certainly consider that.
   

Yes, you're totally right on that (about making sense).

But that is because of the simplified use-case described. A more elaborate
description ..
I have a table with has a set of colums attached to it typically
used for sorting these columns may also be inferred on the table
by a typical join condition and a document that is fts-indexed.
So the actual use-case is that people query for:

give me the 1000 most recent documents matching term

Term may in some cases be hugely trivial, only filtering away 0.001% of the
dataset resulting in a index-scan on a btree date index filtering on the
tsvector column for term.

Term may also be something really specific only returning a single
or a few documents and just pushing a post-sorting to get the ordering.

But in the case where the query-planner falls over to a index-scan
on one of the btree-indices it ends up reading over from the TOAST data.

Will the planner consider doing the index-scan(forward or backwards)
on a btree-index and filter using a gin-index instead of filtering directly
on the tuple-data?
(I haven't been able to enforce an query-plan that looks like that).


Personally I think the issue here has got more to do with the
non-immutability of the single-argument form of to_tsquery, which means
it gets re-evaluated at every row during a seqscan.  Do your results
change if you work with to_tsquery('english', ...)  (or whatever your
default TS config is)?
   


It is english..  and yes it did indeed change the results. So the 
expensive case

dropped from ~60s to ~28s and the cheap case from ~7.3s to ~4.3s, that
is quite surprising that such small change can have that huge impact. The
single-argument version should be forbidden.

But the performance ratio between the two cases is still the same.

The test was actually run with the preliminary gincostestimate-patch from
Oleg Bartunov so the actual cost estimates match way better now, but that
should not impact the actual runtime.

Thanks

--
Jesper

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] dividing money by money

2010-05-31 Thread Andy Balholm
On May 30, 2010, at 6:53 AM, Kevin Grittner wrote:
 You would then generate a diff in context format and post to the
 -hackers list with that file as an attachment.  

Here it is:

dividing-money.diff
Description: Binary data

 Don't forget to add
 it to the CommitFest page:
 
 https://commitfest.postgresql.org/action/commitfest_view/open

I can't add it to the CommitFest page, since I don't have web access, just 
e-mail. Could you please take care of that part? (What is the CommitFest page, 
anyway?)
-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] functional call named notation clashes with SQL feature

2010-05-31 Thread David E. Wheeler
On May 31, 2010, at 8:56 AM, Andrew Dunstan wrote:

 I don't have strong feelings about the timing - I'd be very surprised if := 
 were to be used in this context for any other purpose, so I don't think we'd 
 be biting ourselves too much by just using that now. But if we do that, we 
 should deprecate use of = as an operator now, and definitely remove its use 
 in hstore either now or in 9.1.

+1

David


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] functional call named notation clashes with SQL feature

2010-05-31 Thread Tom Lane
David E. Wheeler da...@kineticode.com writes:
 On May 31, 2010, at 8:56 AM, Andrew Dunstan wrote:
 I don't have strong feelings about the timing - I'd be very surprised if := 
 were to be used in this context for any other purpose, so I don't think we'd 
 be biting ourselves too much by just using that now. But if we do that, we 
 should deprecate use of = as an operator now, and definitely remove its use 
 in hstore either now or in 9.1.

My feeling is that (a) there is no hurry to do anything about an
unreleased draft of the standard, and (b) perhaps Peter could lobby
the committee to change the standard before it does get published.

hstore's use of = is pretty well embedded already; waiting another
release or two before breaking things is not going to make it
significantly more painful.

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] is_absolute_path incorrect on Windows

2010-05-31 Thread Bruce Momjian
Magnus Hagander wrote:
 Here's a thread that incorrectly started on the security list, but really is
 more about functionality. Looking for comments:

I looked into this and it seems to be a serious issue.

 The function is_absolute_path() is incorrect on Windows. As it's implemented,
 it considers the following to be an absolute path:
 * Anything that starts with /
 * Anything that starst with \
 * Anything alphanumerical, followed by a colon, followed by either / or \
 
 Everything else is treated as relative.
 
 However, it misses the case with for example E:foo, which is a perfectly
 valid path on windows. Which isn't absolute *or* relative - it's relative
 to the current directory on the E: drive. Which will be the same as the
 current directory for the process *if* the process current directory is
 on drive E:. In other cases, it's a different directory.

I would argue that E:foo is always relative (which matches
is_absolute_path()).  If E: is the current drive of the process, it is
relative, and if the current drive is not E:, it is relative to the last
current drive on E: for that process, or the top level if there was no
current drive.  (Tested on XP.)

There seem to be three states:

1. absolute - already tested by is_absolute_path()
2. relative to the current directory (current drive)
3. relative on a different drive

We could probably develop code to test all three, but keep in mind that
the path itself can't distinguish between 2 and 3, and while you can
test the current drive, if the current drive changes, a 2 could become a
3, and via versa.

 This function is used in the genfile.c functions to read and list files
 by admin tools like pgadmin - to make sure we can only open files that are
 in our own data directory - by making sure they're either relative, or they're
 absolute but rooted in our own data directory. (It rejects anything with ..
 in it already).

So it is currently broken because you can read other drives?

 The latest step in that thread is this comment from Tom:
 
  Yeah.  I think the fundamental problem is that this code assumes there
  are two kinds of paths: absolute and relative to CWD.  But on Windows
  there's really a third kind, relative with a drive letter.  I believe
  that is_absolute_path is correct on its own terms, namely to identify a
  fully specified path.  If we change it to allow cases that aren't really
  fully specified we will break other uses, such as in make_absolute_path.
 
  I'm inclined to propose adding an additional path test operator, along
  the lines of has_drive_specifier(path) (always false on non-Windows),
  and use that where needed to reject relative-with-drive-letter paths.
 
 I think I agree with this point, but we all agreed that we should throw
 the question out for the wider audience on -hackers for more comments.

So, should this be implemented?

-- 
  Bruce Momjian  br...@momjian.ushttp://momjian.us
  EnterpriseDB http://enterprisedb.com

  + None of us is going to be here forever. +


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Show schema name on REINDEX DATABASE

2010-05-31 Thread Bruce Momjian
Greg Sabino Mullane wrote:
-- Start of PGP signed section.
 Patch attached to show the schema *and* table name when doing 
 a REINDEX DATABASE.

Yea, VACUUM VERBOSE shows the schema name, so REINDEX should as well. 
Patch applied.

-- 
  Bruce Momjian  br...@momjian.ushttp://momjian.us
  EnterpriseDB http://enterprisedb.com

  + None of us is going to be here forever. +


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] fillfactor gets set to zero for toast tables

2010-05-31 Thread Takahiro Itagaki

Tom Lane t...@sss.pgh.pa.us wrote:

 Alvaro Herrera alvhe...@commandprompt.com writes:
  Do we really need default_only entries in user-defined reloptions?

I think we don't, but I also think we don't need it at all even in the
core because it just set a few variables to the default values with
complex code flow. Could you explain why default_only entries idea is
better than adjusting those fields in the toast-specific codes?
It's my understanding that reloption-framework is just a tool to fill
reloption parameters, and it's not responsible for unused fields.

  We have yet to see any indication that anybody is using user-defined
  reloptions at all ...  It'd be good to have an use case at least (if
  only to ensure that the API we're providing is sufficient).

I use it my textsearch_senna extension :-).
But I don't need default_only entries at this time.

 I suggest that 9.0 would be a good time to add an int flags parameter
 to the add_xxx_reloption functions.  The first flag could be
 default_only and we'd have room to add more later without another API
 break.

I agree the idea when we reach a conclusion to introduce default_only.

Regards,
---
Takahiro Itagaki
NTT Open Source Software Center



-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] is_absolute_path incorrect on Windows

2010-05-31 Thread Greg Stark
On Fri, Apr 9, 2010 at 2:16 PM, Magnus Hagander mag...@hagander.net wrote:
 I'm inclined to propose adding an additional path test operator, along
 the lines of has_drive_specifier(path) (always false on non-Windows),
 and use that where needed to reject relative-with-drive-letter paths.

 I think I agree with this point, but we all agreed that we should throw
 the question out for the wider audience on -hackers for more comments.


If you invert the sense then it might not be so windows-specific:


/* NOTE: these two functions aren't complementary under windows,
 * be sure to use the right one */

/* Check path always means the same thing regardless of cwd */
is_absolute_path()
/* Check that path is under cwd */
is_relative_path()

-- 
greg

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] functional call named notation clashes with SQL feature

2010-05-31 Thread Florian Pflug
On Jun 1, 2010, at 0:23 , Tom Lane wrote:
 David E. Wheeler da...@kineticode.com writes:
 On May 31, 2010, at 8:56 AM, Andrew Dunstan wrote:
 I don't have strong feelings about the timing - I'd be very surprised if := 
 were to be used in this context for any other purpose, so I don't think 
 we'd be biting ourselves too much by just using that now. But if we do 
 that, we should deprecate use of = as an operator now, and definitely 
 remove its use in hstore either now or in 9.1.
 
 My feeling is that (a) there is no hurry to do anything about an
 unreleased draft of the standard, and (b) perhaps Peter could lobby
 the committee to change the standard before it does get published.
 
 hstore's use of = is pretty well embedded already; waiting another
 release or two before breaking things is not going to make it
 significantly more painful.


There might be some value in providing an alternative operator though, even if 
there is no definitive plan to deprecate '='.

hstore gained quite a few new features in 9.0 that might attract new users. If 
there is even a slight chance that '=' will be deprecated during the next few 
releases, it'd be nice to save these users the hassle of migration...

For text = text and text[] = text[] I'd propose '||' as an alternative, 
since they both combine their arguments, kind of a like a concatenation.
For hstore = text[] I'd suggest '' since the result's set of keys is the 
intersection of both argument's key-sets.

best regards,
Florian Pflug


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] functional call named notation clashes with SQL feature

2010-05-31 Thread Robert Haas
On Mon, May 31, 2010 at 9:24 PM, Florian Pflug f...@phlo.org wrote:
 On Jun 1, 2010, at 0:23 , Tom Lane wrote:
 David E. Wheeler da...@kineticode.com writes:
 On May 31, 2010, at 8:56 AM, Andrew Dunstan wrote:
 I don't have strong feelings about the timing - I'd be very surprised if 
 := were to be used in this context for any other purpose, so I don't think 
 we'd be biting ourselves too much by just using that now. But if we do 
 that, we should deprecate use of = as an operator now, and definitely 
 remove its use in hstore either now or in 9.1.

 My feeling is that (a) there is no hurry to do anything about an
 unreleased draft of the standard, and (b) perhaps Peter could lobby
 the committee to change the standard before it does get published.

 hstore's use of = is pretty well embedded already; waiting another
 release or two before breaking things is not going to make it
 significantly more painful.


 There might be some value in providing an alternative operator though, even 
 if there is no definitive plan to deprecate '='.

 hstore gained quite a few new features in 9.0 that might attract new users. 
 If there is even a slight chance that '=' will be deprecated during the next 
 few releases, it'd be nice to save these users the hassle of migration...

 For text = text and text[] = text[] I'd propose '||' as an alternative, 
 since they both combine their arguments, kind of a like a concatenation.
 For hstore = text[] I'd suggest '' since the result's set of keys is the 
 intersection of both argument's key-sets.

I was going to propose == across the board.

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise Postgres Company

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] [BUGS] BUG #5487: dblink failed with 63 bytes connection names

2010-05-31 Thread Takahiro Itagaki

Takahiro Itagaki itagaki.takah...@oss.ntt.co.jp wrote:

 Bug reference:  5487
 Logged by:  Takahiro Itagaki
 Email address:  itagaki.takah...@oss.ntt.co.jp
 Description:dblink failed with 63 bytes connection names
 Details: 
 
 Contib/dblink module seems to have a bug in handling
 connection names in NAMEDATALEN-1 bytes.

Here is a patch to fix the bug. I think it comes from wrong usage
of snprintf(NAMEDATALEN - 1). It just copies 62 bytes + \0.

In addition, it should be safe to use pg_mbcliplen() to truncate
extra bytes in connection names because we might return invalid
text when a multibyte character is at 62 or 63 bytes.

Note that the fix should be ported to previous versions, too.


 It cannot use exiting connections with 63 bytes name
 in some cases. For example, we cannot disconnect
 such connections. Also, we can reconnect with the
 same name and will have two connections with the name.
 
 =# SELECT dblink_connect(repeat('1234567890', 6) || 'ABC',
 'host=localhost');
  dblink_connect
 
  OK
 (1 row)
 
 =# SELECT dblink_get_connections();
   dblink_get_connections
 ---
  {123456789012345678901234567890123456789012345678901234567890ABC}
 (1 row)
 
 =# SELECT dblink_disconnect(repeat('1234567890', 6) || 'ABC');
 ERROR:  connection
 123456789012345678901234567890123456789012345678901234567890ABC not
 available

Regards,
---
Takahiro Itagaki
NTT Open Source Software Center



dblink_63bytes.patch
Description: Binary data

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] is_absolute_path incorrect on Windows

2010-05-31 Thread Giles Lean

Greg Stark gsst...@mit.edu wrote:

 /* NOTE: these two functions aren't complementary under windows,
  * be sure to use the right one */
 
 /* Check path always means the same thing regardless of cwd */
 is_absolute_path()
 /* Check that path is under cwd */
 is_relative_path()

Um ... isn't that second function name pretty misleading, if
what you want is what the comment above it says?

Assuming the comment is what you want (presumably, else you'd
just negate a test of is_absolute_path()) then my suggestions
for (IMHO :-) clearer names would be is_subdir_path() if you
still want path in the name, or just is_subdir() if the
meaning will be clear enough from context.

Giles


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] is_absolute_path incorrect on Windows

2010-05-31 Thread Bruce Momjian
Giles Lean wrote:
 
 Greg Stark gsst...@mit.edu wrote:
 
  /* NOTE: these two functions aren't complementary under windows,
   * be sure to use the right one */
  
  /* Check path always means the same thing regardless of cwd */
  is_absolute_path()
  /* Check that path is under cwd */
  is_relative_path()
 
 Um ... isn't that second function name pretty misleading, if
 what you want is what the comment above it says?
 
 Assuming the comment is what you want (presumably, else you'd
 just negate a test of is_absolute_path()) then my suggestions
 for (IMHO :-) clearer names would be is_subdir_path() if you
 still want path in the name, or just is_subdir() if the
 meaning will be clear enough from context.

is_relative_to_cwd()?

-- 
  Bruce Momjian  br...@momjian.ushttp://momjian.us
  EnterpriseDB http://enterprisedb.com

  + None of us is going to be here forever. +

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] functional call named notation clashes with SQL feature

2010-05-31 Thread David E. Wheeler
On May 31, 2010, at 7:40 PM, Robert Haas wrote:

 I was going to propose == across the board.

What about - ?

D

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] functional call named notation clashes with SQL feature

2010-05-31 Thread Tom Lane
David E. Wheeler da...@kineticode.com writes:
 On May 31, 2010, at 7:40 PM, Robert Haas wrote:
 I was going to propose == across the board.

 What about - ?

hstore already uses that for something else.

Robert's idea isn't a bad one if we're forced to rename the operator.
I'd still like to know exactly how hard the concrete has set on the
SQL spec draft, first.  (Peter?)

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] functional call named notation clashes with SQL feature

2010-05-31 Thread Bruce Momjian
Tom Lane wrote:
 David E. Wheeler da...@kineticode.com writes:
  On May 31, 2010, at 7:40 PM, Robert Haas wrote:
  I was going to propose == across the board.
 
  What about - ?
 
 hstore already uses that for something else.
 
 Robert's idea isn't a bad one if we're forced to rename the operator.
 I'd still like to know exactly how hard the concrete has set on the
 SQL spec draft, first.  (Peter?)

I don't know, but based on the fact it matches Oracle, I think it is
pretty well set by now.

If we can't come up with a good syntax (and there isn't an SQL standard
for it), we often review how Oracle or other databases handle such
cases, and my guess is that the SQL committee does the same thing.

-- 
  Bruce Momjian  br...@momjian.ushttp://momjian.us
  EnterpriseDB http://enterprisedb.com

  + None of us is going to be here forever. +

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] functional call named notation clashes with SQL feature

2010-05-31 Thread Robert Haas
On Mon, May 31, 2010 at 11:36 PM, Tom Lane t...@sss.pgh.pa.us wrote:
 David E. Wheeler da...@kineticode.com writes:
 On May 31, 2010, at 7:40 PM, Robert Haas wrote:
 I was going to propose == across the board.

 What about - ?

 hstore already uses that for something else.

 Robert's idea isn't a bad one if we're forced to rename the operator.
 I'd still like to know exactly how hard the concrete has set on the
 SQL spec draft, first.  (Peter?)

Given the way hstore uses -, another reasonable choice might be --

That way we'd have - and -- instead of - and ==

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise Postgres Company

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] functional call named notation clashes with SQL feature

2010-05-31 Thread Tom Lane
Bruce Momjian br...@momjian.us writes:
 Tom Lane wrote:
 I'd still like to know exactly how hard the concrete has set on the
 SQL spec draft, first.  (Peter?)

 I don't know, but based on the fact it matches Oracle, I think it is
 pretty well set by now.

Eh?  The SQL committee has a very long track record of blowing off
any and all Oracle syntaxes.  If we can point to good reasons to adjust
their syntax, they might still listen.  Or at least I'd like to hear
the opinion of our man on the ground before assuming they won't.

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] is_absolute_path incorrect on Windows

2010-05-31 Thread Giles Lean

Bruce Momjian br...@momjian.us wrote:

 is_relative_to_cwd()?

../../../../some/other/place/not/under/cwd

Names are hard, but if I understood the original post, the
revised function is intended to check that the directory is
below the current working directory.

If my understanding is wrong (always possible!) and it only
has to be on the same drive, then your name is probably better
although it doesn't mention 'drive' ... hrm.

is_on_current_drive()?  (Yuck.)
is_on_current_filesystem()?  (Yuck, but at least more general.)

I think we (or at least I) need some clarification from the
original poster about what the code is checking for in detail.

Cheers,

Giles



-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] functional call named notation clashes with SQL feature

2010-05-31 Thread Pavel Stehule
2010/6/1 Tom Lane t...@sss.pgh.pa.us:
 David E. Wheeler da...@kineticode.com writes:
 On May 31, 2010, at 7:40 PM, Robert Haas wrote:
 I was going to propose == across the board.

 What about - ?

 hstore already uses that for something else.

 Robert's idea isn't a bad one if we're forced to rename the operator.
 I'd still like to know exactly how hard the concrete has set on the
 SQL spec draft, first.  (Peter?)


I agree with Tom - we are not hurry. Creating some synonym operator
for hstore like == can be a good idea.

regards
Pavel

                        regards, tom lane

 --
 Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
 To make changes to your subscription:
 http://www.postgresql.org/mailpref/pgsql-hackers


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] tsvector pg_stats seems quite a bit off.

2010-05-31 Thread Jesper Krogh

On 2010-05-31 20:38, Tom Lane wrote:

Jesper Kroghjes...@krogh.cc  writes:
   

Just a small follow up. I tried out the patch (or actually a fresh git
checkout) and it now gives very accurate results for both upper and
lower end of the MCE-histogram with a lower cutoff that doesn't
approach 2.
 

Good.  How much did the ANALYZE time change for your table?
   

1.3m documents.

New code ( 3 runs):
statistics target 1000 = 155s/124s/110s
statictics target 100 = 86s/55s/61s
Old code:
statistics target 1000 = 158s/101s/99s
statistics target 100 = 90s/29s/33s

Somehow I think that the first run is the relevant one, its pretty much 
a dead disk test,
and I wouldn't expect that random sampling of tuples would have any sane 
caching
effect in a production system. But it looks like the algoritm is a bit 
slower.


Thanks again..

Jesper

--
Jesper

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Open Item: pg_controldata - machine readable?

2010-05-31 Thread Greg Smith

Takahiro Itagaki wrote:

The proposal by Joe Conway is adding a new contib module.
http://archives.postgresql.org/message-id/4b959d7a.6010...@joeconway.com
http://github.com/jconway/pg_controldata
  


There's a second one of these floating around now too:  
http://labs.omniti.com/trac/pgtreats/browser/trunk/contrib/control


--
Greg Smith  2ndQuadrant US  Baltimore, MD
PostgreSQL Training, Services and Support
g...@2ndquadrant.com   www.2ndQuadrant.us


--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers