[HACKERS] Inputting relative datetimes

2011-08-25 Thread Dean Rasheed
As background, I have an app that accepts user text input and casts it to a timestamp in order to produce reports. I use PostgreSQL's timestamp input conversion for this, since it gives a lot of flexibility, and can parse pretty much anything the users throw at it. It is also handy that it

Re: [HACKERS] Inputting relative datetimes

2011-08-25 Thread Vik Reykja
On Thu, Aug 25, 2011 at 11:39, Dean Rasheed dean.a.rash...@gmail.comwrote: My first thought was to have some general way of adding or subtracting an interval at the end of an input timestamp, eg. by adding another couple of special values - plus interval and minus interval. This would allow

Re: [HACKERS] Inputting relative datetimes

2011-08-25 Thread Dean Rasheed
On 25 August 2011 10:43, Vik Reykja vikrey...@gmail.com wrote: On Thu, Aug 25, 2011 at 11:39, Dean Rasheed dean.a.rash...@gmail.com wrote: My first thought was to have some general way of adding or subtracting an interval at the end of an input timestamp, eg. by adding another couple of

Re: [HACKERS] cheaper snapshots redux

2011-08-25 Thread Robert Haas
On Thu, Aug 25, 2011 at 1:55 AM, Markus Wanner mar...@bluegap.ch wrote: One difference with snapshots is that only the latest snapshot is of any interest. Theoretically, yes.  But as far as I understood, you proposed the backends copy that snapshot to local memory.  And copying takes some

Re: [HACKERS] patch to slightly improve clarity of a comment in postgresql.conf.sample

2011-08-25 Thread Robert Haas
On Wed, Aug 24, 2011 at 7:33 AM, Dougal Sutherland dou...@gmail.com wrote: The attached change to postgresql.conf.sample makes it more clear at a glance that the default value of listen_addresses is 'localhost', not 'localhost, *'. This would have saved a friend an hour or two of fiddling

Re: [HACKERS] Short document fix

2011-08-25 Thread Robert Haas
On Tue, Aug 23, 2011 at 4:53 PM, Hitoshi Harada umi.tan...@gmail.com wrote: In the CREATE DOMAIN reference page of the current HEAD, it says --- CREATE DOMAIN us_postal_code AS TEXT CHECK(   VALUE ~ '^\\d{5}$' OR VALUE ~ '^\\d{5}-\\d{4}$' ); --- but I believe it should conform the

Re: [HACKERS] patch to slightly improve clarity of a comment in postgresql.conf.sample

2011-08-25 Thread Robert Haas
On Thu, Aug 25, 2011 at 9:30 AM, Dougal Sutherland dou...@gmail.com wrote: Sure, I like that more. OK, done. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to

Re: [HACKERS] cheaper snapshots redux

2011-08-25 Thread Markus Wanner
Robert, On 08/25/2011 03:24 PM, Robert Haas wrote: My hope (and it might turn out that I'm an optimist) is that even with a reasonably small buffer it will be very rare for a backend to experience a wraparound condition. It certainly seems less likely than with the ring-buffer for imessages,

Re: [HACKERS] cheaper snapshots redux

2011-08-25 Thread Robert Haas
On Thu, Aug 25, 2011 at 10:19 AM, Markus Wanner mar...@bluegap.ch wrote: Note, however, that for imessages, I've also had the policy in place that a backend *must* consume its message before sending any.  And that I took great care for all receivers to consume their messages as early as

Re: [HACKERS] cheaper snapshots redux

2011-08-25 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: Well, one long-running transaction that only has a single XID is not really a problem: the snapshot is still small. But one very old transaction that also happens to have a large number of subtransactions all of which have XIDs assigned might be a

Re: [HACKERS] cheaper snapshots redux

2011-08-25 Thread Markus Wanner
Robert, On 08/25/2011 04:48 PM, Robert Haas wrote: What's a typical message size for imessages? Most message types in Postgres-R are just a couple bytes in size. Others, especially change sets, can be up to 8k. However, I think you'll have an easier job guaranteeing that backends consume their

Re: [HACKERS] cheaper snapshots redux

2011-08-25 Thread Markus Wanner
Tom, On 08/25/2011 04:59 PM, Tom Lane wrote: That's a good point. If the ring buffer size creates a constraint on the maximum number of sub-XIDs per transaction, you're going to need a fallback path of some sort. I think Robert envisions the same fallback path we already have:

Re: [HACKERS] cheaper snapshots redux

2011-08-25 Thread Robert Haas
On Thu, Aug 25, 2011 at 11:15 AM, Markus Wanner mar...@bluegap.ch wrote: On 08/25/2011 04:59 PM, Tom Lane wrote: That's a good point.  If the ring buffer size creates a constraint on the maximum number of sub-XIDs per transaction, you're going to need a fallback path of some sort. I think

Re: [HACKERS] Change format of FDW options used in \d* commands

2011-08-25 Thread Robert Haas
2011/8/19 Shigeru Hanada shigeru.han...@gmail.com: BTW, I noticed that pg_dump has same issue since 8.4, initial release of SQL/MED infrastructure.  If a FDW option was defined on any FDW object with a name which contains one of special characters such as space and parentheses, pg_dump

[HACKERS] A couple of issues with psql variable substitution

2011-08-25 Thread Tom Lane
On my way to do something else entirely, I came across a couple of things that are not very nice about psql's lexer: 1. Somebody broke the no-backtracking property back in 9.0 while adding quoted variable substitution. According to the flex manual, use of backtracking creates a performance

Re: [HACKERS] Change format of FDW options used in \d* commands

2011-08-25 Thread Robert Haas
2011/8/19 Shigeru Hanada shigeru.han...@gmail.com: Oops, good catch.  I've fixed psql to use quote_ident() for option_name, and modified regression tests to use special characters in option names.  Please try revised version of patch. This part looks good to me, too. Committed. -- Robert

Re: [HACKERS] A couple of issues with psql variable substitution

2011-08-25 Thread Robert Haas
On Thu, Aug 25, 2011 at 12:47 PM, Tom Lane t...@sss.pgh.pa.us wrote: On my way to do something else entirely, I came across a couple of things that are not very nice about psql's lexer: 1. Somebody broke the no-backtracking property back in 9.0 while adding quoted variable substitution.  

Re: [HACKERS] A couple of issues with psql variable substitution

2011-08-25 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: On Thu, Aug 25, 2011 at 12:47 PM, Tom Lane t...@sss.pgh.pa.us wrote: 1. Somebody broke the no-backtracking property back in 9.0 while adding quoted variable substitution. According to the flex manual, use of backtracking creates a performance penalty.

Re: [HACKERS] A couple of issues with psql variable substitution

2011-08-25 Thread Alvaro Herrera
Excerpts from Tom Lane's message of jue ago 25 14:00:57 -0300 2011: Robert Haas robertmh...@gmail.com writes: On Thu, Aug 25, 2011 at 12:47 PM, Tom Lane t...@sss.pgh.pa.us wrote: 1. Somebody broke the no-backtracking property back in 9.0 while adding quoted variable substitution. According

Re: [HACKERS] A couple of issues with psql variable substitution

2011-08-25 Thread Andrew Dunstan
On 08/25/2011 01:16 PM, Alvaro Herrera wrote: Excerpts from Tom Lane's message of jue ago 25 14:00:57 -0300 2011: Robert Haasrobertmh...@gmail.com writes: On Thu, Aug 25, 2011 at 12:47 PM, Tom Lanet...@sss.pgh.pa.us wrote: 1. Somebody broke the no-backtracking property back in 9.0 while

Re: [HACKERS] A couple of issues with psql variable substitution

2011-08-25 Thread Tom Lane
Andrew Dunstan and...@dunslane.net writes: We could just add -b unconditionally to the flex flags and then count the number of lines in lex.backup. If it's greater that 1 whine loudly, or even fail, otherwise remove lex.backup. Would that avoid locale dependencies? Hm, yeah, seems like

Re: [HACKERS] A couple of issues with psql variable substitution

2011-08-25 Thread Tom Lane
I wrote: Andrew Dunstan and...@dunslane.net writes: We could just add -b unconditionally to the flex flags and then count the number of lines in lex.backup. If it's greater that 1 whine loudly, or even fail, otherwise remove lex.backup. Would that avoid locale dependencies? Hm, yeah,

Re: [HACKERS] WIP: Fast GiST index build

2011-08-25 Thread Heikki Linnakangas
On 24.08.2011 16:57, Alexander Korotkov wrote: I've added some testing results to the wiki page: http://wiki.postgresql.org/wiki/Fast_GiST_index_build_GSoC_2011 There are not all the results I planned for the first chunk because it takes more time than I expect. Some notes about it. Now I see

Re: [HACKERS] Backup's from standby

2011-08-25 Thread Robert Haas
On Fri, Aug 19, 2011 at 9:38 AM, Valentine Gogichashvili val...@gmail.com wrote: What issue we may face if you take a backups(includes data dir + wal files) at standby without LVM snapshot? The backup might be corrupted in arbitrary ways. And what will happen, if one issues a

[HACKERS] tsvector concatenation - backend crash

2011-08-25 Thread Jesper Krogh
Hi Attached SQL files gives (at least in my hands) a reliable backend crash with this stacktrace .. reproduced on both 9.0.4 and HEAD. I'm sorry I cannot provide a more trimmed down set of vectors the reproduces the bug, thus the obsfucated dataset. But even deleting single terms in the

Re: [HACKERS] [v9.1] sepgsql - userspace access vector cache

2011-08-25 Thread Robert Haas
On Fri, Aug 5, 2011 at 2:36 PM, Kohei KaiGai kai...@kaigai.gr.jp wrote: BTW, what is the current status of this patch? The status of contrib/sepgsql part is unclear for me, although we agreed that syscache is suitable mechanism for security labels. Sorry it's taken me a while to get around to

Re: [HACKERS] [GENERAL] pg_upgrade problem

2011-08-25 Thread Bruce Momjian
hubert depesz lubaczewski wrote: hi I have 8.3.11 database, ~ 600GB in size. I want to upgrade it to 9.0. First, I tried with 9.0.4, and when I hit problem (the same) I tried git, head of 9.0 branch. Good. pg_upgrade_dump_db.sql- pg_upgrade_dump_db.sql--- For binary upgrade, must

Re: [HACKERS] [GENERAL] pg_upgrade problem

2011-08-25 Thread hubert depesz lubaczewski
On Thu, Aug 25, 2011 at 04:33:07PM -0400, Bruce Momjian wrote: The problem appears to be that the Postgres catalogs think there is a toast table for 'actions', while the file system doesn't seem to have such a file. I can you look in pg_class and verify that? SELECT reltoastrelid FROM

Re: [HACKERS] [GENERAL] pg_upgrade problem

2011-08-25 Thread Bruce Momjian
hubert depesz lubaczewski wrote: On Thu, Aug 25, 2011 at 04:33:07PM -0400, Bruce Momjian wrote: The problem appears to be that the Postgres catalogs think there is a toast table for 'actions', while the file system doesn't seem to have such a file. I can you look in pg_class and verify

Re: [HACKERS] SSI 2PC coverage

2011-08-25 Thread Alvaro Herrera
Excerpts from Tom Lane's message of mié ago 24 22:11:58 -0300 2011: Alvaro Herrera alvhe...@commandprompt.com writes: After having to play with this, I didn't like it very much, because regression.diffs gets spammed with the (rather massive and completely useless) diff in that test. For

Re: [HACKERS] [GENERAL] pg_upgrade problem

2011-08-25 Thread hubert depesz lubaczewski
On Thu, Aug 25, 2011 at 04:43:02PM -0400, Bruce Momjian wrote: Please check the old cluster. Sure: =# SELECT reltoastrelid FROM pg_class WHERE relname = 'actions';

[HACKERS] PATCH: regular logging of checkpoint progress

2011-08-25 Thread Tomas Vondra
Hello, I'd like to propose a small patch that allows better checkpoint progress monitoring. The patch is quite simple - it adds a new integer GUC checkpoint_update_limit and every time checkpoint writes this number of buffers, it does two things: (a) logs a checkpoint status message into the

Re: [HACKERS] A couple of issues with psql variable substitution

2011-08-25 Thread Tom Lane
While I'm looking at this ... the current implementation has got a number of very inconsistent behaviors with respect to when it will expand a variable reference within a psql meta-command argument. Observe: regression=# \set foo 'value of foo' regression=# \set bar 'value of bar' regression=#

Re: [HACKERS] A couple of issues with psql variable substitution

2011-08-25 Thread Robert Haas
On Thu, Aug 25, 2011 at 5:12 PM, Tom Lane t...@sss.pgh.pa.us wrote: While I'm looking at this ... the current implementation has got a number of very inconsistent behaviors with respect to when it will expand a variable reference within a psql meta-command argument. Observe: regression=#

Re: [HACKERS] A couple of issues with psql variable substitution

2011-08-25 Thread Andrew Dunstan
On 08/25/2011 02:45 PM, Tom Lane wrote: I wrote: Andrew Dunstanand...@dunslane.net writes: We could just add -b unconditionally to the flex flags and then count the number of lines in lex.backup. If it's greater that 1 whine loudly, or even fail, otherwise remove lex.backup. Would that

Re: [HACKERS] Inputting relative datetimes

2011-08-25 Thread Merlin Moncure
On Thu, Aug 25, 2011 at 5:08 AM, Dean Rasheed dean.a.rash...@gmail.com wrote: On 25 August 2011 10:43, Vik Reykja vikrey...@gmail.com wrote: On Thu, Aug 25, 2011 at 11:39, Dean Rasheed dean.a.rash...@gmail.com wrote: My first thought was to have some general way of adding or subtracting an

Re: [HACKERS] Buffering GiST leaf pages too

2011-08-25 Thread Jim Nasby
On Aug 23, 2011, at 2:03 AM, Heikki Linnakangas wrote: While looking at Alexander's GiST fastbuild patch, which adds buffers to internal nodes to avoid random I/O during index build, it occurred to me that inserting the tuples to the leaf pages one at a time is quite inefficient too, even

Re: [HACKERS] Why doesn't psql use the information schema to get ACL description ?

2011-08-25 Thread Jim Nasby
On Aug 23, 2011, at 1:57 PM, Peter Eisentraut wrote: Ultimately my ulterior motive is to make sure the information schema is actually useful and ensure that it maintains that status. The information schema only exposes privileges defined in the SQL standard on objects defined in the SQL

Re: [HACKERS] pg_restore --no-post-data and --post-data-only

2011-08-25 Thread Jim Nasby
On Aug 24, 2011, at 7:43 PM, Josh Berkus wrote: On 8/23/11 1:30 PM, Andrew Dunstan wrote: Attached is an undocumented patch that allows pg_restore to omit post-data items or omit all but post-data items. This has been discussed before, and Simon sent in a patch back on 2008, which has

Re: [HACKERS] pg_restore --no-post-data and --post-data-only

2011-08-25 Thread Andrew Dunstan
On 08/25/2011 06:05 PM, Jim Nasby wrote: On Aug 24, 2011, at 7:43 PM, Josh Berkus wrote: On 8/23/11 1:30 PM, Andrew Dunstan wrote: Attached is an undocumented patch that allows pg_restore to omit post-data items or omit all but post-data items. This has been discussed before, and Simon sent

Re: [HACKERS] PATCH: regular logging of checkpoint progress

2011-08-25 Thread Josh Berkus
Tomas, I'd like to propose a small patch that allows better checkpoint progress monitoring. The patch is quite simple - it adds a new integer GUC checkpoint_update_limit and every time checkpoint writes this number of buffers, it does two things: I'd rather not have a new GUC if we can avoid

Re: [HACKERS] Inputting relative datetimes

2011-08-25 Thread Josh Berkus
Yes, but what I am trying to achieve is a way of entering such relative timestamps using a single input value, so that absolute and relative timestamps can both be bound to a SQL query using just one variable. I think adding a function would be the way to go then. Maybe extending

Re: [HACKERS] cheaper snapshots redux

2011-08-25 Thread Jim Nasby
On Aug 25, 2011, at 8:24 AM, Robert Haas wrote: My hope (and it might turn out that I'm an optimist) is that even with a reasonably small buffer it will be very rare for a backend to experience a wraparound condition. For example, consider a buffer with ~6500 entries, approximately 64 *

Re: [HACKERS] cheaper snapshots redux

2011-08-25 Thread Jim Nasby
On Aug 22, 2011, at 6:22 PM, Robert Haas wrote: With respect to a general-purpose shared memory allocator, I think that there are cases where that would be useful to have, but I don't think there are as many of them as many people seem to think. I wouldn't choose to implement this using a

Re: [HACKERS] PATCH: regular logging of checkpoint progress

2011-08-25 Thread Tomas Vondra
On 26 Srpen 2011, 0:18, Josh Berkus wrote: Tomas, I'd like to propose a small patch that allows better checkpoint progress monitoring. The patch is quite simple - it adds a new integer GUC checkpoint_update_limit and every time checkpoint writes this number of buffers, it does two things:

Re: [HACKERS] PATCH: regular logging of checkpoint progress

2011-08-25 Thread Tomas Vondra
On 26 Srpen 2011, 0:39, Tomas Vondra wrote: On 26 Srpen 2011, 0:18, Josh Berkus wrote: Tomas, I'd like to propose a small patch that allows better checkpoint progress monitoring. The patch is quite simple - it adds a new integer GUC checkpoint_update_limit and every time checkpoint writes

[HACKERS] pg_upgrade problem (fwd)

2011-08-25 Thread Bruce Momjian
I am sending this report to hackers to all hacker subscribers can read the original bug report. --- - Forwarded message from hubert depesz lubaczewski - hi I have 8.3.11 database, ~ 600GB in size. I want to

Re: [HACKERS] Why doesn't psql use the information schema to get ACL description ?

2011-08-25 Thread Dave Cramer
On Thu, Aug 25, 2011 at 5:51 PM, Jim Nasby j...@nasby.net wrote: On Aug 23, 2011, at 1:57 PM, Peter Eisentraut wrote: Ultimately my ulterior motive is to make sure the information schema is actually useful and ensure that it maintains that status. The information schema only exposes

Re: [HACKERS] A couple of issues with psql variable substitution

2011-08-25 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: On Thu, Aug 25, 2011 at 5:12 PM, Tom Lane t...@sss.pgh.pa.us wrote: ISTM the general rule ought to be that we attempt to substitute for a colon-construct regardless of where it appears within an argument, as long as it's not within quotes. My main

Re: [HACKERS] tsvector concatenation - backend crash

2011-08-25 Thread Tom Lane
Jesper Krogh jes...@krogh.cc writes: Attached SQL files gives (at least in my hands) a reliable backend crash with this stacktrace .. reproduced on both 9.0.4 and HEAD. I'm sorry I cannot provide a more trimmed down set of vectors the reproduces the bug, thus the obsfucated dataset. But even

[HACKERS] dropdb and dropuser: IF EXISTS

2011-08-25 Thread Josh Kupershmidt
I noticed a few places where it would be handy if dropdb took a flag like --if-exists which would basically just add in the 'IF EXISTS' clause to the DROP DATABASE statement. For example, scripts like find_static or mbregress.sh use dropdb createdb, but they generate noisy errors from dropdb when

Re: [HACKERS] [GENERAL] pg_upgrade problem

2011-08-25 Thread Bruce Momjian
OK, this was very helpful. I found out that there is a bug in current 9.0.X, 9.1.X, and HEAD that I introduced recently when I excluded temp tables. (The bug is not in any released version of pg_upgrade.) The attached, applied patches should fix it for you. I assume you are running 9.0.X, and

Re: [HACKERS] Removal of useless include references

2011-08-25 Thread Tom Lane
Bruce Momjian br...@momjian.us writes: It has been years since I ran src/tools/pginclude/pgrminclude to remove unnecessary include files. (I have already fixed things so include files can be compiled on their own.) The attached patch removes unneeded include references, and marks some

Re: [HACKERS] Buffering GiST leaf pages too

2011-08-25 Thread Heikki Linnakangas
On 26.08.2011 00:45, Jim Nasby wrote: I've often wondered about the per-tuple overhead of all kinds of operations, not just GiST index builds. For example, if you're doing a seqscan, ISTM it would be a lot more efficient to memcpy an entire page into backend-local memory and operate off of

Re: [HACKERS] tsvector concatenation - backend crash

2011-08-25 Thread Jesper Krogh
On 2011-08-26 05:28, Tom Lane wrote: Jesper Kroghjes...@krogh.cc writes: Attached SQL files gives (at least in my hands) a reliable backend crash with this stacktrace .. reproduced on both 9.0.4 and HEAD. I'm sorry I cannot provide a more trimmed down set of vectors the reproduces the bug,