Re: [HACKERS] [PATCH] add --progress option to pgbench (submission 3)

New submission which put option help in alphabetical position, as per Peter Eisentraut f0ed3a8a99b052d2d5e0b6153a8907b90c486636 This is for reference to the next commitfest. Patch update after conflict induced by pg-indentation, for the next commitfest. -- Fabien.diff --git a/contrib/pgben

Re: [HACKERS] pg_dump with postgis extension dumps rules separately

On Wed, May 29, 2013 at 6:55 PM, Joe Conway wrote: >> OK, simple enough. New patch attached. I still need to do some >> testing to verify this does not break anything, but other than >> that, any complaints (including the notion of backpatching this >> back to 9.1)? > > Here's a cleaned up version

Re: [HACKERS] extensible external toast tuple support

On Thu, May 30, 2013 at 7:42 AM, Andres Freund wrote: > In > http://archives.postgresql.org/message-id/20130216164231.GA15069%40awork2.anarazel.de > I presented the need for 'indirect' toast tuples which point into memory > instead of a toast table. In the comments to that proposal, off-list and >

Re: [HACKERS] Freezing without write I/O

On Fri, May 31, 2013 at 1:26 PM, Bruce Momjian wrote: > On Thu, May 30, 2013 at 10:04:23PM -0400, Robert Haas wrote: >> > Hm. Why? If freezing gets notably cheaper I don't really see much need >> > for keeping that much clog around? If we still run into anti-wraparound >> > areas, there has to be

Re: [HACKERS] removing PD_ALL_VISIBLE

On Fri, May 31, 2013 at 3:45 PM, Josh Berkus wrote: > On 05/31/2013 12:22 PM, Stephen Frost wrote: >> Where I'm going with this whole thing is simply that I do worry a bit >> about using a bitmap for freeze, or similar, information and not being >> able to reconstruct that bitmap from the heap. P

Re: [HACKERS] removing PD_ALL_VISIBLE

On Fri, May 31, 2013 at 1:44 PM, Bruce Momjian wrote: > On Fri, May 31, 2013 at 10:28:12AM -0700, Josh Berkus wrote: >> >> Isn't the visibility map already required for proper return results as >> >> we use it for index-only scans. I think the optimization-only ship has >> >> sailed. >> > >> > At

Re: [HACKERS] fallocate / posix_fallocate for new WAL file creation (etc...)

On Thu, May 30, 2013 at 02:58:26PM +0200, Andres Freund wrote: > > * Andres Freund (and...@2ndquadrant.com) wrote: > > > But really, I am not at all concerned about some obscure values being > > > returned, but about a read() not being successful.. > After a bit of standard perusing writing a sing

Re: [HACKERS] 9.3: Empty arrays returned by array_remove()

On Fri, May 31, 2013 at 08:55:49AM +0100, Dean Rasheed wrote: > On 31 May 2013 08:34, Brendan Jurd wrote: > > On 31 May 2013 02:52, Dean Rasheed wrote: > >> Testing 9.3beta, it seems that array_remove() may return an empty 1-d > >> array whose upper bound is lower than its lower bound. I know tha

Re: [HACKERS] Running pgindent

On Fri, May 31, 2013 at 04:57:20PM -0400, Stephen Frost wrote: > * Bruce Momjian (br...@momjian.us) wrote: > > On Fri, May 31, 2013 at 03:51:30PM -0400, Andrew Dunstan wrote: > > > I spent quite a lot of time trying to make the tool behave the same > > > as the old script. > > > > Yes, and I belie

Re: [HACKERS] Running pgindent

* Bruce Momjian (br...@momjian.us) wrote: > On Fri, May 31, 2013 at 03:51:30PM -0400, Andrew Dunstan wrote: > > I spent quite a lot of time trying to make the tool behave the same > > as the old script. > > Yes, and I believe we tested running the Perl version to make sure it > was the same, so th

Re: [HACKERS] removing PD_ALL_VISIBLE

* Josh Berkus (j...@agliodbs.com) wrote: > On 05/31/2013 12:22 PM, Stephen Frost wrote: > > Where I'm going with this whole thing is simply that I do worry a bit > > about using a bitmap for freeze, or similar, information and not being > > able to reconstruct that bitmap from the heap. Perhaps th

Re: [HACKERS] Unsigned integer types

If you do it, having uint1 (1 byte) would be nice as well. There is a signed 1byte int on PGXN, FWIW: http://pgxn.org/extension/tinyint That's good, thanks for the pointer! However, it is a signed tinyint (-128..127 range), not an unsigned one. -- Fabien. -- Sent via pgsql-hackers maili

Re: [HACKERS] Running pgindent

On Fri, May 31, 2013 at 03:51:30PM -0400, Andrew Dunstan wrote: > > On 05/29/2013 11:41 PM, Bruce Momjian wrote: > >On Wed, May 29, 2013 at 10:08:10PM -0400, Stephen Frost wrote: > >>* Bruce Momjian (br...@momjian.us) wrote: > >>>Wow, uh, yeah, I guess we could do that. I will await more feedback

Re: [HACKERS] detecting binary backup in progress

On 2013-05-31 22:53:14 +0300, Heikki Linnakangas wrote: > On 31.05.2013 22:36, Andres Freund wrote: > >On 2013-05-31 22:29:45 +0300, Heikki Linnakangas wrote: > >>Note that pg_is_in_backup() just checks for presence of > >>$PGDATA/backup_label. Also note that pg_basebackup doesn't create > >>backup

Re: [HACKERS] detecting binary backup in progress

Heikki Linnakangas wrote: > The manual says: > >pg_is_in_backup()boolTrue if an on-line exclusive backup is still in > >progress. > > So clearly that is intentional. That could use some rephrasing, > though; a layman won't know what an "exclusive backup" is. Heck, I don't understand wha

Re: [HACKERS] detecting binary backup in progress

On 31.05.2013 22:36, Andres Freund wrote: On 2013-05-31 22:29:45 +0300, Heikki Linnakangas wrote: Note that pg_is_in_backup() just checks for presence of $PGDATA/backup_label. Also note that pg_basebackup doesn't create backup_label in the server. It's included in the backup that's sent to the c

Re: [HACKERS] Running pgindent

On 05/29/2013 11:41 PM, Bruce Momjian wrote: On Wed, May 29, 2013 at 10:08:10PM -0400, Stephen Frost wrote: * Bruce Momjian (br...@momjian.us) wrote: Wow, uh, yeah, I guess we could do that. I will await more feedback. Please don't. I'm already rather concerned by this one. It looks like t

Re: [HACKERS] removing PD_ALL_VISIBLE

On 05/31/2013 12:22 PM, Stephen Frost wrote: > Where I'm going with this whole thing is simply that I do worry a bit > about using a bitmap for freeze, or similar, information and not being > able to reconstruct that bitmap from the heap. Perhaps that's overly > paranoid, but, well, we also write

Re: [HACKERS] detecting binary backup in progress

-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 05/31/2013 12:36 PM, Andres Freund wrote: > On 2013-05-31 22:29:45 +0300, Heikki Linnakangas wrote: >> Note that pg_is_in_backup() just checks for presence of >> $PGDATA/backup_label. Also note that pg_basebackup doesn't >> create backup_label in t

Re: [HACKERS] detecting binary backup in progress

On 2013-05-31 22:29:45 +0300, Heikki Linnakangas wrote: > On 31.05.2013 21:41, Joe Conway wrote: > >On 05/31/2013 10:46 AM, Heikki Linnakangas wrote: > >>On 31.05.2013 20:38, Joe Conway wrote: > >>>I can check for the presence of $PGDATA/backup_label in order to > >>>detect a backup in progress (i.

Re: [HACKERS] detecting binary backup in progress

-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 05/31/2013 12:29 PM, Heikki Linnakangas wrote: > Note that pg_is_in_backup() just checks for presence of > $PGDATA/backup_label. Also note that pg_basebackup doesn't create > backup_label in the server. It's included in the backup that's sent > to

Re: [HACKERS] detecting binary backup in progress

On 31.05.2013 21:41, Joe Conway wrote: On 05/31/2013 10:46 AM, Heikki Linnakangas wrote: On 31.05.2013 20:38, Joe Conway wrote: I can check for the presence of $PGDATA/backup_label in order to detect a backup in progress (i.e. pg_start_backup() has been run and pg_stop_backup() has not yet been

Re: [HACKERS] removing PD_ALL_VISIBLE

* Bruce Momjian (br...@momjian.us) wrote: > Right, and it is hard to see that losing and adding are somehow > more/less likely, so it seems we already realy on the visibility map > being correct. Yes, but there's also a way to get *back* to a valid state if things go south with the visibility map.

Re: [HACKERS] detecting binary backup in progress

-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 05/31/2013 10:46 AM, Heikki Linnakangas wrote: > On 31.05.2013 20:38, Joe Conway wrote: >> I can check for the presence of $PGDATA/backup_label in order to >> detect a backup in progress (i.e. pg_start_backup() has been run >> and pg_stop_backup() h

Re: [HACKERS] detecting binary backup in progress

-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 05/31/2013 10:42 AM, Andres Freund wrote: > On 2013-05-31 10:38:56 -0700, Joe Conway wrote: >> I can check for the presence of $PGDATA/backup_label in order to >> detect a backup in progress (i.e. pg_start_backup() has been run >> and pg_stop_backup

Re: [HACKERS] removing PD_ALL_VISIBLE

On Fri, May 31, 2013 at 11:00:19AM -0700, Josh Berkus wrote: > Bruce, > > > Roberts statement was: > > > >> Loss or corruption of a single visibility map page means possible loss > >> of half a gigabyte of data. > > I fail to be alarmed at this; currently losing a single page of the clog > cause

Re: [HACKERS] removing PD_ALL_VISIBLE

Bruce, > Roberts statement was: > >> Loss or corruption of a single visibility map page means possible loss >> of half a gigabyte of data. I fail to be alarmed at this; currently losing a single page of the clog causes just as widespread corruption (worse, actually, since it's not confined to on

Re: [HACKERS] Behavior of a pg_trgm index for 2 (or < 3) character LIKE queries

On Fri, May 31, 2013 at 9:41 PM, Amit Langote wrote: > On Sat, Jun 1, 2013 at 1:48 AM, Sawada Masahiko > wrote: > > On Fri, May 31, 2013 at 11:16 AM, Amit Langote > wrote: > >> 2) And if that is so, is there problem in gin_extract_query_trgm(), > >> that is while generating trigrams from a query

Re: [HACKERS] detecting binary backup in progress

On 31.05.2013 20:38, Joe Conway wrote: I can check for the presence of $PGDATA/backup_label in order to detect a backup in progress (i.e. pg_start_backup() has been run and pg_stop_backup() has not yet been run). However there is a period of time after pg_start_backup() is first executed to when

Re: [HACKERS] removing PD_ALL_VISIBLE

On Fri, May 31, 2013 at 10:28:12AM -0700, Josh Berkus wrote: > > >> Isn't the visibility map already required for proper return results as > >> we use it for index-only scans. I think the optimization-only ship has > >> sailed. > > > > At the moment we can remove it without causing corruption. I

Re: [HACKERS] detecting binary backup in progress

On 31 May 2013 18:38, Joe Conway wrote: > -BEGIN PGP SIGNED MESSAGE- > Hash: SHA1 > > I can check for the presence of $PGDATA/backup_label in order to detect > a backup in progress (i.e. pg_start_backup() has been run and > pg_stop_backup() has not yet been run). > > However there is a per

Re: [HACKERS] detecting binary backup in progress

On 2013-05-31 10:38:56 -0700, Joe Conway wrote: > I can check for the presence of $PGDATA/backup_label in order to detect > a backup in progress (i.e. pg_start_backup() has been run and > pg_stop_backup() has not yet been run). > > However there is a period of time after pg_start_backup() is first

Re: [HACKERS] Behavior of a pg_trgm index for 2 (or < 3) character LIKE queries

On Sat, Jun 1, 2013 at 1:48 AM, Sawada Masahiko wrote: > On Fri, May 31, 2013 at 11:16 AM, Amit Langote > wrote: >> 2) And if that is so, is there problem in gin_extract_query_trgm(), >> that is while generating trigrams from a query search term that causes >> trigrams (stored in the index if an

[HACKERS] detecting binary backup in progress

-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 I can check for the presence of $PGDATA/backup_label in order to detect a backup in progress (i.e. pg_start_backup() has been run and pg_stop_backup() has not yet been run). However there is a period of time after pg_start_backup() is first executed t

Re: [HACKERS] removing PD_ALL_VISIBLE

>> Isn't the visibility map already required for proper return results as >> we use it for index-only scans. I think the optimization-only ship has >> sailed. > > At the moment we can remove it without causing corruption. If we were to > use it for freezing we couldn't anymore. So there's a diff

Re: [HACKERS] Freezing without write I/O

On Thu, May 30, 2013 at 10:04:23PM -0400, Robert Haas wrote: > > Hm. Why? If freezing gets notably cheaper I don't really see much need > > for keeping that much clog around? If we still run into anti-wraparound > > areas, there has to be some major operational problem. > > That is true, but we ha

Re: [HACKERS] Unsigned integer types

On May 29, 2013, at 10:48 AM, Fabien COELHO wrote: > If you do it, having uint1 (1 byte) would be nice as well. There is a signed 1byte int on PGXN, FWIW: http://pgxn.org/extension/tinyint Best, David -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes

Re: [HACKERS] removing PD_ALL_VISIBLE

On 2013-05-31 13:14:13 -0400, Bruce Momjian wrote: > On Thu, May 30, 2013 at 09:47:22AM -0400, Robert Haas wrote: > > Well, as Heikki points out, I think that's unacceptably dangerous. > > Loss or corruption of a single visibility map page means possible loss > > of half a gigabyte of data. > > >

Re: [HACKERS] removing PD_ALL_VISIBLE

On Thu, May 30, 2013 at 09:47:22AM -0400, Robert Haas wrote: > Well, as Heikki points out, I think that's unacceptably dangerous. > Loss or corruption of a single visibility map page means possible loss > of half a gigabyte of data. > > Also, if we go that route, looking at the visibility map is n

Re: [HACKERS] Behavior of a pg_trgm index for 2 (or < 3) character LIKE queries

On Fri, May 31, 2013 at 11:16 AM, Amit Langote wrote: > On Fri, May 31, 2013 at 4:25 AM, Alexander Korotkov > wrote: >> On Thu, May 30, 2013 at 12:49 PM, Sawada Masahiko >> wrote: >>> >>> following emails are discussed about partial match of pg_trgm. I hope >>> will this help. >>> >>>

Re: [HACKERS] Freezing without write I/O

On 31.05.2013 00:06, Bruce Momjian wrote: On Thu, May 30, 2013 at 04:33:50PM +0300, Heikki Linnakangas wrote: This would also be the first step in allowing the clog to grow larger than 2 billion transactions, eliminating the need for anti-wraparound freezing altogether. You'd still want to trunc

Re: [HACKERS] 9.3: Empty arrays returned by array_remove()

On 31 May 2013 08:34, Brendan Jurd wrote: > On 31 May 2013 02:52, Dean Rasheed wrote: >> Testing 9.3beta, it seems that array_remove() may return an empty 1-d >> array whose upper bound is lower than its lower bound. I know that we >> discussed allowing this kind of array, but I don't think that

Re: [HACKERS] [PATCH] add --throttle to pgbench (submission 3)

However I'm not sure that pg_stat_replication currently has the necessary information on either side to measure the lag (in time transactions, but how do I know when a transaction was committed? or number of transactions?). The BDR codebase now has a handy function to report when a transaction

Re: [HACKERS] 9.3: Empty arrays returned by array_remove()

On 31 May 2013 02:52, Dean Rasheed wrote: > Testing 9.3beta, it seems that array_remove() may return an empty 1-d > array whose upper bound is lower than its lower bound. I know that we > discussed allowing this kind of array, but I don't think that > discussion reached any conclusion, other than