Re: [HACKERS] Heavily modified big table bloat even in auto vacuum is running

On 07 November 2013 09:42 Amit Kapila wrote: > On Tue, Oct 22, 2013 at 2:09 PM, Haribabu kommi > wrote: > > On 22 October 2013 10:15 Amit Kapila wrote: > >>>On Mon, Oct 21, 2013 at 10:54 AM, Haribabu kommi > wrote: > >>> > >>Actually what I had in mind was to use nkeep to estimate > n_dead_tuples

Re: [HACKERS] Changing pg_dump default file format

On 11/08/2013 11:41 AM, Harold Giménez wrote: > > > > On Thu, Nov 7, 2013 at 7:01 PM, Craig Ringer > wrote: > > > (a) Lots of people only upgrade every two, three, or even more major > versions. I'm dealing with clients on 8.3, and people still pop up on

Re: [HACKERS] TODO: Split out pg_resetxlog output into pre- and post-sections

On Tue, Nov 5, 2013 at 3:20 PM, Rajeev rastogi wrote: > On execution of pg_resetxlog using the option -n > > 1. It will display values in two section. > > 2. First section will be called as "Current pg_control > values or Guess pg_control values". > >

Re: [HACKERS] Changing pg_dump default file format

On Thu, Nov 7, 2013 at 7:01 PM, Craig Ringer wrote: > > (a) Lots of people only upgrade every two, three, or even more major > versions. I'm dealing with clients on 8.3, and people still pop up on > Stack Overflow with 8.1 sometimes! These people don't ever see the > deprecated phase. > Interest

Re: [HACKERS] Changing pg_dump default file format

On 11/08/2013 07:33 AM, Joshua D. Drake wrote: >> > > All I want to start is this simple fix. I don't know who is going to > step up to work on pg_dump/all considering it is the bastard step child > of the project. We only work on it when we absolutely have to (and CMD > is just as guilty as the r

Re: [HACKERS] Changing pg_dump default file format

On 11/08/2013 12:55 AM, Joshua D. Drake wrote: > > We have certainly done worse in the past. 8.3 comes to mind. I am trying > to consider the future here, as we continue to grow. Why leave thorns > for the sake of leaving thorns? 8.3 was _bad_. It's been a major upgrade barrier, and a huge pain.

Re: [HACKERS] Changing pg_dump default file format

On 11/08/2013 12:53 AM, Harold Giménez wrote: > > > Just a thought: If it's the right thing to do, why not do it in small > steps? > For example, couldn't we deprecate the behavior on 9.4, and switch over in > 9.5? By deprecate I mean a warning message on STDERR making it clear > they're relyin

Re: [HACKERS] Changing pg_dump default file format

On 11/07/2013 06:33 PM, Joshua D. Drake wrote: On 11/07/2013 12:42 PM, Josh Berkus wrote: All, I'm amused at how this has become a "let's pile on everything which has ever been missing in pg_dump into one thread". Well it has been broken longer than most of our utilities. Sorry... not bro

Re: [HACKERS] [v9.4] row level security

On 11/07/2013 06:11 PM, Dean Rasheed wrote: > I don't really get this part of the discussion. Why would you want to > make updatable SB views do any of that? I don't, especially. If we're going to use updatable SB views as the basis for RLS then we need the option to skip adding the qual for super

Re: [HACKERS] [v9.4] row level security

On 11/07/2013 09:47 PM, Greg Stark wrote: > Incidentally I still feel this is at root the problem with updateable > views in general. I know it's a bit off to be tossing in concerns from > the peanut gallery when I'm not actually offering to work on it and > others are having putting in serious ef

Re: [HACKERS] stats for network traffic WIP

On Mon, Oct 21, 2013 at 5:14 AM, Nigel Heron wrote: > - can be used to find misbehaving connections. > - can be used in multi-user/multi-database clusters for resource usage > tracking. > - competing databases have such metrics. The most interesting thing that I could see calculating from these

Re: [HACKERS] regclass error reports improperly downcased

Jim Nasby writes: > decibel@decina.cashnetusa=# SELECT 'Moo'::regclass; > ERROR: relation "moo" does not exist at character 8 That's doing what it's supposed to. Compare regression=# select 'Moo'::regclass; ERROR: relation "moo" does not exist LINE 1: select 'Moo'::regclass; ^

Re: [HACKERS] regclass error reports improperly downcased

On Fri, Nov 8, 2013 at 12:29 AM, Jim Nasby wrote: > decibel@decina.cashnetusa=# SELECT 'Moo'::regclass; > ERROR: relation "moo" does not exist at character 8 > > Obviously in this example it doesn't matter, but this can definitely > confuse users. > I would actually say *not* downcasing would b

[HACKERS] regclass error reports improperly downcased

psql (9.3.1) Type "help" for help. decibel@decina.cashnetusa=# SELECT 'Moo'::regclass; ERROR: relation "moo" does not exist at character 8 Obviously in this example it doesn't matter, but this can definitely confuse users. Same in 9.1 and HEAD. -- Jim Nasby, Lead Data Architect (512) 569-94

Re: [HACKERS] [PATCH] warning: suggest braces around empty body in an 'else' statement

Andres Freund writes: > Compiling postgres I currently get the following warning when compiling > without LOCK_DEBUG: > /home/andres/src/postgresql/src/backend/storage/lmgr/lock.c:944:52: > warning: suggest braces around empty body in an 'else' statement > [-Wempty-body] > PROCLOCK_PRINT("Lo

Re: [HACKERS] Disallow pullup of a subquery with a subquery in its targetlist?

I wrote: > Back at > http://www.postgresql.org/message-id/520d221e.2060...@gmail.com > there was a complaint about strange behavior of a query that looks > basically like this: > SELECT ... > FROM > (SELECT ... , > ( SELECT ... > ORDER BY random() >

Re: [HACKERS] Changing pg_dump default file format

On 11/07/2013 12:42 PM, Josh Berkus wrote: All, I'm amused at how this has become a "let's pile on everything which has ever been missing in pg_dump into one thread". Well it has been broken longer than most of our utilities. Sorry... not broken but certainly not complete. It is to be expec

[HACKERS] [PATCH] warning: suggest braces around empty body in an 'else' statement

Hi, Compiling postgres I currently get the following warning when compiling without LOCK_DEBUG: /home/andres/src/postgresql/src/backend/storage/lmgr/lock.c:944:52: warning: suggest braces around empty body in an 'else' statement [-Wempty-body] PROCLOCK_PRINT("LockAcquire: NOWAIT", proclock);

Re: [HACKERS] Using indices for UNION.

On 10/31/13, 6:42 AM, Kyotaro HORIGUCHI wrote: > Hello, This patch might be too complicated (and seems somewhat ad > hoc) for the gain, but more index usage for this kind of > operation should be worth doing. There is a new compiler warning: setrefs.c: In function ‘set_plan_refs’: setrefs.c:782:7

Re: [HACKERS] Get more from indices.

On 10/31/13, 6:43 AM, Kyotaro HORIGUCHI wrote: > Hello, This is the last episode of the 'dance with indices'? > series. Your patch fails the isolation test because of changed query plans: http://pgci.eisentraut.org/jenkins/job/postgresql_commitfest_world/175/artifact/src/test/isolation/regression

Re: [HACKERS] TODO : Allow parallel cores to be used by vacuumdb [ WIP ]

On 07-11-2013 09:42, Dilip kumar wrote: Dilip, this is on my TODO for 9.4. I've already had a half-backed patch for it. Let's see what I can come up with. > Is it required to move the common code for parallel operation of pg_dump and > vacuumdb to one place and reuse it ? > I'm not sure about t

[HACKERS] Gin page deletion bug

Gin page deletion fails to take into account that there might be a search in-flight to the page that is deleted. If the page is reused for something else, the search can get very confused. That's pretty difficult to reproduce in a real system, as the window between releasing a lock on page and

Re: [HACKERS] Changing pg_dump default file format

All, I'm amused at how this has become a "let's pile on everything which has ever been missing in pg_dump into one thread". Agree with Tom that if we're going to create a new program name, we should fix the pg_dumpall issue as well. -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com

Re: [HACKERS] Changing pg_dump default file format

On 11/07/2013 02:27 PM, Jan Lentfer wrote: Btw, how hard would it be, to have pg_restore and now also pg_dump run with -j option do some ordering of work by size of e.g. the tables? E.g. if you run with -j4 it would make sense to start working on the largest tables (and it's indexes) first and

Re: [HACKERS] Changing pg_dump default file format

On 11/07/2013 10:54 AM, Kevin Grittner wrote: That is who I am thinking of. A DBA team may have hundreds of databases to manage, each with many scripts which have been running nicely for years. A change like this is bound to break some of those crontab scripts they may not even remember they

Re: [HACKERS] Changing pg_dump default file format

On 11/7/13, 1:00 PM, Josh Berkus wrote: > If we wanted to change the defaults, I think it would be easier to > create a separate bin name (e.g. pg_backup) than to change the existing > parameters for pg_dump. Note the following code in pg_dump.c: /* Set default options based on progname */

Re: [HACKERS] Changing pg_dump default file format

On 11/07/2013 11:01 AM, Alvaro Herrera wrote: Maybe we could provide a master controller program (pg_backup or whatever name we agree on), which could receive commands much like pg_ctl. $ pg_backup --help pg_backup is a backup handler program for PostgreSQL. Usage: pg_backup backup [-p PAT

Re: [HACKERS] Changing pg_dump default file format

Am 07.11.2013 19:08, schrieb Joshua D. Drake:> > On 11/07/2013 10:00 AM, Josh Berkus wrote: >> If we wanted to change the defaults, I think it would be easier to >> create a separate bin name (e.g. pg_backup) than to change the existing >> parameters for pg_dump. > > I am not opposed to that. Allo

Re: [HACKERS] Changing pg_dump default file format

Tom Lane wrote: > Josh Berkus writes: > > If we wanted to change the defaults, I think it would be easier to > > create a separate bin name (e.g. pg_backup) than to change the existing > > parameters for pg_dump. > > That might work. However, just to be contrary: what about the > functionality o

Re: [HACKERS] Changing pg_dump default file format

Joshua D. Drake wrote: > PITR is wonderful, I use it all the time. It has exactly zero to > do with what we are talking about. We are talking about pg_dump > and its usefulness. Are you suggesting that we tell people to not > use pg_dump and instead use PITR? Yes.  (Well, actually, any of the WA

Re: [HACKERS] Changing pg_dump default file format

On Thu, Nov 07, 2013 at 01:45:20PM -0500, Tom Lane wrote: > Josh Berkus writes: > > If we wanted to change the defaults, I think it would be easier to > > create a separate bin name (e.g. pg_backup) than to change the > > existing parameters for pg_dump. > > That might work. However, just to be

Re: [HACKERS] Changing pg_dump default file format

On Thu, Nov 7, 2013 at 8:53 AM, Joshua D. Drake wrote: > > On 11/07/2013 08:21 AM, Kevin Grittner wrote: > > I humbly request on behalf of those who manage production postgresql >>> instances that we change the default backup file format from -Fp to -Fc. >>> >> >> -1 in the case that output goes

Re: [HACKERS] Changing pg_dump default file format

Josh Berkus writes: > If we wanted to change the defaults, I think it would be easier to > create a separate bin name (e.g. pg_backup) than to change the existing > parameters for pg_dump. That might work. However, just to be contrary: what about the functionality of pg_dumpall? We've heard rep

Re: [HACKERS] Changing pg_dump default file format

On Thu, Nov 7, 2013 at 7:08 PM, Joshua D. Drake wrote: > > On 11/07/2013 10:00 AM, Josh Berkus wrote: >> >> >> On 11/07/2013 08:26 AM, Tom Lane wrote: >>> >>> Magnus Hagander writes: On Thu, Nov 7, 2013 at 5:02 PM, Joshua D. Drake wrote: > > I humbly request on behalf of t

Re: [HACKERS] Changing pg_dump default file format

On 2013-11-07 10:00:53 -0800, Josh Berkus wrote: > If we wanted to change the defaults, I think it would be easier to > create a separate bin name (e.g. pg_backup) than to change the existing > parameters for pg_dump. Provided somebody is actually willing to put in the not inconsiderable amount of

Re: [HACKERS] Changing pg_dump default file format

On 11/07/2013 10:00 AM, Josh Berkus wrote: On 11/07/2013 08:26 AM, Tom Lane wrote: Magnus Hagander writes: On Thu, Nov 7, 2013 at 5:02 PM, Joshua D. Drake wrote: I humbly request on behalf of those who manage production postgresql instances that we change the default backup file format fro

Re: [HACKERS] Changing pg_dump default file format

On 11/07/2013 08:26 AM, Tom Lane wrote: > Magnus Hagander writes: >> On Thu, Nov 7, 2013 at 5:02 PM, Joshua D. Drake >> wrote: >>> I humbly request on behalf of those who manage production postgresql >>> instances that we change the default backup file format from -Fp to -Fc. > >> I'm a bit wor

Re: [HACKERS] better atomics - require PG_USE_INLINE support?

On 2013-11-06 17:24:57 +0100, Andres Freund wrote: > Questions: > * Do we really want to continue to cater to compilers not supporting > PG_USE_INLINE? I've tried to add support for them, but it does make > things considerably more #ifdef-y. > Afaik it's only HPUX's acc that has problems, and

Re: [HACKERS] Changing pg_dump default file format

On 11/07/2013 09:25 AM, Magnus Hagander wrote: PITR is wonderful, I use it all the time. It has exactly zero to do with what we are talking about. We are talking about pg_dump and its usefulness. Are you suggesting that we tell people to not use pg_dump and instead use PITR? For backups, abso

Re: [HACKERS] Changing pg_dump default file format

On Thu, Nov 7, 2013 at 5:53 PM, Joshua D. Drake wrote: > > On 11/07/2013 08:21 AM, Kevin Grittner wrote: > >>> I humbly request on behalf of those who manage production postgresql >>> instances that we change the default backup file format from -Fp to -Fc. >> >> >> -1 in the case that output goes

Re: [HACKERS] alter_table regression test problem

On 2013-11-07 10:10:34 -0500, Tom Lane wrote: > Andres Freund writes: > > On 2013-11-07 06:49:58 -0800, Kevin Grittner wrote: > >> It's up to the committer whether to indent > >> after review and make both substantive and whitespace changes > >> together, but I have definitely seen those done sepa

Re: [HACKERS] Changing pg_dump default file format

On 11/07/2013 08:26 AM, Tom Lane wrote: Magnus Hagander writes: On Thu, Nov 7, 2013 at 5:02 PM, Joshua D. Drake wrote: I humbly request on behalf of those who manage production postgresql instances that we change the default backup file format from -Fp to -Fc. I'm a bit worried about it b

Re: [HACKERS] Changing pg_dump default file format

On Thu, Nov 7, 2013 at 8:26 AM, Tom Lane wrote: > Magnus Hagander writes: > > On Thu, Nov 7, 2013 at 5:02 PM, Joshua D. Drake > wrote: > >> I humbly request on behalf of those who manage production postgresql > >> instances that we change the default backup file format from -Fp to -Fc. > > > I'

Re: [HACKERS] Changing pg_dump default file format

On 11/07/2013 08:21 AM, Kevin Grittner wrote: I humbly request on behalf of those who manage production postgresql instances that we change the default backup file format from -Fp to -Fc. -1 in the case that output goes to stdout. It might make sense for when -f is specified. As a DBA, 90%

Re: [HACKERS] Changing pg_dump default file format

On 11/07/2013 08:17 AM, Andrew Dunstan wrote: At least it gives you a nice message now instead of the rather cryptic message it used to give: [andrew@emma inst.5715]$ bin/pg_dump -f foo.sql [andrew@emma inst.5715]$ bin/pg_restore foo.sql pg_restore: [archiver] input file appears t

Re: [HACKERS] backup.sgml-cmd-v003.patch

On 09/27/2013 03:56 AM, Robert Haas wrote: On Thu, Sep 26, 2013 at 1:15 PM, Ivan Lezhnjov IV wrote: Thanks for a detailed response. I attached a patch file that builds on your corrections and introduces some of the edits discussed above. This patch makes at least five unrelated sets of chan

Re: [HACKERS] Changing pg_dump default file format

Magnus Hagander writes: > On Thu, Nov 7, 2013 at 5:02 PM, Joshua D. Drake > wrote: >> I humbly request on behalf of those who manage production postgresql >> instances that we change the default backup file format from -Fp to -Fc. > I'm a bit worried about it breaking peoples scripts as well, i

Re: [HACKERS] Changing pg_dump default file format

Joshua D. Drake wrote: > For as long as I can remember I have heard complaints about the fact > that pg_dump doesn't use a backup format that is supported by the > pg_restore program. At every single talk that I give that talks about > pg_dump, the people in the majority of the audience agrees th

Re: [HACKERS] Changing pg_dump default file format

On 11/07/2013 11:02 AM, Joshua D. Drake wrote: Hello, For as long as I can remember I have heard complaints about the fact that pg_dump doesn't use a backup format that is supported by the pg_restore program. At every single talk that I give that talks about pg_dump, the people in the major

Re: [HACKERS] Changing pg_dump default file format

On Thu, Nov 7, 2013 at 5:02 PM, Joshua D. Drake wrote: > > Hello, > > For as long as I can remember I have heard complaints about the fact that > pg_dump doesn't use a backup format that is supported by the pg_restore > program. At every single talk that I give that talks about pg_dump, the > peop

[HACKERS] Changing pg_dump default file format

Hello, For as long as I can remember I have heard complaints about the fact that pg_dump doesn't use a backup format that is supported by the pg_restore program. At every single talk that I give that talks about pg_dump, the people in the majority of the audience agrees that this is rather s

[HACKERS] Comment - uniqueness of relfilenode

catalog/catalog.c:GetNewRelFileNode() and its calls indicate that the following change makes sense: diff --git a/src/include/storage/relfilenode.h b/src/include/storage/relfilenode.h index 75f897f..7190974 100644 --- a/src/include/storage/relfilenode.h +++ b/src/include/storage/relfilenode.h @@ -

Re: [HACKERS] alter_table regression test problem

On 11/07/2013 09:59 AM, Andres Freund wrote: On 2013-11-07 06:49:58 -0800, Kevin Grittner wrote: It's up to the committer whether to indent after review and make both substantive and whitespace changes together, but I have definitely seen those done separately, or even left for the next global

Re: [HACKERS] alter_table regression test problem

Andres Freund writes: > On 2013-11-07 06:49:58 -0800, Kevin Grittner wrote: >> It's up to the committer whether to indent >> after review and make both substantive and whitespace changes >> together, but I have definitely seen those done separately, or even >> left for the next global pgindent run

Re: [HACKERS] alter_table regression test problem

On 2013-11-07 06:49:58 -0800, Kevin Grittner wrote: > It's up to the committer whether to indent > after review and make both substantive and whitespace changes > together, but I have definitely seen those done separately, or even > left for the next global pgindent run to fix. Hm. I don't remembe

Re: [HACKERS] alter_table regression test problem

On 2013-11-07 09:55:55 -0500, Tom Lane wrote: > Kevin Grittner writes: > > I think it is pretty much SOP for committers to prefer a patch that > > adds a new pair of braces around 50 lines of code and only changes > > non-whitespace of a few lines within that block to leave the block > > at its ol

Re: [HACKERS] alter_table regression test problem

Kevin Grittner writes: > I think it is pretty much SOP for committers to prefer a patch that > adds a new pair of braces around 50 lines of code and only changes > non-whitespace of a few lines within that block to leave the block > at its old indentation. Yes. It's much easier to review a patch

Re: [HACKERS] alter_table regression test problem

Andres Freund wrote: > On 2013-11-07 06:23:13 -0800, Kevin Grittner wrote: >>   I think this is one of those cases where the large >> chunk of code inside the new "else" block should not be indented >> with the initial patch -- a pgindent-based whitespace-only patch >> should follow so that the s

Re: [HACKERS] alter_table regression test problem

On 2013-11-07 06:23:13 -0800, Kevin Grittner wrote: > Andres Freund wrote: > > > Ok, I've attached a fix for this, which unfortunately is not all > > that small. > > Could either of you commit it? > > Unfortunately, I don't feel I have a good enough grasp of the > caching/invalidation mechanism

Re: [HACKERS] alter_table regression test problem

Andres Freund wrote: > Ok, I've attached a fix for this, which unfortunately is not all > that small. > Could either of you commit it? Unfortunately, I don't feel I have a good enough grasp of the caching/invalidation mechanism to be a committer for this particular patch, but I think you could m

Re: [HACKERS] [v9.4] row level security

On Wed, Nov 6, 2013 at 6:38 AM, Craig Ringer wrote: > That's the approach I've been considering. There are a few wrinkles with > it, though: > > (a) Updatable views are implemented in the rewriter, not the planner. > The rewriter is not re-run when plans are invalidated or when the > session auth

Re: [HACKERS] [v9.4] row level security

On 6 November 2013 19:12, Kohei KaiGai wrote: > 2013/11/6 Craig Ringer : >> On 11/05/2013 09:36 PM, Robert Haas wrote: >>> I haven't studied this patch in detail, but I see why there's some >>> unhappiness about that code: it's an RLS-specific kluge. Just >>> shooting from the hip here, maybe we

Re: [HACKERS] pg_fallocate

On Thu, Oct 31, 2013 at 01:16:44PM +, Mitsumasa KONDO wrote: > --- a/src/backend/storage/file/fd.c > +++ b/src/backend/storage/file/fd.c > @@ -383,6 +383,21 @@ pg_flush_data(int fd, off_t offset, off_t amount) > return 0; > } > > +/* > + * pg_fallocate --- advise OS that the data pre-a

Re: [HACKERS] alter_table regression test problem

On 2013-11-07 00:30:55 +0100, Andres Freund wrote: > On 2013-11-07 00:17:34 +0100, Andres Freund wrote: > > On 2013-11-06 17:00:40 -0300, Alvaro Herrera wrote: > > > Kevin Grittner wrote: > > > > > > > That makes for a pretty simple test for git bisect, even if > > > > everything including initdb