Re: [HACKERS] Badly broken logic in plpython composite-type handling

2015-08-19 Thread Joshua D. Drake
On 08/19/2015 05:05 PM, Tom Lane wrote: Barring objections, I propose to commit and back-patch this. The crash can be demonstrated back to 9.1. regards, tom lane +1 -- Command Prompt, Inc. - http://www.commandprompt.com/ 503-667-4564 PostgreSQL Centered full stack

Re: [HACKERS] Removing unreferenced files

2015-08-06 Thread Joshua D. Drake
On 08/05/2015 11:44 AM, Ron Farrer wrote: Initial questions that had no consensus in previous discussions: 1. Approach on file handling undecided 2. Startup vs standalone tool I think it should be on startup and perhaps also have a function that will do it from user space. If this problem pe

Re: [HACKERS] [PROPOSAL] VACUUM Progress Checker.

2015-07-31 Thread Joshua D. Drake
On 07/31/2015 11:21 AM, Alvaro Herrera wrote: This looks pretty complicated to understand from the user POV, but anything other than this seems to me too simplistic to be of any use. I would agree and I don't think it is all that complicated. This is an RDBMS not a web browser downloading a

Re: [HACKERS] 64-bit XIDs again

2015-07-30 Thread Joshua D. Drake
On 07/30/2015 08:04 AM, Simon Riggs wrote: There is a big downside to expanding xmin/xmax to 64 bits: it takes space. More space means more memory needed for caching, more memory bandwidth, more I/O, etc. My feeling is that the overhead will recede in time. Having a nice, simple c

[HACKERS] RFC: Allow materialized views to use unlogged data store

2015-07-21 Thread Joshua D. Drake
-hackers, What do we think about $subject? In short, the underlying table of an MV would have the option of being unlogged. JD -- Command Prompt, Inc. - http://www.commandprompt.com/ 503-667-4564 PostgreSQL Centered full stack support, consulting and development. Announcing "I'm offended" is

Re: [HACKERS] proposal: multiple psql option -c

2015-07-16 Thread Joshua D. Drake
it is one possible solution too multiple -c option has advantage of simple evaluation of backslash statements .. -c "\l" -c "\dt" - but this advantage is not high important. Or just properly understand the ; ? -c "select * from foo; update bar set baz = 'bing'; vacuum bar;" JD Pavel

Re: [HACKERS] pg_upgrade + Ubuntu

2015-07-10 Thread Joshua D. Drake
On 07/10/2015 12:10 PM, Alvaro Herrera wrote: It seems to me that this is a Debian packaging issue, not an upstream issue, isn't it? If you want to fix the problem in this way, then surely whatever package contains pg_upgrade should also contain pg_config. Why are you not using pg_upgradeclus

Re: [HACKERS] pg_upgrade + Ubuntu

2015-07-10 Thread Joshua D. Drake
On 07/10/2015 11:01 AM, Mike Blackwell wrote: Does pg_config show the correct location? If so, perhaps pg_upgrade could get the .conf location the same way rather than requiring a command line option. Good idea but: postgres@ly19:~$ pg_config You need to install postgresql-server-dev-X.Y for

[HACKERS] pg_upgrade + Ubuntu

2015-07-10 Thread Joshua D. Drake
Hackers, Simple problem (I think): 9.4 version of pg_upgrade said: "/usr/lib/postgresql/9.4/bin/pg_ctl" -w -l "pg_upgrade_server.log" -D "/var/lib/postgresql/9.4/main" -o "-p 9400 -b -c synchronous_commit=off -c fsync=off -c full_page_writes=off -c listen_addresses='' -c unix_socket_permi

Re: [HACKERS] Inconsistent style in pgbench's error messages

2015-07-05 Thread Joshua D. Drake
On 07/05/2015 04:46 PM, Tom Lane wrote: I made a pass over pgbench's error messages to try to make them meet project style guidelines. There was one rather large bit of inconsistency that I didn't try to fix, though: something like half of the messages prepend "pgbench: " to the front, but the

Re: [HACKERS] Raising our compiler requirements for 9.6

2015-07-01 Thread Joshua D. Drake
On 07/01/2015 01:33 PM, Tom Lane wrote: Andres Freund writes: At the very least I think we should start to rely on 'static inline's working. There is not, and hasn't been for a while, any buildfarm animal that does not support it pademelon doesn't. Other reasoning aside, pademelon is runni

Re: [HACKERS] Schedule for 9.5alpha1

2015-06-25 Thread Joshua D. Drake
On 06/25/2015 05:09 PM, Peter Geoghegan wrote: On Thu, Jun 25, 2015 at 4:30 PM, Michael Paquier wrote: I'm tired of having to chase down known bugs when a patch has been around for a long time, and an actual fix is blocking on committer availability -- sometimes I feel the need to privately t

Re: [HACKERS] Should we back-patch SSL renegotiation fixes?

2015-06-25 Thread Joshua D. Drake
On 06/25/2015 06:15 AM, Peter Eisentraut wrote: On 6/25/15 8:03 AM, Andres Freund wrote: Right now if you use streaming rep over ssl, it breaks after a couple hundred megabytes with obscure errors. If it's that bad, then I tend to agree we should turn it off by default. From an "in the wi

Re: [HACKERS] RFC: replace pg_stat_activity.waiting with something more descriptive

2015-06-22 Thread Joshua D. Drake
On 06/22/2015 10:37 AM, Robert Haas wrote: I'm less sure about this next part, but I think we might also want to report ourselves as waiting when we are doing an OS read or an OS write, because it's pretty common for people to think that a PostgreSQL bug is to blame when in fact it's the opera

Re: [HACKERS] 9.5 make world failing due to sgml tools missing

2015-06-17 Thread Joshua D. Drake
On 06/17/2015 01:07 PM, Tom Lane wrote: Keith Fiske writes: The current HEAD of postgres in the git repo is not building when using "make world". It's been like this for about a month or so that I've been aware of. I didn't really need the world build so been making due without it. At PGCon no

Re: [HACKERS] The purpose of the core team

2015-06-11 Thread Joshua D. Drake
On 06/11/2015 10:20 AM, Robert Haas wrote: True but that isn't the fault of core outside of communication. The hackers, reviewers and committers of those patches should be required to communicate with core in a way that expresses the true severity of a situation so core can make a: Management

Re: [HACKERS] The purpose of the core team

2015-06-11 Thread Joshua D. Drake
On 06/11/2015 10:10 AM, Magnus Hagander wrote: Magnus: Committer, primary Windows dude and reviews patches here and there. Not sure that's a fair title at this point. Both Andrew and Michael seem to be doing more of that than me these days, for example. (I do review patches here and t

Re: [HACKERS] The purpose of the core team

2015-06-11 Thread Joshua D. Drake
On 06/11/2015 09:49 AM, Andrew Dunstan wrote: On 06/11/2015 12:29 PM, Joshua D. Drake wrote: JoshB: Advocacy. There is a strong argument that does not need to be a core position. I strongly disagree with this. On the contrary, I think there is a very good argument that FOR such a

Re: [HACKERS] The purpose of the core team

2015-06-11 Thread Joshua D. Drake
On 06/11/2015 07:12 AM, Robert Haas wrote: Hopefully this will be helpful to people. I believe the core team is suffering from a lack of members who are involved in writing, reviewing, and committing patches. Those things are not core functions of the core team, as that charter illustrates.

Re: [HACKERS] The purpose of the core team

2015-06-11 Thread Joshua D. Drake
On 06/11/2015 08:26 AM, Robert Haas wrote: Timing *decisions* are not made by -core, as I've told you in the past. They are made by the packagers who do the actual work, based on suggestions from -core. You have told me that in the past, and I do not accept that it is true. The suggestions f

Re: [HACKERS] skipping pg_log in basebackup (was Re: pg_basebackup and pg_stat_tmp directory)

2015-06-10 Thread Joshua D. Drake
On 06/10/2015 10:22 AM, Robert Haas wrote: On Wed, Jun 10, 2015 at 1:12 PM, Joshua D. Drake wrote: On 06/10/2015 10:01 AM, Andres Freund wrote: On 2015-06-10 09:57:17 -0700, Jeff Janes wrote: Mine goal isn't that. My goal is to have a consistent backup without having to shut dow

Re: [HACKERS] skipping pg_log in basebackup (was Re: pg_basebackup and pg_stat_tmp directory)

2015-06-10 Thread Joshua D. Drake
On 06/10/2015 10:01 AM, Andres Freund wrote: On 2015-06-10 09:57:17 -0700, Jeff Janes wrote: Mine goal isn't that. My goal is to have a consistent backup without having to shut down the server to take a cold one, or having to manually juggle the pg_start_backup, etc. commands. A basebackup

Re: [HACKERS] pg_archivecleanup bug (invalid filename input)

2015-06-09 Thread Joshua D. Drake
On 06/09/2015 05:54 PM, Michael Paquier wrote: Looking at the documentation what is expected is not a path to a segment file, but only a segment file name: http://www.postgresql.org/docs/devel/static/pgarchivecleanup.html So the current behavior is correct, it is actually what SetWALFileNameFor

[HACKERS] pg_archivecleanup bug (invalid filename input)

2015-06-09 Thread Joshua D. Drake
Hello, Trying to use pg_archivecleanup as a: "standalone archive cleaner" Results in an error of: pg_archivecleanup: invalid filename input Try "pg_archivecleanup --help" for more information. /usr/pgsql-9.4/bin/pg_archivecleanup /backups/db1/archive 0001074800B1.00015838.backup

Re: [CORE] [HACKERS] back-branch multixact fixes & 9.5 alpha/beta: schedule

2015-06-08 Thread Joshua D. Drake
On 06/08/2015 12:48 PM, Alvaro Herrera wrote: Well, reputation-wise we're already losing every time somebody's server crashes on 9.4.2 and finds it won't start, where it did start fine with 9.4.1. Maybe they simply wanted to change shared_buffers and the server won't start anymore. Some peopl

Re: [CORE] [HACKERS] back-branch multixact fixes & 9.5 alpha/beta: schedule

2015-06-08 Thread Joshua D. Drake
On 06/08/2015 12:31 PM, Alvaro Herrera wrote: Joshua D. Drake wrote: If we release on Friday that is the 12th, PgCon is starts the 16th and there is a weekend in between. If there is an unknown regression or new bug that is severe, are we going to have the resources to resolve it? ISTM if

Re: [CORE] [HACKERS] back-branch multixact fixes & 9.5 alpha/beta: schedule

2015-06-08 Thread Joshua D. Drake
On 06/08/2015 12:21 PM, Tom Lane wrote: Andres Freund writes: On 2015-06-08 14:18:22 -0400, Tom Lane wrote: As I saw it, on Friday it was not clear whether we would be able to do a release this week. Now it's Monday, and we still have a rather long list of issues Well, these issues aren't

Re: [HACKERS] [CORE] Restore-reliability mode

2015-06-08 Thread Joshua D. Drake
On 06/08/2015 06:21 AM, Geoff Winkless wrote: Wow! I never knew there were all these people out there who would be rushing to help test if only the PG developers released alpha versions. It's funny how they never used to do it when those alphas were done. The type of responses you are providi

Re: [HACKERS] [CORE] Restore-reliability mode

2015-06-06 Thread Joshua D. Drake
On 06/06/2015 07:14 PM, Peter Geoghegan wrote: On Sat, Jun 6, 2015 at 7:07 PM, Robert Haas wrote: Perhaps we're honoring this more in the breech than in the observance, but I'm not making up what Tom has said about this: http://www.postgresql.org/message-id/27310.1251410...@sss.pgh.pa.us htt

Re: [HACKERS] [CORE] Restore-reliability mode

2015-06-06 Thread Joshua D. Drake
On 06/06/2015 07:33 AM, Robert Haas wrote: On Sat, Jun 6, 2015 at 6:47 AM, Geoff Winkless wrote: To play devil's advocate for a moment, is there anyone who would genuinely be prepared to download and install an alpha release who would not already have downloaded one of the nightlies? I only a

Re: [HACKERS] [CORE] Restore-reliability mode

2015-06-06 Thread Joshua D. Drake
On 06/05/2015 08:07 PM, Bruce Momjian wrote: From my side, it is only recently I got some clear answers to my questions about how it worked. I think it is very important that major features have extensive README type documentation with them so the underlying principles used in the development

Re: [HACKERS] [GENERAL] 9.4.1 -> 9.4.2 problem: could not access status of transaction 1

2015-06-05 Thread Joshua D. Drake
On 06/05/2015 01:56 PM, Tom Lane wrote: If we have confidence that we can ship something on Monday that is materially more trustworthy than the current releases, then let's aim to do that; but let's ship only patches we are confident in. We can do another set of releases later that incorporate

Re: [HACKERS] RFC: Remove contrib entirely

2015-06-05 Thread Joshua D. Drake
On 06/05/2015 04:56 AM, Robert Haas wrote: somewhere else. At least not that I can see. 4. Eliminate the EGO of saying "I have a contrib module in core" I've got multiple major features in core. Any ego I may have about my PostgreSQL contributions is not based on pg_prewarm. This was wor

Re: [HACKERS] RFC: Remove contrib entirely

2015-06-04 Thread Joshua D. Drake
On 06/04/2015 11:01 AM, Robert Haas wrote: On Thu, Jun 4, 2015 at 1:57 PM, Joshua D. Drake wrote: I think it's because there are some things we want to include in the core distribution without baking them irrevocably into the server. I have mentioned before isn't really

Re: [HACKERS] RFC: Remove contrib entirely

2015-06-04 Thread Joshua D. Drake
On 06/04/2015 10:34 AM, Robert Haas wrote: On Thu, Jun 4, 2015 at 11:49 AM, Joshua D. Drake wrote: Except, that is kind of the point. Why are we adding to it? If you don't know the answer to that question already, then you probably shouldn't be proposing to get rid of the thing.

Re: [HACKERS] RFC: Remove contrib entirely

2015-06-04 Thread Joshua D. Drake
On 06/04/2015 08:55 AM, Jim Nasby wrote: Personally, I'd rather we publish a list of formally vetted and approved versions of PGXN modules. There are many benefits to that, and the downside of not having that stuff as part of make check would be overcome by the explicit testing we would need to

Re: [HACKERS] RFC: Remove contrib entirely

2015-06-04 Thread Joshua D. Drake
On 06/04/2015 09:00 AM, Andrew Dunstan wrote: No. You keep getting this wrong. The fact that we have extensions doesn't mean that we want to throw out everything that is an extension. It's perfectly reasonable for us to maintain some ourselves, not least as part of eating out own dog food. Y

Re: [HACKERS] RFC: Remove contrib entirely

2015-06-04 Thread Joshua D. Drake
On 06/04/2015 07:34 AM, Robert Haas wrote: I don't think it's very practical to talk about getting rid of contrib when we reliably add to it in every release: Except, that is kind of the point. Why are we adding to it? Contrib (AFAICS) is all things that don't need to be in -core. That is th

Re: [HACKERS] Restore-reliability mode

2015-06-03 Thread Joshua D. Drake
On 06/03/2015 07:18 AM, Andres Freund wrote: On 2015-06-03 09:50:49 -0400, Noah Misch wrote: Second, I would define the subject matter as "bug fixes, testing and review", not "restructuring, testing and review." Different code structures are clearest to different hackers. Restructuring, on av

Re: [HACKERS] pg_xlog -> pg_xjournal?

2015-06-02 Thread Joshua D. Drake
On 06/01/2015 09:35 PM, Michael Nolan wrote: Why not take a simpler approach and create a zero length file in directories that should not be fiddled with by non-experts using a file name something like "DO.NOT.DELETE.THESE.FILES"? +1 No, it won't prevent the incredibly stupid from doing inc

Re: [CORE] [HACKERS] postpone next week's release

2015-05-30 Thread Joshua D. Drake
On 05/30/2015 06:51 PM, David Steele wrote: On 5/30/15 8:38 PM, Joshua D. Drake wrote: On 05/30/2015 03:48 PM, David Steele wrote: On 5/30/15 2:10 PM, Robert Haas wrote: What, in this release, could break things badly? RLS? Grouping sets? Heikki's WAL format changes? That last one s

Re: [CORE] [HACKERS] postpone next week's release

2015-05-30 Thread Joshua D. Drake
On 05/30/2015 03:48 PM, David Steele wrote: On 5/30/15 2:10 PM, Robert Haas wrote: What, in this release, could break things badly? RLS? Grouping sets? Heikki's WAL format changes? That last one sounds really scary to me; it's painful if not impossible to fix the WAL format in a minor release

Re: [CORE] [HACKERS] postpone next week's release

2015-05-30 Thread Joshua D. Drake
On 05/30/2015 06:11 AM, Bruce Momjian wrote: 2017? Really? Is there any need for that hyperbole? Frankly, based on how I feel now, I would have no problem doing 9.5 in 2016 and saying we have a lot of retooling to do. We could say we have gotten too far out ahead of ourselves and we need to

Re: [CORE] [HACKERS] postpone next week's release

2015-05-29 Thread Joshua D. Drake
On 05/29/2015 01:03 PM, Stephen Frost wrote: * Tom Lane (t...@sss.pgh.pa.us) wrote: It's possible that we ought to give up on a pre-conference beta. Certainly a whole lot of time that I'd hoped would go into reviewing 9.5 feature commits has instead gone into back-branch bug chasing this week.

Re: [HACKERS] RFC: Remove contrib entirely

2015-05-29 Thread Joshua D. Drake
On 05/29/2015 12:30 PM, Pavel Stehule wrote: Contrib made sense years ago. It does not any longer. Let's put the old horse down and raise a new herd of ponies on a new pasture. Still there is strong sense - it is a referential implementation of our extension API. We need it to find re

Re: [HACKERS] [CORE] postpone next week's release

2015-05-29 Thread Joshua D. Drake
On 05/29/2015 12:18 PM, Robert Haas wrote: On Fri, May 29, 2015 at 3:09 PM, Magnus Hagander wrote: Do you have any feeling of how likely people are to actually hit the multixact one? I've followed some of that impressive debugging you guys did, and I know it's a pretty critical bug if you hit

Re: [HACKERS] Need Force flag for pg_drop_replication_slot()

2015-05-29 Thread Joshua D. Drake
On 05/29/2015 12:08 PM, Josh Berkus wrote: Now, BDR is good because it sets an application_name which lets me figure out what's using the replication slot. But that's by no means required; other LC plug-ins, I expect, do not do so. So there's no way for the user to figure out which replicatio

Re: [HACKERS] RFC: Remove contrib entirely

2015-05-29 Thread Joshua D. Drake
On 05/29/2015 11:27 AM, Jeff Janes wrote: It would be less confusing for users. Contrib modules seem to be first class extensions, leaving doubt on other extensions. Presumably there are still going to be some extensions maintained by -hackers, and some not. I don't think we are goin

Re: [HACKERS] RFC: Remove contrib entirely

2015-05-29 Thread Joshua D. Drake
On 05/29/2015 11:02 AM, Jeff Janes wrote: Also, removing a feature is a regression, and someone is always bound to complain... What is the real benefit? ISTM that it is a solution that fixes no important problem. Reaching a consensus about what to move here or there will consum

Re: [HACKERS] RFC: Remove contrib entirely

2015-05-29 Thread Joshua D. Drake
On 05/28/2015 11:08 PM, Pavel Stehule wrote: Hi I am not sure if PGXN can substitute contrib - mainly due deployment - It doesn't helps with MS Windows. Installing necessary software for compilation there is terrible. Anyone who is building for Windows won't have that problem. They already ha

Re: [HACKERS] RFC: Remove contrib entirely

2015-05-29 Thread Joshua D. Drake
On 05/28/2015 11:01 PM, Fabien COELHO wrote: Also, removing a feature is a regression, and someone is always bound to complain... We aren't removing any features. These are all items that are NOT installed or functional by default. Sincerely, JD -- The most kicking donkey PostgreSQL Inf

Re: [HACKERS] RFC: Remove contrib entirely

2015-05-28 Thread Joshua D. Drake
On 05/28/2015 08:10 PM, Stephen Frost wrote: JD, This seems reasonable to me. It's in line with the recent move from contrib to bin. It'll just be quite a bit bigger of an undertaking. (50 threads to discuss the merits of each module separately?) Maybe start by picking the top 5 and sort t

Re: [HACKERS] RFC: Remove contrib entirely

2015-05-28 Thread Joshua D. Drake
On 05/28/2015 06:50 PM, Peter Eisentraut wrote: On 5/28/15 3:35 PM, Stephen Frost wrote: What we would need for this is an 'extensions' directory, or similar, and a clear definition of what the requirements are around getting into it are. With that, we could decide for each module currently i

Re: [HACKERS] RFC: Remove contrib entirely

2015-05-28 Thread Joshua D. Drake
On 05/28/2015 06:25 PM, Andrew Dunstan wrote: I'd be ok with installing it by default. But the case that's a lot harder to require to be always installed is pgcrypto, as has often been discussed in the past. It used to be but IIRC we don't have those restrictions anymore. If so, then we n

Re: [HACKERS] RFC: Remove contrib entirely

2015-05-28 Thread Joshua D. Drake
On 05/28/2015 01:11 PM, Andrew Dunstan wrote: This seems to come up regularly. Maybe we should put it in an FAQ somewhere. The barriers to making non-core types into core types are very very high, possibly insurmountable. This is pretty much not an option. O.k., then either: * We install i

Re: [HACKERS] Re: [GENERAL] 9.4.1 -> 9.4.2 problem: could not access status of transaction 1

2015-05-28 Thread Joshua D. Drake
On 05/28/2015 12:56 PM, Robert Haas wrote: FTR: Robert, you have been a Samurai on this issue. Our many thanks. Sincerely, jD -- Command Prompt, Inc. - http://www.commandprompt.com/ 503-667-4564 PostgreSQL Centered full stack support, consulting and development. Announcing "I'm offended"

Re: [HACKERS] RFC: Remove contrib entirely

2015-05-28 Thread Joshua D. Drake
On 05/28/2015 12:35 PM, Stephen Frost wrote: JD, What we would need for this is an 'extensions' directory, or similar, and a clear definition of what the requirements are around getting into it are. With that, we could decide for each module currently in contrib if it should go into the 'ext

[HACKERS] RFC: Remove contrib entirely

2015-05-28 Thread Joshua D. Drake
Hello, This is a topic that has come up in various ways over the years. After the long thread on pg_audit, I thought it might be time to bring it up again. Contrib according to the docs is: "These include porting tools, analysis utilities, and plug-in features that are not part of the core

Re: [HACKERS] [COMMITTERS] pgsql: Add pg_audit, an auditing extension

2015-05-27 Thread Joshua D. Drake
On 05/27/2015 07:02 PM, Stephen Frost wrote: JD, As it is currently an extension, it does not need to be in core. If this extension reaches a point where it needs to be in core to achieve some level of integration not currently provided then we can evaluate that problem. Currently, that is not

Re: [HACKERS] [COMMITTERS] pgsql: Add pg_audit, an auditing extension

2015-05-27 Thread Joshua D. Drake
then we can evaluate that problem. Currently, that is not the case. Sincerely, Joshua D. Drake Thanks! Stephen -- Command Prompt, Inc. - http://www.commandprompt.com/ 503-667-4564 PostgreSQL Centered full stack support, consulting and development. Announcing &q

Re: [HACKERS] [COMMITTERS] pgsql: Add pg_audit, an auditing extension

2015-05-27 Thread Joshua D. Drake
On 05/27/2015 06:11 PM, Stephen Frost wrote: Thank you for your honest comments and your concern. I sincerely hope you're able to be involved in developing auditing for PostgreSQL in the future, as it is a key requirement in any secure environment. Guys, I think we are overlooking the rath

Re: [HACKERS] Add a hint when postgresql.conf hot_standby = 'off' and recovery.conf standby = 'on'

2015-05-22 Thread Joshua D. Drake
On 05/22/2015 11:01 AM, Josh Berkus wrote: On 05/22/2015 12:22 AM, Heikki Linnakangas wrote: It seems worth adding a hint and/or changing the error message to be more descriptive when in this state. Any options about what should be logged before I start putting together a patch? Yeah, might

[HACKERS] Rewriting backup.sgml (patch attached)

2015-05-19 Thread Joshua D. Drake
Hello, Alright, per previous discussions I went through the backup.sgml page. I have gone thoroughly through: sql dump pg_dump pg_restore handling large databases I removed file based backups I didn't really touch the red headed step child that is pg_dumpall (although a word smithed it a li

Re: [HACKERS] a few thoughts on the schedule

2015-05-19 Thread Joshua D. Drake
On 05/19/2015 11:02 AM, Peter Geoghegan wrote: Hasn't every talented reviewer gotten job offers shortly afterwards in the last few years? The ones that accept don't necessarily work that much in the community, but several seem to. And I think in the case of several people the reason they don't

Re: [HACKERS] a few thoughts on the schedule

2015-05-19 Thread Joshua D. Drake
On 05/19/2015 10:44 AM, Andres Freund wrote: I don't know what the solution is but I know I like the idea of a tree freeze except for bug fixes for at least 3 weeks but I would be jumping for joy if we froze the tree except for bug fixes for 6 or 12 weeks. We've done that for pretty much ever

Re: [HACKERS] a few thoughts on the schedule

2015-05-19 Thread Joshua D. Drake
On 05/18/2015 08:52 PM, Andres Freund wrote: Maybe we should forget them and just have monthly 'judgefests' where some poor sod summarizes the current state and direction, and we then collaboratively discuss whether we see things going anywhere and if not, what would need to happen that they do

Re: [HACKERS] a few thoughts on the schedule

2015-05-18 Thread Joshua D. Drake
On 05/18/2015 07:21 PM, Peter Geoghegan wrote: On Mon, May 18, 2015 at 7:10 PM, Andres Freund wrote: So +1 to moving it. +1 I for one would love to see a nice and solid focus on what we have now for a little while versus diverting resources yet again to new development. +1 -- Command

Re: [HACKERS] Triaging the remaining open commitfest items

2015-05-15 Thread Joshua D. Drake
On 05/15/2015 12:32 PM, Josh Berkus wrote: Note that I am not proposing a general delay in feature freeze. I am specifically proposing an additional week for Grouping Sets and *only* for Grouping Sets. Core is in charge of releases. I believe like the other semi and formal organizations aro

Re: [HACKERS] Changes to backup.sgml

2015-05-15 Thread Joshua D. Drake
On 05/15/2015 10:03 AM, Robert Haas wrote: On Thu, May 14, 2015 at 12:53 PM, Joshua D. Drake wrote: 1. File System Level Backup The section should be a note within the larger document. It is largely a legacy section from before 8.3. I agree. I think this section is just plain weird at

Re: [HACKERS] WALWriteLock contention

2015-05-15 Thread Joshua D. Drake
On 05/15/2015 09:06 AM, Robert Haas wrote: 2. I don't really understand why WALWriteLock is set up to prohibit two backends from flushing WAL at the same time. That seems unnecessary. Suppose we've got two backends that flush WAL one after the other. Assume (as is not unlikely) that the seco

Re: [HACKERS] Changes to backup.sgml

2015-05-15 Thread Joshua D. Drake
On 05/15/2015 07:42 AM, Bruce Momjian wrote: 3. Push the rsync paragraph (and edit where appropriate) within the continuous archiving section. 3a. Add information about robocopy (windows rsync) Oh, yes, we should mention robocopy. I had never heard of that. 4. Move continuous arch

[HACKERS] Changes to backup.sgml

2015-05-14 Thread Joshua D. Drake
-hackers, After my brain flatulence last week on backups, I decided to read the docs again. There are some improvements that I would like to make and wanted some feedback: 1. File System Level Backup The section should be a note within the larger document. It is largely a legacy section fr

Re: [HACKERS] a few thoughts on the schedule

2015-05-13 Thread Joshua D. Drake
On 05/13/2015 09:27 AM, Tom Lane wrote: Andres Freund writes: On 2015-05-13 11:52:40 -0400, Tom Lane wrote: One thing that continues to bother me about the commitfest process is that it's created a default expectation that things get committed eventually. Agreed that this is a problem. I t

Re: [HACKERS] a few thoughts on the schedule

2015-05-13 Thread Joshua D. Drake
On 05/13/2015 08:09 AM, Tom Lane wrote: What I think we need to be doing this week is triage. Commit what's ready, punt what's not. I'll post a separate list about that. regards, tom lane +1 JD -- Command Prompt, Inc. - http://www.commandprompt.com/ 503-667-4564

Re: [HACKERS] Multi-xacts and our process problem

2015-05-11 Thread Joshua D. Drake
On 05/11/2015 04:18 PM, Simon Riggs wrote: On 11 May 2015 at 23:47, Bruce Momjian mailto:br...@momjian.us>> wrote: On Mon, May 11, 2015 at 03:42:26PM -0700, Joshua Drake wrote: > >The releases themselves are not the problem, but rather the volume of > >bugs and our slowness in getti

Re: [HACKERS] Multi-xacts and our process problem

2015-05-11 Thread Joshua D. Drake
On 05/11/2015 04:04 PM, Tom Lane wrote: Bruce Momjian writes: On Mon, May 11, 2015 at 03:42:26PM -0700, Joshua Drake wrote: What I am arguing is that the release cycle is at least a big part of the problem. We are trying to get so many new features that bugs are increasing and quality is decr

Re: [HACKERS] Multi-xacts and our process problem

2015-05-11 Thread Joshua D. Drake
increasing and quality is decreasing. If we change the release cycle it will encourage an increase in eyeballs on code we are developing because people aren't going to be in such a rush to "get this feature done for this release". Sincerely, Joshua D. Drake -- Command Prom

Re: [HACKERS] Multi-xacts and our process problem

2015-05-11 Thread Joshua D. Drake
On 05/11/2015 02:00 PM, Bruce Momjian wrote: I think we now know that our inaction didn't serve us well. The question is how can we identify chronic problems and get resources involved sooner. I feel we have been "asleep at the wheel" to some extent on this. Here are some options Slow down

Re: [HACKERS] multixacts woes

2015-05-11 Thread Joshua D. Drake
On 05/11/2015 10:24 AM, Josh Berkus wrote: In terms of adding a new GUC in 9.5: can't we take a stab at auto-tuning this instead of adding a new GUC? We already have a bunch of freezing GUCs which fewer than 1% of our user base has any idea how to set. That is a documentation problem not a u

Re: [HACKERS] add -s to vacuumdb

2015-05-05 Thread Joshua D. Drake
On 05/05/2015 12:22 PM, Andrew Dunstan wrote: On 05/05/2015 03:06 PM, Joshua D. Drake wrote: Hey folks, Just had your standard... our pg_ tables are all bloated out, what is a good way to take care of that. We have -s for reindexdb but not vacuumdb. Thoughts? What command will it run

[HACKERS] add -s to vacuumdb

2015-05-05 Thread Joshua D. Drake
Hey folks, Just had your standard... our pg_ tables are all bloated out, what is a good way to take care of that. We have -s for reindexdb but not vacuumdb. Thoughts? JD -- Command Prompt, Inc. - http://www.commandprompt.com/ 503-667-4564 PostgreSQL Centered full stack support, consulting

Re: [HACKERS] Replication, am I missing something

2015-05-01 Thread Joshua D. Drake
On 05/01/2015 09:28 AM, Andres Freund wrote: On 2015-05-01 09:24:17 -0700, Joshua D. Drake wrote: Origin: select pg_start_backup('my_backup',TRUE); Subscriber: rsync -auvk db1:/var/lib/pgsql/data data Origin: select pg_stop_backup(); Subscriber: remove backup_label Subscri

[HACKERS] Replication, am I missing something

2015-05-01 Thread Joshua D. Drake
nstance, it would be one thing but I am certainly not the only person running into this. Sincerely, Joshua D. Drake -- Command Prompt, Inc. - http://www.commandprompt.com/ 503-667-4564 PostgreSQL Centered full stack support, consulting and development. Announcing "I'm offended&q

Re: [HACKERS] ERROR: unexpected data beyond EOF

2015-04-30 Thread Joshua D. Drake
On 04/30/2015 12:09 PM, Alvaro Herrera wrote: Joshua D. Drake wrote: I take that back, it appears this table is heavily deleted from and also uses the lo_manage() triggers. Well, if it's heavily deleted, then it's probably also heavily vacuumed and from time to time empty pages a

Re: [HACKERS] ERROR: unexpected data beyond EOF

2015-04-30 Thread Joshua D. Drake
I take that back, it appears this table is heavily deleted from and also uses the lo_manage() triggers. -- Command Prompt, Inc. - http://www.commandprompt.com/ 503-667-4564 PostgreSQL Centered full stack support, consulting and development. Announcing "I'm offended" is basically telling the w

Re: [HACKERS] ERROR: unexpected data beyond EOF

2015-04-30 Thread Joshua D. Drake
On 04/30/2015 10:28 AM, Alvaro Herrera wrote: Joshua D. Drake wrote: Alright folks, So I have this error: postgres[21118]: [8-1] ERROR: unexpected data beyond EOF in block 9 of relation base/430666195/430666206 Which produces this lovely hint: postgres[21118]: [8-2] HINT: This has been

[HACKERS] ERROR: unexpected data beyond EOF

2015-04-30 Thread Joshua D. Drake
Alright folks, So I have this error: postgres[21118]: [8-1] ERROR: unexpected data beyond EOF in block 9 of relation base/430666195/430666206 Which produces this lovely hint: postgres[21118]: [8-2] HINT: This has been seen to occur with buggy kernels; consider updating your system. Howev

Re: [HACKERS] pg_dump: largeobject behavior issues (possible bug)

2015-04-24 Thread Joshua D. Drake
On 04/24/2015 03:41 PM, Tom Lane wrote: Given that large objects don't have any individual dependencies, one could envision fixing this by replacing the individual large-object DumpableObjects by a single placeholder to participate in the sort phase, and then when it's time to dump that, scan th

[HACKERS] pg_dump: largeobject behavior issues (possible bug)

2015-04-23 Thread Joshua D. Drake
Hello, I have been working a problem with Andrew Gierth (sp?) in regards to pg_dump. Here is the basic breakdown: FreeBSD 10.1 PostgreSQL 9.3.6 64GB ~ memory 500GB database 228G of largeobjects (106M objects) The database dumps fine as long as we don't dump large objects. However, if we try

Re: [HACKERS] Reducing tuple overhead

2015-04-23 Thread Joshua D. Drake
On 04/23/2015 09:42 AM, Jim Nasby wrote: On 4/23/15 11:24 AM, Andres Freund wrote: I do wonder what, in realistic cases, is actually the bigger contributor to the overhead. The tuple header or the padding we liberally add in many cases... Assuming you're talking about padding between fields.

Re: [HACKERS] Bug #10432 failed to re-find parent key in index

2015-03-31 Thread Joshua D. Drake
On 03/31/2015 11:05 AM, Josh Berkus wrote: I have no intention to backpatch the changes. Too big, too invasive. Perhaps we could consider it after a year or two, once 9.4 is indeed very stable, but at that point you have to wonder if it's really worth the trouble anymore. If someone has runs in

Re: [HACKERS] Bug #10432 failed to re-find parent key in index

2015-03-31 Thread Joshua D. Drake
On 03/31/2015 10:58 AM, Robert Haas wrote: On Tue, Mar 31, 2015 at 1:49 PM, Joshua D. Drake wrote: Perhaps we could consider it after a year or two, once 9.4 is indeed very stable, but at that point you have to wonder if it's really worth the trouble anymore. If someone has runs into

Re: [HACKERS] Bug #10432 failed to re-find parent key in index

2015-03-31 Thread Joshua D. Drake
On 03/31/2015 10:51 AM, Andres Freund wrote: On 2015-03-31 10:49:06 -0700, Joshua D. Drake wrote: On 03/31/2015 04:20 AM, Heikki Linnakangas wrote: Perhaps we could consider it after a year or two, once 9.4 is indeed very stable, but at that point you have to wonder if it's really wort

Re: [HACKERS] Bug #10432 failed to re-find parent key in index

2015-03-31 Thread Joshua D. Drake
On 03/31/2015 04:20 AM, Heikki Linnakangas wrote: I believe that Heikki said he'd backpatch that when 9.4 was considered very stable. I don't think that we've reached that level of confidence in the invasive B-Tree bugfixes that went into 9.4 yet. I have no intention to backpatch the changes.

Re: [HACKERS] Bug #10432 failed to re-find parent key in index

2015-03-30 Thread Joshua D. Drake
Hello, Just wondering if what Peter said was the last word on this? JD -- Command Prompt, Inc. - http://www.commandprompt.com/ 503-667-4564 PostgreSQL Centered full stack support, consulting and development. Announcing "I'm offended" is basically telling the world you can't control your own e

[HACKERS] Bug #10432 failed to re-find parent key in index

2015-03-30 Thread Joshua D. Drake
Hello, We have a database that has run into this problem. The version is 9.1.15 on Linux. I note in this thread: http://www.postgresql.org/message-id/cam-w4hp34ppwegtcwjbznwhq0cmu-lxna62vjku8qrtwlob...@mail.gmail.com That things appear to be fixed in 9.4 but they have not been back-patched?

Re: [HACKERS] Remove fsync ON/OFF as a visible option?

2015-03-21 Thread Joshua D. Drake
On 03/21/2015 12:45 PM, Gavin Flower wrote: How about 2 config files? One marked adult^H^H^H^H^H power users only, or some such, with the really dangerous or unusual options? That has come up before in many threads. I don't know that we need to go down that path again. Consider, power us

Re: [HACKERS] Remove fsync ON/OFF as a visible option?

2015-03-21 Thread Joshua D. Drake
On 03/21/2015 12:32 PM, Gavin Flower wrote: What does ACID mean??? I don't want to trip out on acid, and if I do, I don't want it hanging around. Safer to set this to off!!! I actual do know what ACID means, but some 'children' have write access to a the postgresql.conf file without adequat

Re: [HACKERS] Remove fsync ON/OFF as a visible option?

2015-03-21 Thread Joshua D. Drake
On 03/21/2015 12:00 AM, Mark Kirkwood wrote: -1 Personally I'm against hiding *any* settings. Choosing sensible defaults - yes! Hiding them - that reeks of secret squirrel nonsense and overpaid Oracle dbas that knew the undocumented settings for various capabilities. I think/hope that no open

Re: [HACKERS] Remove fsync ON/OFF as a visible option?

2015-03-21 Thread Joshua D. Drake
On 03/20/2015 11:28 PM, Jaime Casanova wrote: I fought to remove fsync before so i understand JD concerns. and yes, i have seen fsync=off in the field too... what about not removing it but not showing it in postgresql.conf? as a side note, i wonder why trace_sort is not in postgresql.conf..

<    1   2   3   4   5   6   7   8   9   10   >