Re: [HACKERS] pg_basebackup compression TODO item

2016-03-03 Thread Joshua D. Drake
On 03/03/2016 09:23 AM, Jeff Janes wrote: Since SSL compression seems to be a busted flush, I would like to see pg_basebackup be able to do compression on the server end, not just the client end, in order to spare network bandwidth. Any comments on how hard this would be, or why we don't want it

Re: [HACKERS] pg_basebackup compression TODO item

2016-03-03 Thread Joshua D. Drake
On 03/03/2016 09:34 AM, Andres Freund wrote: On 2016-03-03 18:31:03 +0100, Magnus Hagander wrote: I think we want it at protocol level rather than pg_basebackup level. I think we may want both eventually, but I do agree that protocol level has a lot higher "priority" than that. Something like

Re: [HACKERS] Greeting for coming back, and where is PostgreSQL going

2016-03-04 Thread Joshua D. Drake
On 03/04/2016 03:20 AM, MauMau wrote: > I've been visually impaired since birth, and now I'm almost blind (can > only sense the light). I'm using screen reader software to use PCs and > smartphones. As I'm using pgindent, I'm sure the source code style > won't be bad. But I might overlook so

Re: [HACKERS] The plan for FDW-based sharding

2016-03-04 Thread Joshua D. Drake
On 03/04/2016 04:41 PM, Robert Haas wrote: As far as I understand it, Bruce came in near the end of that conversation and now wants to claim credit for something that doesn't really exist yet and, to the extent that it does exist, wasn't even his idea. Robert, This does not sound like Bruce at

Re: [HACKERS] JPUG wants to have a copyright notice on the translated doc

2016-03-04 Thread Joshua D. Drake
On 03/04/2016 05:39 PM, Tatsuo Ishii wrote: JPUG (Japan PostgreSQL Users Group) would like to add a copyright ntice to the Japanese translated docs. http://www.postgresql.jp/document/9.5/html/ Currently "Copyright 1996-2016 The PostgreSQL Global Development Group" is showed on the translated do

Re: [HACKERS] JPUG wants to have a copyright notice on the translated doc

2016-03-04 Thread Joshua D. Drake
On 03/04/2016 06:01 PM, Tatsuo Ishii wrote: Considering they are BSD licensed, I am not sure what abuses could be taken? I imagine kind of an extream case: a bad guy removes "Copyright 1996-2016 The PostgreSQL Global Development Group" and replaces it with his/her copyright. Right but again,

Re: [HACKERS] How can we expand PostgreSQL ecosystem?

2016-03-07 Thread Joshua D. Drake
On 03/07/2016 11:31 AM, MauMau wrote: Why don't we enrich the catalog? I'd like to hear ideas on how to enrich the catalog efficiently. It's ideal for software vendors and users to voluntarily add to the catalog. I think the product/software directory has vastly outlived its purpose. We are

Re: [HACKERS] Proposal: RETURNING primary_key()

2016-03-07 Thread Joshua D. Drake
On 03/07/2016 12:32 PM, Igal @ Lucee.org wrote: The problem is that we do not always know in advance what the Primary Key is, and therefore a solution that was implemented in the pgjdbc I agree that the problem is that you don't always know what the primary key is. I would argue the solutio

Re: [HACKERS] New competition from Microsoft?

2016-03-07 Thread Joshua D. Drake
On 03/07/2016 01:43 PM, Josh berkus wrote: All, http://blogs.microsoft.com/?p=67248 Once SQL Server is available on Linux, we're going to see more people using it as an alternative to PostgreSQL. Especially since they're picking up a lot of our better features, like R support. Yes but: 1.

Re: [HACKERS] silent data loss with ext4 / all current versions

2016-03-08 Thread Joshua D. Drake
On 03/08/2016 02:16 PM, Robert Haas wrote: On Mon, Mar 7, 2016 at 10:18 PM, Andres Freund wrote: Instead of "durable" I think that "persistent" makes more sense. I find durable a lot more descriptive. persistent could refer to retrying the rename or something. Yeah, I like durable, too. T

Re: [HACKERS] Proposal: RETURNING primary_key()

2016-03-10 Thread Joshua D. Drake
On 03/10/2016 08:28 PM, Craig Ringer wrote: On 11 March 2016 at 03:07, Igal @ Lucee.org mailto:[email protected]>> wrote: I noticed that you usually don't put html in the emails here, but I think that it's appropriate here to show the information in a clear way (also, according to my c

Re: [HACKERS] Freeze avoidance of very large table.

2016-03-11 Thread Joshua D. Drake
On 03/11/2016 09:48 AM, Masahiko Sawada wrote: Thank you so much! What I wanted deal with in thread is almost done. I'm going to more test the feature for 9.6 releasing. Nicely done! Regards, -- Masahiko Sawada -- Command Prompt, Inc. http://the.postgres.company/

Re: [HACKERS] Soliciting Feedback on Improving Server-Side Programming Documentation

2016-03-15 Thread Joshua D. Drake
On 03/15/2016 10:02 AM, Corey Huinker wrote: Some of these things may seem obvious/trivial to you. I would argue that they're only obvious in retrospect, and the more obvious-to-you things we robustly document, the quicker we accumulate programmers who are capable of agreeing that it's obvious,

Re: [HACKERS] Soliciting Feedback on Improving Server-Side Programming Documentation

2016-03-15 Thread Joshua D. Drake
On 03/15/2016 10:30 AM, Corey Huinker wrote: On Tue, Mar 15, 2016 at 1:19 PM, Shulgin, Oleksandr mailto:[email protected]>> wrote: There's also a good deal of README files in the source tree, so I would add: 4. find src -name 'README*' That too. But README's don't show

Re: [HACKERS] fd.c doesn't remove files on a crash-restart

2016-03-18 Thread Joshua D. Drake
On 03/16/2016 11:05 AM, Tom Lane wrote: "Joshua D. Drake" writes: Hello, fd.c[1] will remove files from pgsql_tmp on a restart but not a crash-restart per this comment: /* * NOTE: we could, but don't, call this during a post-backend-crash restart * cycle. The argument for

[HACKERS] Make primnodes.h gender neutral

2016-03-19 Thread Joshua D. Drake
Per the twitter verse, here is an updated version of primnodes.h -- Command Prompt, Inc. http://the.postgres.company/ +1-503-667-4564 PostgreSQL Centered full stack support, consulting and development. Everyone appreciates your honesty, until you are honest

Re: [HACKERS] fd.c doesn't remove files on a crash-restart

2016-03-19 Thread Joshua D. Drake
On 03/16/2016 11:08 AM, Andres Freund wrote: Well as Andrew said, we could also create postmaster start option that defaults to don't save. I think these days you'd simply use restart_after_crash = false. For debugging I found that to be rather valuable. That would have created an extended o

[HACKERS] fd.c doesn't remove files on a crash-restart

2016-03-19 Thread Joshua D. Drake
Hello, fd.c[1] will remove files from pgsql_tmp on a restart but not a crash-restart per this comment: /* * NOTE: we could, but don't, call this during a post-backend-crash restart * cycle. The argument for not doing it is that someone might want to examine * the temp files for debugging pu

Re: [HACKERS] fd.c doesn't remove files on a crash-restart

2016-03-19 Thread Joshua D. Drake
On 03/16/2016 10:56 AM, Andres Freund wrote: I understand that this is designed this way. I think it is a bad idea because: 1. The majority crash-restarts in the wild are going to be diagnosed rather easily within the OS itself. They fall into things like OOM killer and out of disk space. I d

Re: [HACKERS] Make primnodes.h gender neutral

2016-03-19 Thread Joshua D. Drake
On 03/17/2016 01:36 PM, Alvaro Herrera wrote: Robert Haas wrote: On Thu, Mar 17, 2016 at 3:31 PM, Joshua D. Drake wrote: Per the twitter verse, here is an updated version of primnodes.h +1. +1 what? Are you saying this patch is good? I don't think it is: the previous sentence is wr

Re: [HACKERS] Make primnodes.h gender neutral

2016-03-19 Thread Joshua D. Drake
On 03/17/2016 02:11 PM, Tom Lane wrote: "Joshua D. Drake" writes: On 03/17/2016 01:36 PM, Alvaro Herrera wrote: +1 what? Are you saying this patch is good? I don't think it is: the previous sentence is written in third person, and the following ones are currently in third

Re: [HACKERS] fd.c doesn't remove files on a crash-restart

2016-03-19 Thread Joshua D. Drake
On 03/16/2016 11:04 AM, Andres Freund wrote: On 2016-03-16 11:02:09 -0700, Joshua D. Drake wrote: 3. The problem can get worse over time. If you have a very long running instance, any time the backend crash-restarts you have to potential to increase disk space used for no purpose. But I think

[HACKERS] Gendered language in source

2016-03-20 Thread Joshua D. Drake
-Hackers, The twitter verse has picked up on the fact that some of the comments in our source code are gendered. We may want to apply a patch for that. Sincerely, JD -- Command Prompt, Inc. http://the.postgres.company/ +1-503-667-4564 PostgreSQL Cente

Re: [HACKERS] large regression for parallel COPY

2016-03-30 Thread Joshua D. Drake
On 03/30/2016 01:10 PM, Andres Freund wrote: On 2016-03-30 15:50:21 -0400, Robert Haas wrote: On Thu, Mar 10, 2016 at 8:29 PM, Andres Freund wrote: Allow to trigger kernel writeback after a configurable number of writes. While testing out Dilip Kumar's relation extension patch today, I disco

[HACKERS] PgConf.US Hackers Track CFP

2016-01-13 Thread Joshua D. Drake
The organization committee for PGConf.US invites all Pg Hackers to the PGConf.US 2016 hackers track. We want to showcase your talents to the general PostgreSQL community, potential new hackers plus provide a forum for feedback and deep discussion by having you present! We have also brought bac

Re: [HACKERS] Releasing in September

2016-01-20 Thread Joshua D. Drake
s delayed is a good thing, not a bad one. I believe we need to be much more willing to push a button that says, "It isn't done." Which goes back to the idea in #1 of having a release "window" versus date. We aren't a company. We don't need to adhere

Re: [HACKERS] Releasing in September

2016-01-20 Thread Joshua D. Drake
lot of people (think about the multixact problem) that will run any software because it is new. Let's put the proper disclaimers on there and let them run it. Sincerely, Joshua D. Drake -- Command Prompt, Inc. http://the.postgres.company/ +1-503-667-

Re: [HACKERS] Releasing in September

2016-01-20 Thread Joshua D. Drake
On 01/20/2016 09:03 AM, Andres Freund wrote: If people don't fix the issues in time, there needs to be direct pushback, leading to much less stuff getting in next time round. We have been slowly moving to a more dictator based release anyway. It used to be that we released "when it's done", t

Re: [HACKERS] Releasing in September

2016-01-20 Thread Joshua D. Drake
On 01/20/2016 09:17 AM, Andres Freund wrote: On 2016-01-20 09:15:01 -0800, Joshua D. Drake wrote: On 01/20/2016 09:03 AM, Andres Freund wrote: If people don't fix the issues in time, there needs to be direct pushback, leading to much less stuff getting in next time round. We have

Re: [HACKERS] Releasing in September

2016-01-20 Thread Joshua D. Drake
On 01/20/2016 09:31 AM, Joel Jacobson wrote: On Wed, Jan 20, 2016 at 5:29 PM, Andres Freund wrote: I think one thing we should work on, is being absolutely religious about requiring, say, 2 reviews for every nontrivial contribution. We currently seem to have a significantly increased submissio

Re: [HACKERS] Releasing in September

2016-01-20 Thread Joshua D. Drake
On 01/20/2016 09:48 AM, David E. Wheeler wrote: On Jan 20, 2016, at 9:42 AM, Joshua D. Drake wrote: 4. Submit a patch, review a patch. Don't review patches? Don't submit patches. There will always be patches desirable-enough that they will be reviewed whether or not the submitte

Re: [HACKERS] Releasing in September

2016-01-20 Thread Joshua D. Drake
On 01/20/2016 10:53 AM, Simon Riggs wrote: On 20 January 2016 at 15:40, Bruce Momjian mailto:[email protected]>> wrote: Many people where happy with our consistent releasing major releases in September, e.g. 9.0 to 9.3: What is the point in having a special mailing list to discuss the r

Re: [HACKERS] pglogical - logical replication contrib module

2016-01-26 Thread Joshua D. Drake
this what we want to do? Sincerely, Joshua D. Drake -- Command Prompt, Inc. http://the.postgres.company/ +1-503-667-4564 PostgreSQL Centered full stack support, consulting and development. -- Sent via pgsql-hackers mailing list ([email protected]

Re: [HACKERS] New committer

2016-01-28 Thread Joshua D. Drake
On 01/28/2016 06:37 AM, Magnus Hagander wrote: Hello! The PostgreSQL core team would like to welcome Dean Rasheed as a new committer for the PostgreSQL project. Dean - welcome! Now let's see how quickly you can break the buildfarm! Congrats! -- Magnus Hagander Me: http://www.hagander.n

Re: [HACKERS] Template for commit messages

2016-01-28 Thread Joshua D. Drake
e all agree to use a fixed format except for you, we might as well just forget the whole thing, because the percentage of commits that are done by you is quite high. Yes, we are either all in or we may as well forgo this. Sincerely, Joshua D. Drake -- Command Prompt, Inc.

Re: [HACKERS] Template for commit messages

2016-01-28 Thread Joshua D. Drake
ided notable review which may or may not have been code review. Sincerely, Joshua D. Drake -- Command Prompt, Inc. http://the.postgres.company/ +1-503-667-4564 PostgreSQL Centered full stack support, consulting and development. -- Sent via pgsql-hackers ma

Re: [HACKERS] Template for commit messages

2016-01-29 Thread Joshua D. Drake
ould do with it, to make sure that what we add is actually useful, and that we don't add noise to the commit messages unnecessarily. - Heikki I think the best question to ask is: "What is the problem we are trying to solve?" "A bunch of work that probably could be automated&q

Re: [HACKERS] PostgreSQL Audit Extension

2016-01-31 Thread Joshua D. Drake
On 01/31/2016 05:07 AM, Alvaro Herrera wrote: David Steele wrote: The attached patch implements audit logging for PostgreSQL as an extension. I believe I have addressed the concerns that were raised at the end of the 9.5 development cycle. This patch got no feedback at all during the commitfe

Re: [HACKERS] Template for commit messages

2016-01-31 Thread Joshua D. Drake
On 01/29/2016 03:05 PM, Alvaro Herrera wrote: Joshua D. Drake wrote: I think the best question to ask is: "What is the problem we are trying to solve?" The problem is alluring more patch reviewers, beta testers and bug reporters. Do we really want patch reviewers, beta teste

Re: [HACKERS] Template for commit messages

2016-01-31 Thread Joshua D. Drake
On 01/31/2016 04:34 PM, Michael Paquier wrote: On Mon, Feb 1, 2016 at 2:44 AM, Joshua D. Drake wrote: On 01/29/2016 03:05 PM, Alvaro Herrera wrote: Joshua D. Drake wrote: One of the offers is to credit them (I'm not exactly clear on what is the group to benefit from this, but the phrasing

Re: [HACKERS] PostgreSQL Audit Extension

2016-02-01 Thread Joshua D. Drake
On 02/01/2016 08:23 PM, Robert Haas wrote: It also appears to me that if we did want to do that, it would need quite a lot of additional cleanup. I haven't dug in enough to have a list of specific issues, but it does look to me like there would be quite a bit. Maybe that'd be worth doing if th

Re: [HACKERS] PostgreSQL Auditing

2016-02-02 Thread Joshua D. Drake
On 02/02/2016 02:47 AM, José Luis Tallón wrote: On 02/02/2016 02:05 AM, Curtis Ruck wrote: [snip] P.S., do you know what sucks, having a highly performant PostGIS database that works great, and being told to move to Oracle or SQL Server (because they have auditing). Even though they charge ext

Re: [HACKERS] PostgreSQL Auditing

2016-02-02 Thread Joshua D. Drake
On 02/02/2016 07:31 AM, [email protected] wrote: Its not available in rpm or premade binary forms in its current instantiation, not a big deal for me, but it raises the barrier to entry. If it was packaged into an RPM, what would be the process to get it added to postgresql's yum repositor

Re: [HACKERS] PostgreSQL Auditing

2016-02-02 Thread Joshua D. Drake
On 02/02/2016 08:13 AM, Michael Banck wrote: On Tue, Feb 02, 2016 at 07:24:23AM -0800, Joshua D. Drake wrote: PostgreSQL has auditing. It is available now, just not in core. Postgis isn't available in core either and it seems to do just fine. I don't really buy that argument. For on

Re: [HACKERS] PostgreSQL Auditing

2016-02-03 Thread Joshua D. Drake
On 02/02/2016 07:26 PM, Robert Haas wrote: On Tue, Feb 2, 2016 at 8:25 PM, Curtis Ruck wrote: Additionally Robert, given your professional status, you are by no means an unbiased contributor in this discussion. Your stance on this matter shows that you don't necessarily want the open source so

Re: [HACKERS] PostgreSQL Audit Extension

2016-02-03 Thread Joshua D. Drake
On 02/03/2016 10:36 AM, Robert Haas wrote: I'll be the first to admit that the design is not the prettiest. Trying It's entirely reasonable for the community NOT to want to privilege one implementation over another. This, not so much. No, this is ABSOLUTELY critical. Suppose EnterpriseDB

Re: [HACKERS] Idle In Transaction Session Timeout, revived

2016-02-03 Thread Joshua D. Drake
On 02/03/2016 02:52 PM, Robert Haas wrote: On Wed, Feb 3, 2016 at 5:36 PM, Jim Nasby wrote: I think killing the session is a perfectly sensible thing to do in this case. Everything meaningful that was done in the session will be rolled back - no need to waste resources keeping the connection o

Re: [HACKERS] a raft of parallelism-related bug fixes

2016-02-08 Thread Joshua D. Drake
On 02/08/2016 10:45 AM, Robert Haas wrote: On Mon, Feb 8, 2016 at 10:17 AM, Andres Freund wrote: On 2016-02-02 15:41:45 -0500, Robert Haas wrote: I realize that this stuff has all been brewing long, and that there's still a lot to do. So you gotta keep moving. And I'm not sure that there's a

Re: [HACKERS] a raft of parallelism-related bug fixes

2016-02-08 Thread Joshua D. Drake
On 02/08/2016 11:24 AM, Robert Haas wrote: On Mon, Feb 8, 2016 at 2:00 PM, Joshua D. Drake wrote: If I am off base, please feel free to yell Latin at me again but isn't this exactly what different trees are for in Git? Would it be possible to say: Robert says, "Hey pull XYZ, run

Re: [HACKERS] enable parallel query by default?

2016-02-08 Thread Joshua D. Drake
On 02/08/2016 01:07 PM, Robert Haas wrote: Hi, One of the questions I have about parallel query is whether it should be enabled by default. That is, should we make the default value of max_parallel_degree to a value higher than 0? Perhaps 1, say? O.k. after some googling where I found your f

Re: [HACKERS] a raft of parallelism-related bug fixes

2016-02-08 Thread Joshua D. Drake
On 02/08/2016 01:11 PM, Peter Geoghegan wrote: On Mon, Feb 8, 2016 at 12:18 PM, Robert Haas wrote: I accept that this might have been a somewhat isolated incident (that I couldn't easily get *at least* a little instant gratification), but it still should be avoided. You've accused me of buryi

Re: [HACKERS] Multi-tenancy with RLS

2016-02-09 Thread Joshua D. Drake
bled or not and I am a super user, it doesn't matter either way. From my perspective, I should not have to enable row security as a non-super user to take a pg_dump. It should just work for what I am allowed to see. In other words: pg_dump -U $non-super_user Should just work, peri

Re: [HACKERS] Multi-tenancy with RLS

2016-02-09 Thread Joshua D. Drake
On 02/09/2016 12:28 PM, Stephen Frost wrote: JD, * Joshua D. Drake ([email protected]) wrote: pg_dump -U $non-super_user Should just work, period. That ship has sailed already, where you're running a pg_dump against objects you don't own and which have RLS enabled on them.

Re: [HACKERS] Defaults for replication/backup

2016-02-13 Thread Joshua D. Drake
On 02/13/2016 05:37 AM, Michael Paquier wrote: On Sat, Feb 13, 2016 at 10:15 PM, Magnus Hagander wrote: So, I suggest the following changes to the defaults: wal_level=hot_standby max_wal_senders=10 max_replication_slots=10 10 seems a bit high. I would think that max_wal_senders and max_replica

Re: [HACKERS] Defaults for replication/backup

2016-02-13 Thread Joshua D. Drake
Hello, I would like to add the idea of having archiving on by default. Not everyone uses streaming replication, some people use PITR. The one that I see is archive_command and I am not sure how to deal with that. Sincerely, JD -- Command Prompt, Inc. http://the.postgres.com

Re: [HACKERS] Relaxing SSL key permission checks

2016-02-18 Thread Joshua D. Drake
On 02/18/2016 08:22 PM, Tom Lane wrote: Now, I have heard it argued that the OpenSSH/L authors are a bunch of idiots who know nothing about security. But it's not like insisting on restrictive permissions on key files is something we invented out of the blue. It's pretty standard practice, AFA

Re: [HACKERS] The plan for FDW-based sharding

2016-02-26 Thread Joshua D. Drake
On 02/26/2016 08:06 AM, Robert Haas wrote: On Fri, Feb 26, 2016 at 7:21 PM, Oleg Bartunov wrote: Right now tm is hardcoded and it's doesn't matter "if other people might need" at all. We at least provide developers ("other people") ability to work on their implementations and the patch is s

Re: [HACKERS] Renaming of pg_xlog and pg_clog

2016-08-26 Thread Joshua D. Drake
On 08/26/2016 03:48 AM, Magnus Hagander wrote: Same reason I'm also +1 for Stephens suggestion to put all things that should not be in a base backup into the same directory. That may break things now, but it will simplify things down the road. And doing it at the same time as renaming these thi

Re: [HACKERS] Renaming of pg_xlog and pg_clog

2016-08-26 Thread Joshua D. Drake
On 08/25/2016 07:39 PM, Michael Paquier wrote: Hi all, I am relaunching $subject as 10 development will begin soon. As far as I know, there is agreement that we can do something here. Among the different proposals I have found: - pg_clog renamed to pg_commit_status, pg_xact or pg_commit - pg_xlo

Re: [HACKERS] Renaming of pg_xlog and pg_clog

2016-08-26 Thread Joshua D. Drake
On 08/26/2016 09:28 AM, Tom Lane wrote: Magnus Hagander writes: On Aug 26, 2016 5:54 PM, "Peter Eisentraut" < [email protected]> wrote: If we're going to do some renaming, then I suggest we do a mini-file-system structure under $PGDATA, like $PGDATA/etc $PGDATA/log $PGDATA/run

Re: [HACKERS] pg_dump with tables created in schemas created by extensions

2016-08-26 Thread Joshua D. Drake
On 08/26/2016 04:15 PM, Tomas Vondra wrote: On 08/27/2016 12:37 AM, Tom Lane wrote: =?UTF-8?B?TWFydMOtbiBNYXJxdcOpcw==?= writes: Looking at this issue today, I found that we are not setting a dependency for an index created inside an extension. Surely the index has a dependency on a table, w

Re: [HACKERS] Renaming of pg_xlog and pg_clog

2016-08-29 Thread Joshua D. Drake
On 08/27/2016 11:11 AM, Tom Lane wrote: Alvaro Herrera writes: I'm for renaming too, but I'd go with Peter E's suggestion: move pg_xlog to something like $PGDATA/var/wal or $PGDATA/srv/wal or something like that. I think that would make sense if we were to relocate *everything* under PGDATA i

Re: [HACKERS] Renaming of pg_xlog and pg_clog

2016-08-29 Thread Joshua D. Drake
On 08/29/2016 12:04 AM, Magnus Hagander wrote: On Mon, Aug 29, 2016 at 2:45 AM, Jim Nasby mailto:[email protected]>> wrote: On 8/26/16 4:08 PM, Andres Freund wrote: Splitting of ephemeral data seems to have a benefit, the rest seems more like rather noisy busy

Re: [HACKERS] Renaming of pg_xlog and pg_clog

2016-08-29 Thread Joshua D. Drake
On 08/29/2016 06:42 AM, Daniel Verite wrote: Aside from that, we might also question how much of the excuse "pg_xlog looked like it was just deletable logs" is a lie made up after the fact, because anybody wrecking a database is not against deflecting a bit of the blame to the software, that's h

Re: [HACKERS] Renaming of pg_xlog and pg_clog

2016-08-29 Thread Joshua D. Drake
On 08/29/2016 08:07 AM, Tom Lane wrote: "Joshua D. Drake" writes: Also as a note to the idea that we make break things for external user space; the next version being v10 is the exact time to do that. Let's please drop this meme that "v10 is a great time to break things&

Re: [HACKERS] Renaming of pg_xlog and pg_clog

2016-08-29 Thread Joshua D. Drake
On 08/29/2016 10:00 AM, Daniel Verite wrote: Let's imagine that pg_xlog is named wal instead. How does that help our user with the disk space problem? Does that point to a path of resolution? I don't see it. What do we think that user's next move will be? After all, WAL means Write Ahead *Log*.

[HACKERS] restoration after crash slowness, any way to improve?

2016-08-31 Thread Joshua D. Drake
-hackers, So this is more of a spit balling thread than anything. As I understand it, if we have a long running transaction or a large number of wal logs and we crash, we then have to restore those logs on restart to the last known good transaction. No problem. I recently ran a very long tra

[HACKERS] search path security issue?

2017-10-05 Thread Joshua D. Drake
-hackers, Please see the below: """ postgres=# create user foo; CREATE ROLE postgres=# create schema foo; CREATE SCHEMA postgres=# alter role foo set search_path to 'foo'; ALTER ROLE postgres=# \q jd@jd-wks:~$ psql -U foo postgres psql (9.6.5) Type "help" for help. postgres=> show search_path;

Re: [HACKERS] search path security issue?

2017-10-05 Thread Joshua D. Drake
On 10/05/2017 02:54 PM, David G. Johnston wrote: On Thu, Oct 5, 2017 at 2:37 PM, Joshua D. Drake <mailto:[email protected]>>wrote: I get being able to change my search_path on the fly but it seems odd that as user foo I can change my default search path? Seems d

Re: [HACKERS] Columnar storage support

2017-10-09 Thread Joshua D. Drake
On 10/09/2017 01:03 PM, legrand legrand wrote: Is there a chance that pluggable storage permits creation of a columnar rdbms as monetDB in PostgreSQL ? Thanks un advance for thé answer The extension C-Store from Citus is probably what you are looking for. jD -- Sent from: http://www.postg

[HACKERS] Determine state of cluster (HA)

2017-10-12 Thread Joshua D. Drake
-Hackers, I had a long call with a firm developing front end proxy/cache/HA for Postgres today. Essentially the software is a replacement for PGPool in entirety but also supports analytics etc... When I was asking them about pain points they talked about the below and I was wondering if this i

Re: [HACKERS] Determine state of cluster (HA)

2017-10-13 Thread Joshua D. Drake
On 10/12/2017 05:50 PM, Joshua D. Drake wrote: -Hackers, Bumping this. I had a long call with a firm developing front end proxy/cache/HA for Postgres today. Essentially the software is a replacement for PGPool in entirety but also supports analytics etc... When I was asking them about

Re: [HACKERS] Determine state of cluster (HA)

2017-10-16 Thread Joshua D. Drake
On 10/15/2017 07:39 PM, Craig Ringer wrote: On 13 October 2017 at 08:50, Joshua D. Drake wrote: -Hackers, I had a long call with a firm developing front end proxy/cache/HA for Postgres today. Essentially the software is a replacement for PGPool in entirety but also supports analytics etc

Re: [HACKERS] Determine state of cluster (HA)

2017-10-16 Thread Joshua D. Drake
On 10/16/2017 03:55 AM, Magnus Hagander wrote: On Mon, Oct 16, 2017 at 4:39 AM, Craig Ringer <mailto:[email protected]>> wrote: On 13 October 2017 at 08:50, Joshua D. Drake mailto:[email protected]>> wrote: > 5.  There is no way to connect to a db node with

Re: [HACKERS] Determine state of cluster (HA)

2017-10-17 Thread Joshua D. Drake
On 10/16/2017 07:31 PM, Craig Ringer wrote: On 17 October 2017 at 01:02, Joshua D. Drake wrote: On 10/15/2017 07:39 PM, Craig Ringer wrote: - Get info about master. We should finish merging recovery.conf into postgresql.conf. Definitely. There's a patch from Abhijit Menon-Sen you

Re: [HACKERS] Indirect indexes

2016-10-18 Thread Joshua D. Drake
On 10/18/2016 11:28 AM, Alvaro Herrera wrote: Vacuuming presents an additional challenge: in order to remove index items from an indirect index, it's critical to scan the PK index first and collect the PK values that are being removed. Then scan the indirect index and remove any items that matc

[HACKERS] Disable autovacuum guc?

2016-10-19 Thread Joshua D. Drake
Hello, After all these years, we are still regularly running into people who say, "performance was bad so we disabled autovacuum". I am not talking about once in a while, it is often. I would like us to consider removing the autovacuum option. Here are a few reasons: 1. It does not hurt anyo

[HACKERS] Remove autovacuum GUC?

2016-10-20 Thread Joshua D. Drake
Hello, After all these years, we are still regularly running into people who say, "performance was bad so we disabled autovacuum". I am not talking about once in a while, it is often. I would like us to consider removing the autovacuum option. Here are a few reasons: 1. It does not hurt anyo

Re: [HACKERS] Disable autovacuum guc?

2016-10-20 Thread Joshua D. Drake
On 10/19/2016 07:22 PM, Josh Berkus wrote: On 10/19/2016 06:27 PM, Joshua D. Drake wrote: Hello, After all these years, we are still regularly running into people who say, "performance was bad so we disabled autovacuum". I am not talking about once in a while, it is often. I would

Re: [HACKERS] Indirect indexes

2016-10-20 Thread Joshua D. Drake
On 10/20/2016 06:39 AM, Alvaro Herrera wrote: Bruce Momjian wrote: Also, it seems indirect indexes would be useful for indexing columns that are not updated frequently on tables that are updated frequently, and whose primary key is not updated frequently. That's quite a logic problem for user

Re: [HACKERS] Remove autovacuum GUC?

2016-10-20 Thread Joshua D. Drake
On 10/20/2016 07:12 AM, Robert Haas wrote: On Wed, Oct 19, 2016 at 9:24 PM, Joshua D. Drake wrote: Setting autovacuum=off is at least useful for testing purposes and I've used it that way. On the other hand, I haven't seen a customer disable this unintentionally in years. Gene

Re: [HACKERS] Disable autovacuum guc?

2016-10-20 Thread Joshua D. Drake
On 10/20/2016 08:54 AM, Josh Berkus wrote: On 10/20/2016 06:34 AM, Joshua D. Drake wrote: On 10/19/2016 07:22 PM, Josh Berkus wrote: On 10/19/2016 06:27 PM, Joshua D. Drake wrote: Hrm, true although that is by far a minority of our users. What if we made it so we disabled the autovacuum guc

Re: [HACKERS] Renaming of pg_xlog and pg_clog

2016-10-20 Thread Joshua D. Drake
On 10/20/2016 09:12 AM, Stephen Frost wrote: * Tom Lane ([email protected]) wrote: Robert Haas writes: That said, I'd also like to see a --force or similar option or mechanism put in place to reduce the risk of users trashing their system because they think pg_resetwal is "safe." ("It's jus

Re: [HACKERS] Remove autovacuum GUC?

2016-10-20 Thread Joshua D. Drake
On 10/20/2016 09:24 AM, Robert Haas wrote: On Thu, Oct 20, 2016 at 11:53 AM, Joshua D. Drake wrote: The right answer isn't the answer founded in the reality for many if not most of our users. I think that's high-handed nonsense. Sure, there are some unsophisticated users who do

Re: [HACKERS] Remove autovacuum GUC?

2016-10-20 Thread Joshua D. Drake
Hello, What about a simpler solution to all of this. Let's just remove it from postgresql.conf. Out of sight. If someone needs to test they can but a uneducated user won't immediately know what to do about that "autovacuum process" and when they look it up the documentation is exceedingly blun

[HACKERS] Does it make sense to add a -q (quiet) flag to initdb?

2016-10-25 Thread Joshua D. Drake
Hello, Per: https://www.commandprompt.com/blog/can_i_make_initdb_quiet/ This was a question that was asked on #postgresql. Obviously we found a work around but I wonder if it makes sense to add a -q to solve some of these issues? (I could see it being useful with automation). Sincerely, JD

[HACKERS] Issues with building snap packages and psql

2016-10-26 Thread Joshua D. Drake
Hello, CMD has been working on building snap packages. It has been an adventure and we are very close to having them complete. In fact they are complete enough that we have published them in the snap repo. jd@jd-wks:~$ snap find postgres NameVersion Developer Notes Summary post

Re: [HACKERS] Issues with building snap packages and psql

2016-10-26 Thread Joshua D. Drake
On 10/26/2016 10:16 AM, Tom Lane wrote: "Joshua D. Drake" writes: Is psql getting the home directory from /etc/passwd? Yes, of course. If so what can we do about that? Fix your /etc/passwd entry? Seems unlikely that psql is the only program that believes that entry. It

Re: [HACKERS] Issues with building snap packages and psql

2016-10-26 Thread Joshua D. Drake
On 10/26/2016 10:26 AM, Andres Freund wrote: On October 26, 2016 8:09:11 PM GMT+03:00, "Joshua D. Drake" wrote: postgres=# \! export export HOME='/home/jd/snap/postgresql96/1' That doesn't do much. Isn't that just spawning a shell in which you set the va

Re: [HACKERS] Issues with building snap packages and psql

2016-10-26 Thread Joshua D. Drake
Hello, After some further investigation and collaboration with #postgresql it looks like what we have here is an actual bug or at least improper implementation. Apparently, we use getpwuid on the euid to locate the 'home directory'. This is incorrect (as well as inconsistent to our own docume

Re: [HACKERS] Issues with building snap packages and psql

2016-10-26 Thread Joshua D. Drake
On 10/26/2016 11:33 AM, Robert Haas wrote: On Wed, Oct 26, 2016 at 2:22 PM, Joshua D. Drake wrote: After some further investigation and collaboration with #postgresql it looks like what we have here is an actual bug or at least improper implementation. Apparently, we use getpwuid on the euid

Re: [HACKERS] pg_hba_file_settings view patch

2016-10-26 Thread Joshua D. Drake
On 10/26/2016 12:54 PM, Josh Berkus wrote: On 10/26/2016 12:24 PM, Tom Lane wrote: Robert Haas writes: FWIW, I'm -1 on using JSON here. I don't believe that we should start using JSON all over the place just because we can. If we do that, we'll end up with a mishmash of styles, and maybe loo

Re: [HACKERS] Issues with building snap packages and psql

2016-10-27 Thread Joshua D. Drake
On 10/26/2016 12:10 PM, Joshua D. Drake wrote: On 10/26/2016 11:33 AM, Robert Haas wrote: On Wed, Oct 26, 2016 at 2:22 PM, Joshua D. Drake It is also inconsistent with how other programs behave. For example if psql uses readline, and you change the value of $HOME, then readline uses $HOME

Re: [HACKERS] Fix checkpoint skip logic on idle systems by tracking LSN progress

2016-11-10 Thread Joshua D. Drake
On 11/10/2016 09:33 AM, David Steele wrote: On 11/10/16 10:28 AM, Stephen Frost wrote: diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c [...] + if (log_checkpoints) + ereport(LOG, (errmsg("checkpoint skippe

Re: [HACKERS] Mail thread references in commits

2016-11-19 Thread Joshua D. Drake
On 11/17/2016 01:02 PM, Andrew Dunstan wrote: I love seeing references to email threads in commit messages. It would make them a lot friendlier though if a full http URL were included instead of just a Message-ID, i.e. instead of put . I kno

Re: [HACKERS] Documentation improvements for partitioning

2017-02-23 Thread Joshua D. Drake
On 02/23/2017 09:27 AM, Peter Geoghegan wrote: On Thu, Feb 23, 2017 at 8:08 AM, Simon Riggs wrote: * "Good work so far, but there is still a very significant amount of work to do." There is always more work to do, so why say so? I think that the implication is that this isn't complete as a f

Re: [HACKERS] I propose killing PL/Tcl's "modules" infrastructure

2017-02-25 Thread Joshua D. Drake
On 02/25/2017 10:14 AM, Tom Lane wrote: Now, we could try to fix this bug, and add the regression test coverage that the code clearly lacks, and upgrade the documentation about it from its currently very sad state. But I think the right answer is just to remove the feature altogether. It's evi

Re: [HACKERS] Replication vs. float timestamps is a disaster

2017-02-27 Thread Joshua D. Drake
On 02/22/2017 02:45 PM, Tom Lane wrote: Andres Freund writes: On 2017-02-22 08:43:28 -0500, Tom Lane wrote: (To be concrete, I'm suggesting dropping --disable-integer-datetimes in HEAD, and just agreeing that in the back branches, use of replication protocol with float-timestamp servers is not

[HACKERS] Should we eliminate or reduce HUP from docs?

2017-03-10 Thread Joshua D. Drake
Hello, I am a bad speaker, I am writing a talk three weeks before the conference (as opposed to on the plane). I noticed in the docs we still reference the passing of SIGHUP for reloading conf file but we now have pg_reload_conf(); It seems the use of pg_reload_conf() would provide a better

[HACKERS] Odd listen_addresses behavior

2017-03-15 Thread Joshua D. Drake
-hackers, I found this today: jd@jd-wks:~/snap/postgresql96/common/data$ /snap/postgresql96/19/usr/bin/pg_ctl -D data stop pg_ctl: directory "data" does not exist jd@jd-wks:~/snap/postgresql96/common/data$ cd .. jd@jd-wks:~/snap/postgresql96/common$ /snap/postgresql96/19/usr/bin/pg_ctl -D d

  1   2   3   4   5   6   7   8   9   10   >