Re: [HACKERS] TAP test breakage on MacOS X

2014-10-29 Thread Noah Misch
On Wed, Oct 29, 2014 at 08:14:07PM -0400, Peter Eisentraut wrote: > On 10/28/14 9:09 PM, Peter Eisentraut wrote: > > I have looked into IPC::Cmd, but the documentation keeps telling me that > > to do anything interesting I have to have IPC::Run anyway. I'll give it > > a try, though. > > I tried

Re: [HACKERS] Lockless StrategyGetBuffer() clock sweep

2014-10-29 Thread Amit Kapila
On Thu, Oct 30, 2014 at 12:39 AM, Andres Freund wrote: > On 2014-10-29 14:18:33 -0400, Robert Haas wrote: > > > The interaction between this and the bgreclaimer idea is interesting. > > We can't making popping the freelist lockless without somehow dealing > > with the resulting A-B-A problem (name

Re: [HACKERS] group locking: incomplete patch, just for discussion

2014-10-29 Thread Amit Kapila
On Wed, Oct 29, 2014 at 7:38 PM, Simon Riggs wrote: > On 29 October 2014 12:08, Amit Kapila wrote: > > On Wed, Oct 29, 2014 at 2:18 PM, Simon Riggs wrote: > >> > >> CREATE INDEX has a lock on the table. Worker tasks can be simply > >> banned from acquiring new locks and doing many other tasks. >

Re: [HACKERS] Validating CHECK constraints with SPI

2014-10-29 Thread Noah Misch
On Wed, Oct 29, 2014 at 10:24:26AM -0400, Tom Lane wrote: > Dan Robinson writes: > > Since the table is locked to updates while the constraint is validating, > > this means you have to jump through hoops if you want to add a CHECK > > constraint to a large table in a production setting. This valid

Re: [HACKERS] Improve automatic analyze messages for inheritance trees

2014-10-29 Thread Etsuro Fujita
(2014/10/17 18:35), Etsuro Fujita wrote: (2014/10/16 17:17), Simon Riggs wrote: Would it be useful to keep track of how many tables just got analyzed? i.e. analyze of foo (including N inheritance children) I think that's a good idea. So, I'll update the patch. Done. Attached is an updated

Re: [HACKERS] Validating CHECK constraints with SPI

2014-10-29 Thread Dan Robinson
On Wed, Oct 29, 2014 at 7:17 AM, Alvaro Herrera wrote: > Dan Robinson wrote: > > Hi all, > > > > If I'm reading correctly in src/backend/commands/tablecmds.c, it looks > like > > PostgreSQL does a full table scan in validateCheckConstraint and in the > > constraint validation portion of ATRewrite

Re: [HACKERS] TAP test breakage on MacOS X

2014-10-29 Thread Peter Eisentraut
On 10/28/14 9:09 PM, Peter Eisentraut wrote: > I agree we should get rid of subplans. I removed subtests. Everything now runs (without skipping) with Perl 5.8+ plus IPC::Run. > I have looked into IPC::Cmd, but the documentation keeps telling me that > to do anything interesting I have to have IP

Re: [HACKERS] Directory/File Access Permissions for COPY and Generic File Access Functions

2014-10-29 Thread Peter Eisentraut
On 10/29/14 3:41 PM, Jim Nasby wrote: > On 10/29/14, 2:33 PM, Tom Lane wrote: >> Capture the postmaster log. Keep on capturing it till somebody >> fat-fingers their login to the extent of swapping the username and >> password (yeah, I've done that, haven't you?). > > Which begs the question: why

[HACKERS] Review of Refactoring code for sync node detection

2014-10-29 Thread Jim Nasby
Original message (sorry, don't have a copy to reply to :( ): http://www.postgresql.org/message-id/CAB7nPqThX=WvuGA1J-_CQ293dK3FmUivuYkNvHR0W5xjEc=o...@mail.gmail.com Commitfest URL: https://commitfest.postgresql.org/action/patch_view?id=1579 Summary: I'd like someone to chime in on the potential

[HACKERS] Re: Directory/File Access Permissions for COPY and Generic File Access Functions

2014-10-29 Thread Robert Haas
On Wed, Oct 29, 2014 at 5:22 PM, Greg Stark wrote: > On Wed, Oct 29, 2014 at 3:52 PM, Robert Haas wrote: >> And it still doesn't protect against the case where you hardlink to a file >> and then the permissions on that file are later changed. > > Fwiw that's not how hard links work, at least UFS

[HACKERS] Re: Directory/File Access Permissions for COPY and Generic File Access Functions

2014-10-29 Thread Greg Stark
On Wed, Oct 29, 2014 at 3:52 PM, Robert Haas wrote: > And it still doesn't protect against the case where you hardlink to a file > and then the permissions on that file are later changed. Fwiw that's not how hard links work, at least UFS semantics permissions such as ext2 etc. Hard links are link

Re: [HACKERS] Directory/File Access Permissions for COPY and Generic File Access Functions

2014-10-29 Thread Kevin Grittner
Tom Lane wrote: > If you are letting untrustworthy people read the postmaster log > you've already got security problems, as per other recent > threads. That seems like a rather naive perspective. In a large operation there are different roles. To draw some analogies, you might trust your atto

Re: [HACKERS] foreign data wrapper option manipulation during Create foreign table time?

2014-10-29 Thread Demai Ni
Robert and Ronan, many thanks for your response. I realized there is no clean way/api for it. maybe a hacking of ptree can do the trick.. :-) I will also take a look at IMPORT FOREIGN SCHEMA. However, for this requirement, I still need the user to input filename or filefolder, and I'd like to pr

Re: [HACKERS] Support UPDATE table SET(*)=...

2014-10-29 Thread Marko Tiikkaja
Hi Atri, Sorry for the delay. With pgconf.eu and all, it's been very hard to find the time to look at this. On 10/15/14, 10:02 AM, Atri Sharma wrote: Please find attached a patch which implements support for UPDATE table1 SET(*)=... The patch supports both UPDATE table SET(*)=(a,b,c) and UPD

Re: [HACKERS] Directory/File Access Permissions for COPY and Generic File Access Functions

2014-10-29 Thread Stephen Frost
* Robert Haas (robertmh...@gmail.com) wrote: > On Wed, Oct 29, 2014 at 3:31 PM, Stephen Frost wrote: > > I still don't particularly like it and, frankly, the limitations we've > > come up with thus far are not issues for my use-cases and I'd rather > > have them and be able to say "yes, you can us

Re: [HACKERS] pg_background (and more parallelism infrastructure patches)

2014-10-29 Thread Andres Freund
On 2014-10-29 16:38:59 -0400, Robert Haas wrote: > On Wed, Oct 29, 2014 at 3:28 PM, Andres Freund wrote: > >> > Hm. So every user can do this once the extension is created as the > >> > functions are most likely to be PUBLIC. Is this a good idea? > >> > >> Why not? If they can log in, they could

Re: [HACKERS] Directory/File Access Permissions for COPY and Generic File Access Functions

2014-10-29 Thread Stephen Frost
* Tom Lane (t...@sss.pgh.pa.us) wrote: > Stephen Frost writes: > > * Tom Lane (t...@sss.pgh.pa.us) wrote: > >> If your ETL process can be restricted that much, can't it use file_fdw or > >> some such to access a fixed filename set by somebody with more privilege? > > > We currently have the ETL f

Re: [HACKERS] Directory/File Access Permissions for COPY and Generic File Access Functions

2014-10-29 Thread Robert Haas
On Wed, Oct 29, 2014 at 3:31 PM, Stephen Frost wrote: > I still don't particularly like it and, frankly, the limitations we've > come up with thus far are not issues for my use-cases and I'd rather > have them and be able to say "yes, you can use this with some confidence > that it won't trivially

Re: [HACKERS] Directory/File Access Permissions for COPY and Generic File Access Functions

2014-10-29 Thread Tom Lane
Stephen Frost writes: > * Tom Lane (t...@sss.pgh.pa.us) wrote: >> If your ETL process can be restricted that much, can't it use file_fdw or >> some such to access a fixed filename set by somebody with more privilege? > We currently have the ETL figure out what the filename is on a daily > basis a

Re: [HACKERS] pg_background (and more parallelism infrastructure patches)

2014-10-29 Thread Robert Haas
On Wed, Oct 29, 2014 at 3:28 PM, Andres Freund wrote: >> > Hm. So every user can do this once the extension is created as the >> > functions are most likely to be PUBLIC. Is this a good idea? >> >> Why not? If they can log in, they could start separate sessions with >> similar effect. > > Connect

Re: [HACKERS] BRIN indexes - TRAP: BadArgument

2014-10-29 Thread Alvaro Herrera
Heikki Linnakangas wrote: > On 10/07/2014 01:33 AM, Alvaro Herrera wrote: > >I added an USE_ASSERTION-only block in brininsert that runs the union > >support proc and compares the output with the one from regular addValue. > >I haven't tested this too much yet. > > Ok, that's better than nothing.

Re: [HACKERS] Add shutdown_at_recovery_target option to recovery.conf

2014-10-29 Thread Petr Jelinek
On 29/10/14 20:27, Asif Naeem wrote: I have spent sometime to review the patch, overall patch looks good, it applies fine and make check run without issue. If recovery target is specified and shutdown_at_recovery_target is set to true, it shutdown the server at specified recovery point. I do have

Re: [HACKERS] Directory/File Access Permissions for COPY and Generic File Access Functions

2014-10-29 Thread Stephen Frost
* Tom Lane (t...@sss.pgh.pa.us) wrote: > Stephen Frost writes: > > This "ad-hoc data load for Joe" use-case isn't where I had been going > > with this feature, and I do trust the ETL processes that are behind the > > use-case that I've proposed for the most part, but there's also no > > reason for

Re: [HACKERS] Directory/File Access Permissions for COPY and Generic File Access Functions

2014-10-29 Thread Tom Lane
Stephen Frost writes: > This "ad-hoc data load for Joe" use-case isn't where I had been going > with this feature, and I do trust the ETL processes that are behind the > use-case that I've proposed for the most part, but there's also no > reason for those files to be symlinks or have hard-links or

Re: [HACKERS] Directory/File Access Permissions for COPY and Generic File Access Functions

2014-10-29 Thread Stephen Frost
* Tom Lane (t...@sss.pgh.pa.us) wrote: > Well, the issues we've thought of so far require that the attacker have > his own shell-level access to the filesystem, but I would not like to > posit that there are none that don't require it. Race conditions, for > example, could be exploited without a s

Re: [HACKERS] Replication identifiers, take 3

2014-10-29 Thread Simon Riggs
On 2 October 2014 09:49, Heikki Linnakangas wrote: >> What I've previously suggested (and which works well in BDR) is to add >> the internal id to the XLogRecord struct. There's 2 free bytes of >> padding that can be used for that purpose. > > > Adding a field to XLogRecord for this feels wrong.

Re: [HACKERS] Directory/File Access Permissions for COPY and Generic File Access Functions

2014-10-29 Thread Jim Nasby
On 10/29/14, 2:33 PM, Tom Lane wrote: Capture the postmaster log. Keep on capturing it till somebody fat-fingers their login to the extent of swapping the username and password (yeah, I've done that, haven't you?). Which begs the question: why on earth do we log passwords at all? This is a pr

Re: [HACKERS] WIP: Access method extendability

2014-10-29 Thread Andres Freund
On 2014-10-29 14:33:00 -0500, Jim Nasby wrote: > On 10/28/14, 3:27 PM, Simon Riggs wrote: > >On 28 October 2014 17:50, Jim Nasby wrote: > >>On 10/28/14, 9:22 AM, Simon Riggs wrote: > >>> > >>>2. Some additional code in Autovacuum to rebuild corrupt indexes at > >>>startup, using AV worker processe

Re: [HACKERS] WIP: Access method extendability

2014-10-29 Thread Jim Nasby
On 10/28/14, 3:27 PM, Simon Riggs wrote: On 28 October 2014 17:50, Jim Nasby wrote: On 10/28/14, 9:22 AM, Simon Riggs wrote: 2. Some additional code in Autovacuum to rebuild corrupt indexes at startup, using AV worker processes to perform a REINDEX CONCURRENTLY. I don't think loading more

Re: [HACKERS] Directory/File Access Permissions for COPY and Generic File Access Functions

2014-10-29 Thread Tom Lane
Stephen Frost writes: > * Kevin Grittner (kgri...@ymail.com) wrote: >> What's interesting and disappointing here is that not one of these >> suggested vulnerabilities seems like a possibility on a database >> server managed in what I would consider a sane and secure manner[1]. > For my part- I ag

Re: [HACKERS] pg_background (and more parallelism infrastructure patches)

2014-10-29 Thread Petr Jelinek
On 29/10/14 20:00, Robert Haas wrote: After reviewing all of those possibilities with the sort of laser-like focus the situation demands, I'm inclined to endorse Alvaro's proposal to rename the existing dsm_keep_mapping() function to dsm_pin_mapping() and the existing dsm_keep_segment() function

Re: [HACKERS] Directory/File Access Permissions for COPY and Generic File Access Functions

2014-10-29 Thread Stephen Frost
* Tom Lane (t...@sss.pgh.pa.us) wrote: > So heaven help you if you grant user joe access in directory > /home/joe/copydata, or any other directory whose parent is writable by > him. He can just remove the directory and replace it with a symlink to > whatever directory contains files he'd like the

Re: [HACKERS] pg_background (and more parallelism infrastructure patches)

2014-10-29 Thread Andres Freund
On 2014-10-22 19:03:28 -0400, Robert Haas wrote: > On Wed, Oct 8, 2014 at 6:32 PM, Andres Freund wrote: > > I got to ask: Why is it helpful that we have this in contrib? I have a > > good share of blame to bear for that, but I think we need to stop > > dilluting contrib evermore with test programs

Re: [HACKERS] Add shutdown_at_recovery_target option to recovery.conf

2014-10-29 Thread Asif Naeem
Hi Petr, I have spent sometime to review the patch, overall patch looks good, it applies fine and make check run without issue. If recovery target is specified and shutdown_at_recovery_target is set to true, it shutdown the server at specified recovery point. I do have few points to share i.e. 1.

Re: [HACKERS] pg_receivexlog --status-interval add fsync feedback

2014-10-29 Thread Robert Haas
On Wed, Oct 22, 2014 at 9:26 AM, Heikki Linnakangas wrote: > We seem to be going in circles. You suggested having two options, > --feedback, and --fsync, which is almost exactly what Furuya posted > originally. I objected to that, because I think that user interface is too > complicated. Instead,

Re: [HACKERS] Lockless StrategyGetBuffer() clock sweep

2014-10-29 Thread Andres Freund
On 2014-10-29 14:18:33 -0400, Robert Haas wrote: > On Mon, Oct 27, 2014 at 9:32 AM, Andres Freund wrote: > > I've previously posted a patch at > > http://archives.postgresql.org/message-id/20141010160020.GG6670%40alap3.anarazel.de > > that reduces contention in StrategyGetBuffer() by making the cl

Re: [HACKERS] lag_until_you_get_something() OVER () window function

2014-10-29 Thread Merlin Moncure
On Wed, Oct 29, 2014 at 12:04 PM, Kirk Roybal wrote: > This [custom aggregate gapfill] is a pretty elegant way of getting there. > > It also does a better job of respecting the window frame. > > I'll use this until this > https://commitfest.postgresql.org/action/patch_view?id=1096 shows up. Yes.

Re: [HACKERS] pg_background (and more parallelism infrastructure patches)

2014-10-29 Thread Andres Freund
On 2014-10-29 15:00:36 -0400, Robert Haas wrote: > 1. Does anyone strongly object to that course of action? I don't. > 2. Does anyone wish to argue for or against back-patching the name > changes to 9.4? +1. Greetings, Andres Freund -- Andres Freund http://www.2ndQuadran

Re: [HACKERS] pg_background (and more parallelism infrastructure patches)

2014-10-29 Thread Robert Haas
On Sat, Oct 25, 2014 at 7:01 AM, Alvaro Herrera wrote: > I do think that dsm_keep_mapping is a strange name for what it does. OK, so let me see if I can summarize the votes so far on this (highly important?) naming issue: - Andres doesn't like "unkeep". He suggests dsm_manage_mapping(), dsm_en

Re: [HACKERS] proposal: CREATE DATABASE vs. (partial) CHECKPOINT

2014-10-29 Thread Tomas Vondra
On 29.10.2014 18:31, Robert Haas wrote: > On Mon, Oct 27, 2014 at 8:01 PM, Tomas Vondra wrote: >> (3) write-heavy workloads / large template database >> >> Current approach wins, for two reasons: (a) for large databases the >> WAL-logging overhead may generate much more I/O than a checkpoi

Re: [HACKERS] Directory/File Access Permissions for COPY and Generic File Access Functions

2014-10-29 Thread Stephen Frost
Kevin, * Kevin Grittner (kgri...@ymail.com) wrote: > Tom Lane wrote: > > So at this point we've decided that we must forbid access to symlinked or > > hardlinked files, which is a significant usability penalty; we've also > > chosen to blow off most older platforms entirely; and we've only spent

Re: [HACKERS] WIP: Access method extendability

2014-10-29 Thread Simon Riggs
On 29 October 2014 09:27, Simon Riggs wrote: > The current system does not allow for the possibility of a corruption > bug. If one occurs, the only thing an AM can do is PANIC. It has no > mechanism to isolate the problem and deal with it, which affects the > whole server. > > So the issue is one

Re: [HACKERS] Directory/File Access Permissions for COPY and Generic File Access Functions

2014-10-29 Thread Tom Lane
I wrote: > ... and we've only spent > about five minutes actually looking for security issues, with no good > reason to assume there are no more. Oh, here's another one: what I read in RHEL6's open(2) man page is O_NOFOLLOW If pathname is a symbolic link, then the open fails.

Re: [HACKERS] Materialized views don't show up in information_schema

2014-10-29 Thread Robert Haas
On Wed, Oct 29, 2014 at 1:57 PM, Stephen Frost wrote: >> No. Materialized views don't have column defaults, and marking them >> security_barrier does nothing. > > I'm a bit confused by this- views have column defaults? Yep. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterpris

Re: [HACKERS] Lockless StrategyGetBuffer() clock sweep

2014-10-29 Thread Robert Haas
On Mon, Oct 27, 2014 at 9:32 AM, Andres Freund wrote: > I've previously posted a patch at > http://archives.postgresql.org/message-id/20141010160020.GG6670%40alap3.anarazel.de > that reduces contention in StrategyGetBuffer() by making the clock sweep > lockless. Robert asked me to post it to a ne

Re: [HACKERS] Directory/File Access Permissions for COPY and Generic File Access Functions

2014-10-29 Thread Kevin Grittner
Tom Lane wrote: > So at this point we've decided that we must forbid access to symlinked or > hardlinked files, which is a significant usability penalty; we've also > chosen to blow off most older platforms entirely; and we've only spent > about five minutes actually looking for security issues,

Re: [HACKERS] Directory/File Access Permissions for COPY and Generic File Access Functions

2014-10-29 Thread Tom Lane
Andres Freund writes: > On 2014-10-29 16:38:44 +, Jeremy Harris wrote: >> On 29/10/14 16:11, Andres Freund wrote: >>> I do think checking the link count to >>> be 1 is safe though. >> You will fail against certain styles of online-backup. > Meh. I don't think that's really a problem for the

Re: [HACKERS] Materialized views don't show up in information_schema

2014-10-29 Thread Stephen Frost
* Robert Haas (robertmh...@gmail.com) wrote: > On Wed, Oct 29, 2014 at 1:26 PM, Stephen Frost wrote: > > I agree with this, certainly, but these are not considerations that the > > SQL spec takes into account. I've always found it odd of the spec to > > avoid these considerations and concerns, bu

Re: [HACKERS] Add CREATE support to event triggers

2014-10-29 Thread Robert Haas
On Tue, Oct 28, 2014 at 6:00 AM, Andres Freund wrote: >> Uhm. Obviously we didn't have jsonb when I started this and we do have >> them now, so I could perhaps see about updating the patch to do things >> this way; but I'm not totally sold on that idea, as my ObjTree stuff is >> a lot easier to m

Re: [HACKERS] Materialized views don't show up in information_schema

2014-10-29 Thread Robert Haas
On Wed, Oct 29, 2014 at 1:26 PM, Stephen Frost wrote: > I agree with this, certainly, but these are not considerations that the > SQL spec takes into account. I've always found it odd of the spec to > avoid these considerations and concerns, but it is the spec and it's > viewpoint that we're disc

Re: [HACKERS] proposal: CREATE DATABASE vs. (partial) CHECKPOINT

2014-10-29 Thread Robert Haas
On Mon, Oct 27, 2014 at 8:01 PM, Tomas Vondra wrote: > (3) write-heavy workloads / large template database > > Current approach wins, for two reasons: (a) for large databases the > WAL-logging overhead may generate much more I/O than a checkpoint, > and (b) it may generate so many WAL

Re: [HACKERS] Materialized views don't show up in information_schema

2014-10-29 Thread Stephen Frost
* Robert Haas (robertmh...@gmail.com) wrote: > On Mon, Oct 27, 2014 at 11:45 AM, Stephen Frost wrote: > > I don't think Kevin was wrong to use a different relkind, but I don't > > buy into the argument that a different relkind means it's not a view. > > As for the other comments, I agree that a ma

Re: [HACKERS] Directory/File Access Permissions for COPY and Generic File Access Functions

2014-10-29 Thread Tom Lane
Stephen Frost writes: > * Tom Lane (t...@sss.pgh.pa.us) wrote: >> This points up the fact that platform-specific security holes are likely >> to be a huge part of the problem. I won't even speculate about our odds >> of building something that's secure on Windows. > Andres' suggestion to only pr

Re: [HACKERS] Directory/File Access Permissions for COPY and Generic File Access Functions

2014-10-29 Thread Stephen Frost
* Jeremy Harris (j...@wizmail.org) wrote: > On 29/10/14 16:11, Andres Freund wrote: > > I do think checking the link count to > > be 1 is safe though. > > You will fail against certain styles of online-backup. Fail-safe though, no? For my part, I'm not particularly bothered by that; we'd have t

Re: [HACKERS] Autovacuum fails to keep visibility map up-to-date in mostly-insert-only-tables

2014-10-29 Thread Robert Haas
On Mon, Oct 27, 2014 at 5:51 PM, Alvaro Herrera wrote: > Jeff Janes wrote: >> It is only a page read if you have to read the page. It would seem optimal >> to have bgwriter adventitiously set hint bits and vm bits, because that is >> the last point at which the page can be changed without risking

Re: [HACKERS] Materialized views don't show up in information_schema

2014-10-29 Thread Robert Haas
On Mon, Oct 27, 2014 at 11:45 AM, Stephen Frost wrote: >> But I think it's the wrong thing anyway, because it presumes that, >> when Kevin chose to make materialized views a different relkind and a >> different object type, rather than just a property of an object, he >> made the wrong call, and I

Re: [HACKERS] Directory/File Access Permissions for COPY and Generic File Access Functions

2014-10-29 Thread Andres Freund
On 2014-10-29 16:38:44 +, Jeremy Harris wrote: > On 29/10/14 16:11, Andres Freund wrote: > > I do think checking the link count to > > be 1 is safe though. > > You will fail against certain styles of online-backup. Meh. I don't think that's really a problem for the usecases for COPY FROM. G

Re: [HACKERS] lag_until_you_get_something() OVER () window function

2014-10-29 Thread Kirk Roybal
This is a pretty elegant way of getting there. It also does a better job of respecting the window frame. I'll use this until this https://commitfest.postgresql.org/action/patch_view?id=1096 [1] shows up. Thanks On 2014-10-28 17:35, Merlin Moncure wrote: > On Tue, Oct 28, 2014 at 12:40

Re: [HACKERS] foreign data wrapper option manipulation during Create foreign table time?

2014-10-29 Thread Ronan Dunklau
Le mercredi 29 octobre 2014 12:49:12 Robert Haas a écrit : > On Tue, Oct 28, 2014 at 5:26 PM, Demai Ni wrote: > > I am looking for a couple pointers here about fdw, and how to change the > > option values during CREATE table time. > > > > I am using postgres-xc-1.2.1 right now. For example, it co

Re: [HACKERS] lag_until_you_get_something() OVER () window function

2014-10-29 Thread Kirk Roybal
This is cleaner and better. Thanks for the link, I hope to see it in a commitfest some time soon. /Kirk On 2014-10-28 16:34, Vladimir Sitnikov wrote: > There is already a patch for that (ignore/respect nulls in lead/lag): > https://commitfest.postgresql.org/action/patch_view?id=1096 [1]

Re: [HACKERS] Proposal: Log inability to lock pages during vacuum

2014-10-29 Thread Jim Nasby
On 10/21/14, 6:05 PM, Tom Lane wrote: Jim Nasby writes: - What happens if we run out of space to remember skipped blocks? You forget some, and are no worse off than today. (This might be an event worthy of logging, if the array is large enough that we don't expect it to happen often ...) M

Re: [HACKERS] Directory/File Access Permissions for COPY and Generic File Access Functions

2014-10-29 Thread Jeremy Harris
On 29/10/14 16:11, Andres Freund wrote: > I do think checking the link count to > be 1 is safe though. You will fail against certain styles of online-backup. -- Cheers, Jeremy -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http:/

Re: [HACKERS] foreign data wrapper option manipulation during Create foreign table time?

2014-10-29 Thread Robert Haas
On Tue, Oct 28, 2014 at 5:26 PM, Demai Ni wrote: > I am looking for a couple pointers here about fdw, and how to change the > option values during CREATE table time. > > I am using postgres-xc-1.2.1 right now. For example, it contains file_fdw, > whose create-table-stmt looks like: > CREATE FOREIG

Re: [HACKERS] Failback to old master

2014-10-29 Thread Robert Haas
On Wed, Oct 29, 2014 at 12:43 PM, Maeldron T. wrote: > Thank you, Robert. > > I thought that removing the recovery.conf file makes the slave master only > after the slave was restarted. (Unlike creating the a trigger_file). Isn't > this true? Yes, but after the restart, the slave will also rewind

Re: [HACKERS] Directory/File Access Permissions for COPY and Generic File Access Functions

2014-10-29 Thread Robert Haas
On Wed, Oct 29, 2014 at 12:36 PM, Adam Brightwell wrote: > Robert, > >> To articular my own concerns perhaps a bit better, there are two major >> things I don't like about the whole DIRALIAS proposal. Number one, >> you're creating this SQL object whose name is not actually used for >> anything o

Re: [HACKERS] Failback to old master

2014-10-29 Thread Maeldron T.
Thank you, Robert. I thought that removing the recovery.conf file makes the slave master only after the slave was restarted. (Unlike creating the a trigger_file). Isn't this true? I also thought that if there was a crash on the original master and it applied WAL entries on itself that are not pre

Re: [HACKERS] Directory/File Access Permissions for COPY and Generic File Access Functions

2014-10-29 Thread Stephen Frost
Adam, * Adam Brightwell (adam.brightw...@crunchydatasolutions.com) wrote: > Pardon my ignorance, but can you help me understand the advantage of not > having absolute path names in the COPY command? If you're writing ETL processes and/or PL/PgSQL code which embeds the COPY command and you migrate

Re: [HACKERS] Deferring some AtStart* allocations?

2014-10-29 Thread Robert Haas
On Tue, Oct 28, 2014 at 10:16 AM, Andres Freund wrote: > On 2014-10-24 11:25:23 -0400, Robert Haas wrote: >> On Fri, Oct 24, 2014 at 10:10 AM, Andres Freund >> wrote: >> > What I was thinking was that you'd append the messages to the layer one >> > level deeper than the parent. Then we'd missed

Re: [HACKERS] Allow peer/ident to fall back to md5?

2014-10-29 Thread Jim Nasby
On 10/29/14, 11:23 AM, Josh Berkus wrote: I don't see a problem with having a "continue" directive, and documenting that it only works with peer and ident. Maybe someday (protocol bump) we can have a way to make other methods continue, and then nobody will need to change their files to support t

Re: [HACKERS] Directory/File Access Permissions for COPY and Generic File Access Functions

2014-10-29 Thread Adam Brightwell
Alvaro, I think it would make more sense if the file-accessing command specified > the DIRALIAS (or DIRECTORY, whatever we end up calling this) and a > pathname relative to the base one. Something like > > postgres=# CREATE DIRECTORY logdir ALIAS FOR '/pgsql/data/pg_log'; Following this, what d

Re: [HACKERS] Directory/File Access Permissions for COPY and Generic File Access Functions

2014-10-29 Thread Adam Brightwell
Robert, > To articular my own concerns perhaps a bit better, there are two major > things I don't like about the whole DIRALIAS proposal. Number one, > you're creating this SQL object whose name is not actually used for > anything other than manipulating the alias you created. The users are > s

Re: [HACKERS] Directory/File Access Permissions for COPY and Generic File Access Functions

2014-10-29 Thread Stephen Frost
* Tom Lane (t...@sss.pgh.pa.us) wrote: > This points up the fact that platform-specific security holes are likely > to be a huge part of the problem. I won't even speculate about our odds > of building something that's secure on Windows. Andres' suggestion to only provide it on platforms which su

Re: [HACKERS] group locking: incomplete patch, just for discussion

2014-10-29 Thread Simon Riggs
On 29 October 2014 15:43, Robert Haas wrote: > On Wed, Oct 29, 2014 at 10:21 AM, Simon Riggs wrote: >> There is a real danger that your "ta-dah" moment sometime in the >> future contains flaws which need to be addressed, but we now have >> piles of questionable infrastructure lieing around. If yo

Re: [HACKERS] Directory/File Access Permissions for COPY and Generic File Access Functions

2014-10-29 Thread Tom Lane
Stephen Frost writes: > * Tom Lane (t...@sss.pgh.pa.us) wrote: >> No such file in RHEL 6.6 :-(. > Ouch. Although- have you tested when happens there? Pretty much exactly the same thing I just saw on OSX, ie, nothing. [tgl@sss1 zzz]$ touch foo [tgl@sss1 zzz]$ ls -l total 0 -rw-rw-r--. 1 tgl tgl

Re: [HACKERS] Allow peer/ident to fall back to md5?

2014-10-29 Thread Josh Berkus
On 10/29/2014 02:52 AM, Craig Ringer wrote: > On 10/29/2014 05:46 PM, Andres Freund wrote: >> I like this one. But then I perhaps edited too many pam configuration >> files. > > It seems good to me too. I haven't looked at how viable it is in > implementation terms. > > I think we could only prop

Re: [HACKERS] Directory/File Access Permissions for COPY and Generic File Access Functions

2014-10-29 Thread Stephen Frost
* Tom Lane (t...@sss.pgh.pa.us) wrote: > Stephen Frost writes: > > * Alvaro Herrera (alvhe...@2ndquadrant.com) wrote: > >> Users cannot create a hard link to a file they can't already access. > > > The specifics actually depend on (on Linux, at least) the value of > > /proc/sys/fs/protected_hardl

Re: [HACKERS] Directory/File Access Permissions for COPY and Generic File Access Functions

2014-10-29 Thread Tom Lane
Stephen Frost writes: > * Alvaro Herrera (alvhe...@2ndquadrant.com) wrote: >> Users cannot create a hard link to a file they can't already access. > The specifics actually depend on (on Linux, at least) the value of > /proc/sys/fs/protected_hardlink, which has existed in upstream since 3.6 > (not

Re: [HACKERS] Directory/File Access Permissions for COPY and Generic File Access Functions

2014-10-29 Thread Andres Freund
On 2014-10-29 12:09:00 -0400, Tom Lane wrote: > Stephen Frost writes: > > * Robert Haas (robertmh...@gmail.com) wrote: > >> I think the question is "just how innumerable are those attack > >> routes"? So, we can prevent a symlink from being used via O_NOFOLLOW. > >> But what about hard links? >

Re: [HACKERS] Directory/File Access Permissions for COPY and Generic File Access Functions

2014-10-29 Thread Stephen Frost
* Andres Freund (and...@2ndquadrant.com) wrote: > On 2014-10-29 12:03:54 -0400, Robert Haas wrote: > > I don't see how you can draw an arbitrary line there. We either > > guarantee that the logged-in user can't usurp the server's > > permissions, or we don't. Making it happen only sometimes in ca

Re: [HACKERS] Directory/File Access Permissions for COPY and Generic File Access Functions

2014-10-29 Thread Tom Lane
Stephen Frost writes: > * Robert Haas (robertmh...@gmail.com) wrote: >> I think the question is "just how innumerable are those attack >> routes"? So, we can prevent a symlink from being used via O_NOFOLLOW. >> But what about hard links? > You can't hard link to files you don't own. That restri

Re: [HACKERS] Directory/File Access Permissions for COPY and Generic File Access Functions

2014-10-29 Thread Andres Freund
On 2014-10-29 12:03:54 -0400, Robert Haas wrote: > >> And it > >> still doesn't protect against the case where you hardlink to a file > >> and then the permissions on that file are later changed. > > > > Imo that's simply not a problem that we need to solve - it's much more > > general and independ

Re: [HACKERS] Directory/File Access Permissions for COPY and Generic File Access Functions

2014-10-29 Thread Robert Haas
On Wed, Oct 29, 2014 at 12:00 PM, Andres Freund wrote: > It's possible to do this securely by doing a fstat() and checking the > link count. Good point. >> And it >> still doesn't protect against the case where you hardlink to a file >> and then the permissions on that file are later changed. >

Re: [HACKERS] Directory/File Access Permissions for COPY and Generic File Access Functions

2014-10-29 Thread Andres Freund
On 2014-10-29 11:52:43 -0400, Robert Haas wrote: > On Wed, Oct 29, 2014 at 11:34 AM, Stephen Frost wrote: > > The specifics actually depend on (on Linux, at least) the value of > > /proc/sys/fs/protected_hardlink, which has existed in upstream since 3.6 > > (not sure about the RHEL kernels, though

Re: [HACKERS] Trailing comma support in SELECT statements

2014-10-29 Thread Robert Haas
On Tue, Oct 28, 2014 at 7:59 PM, David Johnston wrote: > I'd be much more inclined to favor this if the user is provided a capability > to have warnings emitted whenever extraneous commas are present - either via > some form of strict mode or linting configuration. My experience with this kind of

Re: [HACKERS] Directory/File Access Permissions for COPY and Generic File Access Functions

2014-10-29 Thread Robert Haas
On Wed, Oct 29, 2014 at 11:34 AM, Stephen Frost wrote: > The specifics actually depend on (on Linux, at least) the value of > /proc/sys/fs/protected_hardlink, which has existed in upstream since 3.6 > (not sure about the RHEL kernels, though I expect they've incorporated > it also at some point al

Re: [HACKERS] group locking: incomplete patch, just for discussion

2014-10-29 Thread Robert Haas
On Wed, Oct 29, 2014 at 10:21 AM, Simon Riggs wrote: > There is a real danger that your "ta-dah" moment sometime in the > future contains flaws which need to be addressed, but we now have > piles of questionable infrastructure lieing around. If you have > similar doubts about anything I'm doing, p

Re: [HACKERS] Directory/File Access Permissions for COPY and Generic File Access Functions

2014-10-29 Thread Tom Lane
Stephen Frost writes: > * Tom Lane (t...@sss.pgh.pa.us) wrote: >> The larger point though is that this is just one of innumerable attack >> routes for anyone with the ability to make the server do filesystem reads >> or writes of his choosing. If you think that's something you can safely >> give

Re: [HACKERS] Directory/File Access Permissions for COPY and Generic File Access Functions

2014-10-29 Thread Stephen Frost
* Alvaro Herrera (alvhe...@2ndquadrant.com) wrote: > Robert Haas wrote: > > On Wed, Oct 29, 2014 at 10:52 AM, Andres Freund > > wrote: > > >> The larger point though is that this is just one of innumerable attack > > >> routes for anyone with the ability to make the server do filesystem reads > >

Re: [HACKERS] Directory/File Access Permissions for COPY and Generic File Access Functions

2014-10-29 Thread Alvaro Herrera
Robert Haas wrote: > On Wed, Oct 29, 2014 at 10:52 AM, Andres Freund > wrote: > >> The larger point though is that this is just one of innumerable attack > >> routes for anyone with the ability to make the server do filesystem reads > >> or writes of his choosing. If you think that's something y

Re: [HACKERS] Directory/File Access Permissions for COPY and Generic File Access Functions

2014-10-29 Thread Stephen Frost
* Robert Haas (robertmh...@gmail.com) wrote: > On Wed, Oct 29, 2014 at 10:52 AM, Andres Freund > wrote: > >> The larger point though is that this is just one of innumerable attack > >> routes for anyone with the ability to make the server do filesystem reads > >> or writes of his choosing. If yo

Re: [HACKERS] Directory/File Access Permissions for COPY and Generic File Access Functions

2014-10-29 Thread Stephen Frost
Tom, * Tom Lane (t...@sss.pgh.pa.us) wrote: > Stephen Frost writes: > > Agreed- additional input from others would be great. > > I think this entire concept is a bad idea that will be a never-ending > source of security holes. There are too many things that a user with > filesystem access can d

Re: [HACKERS] Directory/File Access Permissions for COPY and Generic File Access Functions

2014-10-29 Thread Robert Haas
On Wed, Oct 29, 2014 at 10:52 AM, Andres Freund wrote: >> The larger point though is that this is just one of innumerable attack >> routes for anyone with the ability to make the server do filesystem reads >> or writes of his choosing. If you think that's something you can safely >> give to peopl

Re: [HACKERS] Directory/File Access Permissions for COPY and Generic File Access Functions

2014-10-29 Thread Andres Freund
On 2014-10-29 10:47:58 -0400, Tom Lane wrote: > Here is one trivial example: you want to let user joe import COPY > data quickly, so you give him read access in directory foo, which he > has write access on from his own account. Surely that's right in the > middle of use cases you had in mind, or

Re: [HACKERS] Directory/File Access Permissions for COPY and Generic File Access Functions

2014-10-29 Thread Tom Lane
Stephen Frost writes: > Agreed- additional input from others would be great. I think this entire concept is a bad idea that will be a never-ending source of security holes. There are too many things that a user with filesystem access can do to get superuser-equivalent status. Here is one trivia

Re: [HACKERS] pg_basebackup fails with long tablespace paths

2014-10-29 Thread Robert Haas
On Tue, Oct 28, 2014 at 8:29 PM, Peter Eisentraut wrote: > On 10/20/14 2:59 PM, Tom Lane wrote: >> My Salesforce colleague Thomas Fanghaenel observed that the TAP tests >> for pg_basebackup fail when run in a sufficiently deeply-nested directory >> tree. > > As for the test, we can do something li

Re: [HACKERS] WIP: Access method extendability

2014-10-29 Thread Robert Haas
On Tue, Oct 28, 2014 at 7:25 PM, Andres Freund wrote: > To me this is a pretty independent issue. I quite agree. As Stephen was at pains to remind me last night on another thread, we cannot force people to write the patches we think they should write. They get to pursue what they think the prio

Re: [HACKERS] Failback to old master

2014-10-29 Thread Robert Haas
On Wed, Oct 29, 2014 at 6:21 AM, Maeldron T. wrote: > I swear I have read a couple of old threads. Yet I am not sure if it safe to > failback to the old master in case of async replication without base backup. > > Considering: > I have the latest 9.3 server > A: master > B: slave > B is actively c

Re: [HACKERS] Validating CHECK constraints with SPI

2014-10-29 Thread Tom Lane
Dan Robinson writes: > Since the table is locked to updates while the constraint is validating, > this means you have to jump through hoops if you want to add a CHECK > constraint to a large table in a production setting. This validation could > be considerably faster if we enabled it to use relev

Re: [HACKERS] group locking: incomplete patch, just for discussion

2014-10-29 Thread Simon Riggs
On 29 October 2014 12:28, Robert Haas wrote: > I care much more about getting the general infrastructure in place to > make parallel programming feasible in PostgreSQL than I do about > getting one particular case working. And more than feasible: I want > it to be relatively straightforward. Th

Re: [HACKERS] Validating CHECK constraints with SPI

2014-10-29 Thread Alvaro Herrera
Dan Robinson wrote: > Hi all, > > If I'm reading correctly in src/backend/commands/tablecmds.c, it looks like > PostgreSQL does a full table scan in validateCheckConstraint and in the > constraint validation portion of ATRewriteTable. > > Since the table is locked to updates while the constraint

  1   2   >