Re: [HACKERS] Let file_fdw access COPY FROM PROGRAM

2016-09-29 Thread Tom Lane
Corey Huinker writes: > [ file_fdw_program_v3.diff ] Pushed with cosmetic adjustments, mostly more work on the comments and documentation. I did not push the proposed test case; it's unportable. The upthread suggestion to add a TAP test would have been all right, because --enable-tap-tests requ

Re: [HACKERS] Let file_fdw access COPY FROM PROGRAM

2016-09-12 Thread Amit Langote
On 2016/09/13 2:01, Corey Huinker wrote: > Thanks for the review! > > I agree with all the code cleanups suggested and have made then in the > attached patch, to save the committer some time. Thanks. Have already marked the patch as ready for the committer. > Also in this patch, I changed sgml

Re: [HACKERS] Let file_fdw access COPY FROM PROGRAM

2016-09-12 Thread Corey Huinker
On Mon, Sep 12, 2016 at 1:59 AM, Amit Langote wrote: > On 2016/09/11 8:04, Corey Huinker wrote: > > V2 of this patch: > > > > Changes: > > * rebased to most recent master > > * removed non-tap test that assumed the existence of Unix sed program > > * added non-tap test that assumes the existence

Re: [HACKERS] Let file_fdw access COPY FROM PROGRAM

2016-09-12 Thread Corey Huinker
Thanks for the review! I agree with all the code cleanups suggested and have made then in the attached patch, to save the committer some time. Also in this patch, I changed sgml para to Changing table-level options requires superuser privileges, for security reasons: only a superuser should

Re: [HACKERS] Let file_fdw access COPY FROM PROGRAM

2016-09-11 Thread Amit Langote
On 2016/09/11 8:04, Corey Huinker wrote: > V2 of this patch: > > Changes: > * rebased to most recent master > * removed non-tap test that assumed the existence of Unix sed program > * added non-tap test that assumes the existence of perl > * switched from filename/program to filename/is_program to

Re: [HACKERS] Let file_fdw access COPY FROM PROGRAM

2016-09-10 Thread Corey Huinker
V2 of this patch: Changes: * rebased to most recent master * removed non-tap test that assumed the existence of Unix sed program * added non-tap test that assumes the existence of perl * switched from filename/program to filename/is_program to more closely follow patterns in copy.c * slight wordin

Re: [HACKERS] Let file_fdw access COPY FROM PROGRAM

2016-09-08 Thread Corey Huinker
On Thu, Sep 8, 2016 at 6:59 PM, Craig Ringer wrote: > On 9 Sep. 2016 03:45, "Corey Huinker" wrote: > > > > > > > Stylistically, would a separate .pl file for the emitter be preferable > to something inline like > > > >> perl -e 'print "a\tb\tcc\t4\n"; print "b\tc\tdd\t5\n"' > > I'd be fine with

Re: [HACKERS] Let file_fdw access COPY FROM PROGRAM

2016-09-08 Thread Craig Ringer
On 9 Sep. 2016 03:45, "Corey Huinker" wrote: > > > Stylistically, would a separate .pl file for the emitter be preferable to something inline like > >> perl -e 'print "a\tb\tcc\t4\n"; print "b\tc\tdd\t5\n"' I'd be fine with that and a suitable comment. Just be careful with different platforms' s

Re: [HACKERS] Let file_fdw access COPY FROM PROGRAM

2016-09-08 Thread Corey Huinker
On Tue, Sep 6, 2016 at 11:44 PM, Craig Ringer wrote: > On 7 September 2016 at 11:37, Corey Huinker > wrote: > > On Tue, Sep 6, 2016 at 11:24 PM, Craig Ringer < > craig.rin...@2ndquadrant.com> > > wrote: > >> > >> On 7 September 2016 at 11:21, Corey Huinker > >> wrote: > >> > On Tue, Sep 6, 2016

Re: [HACKERS] Let file_fdw access COPY FROM PROGRAM

2016-09-06 Thread Amit Langote
On 2016/09/07 12:29, Corey Huinker wrote: > On Tue, Sep 6, 2016 at 9:46 PM, Amit Langote wrote: >> OK. > Well...maybe not, depending on what Craig and other can do to educate me > about the TAP tests. Sure. >>> Changing table-level options requires superuser privileges, for security >>> reasons:

Re: [HACKERS] Let file_fdw access COPY FROM PROGRAM

2016-09-06 Thread Craig Ringer
On 7 September 2016 at 11:37, Corey Huinker wrote: > On Tue, Sep 6, 2016 at 11:24 PM, Craig Ringer > wrote: >> >> On 7 September 2016 at 11:21, Corey Huinker >> wrote: >> > On Tue, Sep 6, 2016 at 6:53 PM, Craig Ringer >> > >> >> > And the TAP test would detect the operating system and know to c

Re: [HACKERS] Let file_fdw access COPY FROM PROGRAM

2016-09-06 Thread Corey Huinker
On Tue, Sep 6, 2016 at 11:24 PM, Craig Ringer wrote: > On 7 September 2016 at 11:21, Corey Huinker > wrote: > > On Tue, Sep 6, 2016 at 6:53 PM, Craig Ringer < > craig.rin...@2ndquadrant.com> > > > And the TAP test would detect the operating system and know to create an > FDW > > that has the PRO

Re: [HACKERS] Let file_fdw access COPY FROM PROGRAM

2016-09-06 Thread Corey Huinker
On Tue, Sep 6, 2016 at 9:46 PM, Amit Langote wrote: > On 2016/09/07 3:12, Corey Huinker wrote: > > On Fri, Sep 2, 2016 at 5:07 AM, Amit Langote wrote: > >> I am not familiar with win32 stuff too, so I don't have much to say > about > >> that. Maybe someone else can chime in to help with that. >

Re: [HACKERS] Let file_fdw access COPY FROM PROGRAM

2016-09-06 Thread Craig Ringer
On 7 September 2016 at 11:21, Corey Huinker wrote: > On Tue, Sep 6, 2016 at 6:53 PM, Craig Ringer > And the TAP test would detect the operating system and know to create an FDW > that has the PROGRAM value 'cat test_data.csv' on Unix, 'type test_data.csv' > on windows, and 'type test_data.csv;1'

Re: [HACKERS] Let file_fdw access COPY FROM PROGRAM

2016-09-06 Thread Corey Huinker
On Tue, Sep 6, 2016 at 6:53 PM, Craig Ringer wrote: > On 7 Sep. 2016 02:14, "Corey Huinker" wrote: > > > > > Having regression tests for this is extremely problematic, because the > program invoked would need to be an invokable command on any architecture > supported by postgres. I'm pretty sure

Re: [HACKERS] Let file_fdw access COPY FROM PROGRAM

2016-09-06 Thread Amit Langote
On 2016/09/07 3:12, Corey Huinker wrote: > On Fri, Sep 2, 2016 at 5:07 AM, Amit Langote wrote: >> I am not familiar with win32 stuff too, so I don't have much to say about >> that. Maybe someone else can chime in to help with that. > > The regressions basically *can't* test this because we'd need

Re: [HACKERS] Let file_fdw access COPY FROM PROGRAM

2016-09-06 Thread Michael Paquier
On Wed, Sep 7, 2016 at 7:53 AM, Craig Ringer wrote: > On 7 Sep. 2016 02:14, "Corey Huinker" wrote: >> > >> Having regression tests for this is extremely problematic, because the >> program invoked would need to be an invokable command on any architecture >> supported by postgres. I'm pretty sure

Re: [HACKERS] Let file_fdw access COPY FROM PROGRAM

2016-09-06 Thread Craig Ringer
On 7 Sep. 2016 02:14, "Corey Huinker" wrote: > > Having regression tests for this is extremely problematic, because the program invoked would need to be an invokable command on any architecture supported by postgres. I'm pretty sure no such command exists. Your best bet will be using the TAP fra

Re: [HACKERS] Let file_fdw access COPY FROM PROGRAM

2016-09-06 Thread Corey Huinker
On Fri, Sep 2, 2016 at 5:07 AM, Amit Langote wrote: > > I am not familiar with win32 stuff too, so I don't have much to say about > that. Maybe someone else can chime in to help with that. > The regressions basically *can't* test this because we'd need a shell command we know works on any archi

Re: [HACKERS] Let file_fdw access COPY FROM PROGRAM

2016-09-02 Thread Amit Langote
Hi Corey, Here are some comments and a review of the patch. On 2016/06/03 5:48, Corey Huinker wrote: > A while back, there was a push to make COPY gzip-aware. That didn't happen, > but COPY FROM PROGRAM did, and it scratches the same itch. > > I have a similar need, but with file_fdw foreign tab

Re: [HACKERS] Let file_fdw access COPY FROM PROGRAM

2016-06-06 Thread Adam Gomaa
I'm fine with the code being released under the PostgreSQL license. Thanks, Adam -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Let file_fdw access COPY FROM PROGRAM

2016-06-06 Thread Robert Haas
On Thu, Jun 2, 2016 at 4:48 PM, Corey Huinker wrote: > A while back, there was a push to make COPY gzip-aware. That didn't happen, > but COPY FROM PROGRAM did, and it scratches the same itch. > > I have a similar need, but with file_fdw foreign tables. I have .csv.gz > files downloaded to the serv

Re: [HACKERS] Let file_fdw access COPY FROM PROGRAM

2016-06-03 Thread Corey Huinker
On Fri, Jun 3, 2016 at 1:06 AM, Craig Ringer wrote: > On 3 June 2016 at 04:48, Corey Huinker wrote: > >> A while back, there was a push to make COPY gzip-aware. That didn't >> happen, but COPY FROM PROGRAM did, and it scratches the same itch. >> > > >> - writing unwanted columns to a temp/work t

Re: [HACKERS] Let file_fdw access COPY FROM PROGRAM

2016-06-02 Thread Craig Ringer
On 3 June 2016 at 04:48, Corey Huinker wrote: > A while back, there was a push to make COPY gzip-aware. That didn't > happen, but COPY FROM PROGRAM did, and it scratches the same itch. > > - writing unwanted columns to a temp/work table via COPY, and then > immediately re-reading them > Withou