I wrote:
> > I'll reimplement the feature using the PROGRAM keyword:
> >
> > > COPY TABLE FROM PROGRAM 'command line';
> > > COPY TABLE TO PROGRAM 'command line';
>
> I've reimplemented the feature. Attached is an updated version of the patch.
I fixed bugs in the previous version of the patc
I wrote:
> > The biggest problem this patch has had from the very beginning is
> > overdesign, and this is more of the same. Let's please just define the
> > feature as "popen, not fopen, the given string" and have done. You can
> > put all the warning verbiage you want in the documentation. (Bu
Robert Haas writes:
> On Thu, Nov 15, 2012 at 2:35 PM, Tom Lane wrote:
>> The biggest problem this patch has had from the very beginning is
>> overdesign, and this is more of the same. Let's please just define the
>> feature as "popen, not fopen, the given string" and have done.
> ... If we giv
On Thu, Nov 15, 2012 at 2:35 PM, Tom Lane wrote:
> Robert Haas writes:
>> Yeah. If we're going to do this at all, and I'm not convinced it's
>> worth the work, I think it's definitely good to support a variant
>> where we specify exactly the things that will be passed to exec().
>> There's just
> From: Tom Lane [mailto:t...@sss.pgh.pa.us]
> Robert Haas writes:
> > Yeah. If we're going to do this at all, and I'm not convinced it's
> > worth the work, I think it's definitely good to support a variant
> > where we specify exactly the things that will be passed to exec().
> > There's just
Craig Ringer writes:
> I'd still be much happier to have COPY ... FROM PROGRAM - or something -
> to clearly make the two different, for clarity as much as security.
I don't object to using a PROGRAM keyword rather than something inside
the string to select this behavior.
On 11/16/2012 03:35 AM, Tom Lane wrote:
> The biggest problem this patch has had from the very beginning is
> overdesign, and this is more of the same. Let's please just define the
> feature as "popen, not fopen, the given string" and have done. You can
> put all the warning verbiage you want in t
Robert Haas writes:
> Yeah. If we're going to do this at all, and I'm not convinced it's
> worth the work, I think it's definitely good to support a variant
> where we specify exactly the things that will be passed to exec().
> There's just too many ways to accidentally shoot yourself in the foot
On Wed, Nov 14, 2012 at 10:14 PM, Craig Ringer wrote:
>> So? You're already handing the keys to the kingdom to anybody who can
>> control the contents of that command line, even if it's only to point at
>> the wrong program. And one man's "unexpected side-effect" is another
>> man's "essential f
On 11/15/2012 10:19 AM, Tom Lane wrote:
>
> I disagree very very strongly with that. If we prevent use of shell
> syntax, we will lose a lot of functionality, for instance
>
> copy ... from program 'foocopy ... from program 'foo | bar'
>
> unless you're imagining that we will reimple
Craig Ringer writes:
> On 11/15/2012 12:46 AM, Tom Lane wrote:
>> Agreed, and there's also the question of passing switches etc to the
>> program, so the string can't be a bare file name anyway. I proposed
>> pipe symbols (|) in the string previously, but if you find that too
>> Unix-centric I su
On 11/15/2012 12:46 AM, Tom Lane wrote:
>
> Agreed, and there's also the question of passing switches etc to the
> program, so the string can't be a bare file name anyway. I proposed
> pipe symbols (|) in the string previously, but if you find that too
> Unix-centric I suppose we could do
>
>
On 11/14/2012 02:37 PM, Tom Lane wrote:
What I'm imagining is a very very simple addition to COPY that just
allows it to execute popen() instead of fopen() to read or write the
data source/sink. What you suggest would require hundreds of lines and
create many opportunities for new bugs.
Andrew Dunstan writes:
> On 11/14/2012 02:05 PM, Peter Eisentraut wrote:
>> Why don't you filter the data before it gets to stdin? Some program is
>> feeding the data to "stdin" on the client side. Why doesn't that do the
>> filtering? I don't see a large advantage in having the data be sent
>>
On 11/14/2012 02:05 PM, Peter Eisentraut wrote:
On 11/14/12 11:50 AM, Andrew Dunstan wrote:
COPY table FROM 'some command line |';
COPY table TO '| some command line';
I'd like to be able to filter STDIN if possible - with this syntax how
is COPY going to know to hook up STDIN to t
On 11/14/12 11:50 AM, Andrew Dunstan wrote:
>> COPY table FROM 'some command line |';
>> COPY table TO '| some command line';
>>
>
>
> I'd like to be able to filter STDIN if possible - with this syntax how
> is COPY going to know to hook up STDIN to the program?
Why don't you filter the
On 11/14/2012 11:56 AM, Tom Lane wrote:
Andrew Dunstan writes:
On 11/14/2012 11:39 AM, Tom Lane wrote:
What happened to the previous proposal of treating the COPY
target as a pipe specification, ie
I'd like to be able to filter STDIN if possible - with this syntax how
is COPY going to know t
Andrew Dunstan writes:
> On 11/14/2012 11:39 AM, Tom Lane wrote:
>> What happened to the previous proposal of treating the COPY
>> target as a pipe specification, ie
> I'd like to be able to filter STDIN if possible - with this syntax how
> is COPY going to know to hook up STDIN to the program?
On 11/14/2012 11:39 AM, Tom Lane wrote:
Andrew Dunstan writes:
Yeah, I agree, although the syntax looks a bit unclean.
Oh, I had not looked at the syntax closely. I agree, that basically
sucks: it's overcomplicated and under-featured, because you can't
control the actual program command line
Simon Riggs writes:
> On 14 November 2012 15:09, Fujii Masao wrote:
>> I'd just like to execute
>> COPY vmstat_table FROM 'vmstat' WITH ...
> If we go ahead with this, I think it needs additional keyword to
> indicate that we will execute the file rather than read from it. I
> don't think we sho
Andrew Dunstan writes:
> Yeah, I agree, although the syntax looks a bit unclean.
Oh, I had not looked at the syntax closely. I agree, that basically
sucks: it's overcomplicated and under-featured, because you can't
control the actual program command line very conveniently. Nor do I see
a reason
On 14 November 2012 15:09, Fujii Masao wrote:
> What about further extending the COPY IN syntax to the following?
>
> COPY table_name FROM 'progname [ option, ... ]' WITH ...
>
> I'd just like to execute
>
> COPY vmstat_table FROM 'vmstat' WITH ...
If we go ahead with this, I think it ne
On 14 November 2012 16:20, Tom Lane wrote:
> Alvaro Herrera writes:
>> Simon Riggs escribió:
>>> On 14 November 2012 15:09, Fujii Masao wrote:
Here, progname for COPY IN is the user-supplied program that takes
filename as
its argument and that writes on standard output.
>
>>> I t
On 11/14/2012 11:20 AM, Tom Lane wrote:
I disagree with Simon's objection also, because neither reading from
nor writing to an external program is likely to fit the model of
reading/updating a table very well. For instance, there's no good
reason to suppose that reading twice will give the same
Alvaro Herrera writes:
> Simon Riggs escribió:
>> On 14 November 2012 15:09, Fujii Masao wrote:
>>> Here, progname for COPY IN is the user-supplied program that takes filename
>>> as
>>> its argument and that writes on standard output.
>> I think we should be using FDWs/SRFs here, not inventing
Simon Riggs escribió:
> On 14 November 2012 15:09, Fujii Masao wrote:
>
> >> Here, progname for COPY IN is the user-supplied program that takes
> >> filename as
> >> its argument and that writes on standard output.
> >
> > What about further extending the COPY IN syntax to the following?
> >
> >
On Thu, Nov 15, 2012 at 12:31 AM, Simon Riggs wrote:
> On 14 November 2012 15:09, Fujii Masao wrote:
>
>>> Here, progname for COPY IN is the user-supplied program that takes filename
>>> as
>>> its argument and that writes on standard output.
>>
>> What about further extending the COPY IN syntax
On 14 November 2012 15:09, Fujii Masao wrote:
>> Here, progname for COPY IN is the user-supplied program that takes filename
>> as
>> its argument and that writes on standard output.
>
> What about further extending the COPY IN syntax to the following?
>
> COPY table_name FROM 'progname [ op
On Wed, Nov 14, 2012 at 8:30 PM, Etsuro Fujita
wrote:
> I wrote:
>> > From: Tom Lane [mailto:t...@sss.pgh.pa.us]
>>
>> > I wrote:
>> > > "Etsuro Fujita" writes:
>> > >> I have a question. I think it would be also better to extend the syntax
>> > >> for the SQL COPY command in the same way, ie,
>
On 13 September 2012 10:13, Etsuro Fujita wrote:
> I'd like to add the following options to the SQL COPY command and the psql
> \copy
> instruction:
>
> * PREPROCESSOR: Specifies the user-supplied program for COPY IN. The data
> from an input file is preprocessed by the program before the da
I wrote:
> > From: Tom Lane [mailto:t...@sss.pgh.pa.us]
>
> > I wrote:
> > > "Etsuro Fujita" writes:
> > >> I have a question. I think it would be also better to extend the syntax
> > >> for the SQL COPY command in the same way, ie,
> > >> COPY foo from '/home/pgsql/decompress.sh /home/pgsql/foo
> From: Tom Lane [mailto:t...@sss.pgh.pa.us]
> I wrote:
> > "Etsuro Fujita" writes:
> >> I have a question. I think it would be also better to extend the syntax
> >> for the SQL COPY command in the same way, ie,
> >> COPY foo from '/home/pgsql/decompress.sh /home/pgsql/foo.csv.gz |' with
> >> fo
I wrote:
> "Etsuro Fujita" writes:
>> I have a question. I think it would be also better to extend the syntax for
>> the
>> SQL COPY command in the same way, ie,
>> COPY foo from '/home/pgsql/decompress.sh /home/pgsql/foo.csv.gz |' with
>> format
>> 'csv'
> Yeah, sure --- that case is already
"Etsuro Fujita" writes:
>> From: Tom Lane [mailto:t...@sss.pgh.pa.us]
>> I think it would be better to present this as something like
>> \copy foo from '/home/pgsql/decompress.sh /home/pgsql/foo.csv.gz |' with
>> format 'csv'
> I have a question. I think it would be also better to extend the syn
> From: Tom Lane [mailto:t...@sss.pgh.pa.us]
> "Etsuro Fujita" writes:
> > Maybe my explanation was insufficient. Let me add one thing to my earlier
> > explanation. The submitted patch allows the psql \copy instruction to be
> > executed like:
>
> > $ echo '/bin/gunzip -c $1' > decompress.sh
"Etsuro Fujita" writes:
> Maybe my explanation was insufficient. Let me add one thing to my earlier
> explanation. The submitted patch allows the psql \copy instruction to be
> executed like:
> $ echo '/bin/gunzip -c $1' > decompress.sh
> $ chmod +x decompress.sh
> postgres=# \copy foo FROM '/
> From: Craig Ringer [mailto:ring...@ringerc.id.au]
> On 09/13/2012 10:25 PM, Tom Lane wrote:
> > I think it would be a lot better if this were designed so that the
> > processor programs executed on client side. Which would probably make
> > it not a COPY patch at all, but something in psql.
Ma
On 09/13/2012 10:25 PM, Tom Lane wrote:
I think it would be a lot better if this were designed so that the
processor programs executed on client side. Which would probably make
it not a COPY patch at all, but something in psql.
Either that, or allow the pre- and post- processors to be programs
; pgsql-hackers@postgresql.org
> Subject: Re: [HACKERS] WIP patch: add (PRE|POST)PROCESSOR options to COPY
>
>
> On 09/13/2012 01:20 PM, Dimitri Fontaine wrote:
> > Tom Lane writes:
> >> I think it would be a lot better if this were designed so that the
> >> proce
On 09/13/2012 01:20 PM, Dimitri Fontaine wrote:
Tom Lane writes:
I think it would be a lot better if this were designed so that the
processor programs executed on client side. Which would probably make
it not a COPY patch at all, but something in psql.
And pgloader, which already has a part
Tom Lane writes:
> I think it would be a lot better if this were designed so that the
> processor programs executed on client side. Which would probably make
> it not a COPY patch at all, but something in psql.
And pgloader, which already has a part of that feature with the per
column reformatin
"Etsuro Fujita" writes:
> I'd like to add the following options to the SQL COPY command and the psql
> \copy
> instruction:
> * PREPROCESSOR: Specifies the user-supplied program for COPY IN. The data
> from an input file is preprocessed by the program before the data is loaded
> into
> a p
2012/9/13 Alvaro Herrera :
> Excerpts from Etsuro Fujita's message of jue sep 13 06:13:26 -0300 2012:
>> I'd like to add the following options to the SQL COPY command and the psql
>> \copy
>> instruction:
>>
>> * PREPROCESSOR: Specifies the user-supplied program for COPY IN. The
>> data
>> f
Excerpts from Etsuro Fujita's message of jue sep 13 06:13:26 -0300 2012:
> I'd like to add the following options to the SQL COPY command and the psql
> \copy
> instruction:
>
> * PREPROCESSOR: Specifies the user-supplied program for COPY IN. The data
> from an input file is preprocessed by t
I'd like to add the following options to the SQL COPY command and the psql \copy
instruction:
* PREPROCESSOR: Specifies the user-supplied program for COPY IN. The data
from an input file is preprocessed by the program before the data is loaded into
a postgres table.
* POSTPROCESSOR: Speci
45 matches
Mail list logo
