Re: [HACKERS] CSV mode option for pg_dump

2006-06-14 Thread Andrew Dunstan
I wrote: The main use case seems to me to be where you are exporting a whole database or most of it with a very large number of tables, and it is convenient to have all the CSVs created for you rather than have to make them manually one at a time. You could get these out of, say, a tar for

Re: [HACKERS] CSV mode option for pg_dump

2006-06-14 Thread Andrew Dunstan
Greg Stark wrote: And before you ask, yes, virtually every version of Excel has changed its csv file format. I'd be interested to hear of Excel variants we haven't catered for - our CSV parsing is some of the most flexible and complete around, IMNSHO. BTW, I suspect we won't see too

Re: [HACKERS] CSV mode option for pg_dump

2006-06-14 Thread Bruce Momjian
Andrew Dunstan wrote: > > > Bruce Momjian wrote: > > > > >As was stated before, the use-case for this is by people we don't > >normally have contact with. > > > > > > > > I do think we need a use case for what we do. > > The main use case seems to me to be where you are exporting a whole >

Re: [HACKERS] CSV mode option for pg_dump

2006-06-14 Thread Andrew Dunstan
Bruce Momjian wrote: As was stated before, the use-case for this is by people we don't normally have contact with. I do think we need a use case for what we do. The main use case seems to me to be where you are exporting a whole database or most of it with a very large number of table

Re: [HACKERS] CSV mode option for pg_dump

2006-06-14 Thread Greg Stark
"Andrew Dunstan" <[EMAIL PROTECTED]> writes: > The CSV definition and its lack of formality is a complete red herring in > this, as are references to Microsoft. I think then you've missed the real issue. > The real issue in what you say here is your suggestion that we might not be > able to rep

Re: [HACKERS] CSV mode option for pg_dump

2006-06-14 Thread Martijn van Oosterhout
On Wed, Jun 14, 2006 at 10:55:04AM -0400, Bruce Momjian wrote: > > Once again: give me a plausible use-case for pulling this data out of a > > pg_dump output file. You'd need data-massaging code anyway just to > > extract the data, so why not expect that code to convert to CSV or > > whatever othe

Re: [HACKERS] CSV mode option for pg_dump

2006-06-14 Thread Martijn van Oosterhout
On Tue, Jun 13, 2006 at 09:14:48AM -0700, Joshua D. Drake wrote: > >I don't get it. If you can use psql then you already have csv support. > > > >psql -c 'COPY pg_class TO STDOUT WITH CSV' postgres > pg_class.csv > > If you data looks like this: > > foo barbaz bing > > You are o.k. You have t

Re: [HACKERS] CSV mode option for pg_dump

2006-06-14 Thread Bruce Momjian
Tom Lane wrote: > Bruce Momjian writes: > > As was stated before, the use-case for this is by people we don't > > normally have contact with. > > Or who don't exist. > > Once again: give me a plausible use-case for pulling this data out of a > pg_dump output file. You'd need data-massaging code

Re: [HACKERS] CSV mode option for pg_dump

2006-06-14 Thread Tom Lane
Bruce Momjian writes: > As was stated before, the use-case for this is by people we don't > normally have contact with. Or who don't exist. Once again: give me a plausible use-case for pulling this data out of a pg_dump output file. You'd need data-massaging code anyway just to extract the data

Re: [HACKERS] CSV mode option for pg_dump

2006-06-14 Thread Bruce Momjian
Tom Lane wrote: > Bruce Momjian writes: > > I agree with Andrew. If there are bugs in CSV, then we should fix them, > > not avoid give users this usability improvement. > > The case for it being a usability improvement seems very weak to me; > no one has yet demonstrated an actual use-case where

Re: [HACKERS] CSV mode option for pg_dump

2006-06-14 Thread Tom Lane
Bruce Momjian writes: > I agree with Andrew. If there are bugs in CSV, then we should fix them, > not avoid give users this usability improvement. The case for it being a usability improvement seems very weak to me; no one has yet demonstrated an actual use-case where someone would pull CSV data

Re: [HACKERS] CSV mode option for pg_dump

2006-06-14 Thread Bruce Momjian
I agree with Andrew. If there are bugs in CSV, then we should fix them, not avoid give users this usability improvement. I know I have analyzed the code and fixed a few problems with it in the past. As far as pg_dump, I am thinking the most flexible solution would be to allow an arbitrary WITH

Re: [HACKERS] CSV mode option for pg_dump

2006-06-14 Thread Andrew Dunstan
Tom Lane said: > Andrew Dunstan <[EMAIL PROTECTED]> writes: >> This is just nonsense. There is not the slightest reason that CSV data >> cannot be embedded in a text dump nor exist as the data members of a >> custom or tar dump with the corresponding COPY statements modified >> accordingly. > > We

Re: [HACKERS] CSV mode option for pg_dump

2006-06-13 Thread Tom Lane
Andrew Dunstan <[EMAIL PROTECTED]> writes: > This is just nonsense. There is not the slightest reason that CSV data > cannot be embedded in a text dump nor exist as the data members of a > custom or tar dump with the corresponding COPY statements modified > accordingly. Well, the really *core*

Re: [HACKERS] CSV mode option for pg_dump

2006-06-13 Thread Andrew Dunstan
Bruce Momjian wrote: pg_dump CSV TODO item removed until we come up with something everyone can agree on. That's a pity. Just to show you how little is involved in what I was suggesting, a prototype patch is attached - it's 182 lines of context diff, which is pretty small for a new fe

Re: [HACKERS] CSV mode option for pg_dump

2006-06-13 Thread Bruce Momjian
pg_dump CSV TODO item removed until we come up with something everyone can agree on. --- Joshua D. Drake wrote: > > > > Would that be adequate, or do we really want to reimplement and maintain > > all > > the output format

Re: [HACKERS] CSV mode option for pg_dump

2006-06-13 Thread Andrew Dunstan
Steve Atkins wrote: Would that be adequate, or do we really want to reimplement and maintain all the output format complexity in our own code, in C? Code to produce CSVs is there, now, today, and has been since 8.0. That is *not* what is at issue here. If you want to debate whether o

Re: [HACKERS] CSV mode option for pg_dump

2006-06-13 Thread Steve Atkins
On Jun 13, 2006, at 9:47 AM, Joshua D. Drake wrote: Would that be adequate, or do we really want to reimplement and maintain all the output format complexity in our own code, in C? I think the point is that we should provide a native implementation because not everyone is crazy enough to

Re: [HACKERS] CSV mode option for pg_dump

2006-06-13 Thread Bill Bartlett
of which have also been discussed for pg_dump recently. - Bill > > > > -Original Message- > > > From: Joshua D. Drake [mailto:[EMAIL PROTECTED] > > > Sent: Monday, June 12, 2006 4:15 PM > > > To: Bill Bartlett > > > Cc: 'Andrew Dunst

Re: [HACKERS] CSV mode option for pg_dump

2006-06-13 Thread Joshua D. Drake
Would that be adequate, or do we really want to reimplement and maintain all the output format complexity in our own code, in C? I think the point is that we should provide a native implementation because not everyone is crazy enough to use perl (blatant jab ;)). I would never expect a cust

Re: [HACKERS] CSV mode option for pg_dump

2006-06-13 Thread Andrew Dunstan
PFC wrote: From what I gather, the CSV format dump would only contain data. I think pg_dump is the friend of pg_restore. It dumps everything including user defined functions, types, schemas etc. CSV does not fit with this. This is just nonsense. There is not the slightest

Re: [HACKERS] CSV mode option for pg_dump

2006-06-13 Thread PFC
From what I gather, the CSV format dump would only contain data. I think pg_dump is the friend of pg_restore. It dumps everything including user defined functions, types, schemas etc. CSV does not fit with this. Besides, people will probably want to dump into CSV the result of any

Re: [HACKERS] CSV mode option for pg_dump

2006-06-13 Thread Joshua D. Drake
baz" "bing" Sincerely, Joshua D. Drake -Original Message- From: Joshua D. Drake [mailto:[EMAIL PROTECTED] Sent: Monday, June 12, 2006 4:15 PM To: Bill Bartlett Cc: 'Andrew Dunstan'; 'Tom Lane'; 'PG Hackers' Subject: Re: [HACKERS] CSV mode opt

Re: [HACKERS] CSV mode option for pg_dump

2006-06-13 Thread Steve Atkins
On Jun 13, 2006, at 7:34 AM, Martijn van Oosterhout wrote: What this tells me is that we need a tool somewhere between psql and pg_dump, say, pgquery. It's sole purpose in life is to generate output from various queries. Because it's a seperate tool there's no question of psql or pg_dump being

Re: [HACKERS] CSV mode option for pg_dump

2006-06-13 Thread Rod Taylor
--- > > From: Joshua D. Drake [mailto:[EMAIL PROTECTED] > > Sent: Monday, June 12, 2006 4:15 PM > > To: Bill Bartlett > > Cc: 'Andrew Dunstan'; 'Tom Lane'; 'PG Hackers' > > Subject: Re: [HACKERS] CSV mode option for pg_dump > > >

Re: [HACKERS] CSV mode option for pg_dump

2006-06-13 Thread Andrew Dunstan
Martijn van Oosterhout wrote: Side note: In my experience Excel happily slurps up tab delimited output, so I'm not sure why all of this is an issue in the first place. I guess you experience doesn't run to data that has embedded tabs, for example. There really is a reason we did this

Re: [HACKERS] CSV mode option for pg_dump

2006-06-13 Thread Andrew Dunstan
Volkan YAZICI wrote: On Jun 13 10:20, Bruce Momjian wrote: Good point. The number of CSV options would be hard to support for pg_dump. Any thoughts from anyone on how to do that cleanly? Could we just support the default behavior? IMHO, it might be better if we'd support a syntax

Re: [HACKERS] CSV mode option for pg_dump

2006-06-13 Thread Joshua D. Drake
Bruce Momjian wrote: Good point. The number of CSV options would be hard to support for pg_dump. Any thoughts from anyone on how to do that cleanly? Could we just support the default behavior? Although I don't see a real need for the feature, I do think that if we were to support "1" (well

Re: [HACKERS] CSV mode option for pg_dump

2006-06-13 Thread Andrew Dunstan
Tom Lane wrote: there are half a dozen more sub-options, and it seems like every time we turn around someone is finding a reason for another one. This is a bit unfair. The feature was introduced in 8.0, and slightly enhanced in 8.1. There have not been any additional CSV features this

Re: [HACKERS] CSV mode option for pg_dump

2006-06-13 Thread Volkan YAZICI
On Jun 13 10:20, Bruce Momjian wrote: > > Good point. The number of CSV options would be hard to support for > pg_dump. Any thoughts from anyone on how to do that cleanly? Could we > just support the default behavior? IMHO, it might be better if we'd support a syntax like pg_dump --csv=opt0

Re: [HACKERS] CSV mode option for pg_dump

2006-06-13 Thread Martijn van Oosterhout
On Tue, Jun 13, 2006 at 10:20:53AM -0400, Bruce Momjian wrote: > > Good point. The number of CSV options would be hard to support for > pg_dump. Any thoughts from anyone on how to do that cleanly? Could we > just support the default behavior? What this tells me is that we need a tool somewhere

Re: [HACKERS] CSV mode option for pg_dump

2006-06-13 Thread Bruce Momjian
Good point. The number of CSV options would be hard to support for pg_dump. Any thoughts from anyone on how to do that cleanly? Could we just support the default behavior? --- Tom Lane wrote: > "Andrew Dunstan" <[EMAIL PR

Re: [HACKERS] CSV mode option for pg_dump

2006-06-13 Thread Tom Lane
"Andrew Dunstan" <[EMAIL PROTECTED]> writes: > I wish I could understand why people are so keen to make other people turn > handsprings in order to avoid a feature which, as Bruce points out, is > already on the TODO list, and which, by my 10 minute analysis, would involve > almost trivial code imp

Re: [HACKERS] CSV mode option for pg_dump

2006-06-13 Thread Bruce Momjian
Andrew Dunstan wrote: > Matthew T. OConnor said: > > Bill Bartlett wrote: > >> Can't -- the main production database is over at a CoLo site with > >> access only available via SSH, and tightly-restricted SSH at that. > >> Generally one of the developers will SSH over to the server, pull out > >> wh

Re: [HACKERS] CSV mode option for pg_dump

2006-06-13 Thread Andrew Dunstan
Matthew T. OConnor said: > Bill Bartlett wrote: >> Can't -- the main production database is over at a CoLo site with >> access only available via SSH, and tightly-restricted SSH at that. >> Generally one of the developers will SSH over to the server, pull out >> whatever data is needed into a text

Re: [HACKERS] CSV mode option for pg_dump

2006-06-13 Thread Matthew T. O'Connor
Bill Bartlett wrote: Can't -- the main production database is over at a CoLo site with access only available via SSH, and tightly-restricted SSH at that. Generally one of the developers will SSH over to the server, pull out whatever data is needed into a text file via psql or pg_dump, scp the fil

Re: [HACKERS] CSV mode option for pg_dump

2006-06-12 Thread Bruce Momjian
Already on TODO: pg_dump: o %Add CSV output format --- Tom Lane wrote: > Andrew Dunstan <[EMAIL PROTECTED]> writes: > > Something someone said on IRC just now triggered a little memory ... I > > think we shoul

Re: [HACKERS] CSV mode option for pg_dump

2006-06-12 Thread Bill Bartlett
the ability to export to CSV would be handy. - Bill > -Original Message- > From: Joshua D. Drake [mailto:[EMAIL PROTECTED] > Sent: Monday, June 12, 2006 4:15 PM > To: Bill Bartlett > Cc: 'Andrew Dunstan'; 'Tom Lane'; 'PG Hackers' > Subje

Re: [HACKERS] CSV mode option for pg_dump

2006-06-12 Thread Joshua D. Drake
Bill Bartlett wrote: Here's me speaking up -- I'd definitely use it! As a quick way to pull data into Excel to do basic reports or analysis, a CSV format would be great. Why not just use ODBC? Joshua D. Drake -- === The PostgreSQL Company: Command Prompt, Inc. === Sales/S

Re: [HACKERS] CSV mode option for pg_dump

2006-06-12 Thread Bill Bartlett
Monday, June 12, 2006 2:16 PM > To: Tom Lane > Cc: PG Hackers > Subject: Re: [HACKERS] CSV mode option for pg_dump > > > Tom Lane wrote: > > Andrew Dunstan <[EMAIL PROTECTED]> writes: > > > >> Something someone said on IRC just now triggered a l

Re: [HACKERS] CSV mode option for pg_dump

2006-06-12 Thread Jim C. Nasby
On Mon, Jun 12, 2006 at 02:15:59PM -0400, Andrew Dunstan wrote: > Naturally you won't have a use for it, but I suspect others might (in > which case they had better speak up ;-) ) I'd bet that those who would find this useful are far more likely to be on -general and not in here. -- Jim C. Nasby

Re: [HACKERS] CSV mode option for pg_dump

2006-06-12 Thread Andrew Dunstan
Tom Lane wrote: Andrew Dunstan <[EMAIL PROTECTED]> writes: Something someone said on IRC just now triggered a little memory ... I think we should provide an option to have pg_dump work in CSV mode rather than text mode. This probably doesn't have much importance in the case of text dumps,

Re: [HACKERS] CSV mode option for pg_dump

2006-06-12 Thread Joshua D. Drake
No it won't, not if there are tabs in the data. Hmmm then would just double quoting the data work? At least in OOCalc (and IIRC Excel) there is the ability to select a text delimiter. Joshua D. Drake -- === The PostgreSQL Company: Command Prompt, Inc. === Sales/Suppor

Re: [HACKERS] CSV mode option for pg_dump

2006-06-12 Thread Tom Lane
Andrew Dunstan <[EMAIL PROTECTED]> writes: > Something someone said on IRC just now triggered a little memory ... I > think we should provide an option to have pg_dump work in CSV mode > rather than text mode. This probably doesn't have much importance in the > case of text dumps, but in custom

Re: [HACKERS] CSV mode option for pg_dump

2006-06-12 Thread Josh Berkus
Josh, > > This should be a pretty low cost item, I expect (good newbie project?) > > Uhh... just about any application that can import CSV can import our > dumps. It just tell it the delimiter is a tab. Actually, there was an Summer of Code applcation to do this, but with all the other nifty stu

Re: [HACKERS] CSV mode option for pg_dump

2006-06-12 Thread Andrew Dunstan
Joshua D. Drake wrote: Andrew Dunstan wrote: Something someone said on IRC just now triggered a little memory ... I think we should provide an option to have pg_dump work in CSV mode rather than text mode. This probably doesn't have much importance in the case of text dumps, but in custom o

Re: [HACKERS] CSV mode option for pg_dump

2006-06-12 Thread Joshua D. Drake
Andrew Dunstan wrote: Something someone said on IRC just now triggered a little memory ... I think we should provide an option to have pg_dump work in CSV mode rather than text mode. This probably doesn't have much importance in the case of text dumps, but in custom or tar dumps where you mi

[HACKERS] CSV mode option for pg_dump

2006-06-12 Thread Andrew Dunstan
Something someone said on IRC just now triggered a little memory ... I think we should provide an option to have pg_dump work in CSV mode rather than text mode. This probably doesn't have much importance in the case of text dumps, but in custom or tar dumps where you might want to get at ind