Re: [HACKERS] PQputCopyData dont signal error

2014-04-07 Thread Martijn van Oosterhout
On Tue, Apr 01, 2014 at 01:53:13PM -0400, Robert Haas wrote: One of the things you mentioned is I often find it necessary to refer to existing examples of code when trying to figure out how to do things correctly. I couldn't agree more. Haven't seen one yet, but found plenty of

Re: [HACKERS] PQputCopyData dont signal error

2014-04-07 Thread Tom Lane
Martijn van Oosterhout klep...@svana.org writes: To move the conversation along: https://github.com/postgres/postgres/blob/master/src/bin/psql/copy.c#L664 Seems possibly even more robust than most people will code, but it's had a lot of real world testing. Note that the looping behavior

Re: [HACKERS] PQputCopyData dont signal error

2014-04-01 Thread Robert Haas
On Mon, Mar 31, 2014 at 4:21 PM, steve k steven.c.koh...@nasa.gov wrote: I'd love to see an actual working example where an executing C++ program was able to in fact determine that copy data containing bad data that was sent by the C++ program was rejected by the server and subsequently the C++

Re: [HACKERS] PQputCopyData dont signal error

2014-04-01 Thread steve k
Thanks Robert, I'm already there. Obviously I'm the only one in the room that didn't get the memo. I've had some time to reflect on what might be done differently, just not any time to try it. If I get it to work I'll let everyone know. The code I was working with went away when the Network

Re: [HACKERS] PQputCopyData dont signal error

2014-04-01 Thread Robert Haas
On Tue, Apr 1, 2014 at 12:14 PM, steve k steven.c.koh...@nasa.gov wrote: I'm already there. Obviously I'm the only one in the room that didn't get the memo. I've had some time to reflect on what might be done differently, just not any time to try it. If I get it to work I'll let everyone

Re: [HACKERS] PQputCopyData dont signal error

2014-03-31 Thread steve k
http://postgresql.1045698.n5.nabble.com/file/n5798002/PG_man_excerpt.png These were my results: http://postgresql.1045698.n5.nabble.com/file/n5798002/PG_embedded_copy_log_excerpt.png I'd advise anyone contemplating using this feature to seriously seriously seriously test this and examine

Re: [HACKERS] PQputCopyData dont signal error

2014-03-31 Thread Andrew Dunstan
On 03/31/2014 10:18 AM, steve k wrote: http://postgresql.1045698.n5.nabble.com/file/n5798002/PG_man_excerpt.png These were my results: http://postgresql.1045698.n5.nabble.com/file/n5798002/PG_embedded_copy_log_excerpt.png I'd advise anyone contemplating using this feature to seriously

Re: [HACKERS] PQputCopyData dont signal error

2014-03-31 Thread steve k
Am I to understand then that I should expect no error feedback if copy fails because of something like attempting to insert alphabetic into a numeric? I apologize for my ignorance, but all my return codes were always successful (PGRES_COMMAND_OK) even if nothing was copied due to garbage data.

Re: [HACKERS] PQputCopyData dont signal error

2014-03-31 Thread Alvaro Herrera
steve k wrote: I am examining other ways to do mass inserts/writes that allow for notification if some of the data contained within for some reason fails to copy/insert so that the cause of the bad data can be examined and remedied as soon as it occurs as well as writing the offending data to

Re: [HACKERS] PQputCopyData dont signal error

2014-03-31 Thread David Johnston
steve k wrote Am I to understand then that I should expect no error feedback if copy fails because of something like attempting to insert alphabetic into a numeric? I apologize for my ignorance, but all my return codes were always successful (PGRES_COMMAND_OK) even if nothing was copied

Re: [HACKERS] PQputCopyData dont signal error

2014-03-31 Thread steve k
Hi Alvaro, Thanks for the prompt response. PGLoader looks like an awesome project and I especially liked this part: /Handling PostgreSQL errors Some data will get rejected by PostgreSQL, even after being carefully prepared by the transformation functions you can attach to pgloader. Then

Re: [HACKERS] PQputCopyData dont signal error

2014-03-31 Thread steve k
I started with this: DBInsert_excerpts6_test_cpdlc.cpp http://postgresql.1045698.n5.nabble.com/file/n5798049/DBInsert_excerpts6_test_cpdlc.cpp Due to a cut and paste error I was originally querying the wrong sequence which led to me wondering why data wasn't in the table that the log

Re: [HACKERS] PQputCopyData dont signal error

2014-03-31 Thread David Johnston
steve k wrote I started with this: DBInsert_excerpts6_test_cpdlc.cpp http://postgresql.1045698.n5.nabble.com/file/n5798049/DBInsert_excerpts6_test_cpdlc.cpp Can you point out to me where in that code you've followed this instruction from the documentation: After successfully calling

Re: [HACKERS] PQputCopyData dont signal error

2014-03-31 Thread steve k
David, The code I posted is what I started/finished with. As I tried to figure things out I kept adding PQgetResult, PQresultStatus and PQresultErrorMessage calls and writing the results to the log file. By the time I was done it was so messy that I stripped all that messaging/logging back

Re: [HACKERS] PQputCopyData dont signal error

2014-03-31 Thread David Johnston
steve k wrote Sorry I can't provide more information but I do appreciate your time. If you can't get any further with it I understand and don't expect another reply. For the benefit of others I'm reading this as basically you've found a better way to do this so you are no longer concerned

Re: [HACKERS] PQputCopyData dont signal error

2014-03-31 Thread Michael Paquier
On Tue, Apr 1, 2014 at 1:33 AM, Alvaro Herrera alvhe...@2ndquadrant.com wrote: steve k wrote: I am examining other ways to do mass inserts/writes that allow for notification if some of the data contained within for some reason fails to copy/insert so that the cause of the bad data can be

Re: [HACKERS] PQputCopyData dont signal error

2014-03-29 Thread steve k
I realize this is an old thread, but seems to be the only discussion I can find on this topic I have a problem with PQputCopyData function. It doesn't signal some error. I am using from within a c++ program: PQexec(m_pConn, COPY... ...FROM stdin), followed by

Re: [HACKERS] PQputCopyData dont signal error

2014-03-29 Thread Tom Lane
steve k steven.c.koh...@nasa.gov writes: I realize this is an old thread, but seems to be the only discussion I can find on this topic I have a problem with PQputCopyData function. It doesn't signal some error. PQputCopyData/PQputCopyEnd are only concerned with transferring data. After

Re: [HACKERS] PQputCopyData dont signal error

2014-03-29 Thread David Johnston
steve k wrote I realize this is an old thread, but seems to be the only discussion I can find on this topic I have a problem with PQputCopyData function. It doesn't signal some error. I am using from within a c++ program: PQexec(m_pConn, COPY... ...FROM stdin), followed

Re: [HACKERS] PQputCopyData dont signal error

2014-02-04 Thread Jeff Davis
On Thu, 2011-04-14 at 10:50 +0300, Heikki Linnakangas wrote: On 14.04.2011 10:15, Pavel Stehule wrote: Hello I have a problem with PQputCopyData function. It doesn't signal some error. while ((row = mysql_fetch_row(res)) != NULL) {

Re: [HACKERS] PQputCopyData dont signal error

2014-02-04 Thread Tom Lane
Jeff Davis pg...@j-davis.com writes: On Thu, 2011-04-14 at 10:50 +0300, Heikki Linnakangas wrote: I think you'll need to send all the data and finish the COPY until you get an error. If you have a lot of data to send, you might want to slice it into multiple COPY statements of say 50MB each,

[HACKERS] PQputCopyData dont signal error

2011-04-14 Thread Pavel Stehule
Hello I have a problem with PQputCopyData function. It doesn't signal some error. while ((row = mysql_fetch_row(res)) != NULL) { snprintf(buffer, sizeof(buffer), %s%s\n, row[0], row[1]); copy_result =

Re: [HACKERS] PQputCopyData dont signal error

2011-04-14 Thread Pavel Stehule
Hello The way COPY works is that PQputCopyData just sends the data to the server, and the server will buffer it in its internal buffer and processes it when it feels like it. The PQputCopyData() calls don't even need to match line boundaries. Yes, it is current behave - then documentation

Re: [HACKERS] PQputCopyData dont signal error

2011-04-14 Thread Heikki Linnakangas
On 14.04.2011 10:15, Pavel Stehule wrote: Hello I have a problem with PQputCopyData function. It doesn't signal some error. while ((row = mysql_fetch_row(res)) != NULL) { snprintf(buffer, sizeof(buffer), %s%s\n, row[0], row[1]);