Libpq copy speedup patch attached. No input buffer logic involved, just
removing the expensive PQconsumeInput call per putCopyData call, and leaving
parseInput as is, as discussed before.
Alon.
libpq_copy.patch
Description: Binary data
---(end of broadcast)-
> You mean something like
>
> if (input-buffer-not-empty)
> parseInput();
>
> ? This still bothers me a bit since it's a mixing of logic levels;
> PQputCopyData is an output routine, it shouldn't be getting its fingers
> dirty with input buffer contents. I'm willing to tolerate this if it
> can
Tom,
>> Here is a patch against today's code 1/24. As discussed in -hackers
>> consumeInput/parse is removed from being called every single time. It's
>> still there for only when the data is sent to the server.
>
> This appears to be the exact same patch you sent before. Did you
> test my sugge
Here is a patch against today's code 1/24. As discussed in -hackers
consumeInput/parse is removed from being called every single time. It's
still there for only when the data is sent to the server.
Alon.
pq_put_copy_data.patch
Description: Binary data
---(end of broadca
I did some performance checks after the recent code commit.
The good news is that the parsing speed of COPY is now MUCH faster, which is
great. It is about 5 times faster - about 100MB/sec on my machine
(previously 20MB/sec at best, usually less).
The better news is that my original patch parsing
Tom,
Thanks for pointing it out. I made the small required modifications to match
copy.c version 1.247 and sent it to -patches list. New patch is V16.
Alon.
On 8/1/05 7:51 PM, "Tom Lane" <[EMAIL PROTECTED]> wrote:
> "Alon Goldshuv" <[EMAIL PROTECTED]> writes
markw 50 136M Jul 8 15:03 supplier.tbl
>
> Number of rows:
> # wc -l *.tbl
> 1500 customer.tbl
>600037902 lineitem.tbl
> 25 nation.tbl
>15000 orders.tbl
> 2000 part.tbl
> 8000 partsupp.tbl
>5 region.tbl
>
speed
etc... This is for TEXT format. As for CSV, it may be faster but not as much
as I specified here. BINARY will stay the same as before.
HTH
Alon.
On 7/19/05 12:54 PM, "Mark Wong" <[EMAIL PROTECTED]> wrote:
> On Thu, 14 Jul 2005 17:22:18 -0700
> "Alon Goldshu
Alon.
On 6/25/05 4:27 PM, "Tom Lane" <[EMAIL PROTECTED]> wrote:
> "Alon Goldshuv" <[EMAIL PROTECTED]> writes:
>> A struct "bytebuffer" is used instead of a StringInfoData for storing the
>> line and attributes. A StringInfoData is act
e bytebuffer is more straightfoward and faster.
Alon.
On 6/25/05 3:52 PM, "Alon Goldshuv" <[EMAIL PROTECTED]> wrote:
>
>
>> I haven't seen any demand for users to be allowed to specify an escape char
>> in text mode (BTW, that's what the docs call what
> I haven't seen any demand for users to be allowed to specify an escape char
> in text mode (BTW, that's what the docs call what you have called delimited
> mode). Maybe there's a case for it, but I somewhat doubt it.
I ran into users that could not load their data using COPY because their
data
> What about this one:
> case COPY_OLD_FE:
> +ereport(ERROR,
> + (errcode(ERRCODE_CONNECTION_FAILURE),
> + errmsg("Old FE protocal not yet supported in fast COPY processing
> (CopyGetData())")));
> +
>
> This can't stay like it is, can it?
> Who uses the old FE protocol for CO
Andrew,
Thanks for your initial feedback.
> 2. This comment raises a flag in my mind:
>
> + * each attribute begins. If a specific attribute is not used for this
> + * COPY command (ommitted from the column list), a value of 0 will be
> assigned.+ * For example: for table foo(a,b,c,d,e) and COPY
Neil,
Right; I really dislike the idea of having two separate code paths for
COPY. When you say this approach is "temporary", are you suggesting that
you intend to reimplement your changes as improvements/replacements of
the existing COPY code path rather than as a parallel code path?
My tho
14 matches
Mail list logo
