Joshua D. Drake wrote:
Heikki Linnakangas wrote:
We read 64 KB at a time, and then CopyReadLineText returns one line
at a time from that buffer.
O.k. I am sure I am oversimplifying things but why are we returning
one line at a time? That reads expensive to me. Just following the
general, don't do inserts one at a time, do them in batch idea for
example.
Quite simply because one line corresponds to one record. And yes, I
believe you are oversimplifying, or under several misapprehensions about
what can be done at this level.
I would also question the 64KB at a time. Why not a 1024KB (arbitrary)
at a time? Is it a resource issue? In the old days when we actually
had people trying to run postgresql on 128 and 256 megs of ram, o.k.
but now?
It would be simple enough to change. Try it and see if it actually makes
a difference. All you have to change is the define of RAW_BUF_SIZE.
In reading:
http://doxygen.postgresql.org/backend_2commands_2copy_8c-source.html
It looks like CopyReadAttributesText is used as part of the column
breakup. It also appears that this happens "before" insert right? So
if that is the case we are going to pay an additional penalty on the
data checking.
What? I don't understand what you are talking about.
cheers
andrew
---------------------------(end of broadcast)---------------------------
TIP 2: Don't 'kill -9' the postmaster
