You can use COPY from a stored procedure, but only to and from files.
I think that's in the chocolate fireguard realm though as far as
efficiency for this sort of scenario goes, even if its handled by
retaining an mmap'd file as workspace.
If SPI provided a way to perform a copy to a temp table and then some callback
on an iterator that yields rows to it, that would do the trick I guess.
SPI is useful, but it's certainly possible to avoid its use. After all, that
what almost the whole backend does, including the COPY code. Of course, it's a
lot harder to write that way, which is part of why SPI exists. Efficiency has
its price.
So it is possible to use a lower level interface from a C stored proc?
SPI is the (only) documented direct function extension API isn't it?
Is the issue with using the JSON data-to-record set that the parsing can
be costly? Perhaps it can be achieved with B64 of compressed protobuf,
or such. I don't mind if it seems a bit messy - the code can be
generated from the table easily enough, especially if I can use C++. I
guess an allocator that uses SPI_palloc would solve issues with memory
management on error?
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
