On Fri, Jun 3, 2011 at 12:04 PM, Heikki Linnakangas <heikki.linnakan...@enterprisedb.com> wrote: > On 03.06.2011 19:19, Radosław Smogura wrote: >> >> Hello, >> >> Sorry for short introduction about this, and plese as far as possible, >> disconnet it from LOBs, as it on top of LOB. >> >> Idea of streaming is to reduce memory copy mainly during receiving and >> sending >> tuples. Currently receive works as follows >> 1. Read bytes of tuple (allocate x memory). >> 2. Eventually convert it to database encoding. >> 3. Use this data and create datum (which is little changed copy of 1 or 2) >> 4. Streaming will be allowed only in binary mode, and actually stream >> in/out >> will return binary data. > > Hmm, I was thinking that streaming would be a whole new mode, alongside the > current text and binary mode. > >> Look for example at execution chain from textrecv. >> >> Idea is to add stream_in, stream_out columns pg_type. >> >> When value is to be serialized the sin/sout is called. Struct must pass >> len of >> data, and struct of stream (simillar to C FILE*). >> >> Caller should validate if all bytes has been consumed (expose simple >> methods >> for this) >> >> To implement(code API requirements): >> First stream is buffered socekt reader. >> >> Fast substreams - create fast stream limited to x bytes basing on other >> stream >> >> Skipping bytes + skipAll() >> >> Stream filtering - do fast (faster will be if conversion will occure in >> buffered chunks) encoding conversion. >> >> Support for direct PG printf() version. Linux has ability to create cookie >> streams and use it with fprintf(), so its greate advantage to format huge >> strings. Other system should buffer output. Problem is if Linux cookie >> will >> fail will it write something to output? Windows proxy will push value to >> temp >> buffer. > > This is pretty low-level stuff, I think we should focus on the protocol > changes and user-visible libpq API first.
+1. in particular, I'd like to see the libpq api changes. merlin -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
