* Tom Lane ([EMAIL PROTECTED]) wrote: > Stephen Frost <[EMAIL PROTECTED]> writes: > > It's only the functional equivalent when you think all the world is a > > Postgres app, which is just not the case. > > If we are dumping data into a simple memory block in a format dictated > by libpq, then we haven't done a thing to make the app's use of that > data independent of libpq. Furthermore, because that format has to be > generalized (variable-length fields, etc), it will not be noticeably > easier to use than the existing PQresult API.
The format of the structure *isn't* really dictated by libpq. The offsets, value length and record size is intended to support most any C array structure. Variable length fields have a max size which, if it goes over, an error is returned or indicated throgh the indicator array. It also gets it into the structure quite a few applications would like to have it in (which is certainly not PQresult). > What I would envision as a typical use of a callback is to convert the > data and store it in a C struct designed specifically for a particular > query's known result structure (say, a few ints, a string of a known > maximum length, etc). libpq can't do that, but a callback could do it > easily. Heh, this is exactly what I'm proposed we make libpq capable of doing, which is a relatively simple thing to do. I agree that it's often a goal of application devlopers to get it into this structure. The one downside is that at the moment I think the binary results from libpq come back in network-byte-order instead of host-byte-order. Oracle provided a way to indicate the types of the fields in the structure and performed some conversions (such as these) for you. The constants they used started with "SQL_" but I'm not entirely sure if they were actually defined in the standard or not. > The fixed-memory-block approach also falls over when considering results > of uncertain maximum size. Lastly, it doesn't seem to me to respond at > all to the ODBC needs that started this thread: IIUC, they want each row > separately malloc'd so that they can free selected rows from the > completed resultset. Results of uncertain maximum size aren't a problem at all... The caller can do the exact same thing libpq does (realloc), or it could allocate another array. *Each* call to the libpq function would return the number of elements actually populated into the memory-block; the caller would then be expected to pass in a *fresh* memory block for the next call (which could just be a simply calculated offset into the block they allocated, or could be a realloc'd block + offset, or a brand new block, etc...). I'm really not why there seem to be this "this won't work!" reaction. This isn't something I came up with out of whole cloth, it's an API that isn't unlike PQexecParams, is similar to something Oracle does (which I've used quite a bit for doing *exactly* what's mentioned above- I've got an array of pre-defined C structs that I know match the query and I want that array filled in) and is really not that complicated. > > For one thing, it's certainly possible the callback (to do a data > > transform like you're suggesting) would want access to the other > > information in a given tuple. Having to store a partial tuple in a > > temporary area which has to be built up to the full tuple before you can > > actually process it wouldn't be all that great. > > So instead, you'd prefer to *always* store partial tuples in a temporary > area, thereby making sure the independent-field-conversions case has > performance just as bad as the dependent-conversions case. > I can't follow that reasoning. I havn't been ruling out providing a callback mechanism as well but I think it's the 10% case and the 90% case is being shoe-horned into the 10% case with a performance degredation to boot. They're also not partial tuples, it's not a temporary area, and there's demonstratably less copying around of the data. It seems ODBC may be in the 10% piece here but I havn't looked at the ODBC source code yet. Thanks, Stephen
signature.asc
Description: Digital signature