lidavidm commented on PR #4544:
URL: https://github.com/apache/arrow-adbc/pull/4544#issuecomment-5290153012
> > I think I'm leaning towards having all options/filters be set by
SetOption.
>
> This is kind of ugly but I see what you're getting at. GDAL handles this
with `const char** options` where options is NULL terminated list of
`KEY=arbitrary value`, with the downside that everything is serialized. If this
is a get/set option on a connection is seems more problematic (on a statement
it's more clear that options have a specific scope).
I agree that the scoping of the option feels a bit awkward. @paleolimbot do
you mean that it's a flat list of alternating `[key, value, key, value, key,
value]`? That could work well enough.
>
> > This would be useful if we do want to support Iceberg-style catalogs, as
we need a way to pass a list of namespace parts
>
> It's a hack, but some REST APIs use the unit separator (`\x1f`) or have a
user-specified separator and just have "schemas" with potentially multiple
parts.
I suppose the separator works but I dislike this sort of in-band
encoding...that said we could also figure out some convention to encode
multiple variadic arguments into a parameter list (e.g. terminating a variadic
argument with a NULL element)
>
> I'm guessing you don't want to go this direction, but I can't help but
notice there's a large amount of complexity associated with stuffing these
concepts into Arrow arrays that are highly nested and very difficult to parse.
I'll throw out that you could do something like
>
> ```c
> struct AdbcCatalogNode {
> AdbcStatus (*get_property)(struct AdbcCatalogNode* self, const char*
what, struct ArrowSchema* out_schema, struct ArrowArray* out_array, AdbcError*
err);
> AdbcStatus (*get_child)(struct AdbcCatalogNode* self, const char* what,
AdbcError* err);
> void* private_data;
> void* private_driver;
> }
>
> AdbcStatus AdbcConnectionGetCatalogs(struct AdbcConnection* connection,
const char** options, struct AdbcCatalogNode* out, AdbcError* err);
> ```
>
> Not perfect, but maybe lets some of this complexity get pushed onto the
driver instead of on the consumer since drivers might have abstractions for
some of this already.
The new collection definitions don't nest the data anymore, to hopefully
avoid this.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
