zeroshade commented on code in PR #4544:
URL: https://github.com/apache/arrow-adbc/pull/4544#discussion_r3780300834
##########
c/include/arrow-adbc/adbc.h:
##########
@@ -2122,6 +2134,360 @@ AdbcStatusCode AdbcConnectionGetObjects(struct
AdbcConnection* connection, int d
struct ArrowArrayStream* out,
struct AdbcError* error);
+/// \brief Fetch (catalog) metadata from the database.
+///
+/// The metadata to fetch is defined by the `collection` parameter. The result
+/// is an Arrow dataset with a schema defined by the collection. For example,
+/// a client may request a list of tables in the database, or a list of
+/// supported data types. Drivers may implement collections beyond those
+/// defined by ADBC, but must use a vendor-specific prefix
+/// (e.g. `postgresql.`) to avoid conflicts with future standardized
+/// collections. Drivers must not use the `adbc.` prefix.
+///
+/// The result may be filtered by `filters`, which is an array of (nullable)
+/// strings. `num_filters` must be set to the number of filter arguments
+/// passed.
+///
+/// All drivers must implement a collection called "meta" (which is aliased to
+/// NULL and blank string) that defines the available collections. See
+/// ADBC_METADATA_COLLECTION_META.
+///
+/// Drivers may add more fields at the end of standard schemas to reflect
+/// vendor-specific metadata. Applications must access these using an offset
+/// from the end of the schema and cannot assume that the index of the field
+/// will remain stable. Drivers must add the fields at the end should prefix
+/// field names with the vendor/driver name to differentiate them
+/// (e.g. 'POSTGRESQL:owner').
+///
+/// Similarly, future standard revisions may add more fields to existing
+/// standard schemas. Applications must not assume the number of fields is
+/// fixed.
+///
+/// This AdbcConnection must outlive the returned ArrowArrayStream.
+///
+/// \param[in] connection The database connection.
+/// \param[in] collection The collection to fetch.
+/// \param[out] out The result set.
Review Comment:
The filters params are missing from the docs here
##########
c/include/arrow-adbc/adbc.h:
##########
@@ -2122,6 +2134,360 @@ AdbcStatusCode AdbcConnectionGetObjects(struct
AdbcConnection* connection, int d
struct ArrowArrayStream* out,
struct AdbcError* error);
+/// \brief Fetch (catalog) metadata from the database.
+///
+/// The metadata to fetch is defined by the `collection` parameter. The result
+/// is an Arrow dataset with a schema defined by the collection. For example,
+/// a client may request a list of tables in the database, or a list of
+/// supported data types. Drivers may implement collections beyond those
+/// defined by ADBC, but must use a vendor-specific prefix
+/// (e.g. `postgresql.`) to avoid conflicts with future standardized
+/// collections. Drivers must not use the `adbc.` prefix.
+///
+/// The result may be filtered by `filters`, which is an array of (nullable)
+/// strings. `num_filters` must be set to the number of filter arguments
+/// passed.
+///
+/// All drivers must implement a collection called "meta" (which is aliased to
+/// NULL and blank string) that defines the available collections. See
+/// ADBC_METADATA_COLLECTION_META.
+///
+/// Drivers may add more fields at the end of standard schemas to reflect
+/// vendor-specific metadata. Applications must access these using an offset
+/// from the end of the schema and cannot assume that the index of the field
+/// will remain stable. Drivers must add the fields at the end should prefix
+/// field names with the vendor/driver name to differentiate them
+/// (e.g. 'POSTGRESQL:owner').
+///
+/// Similarly, future standard revisions may add more fields to existing
+/// standard schemas. Applications must not assume the number of fields is
+/// fixed.
+///
+/// This AdbcConnection must outlive the returned ArrowArrayStream.
+///
+/// \param[in] connection The database connection.
+/// \param[in] collection The collection to fetch.
+/// \param[out] out The result set.
+/// \param[out] error Error details, if an error occurs.
+/// \since ADBC API revision 1.2.0
+ADBC_EXPORT
+AdbcStatusCode AdbcConnectionGetMetadataCollection(
+ struct AdbcConnection* connection, const char* collection, size_t
num_filters,
+ const char** filters, struct ArrowArrayStream* out, struct AdbcError*
error);
+
+/// \brief The "meta" collection returns the available metadata collections.
+///
+/// | Field Name | Field Type | Comments |
+/// |--------------------------|------------------------------|----------|
+/// | collection_name | utf8 not null | |
+/// | collection_description | utf8 | |
+/// | collection_schema | extension<arrow.schema_json> | |
+/// | collection_filters | list<FILTER_SCHEMA> | |
+///
+/// FILTER_SCHEMA is a Struct with fields:
+///
+/// | Field Name | Field Type | Comments |
+/// |--------------------------|------------------------------|----------|
+/// | filter_description | utf8 | |
+/// | required | bool not null | |
+#define ADBC_METADATA_COLLECTION_META "meta"
+
+/// \brief The "catalogs" collection returns the catalogs defined in the
+/// database.
+///
+/// Some systems may not have the concept of catalogs, in which case this
+/// collection should contain a single entry with an empty, non-null name.
+///
+/// | Field Name | Field Type | Comments |
+/// |--------------------------|------------------------------|----------|
+/// | catalog_name | utf8 | |
+/// | catalog_remarks | utf8 | (1) |
+///
+/// (1) A description of the catalog.
+///
+/// Filters:
+/// 1. The catalog name to filter by. May be a search pattern.
+#define ADBC_METADATA_COLLECTION_CATALOGS "catalogs"
+
+/// \brief The "schemas" collection returns the schemas defined in the
+/// database.
+///
+/// Some systems may not have the concept of schemas, in which case this
+/// collection should contain a single entry per catalog with an empty,
+/// non-null name.
+///
+/// | Field Name | Field Type | Comments |
+/// |--------------------------|------------------------------|----------|
+/// | catalog_name | utf8 | (R) |
+/// | db_schema_name | utf8 | |
+/// | db_schema_remarks | utf8 | (1) |
+///
+/// (R) This field is run-length encoded by default; it can be disabled via
Review Comment:
Why REE instead of dictionary encoded (which is more widely supported among
arrow implementations)?
##########
c/include/arrow-adbc/adbc.h:
##########
@@ -2122,6 +2134,360 @@ AdbcStatusCode AdbcConnectionGetObjects(struct
AdbcConnection* connection, int d
struct ArrowArrayStream* out,
struct AdbcError* error);
+/// \brief Fetch (catalog) metadata from the database.
+///
+/// The metadata to fetch is defined by the `collection` parameter. The result
+/// is an Arrow dataset with a schema defined by the collection. For example,
+/// a client may request a list of tables in the database, or a list of
+/// supported data types. Drivers may implement collections beyond those
+/// defined by ADBC, but must use a vendor-specific prefix
+/// (e.g. `postgresql.`) to avoid conflicts with future standardized
+/// collections. Drivers must not use the `adbc.` prefix.
+///
+/// The result may be filtered by `filters`, which is an array of (nullable)
+/// strings. `num_filters` must be set to the number of filter arguments
+/// passed.
+///
+/// All drivers must implement a collection called "meta" (which is aliased to
+/// NULL and blank string) that defines the available collections. See
+/// ADBC_METADATA_COLLECTION_META.
+///
+/// Drivers may add more fields at the end of standard schemas to reflect
+/// vendor-specific metadata. Applications must access these using an offset
+/// from the end of the schema and cannot assume that the index of the field
+/// will remain stable. Drivers must add the fields at the end should prefix
+/// field names with the vendor/driver name to differentiate them
+/// (e.g. 'POSTGRESQL:owner').
+///
+/// Similarly, future standard revisions may add more fields to existing
+/// standard schemas. Applications must not assume the number of fields is
+/// fixed.
+///
+/// This AdbcConnection must outlive the returned ArrowArrayStream.
+///
+/// \param[in] connection The database connection.
+/// \param[in] collection The collection to fetch.
+/// \param[out] out The result set.
+/// \param[out] error Error details, if an error occurs.
+/// \since ADBC API revision 1.2.0
+ADBC_EXPORT
+AdbcStatusCode AdbcConnectionGetMetadataCollection(
+ struct AdbcConnection* connection, const char* collection, size_t
num_filters,
+ const char** filters, struct ArrowArrayStream* out, struct AdbcError*
error);
Review Comment:
Did I miss it or does the docs not define the semantics of the filters in
terms of prefix, a LIKE pattern, regex, etc.?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
