Re: [PR] feat(c/driver_manager): add connection profile interface [arrow-adbc]
lidavidm commented on code in PR #3876:
URL: https://github.com/apache/arrow-adbc/pull/3876#discussion_r2791512949
##
c/include/arrow-adbc/adbc_driver_manager.h:
##
@@ -175,6 +175,171 @@ AdbcStatusCode
AdbcDriverManagerDatabaseSetAdditionalSearchPathList(
ADBC_EXPORT
const char* AdbcStatusCodeMessage(AdbcStatusCode code);
+/// \defgroup adbc-driver-manager-connection-profile Connection Profiles
+/// Similar to odbc.ini, the ADBC driver manager can support "connection
profiles"
+/// that specify a driver and options to use when connecting. This allows
users to
+/// specify connection information in a file or environment variable, and have
the
+/// driver manager load the appropriate driver and set options accordingly.
+///
+/// This allows creating reusable connection configurations for sharing and
distribution
+/// without needing to hardcode driver names and options in application code.
Profiles
+/// will be loaded during DatabaseInit before attempting to initialize the
driver. Any
+/// options specified by the profile will be applied but will not override
options
+/// that have already been set using DatabaseSetOption.
+///
+/// To facilitate customization, we define an interface for implementing a
Connection
+/// Profile object along with a provider function definition which can be set
into
+/// the driver manager to allow for customized profile loading.
+///
+/// A profile can be specified to the Driver Manager in one of two ways,
+/// which will invoke the profile provider during the call to DatabaseInit:
+///
+/// 1. The "profile" option can be set using DatabaseSetOption with the name
of the
+/// profile to load.
+/// 2. The "uri" being used can have the form "profile://"
+///
+/// @{
+
+/// \brief Abstract interface for connection profile providers
+struct ADBC_EXPORT AdbcConnectionProfile {
+ /// \brief Opaque implementation-defined state.
+ /// This field is NULL if the profile is uninitialized/freed (but
+ /// it need not have a value even if the profile is initialized).
+ void* private_data;
+
+ /// \brief Release the profile and perform any cleanup.
+ void (*release)(struct AdbcConnectionProfile* profile);
+
+ /// \brief Get the driver to use as specified by this profile.
+ ///
+ /// It is not required that a profile specify a driver. If the options
+ // can be reusable across drivers, then the profile does not need to specify
+ /// a driver (if this provides an empty string or nullptr then the driver
+ /// must be defined by other means, e.g. by the driver / uri options).
+ ///
+ /// \param[in] profile The profile to query.
+ /// \param[out] driver_name The name of the driver to use, or NULL if not
specified.
+ /// \param[out] error An optional location to return an error message
+ AdbcStatusCode (*GetDriverName)(struct AdbcConnectionProfile* profile,
+ const char** driver_name, struct AdbcError*
error);
Review Comment:
It would be nice to have other people chime in as well ^_^
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
Re: [PR] feat(c/driver_manager): add connection profile interface [arrow-adbc]
zeroshade commented on code in PR #3876:
URL: https://github.com/apache/arrow-adbc/pull/3876#discussion_r2791336069
##
c/include/arrow-adbc/adbc_driver_manager.h:
##
@@ -175,6 +175,171 @@ AdbcStatusCode
AdbcDriverManagerDatabaseSetAdditionalSearchPathList(
ADBC_EXPORT
const char* AdbcStatusCodeMessage(AdbcStatusCode code);
+/// \defgroup adbc-driver-manager-connection-profile Connection Profiles
+/// Similar to odbc.ini, the ADBC driver manager can support "connection
profiles"
+/// that specify a driver and options to use when connecting. This allows
users to
+/// specify connection information in a file or environment variable, and have
the
+/// driver manager load the appropriate driver and set options accordingly.
+///
+/// This allows creating reusable connection configurations for sharing and
distribution
+/// without needing to hardcode driver names and options in application code.
Profiles
+/// will be loaded during DatabaseInit before attempting to initialize the
driver. Any
+/// options specified by the profile will be applied but will not override
options
+/// that have already been set using DatabaseSetOption.
+///
+/// To facilitate customization, we define an interface for implementing a
Connection
+/// Profile object along with a provider function definition which can be set
into
+/// the driver manager to allow for customized profile loading.
+///
+/// A profile can be specified to the Driver Manager in one of two ways,
+/// which will invoke the profile provider during the call to DatabaseInit:
+///
+/// 1. The "profile" option can be set using DatabaseSetOption with the name
of the
+/// profile to load.
+/// 2. The "uri" being used can have the form "profile://"
+///
+/// @{
+
+/// \brief Abstract interface for connection profile providers
+struct ADBC_EXPORT AdbcConnectionProfile {
+ /// \brief Opaque implementation-defined state.
+ /// This field is NULL if the profile is uninitialized/freed (but
+ /// it need not have a value even if the profile is initialized).
+ void* private_data;
+
+ /// \brief Release the profile and perform any cleanup.
+ void (*release)(struct AdbcConnectionProfile* profile);
+
+ /// \brief Get the driver to use as specified by this profile.
+ ///
+ /// It is not required that a profile specify a driver. If the options
+ // can be reusable across drivers, then the profile does not need to specify
+ /// a driver (if this provides an empty string or nullptr then the driver
+ /// must be defined by other means, e.g. by the driver / uri options).
+ ///
+ /// \param[in] profile The profile to query.
+ /// \param[out] driver_name The name of the driver to use, or NULL if not
specified.
+ /// \param[out] error An optional location to return an error message
+ AdbcStatusCode (*GetDriverName)(struct AdbcConnectionProfile* profile,
+ const char** driver_name, struct AdbcError*
error);
Review Comment:
We can, I just didn't based on the way I implemented it. If both of you
think it's a good way to go, I don't have a good argument against it so I can
add it to the profile object. Sure.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
Re: [PR] feat(c/driver_manager): add connection profile interface [arrow-adbc]
lidavidm commented on code in PR #3876:
URL: https://github.com/apache/arrow-adbc/pull/3876#discussion_r2791300566
##
c/include/arrow-adbc/adbc_driver_manager.h:
##
@@ -175,6 +175,171 @@ AdbcStatusCode
AdbcDriverManagerDatabaseSetAdditionalSearchPathList(
ADBC_EXPORT
const char* AdbcStatusCodeMessage(AdbcStatusCode code);
+/// \defgroup adbc-driver-manager-connection-profile Connection Profiles
+/// Similar to odbc.ini, the ADBC driver manager can support "connection
profiles"
+/// that specify a driver and options to use when connecting. This allows
users to
+/// specify connection information in a file or environment variable, and have
the
+/// driver manager load the appropriate driver and set options accordingly.
+///
+/// This allows creating reusable connection configurations for sharing and
distribution
+/// without needing to hardcode driver names and options in application code.
Profiles
+/// will be loaded during DatabaseInit before attempting to initialize the
driver. Any
+/// options specified by the profile will be applied but will not override
options
+/// that have already been set using DatabaseSetOption.
+///
+/// To facilitate customization, we define an interface for implementing a
Connection
+/// Profile object along with a provider function definition which can be set
into
+/// the driver manager to allow for customized profile loading.
+///
+/// A profile can be specified to the Driver Manager in one of two ways,
+/// which will invoke the profile provider during the call to DatabaseInit:
+///
+/// 1. The "profile" option can be set using DatabaseSetOption with the name
of the
+/// profile to load.
+/// 2. The "uri" being used can have the form "profile://"
+///
+/// @{
+
+/// \brief Abstract interface for connection profile providers
+struct ADBC_EXPORT AdbcConnectionProfile {
+ /// \brief Opaque implementation-defined state.
+ /// This field is NULL if the profile is uninitialized/freed (but
+ /// it need not have a value even if the profile is initialized).
+ void* private_data;
+
+ /// \brief Release the profile and perform any cleanup.
+ void (*release)(struct AdbcConnectionProfile* profile);
+
+ /// \brief Get the driver to use as specified by this profile.
+ ///
+ /// It is not required that a profile specify a driver. If the options
+ // can be reusable across drivers, then the profile does not need to specify
+ /// a driver (if this provides an empty string or nullptr then the driver
+ /// must be defined by other means, e.g. by the driver / uri options).
+ ///
+ /// \param[in] profile The profile to query.
+ /// \param[out] driver_name The name of the driver to use, or NULL if not
specified.
+ /// \param[out] error An optional location to return an error message
+ AdbcStatusCode (*GetDriverName)(struct AdbcConnectionProfile* profile,
+ const char** driver_name, struct AdbcError*
error);
Review Comment:
I guess what I'm wondering (and presumably what Dewey is wondering) is why
we can't just let profiles specify an init_func instead of _requiring_ them to
defer to the driver search process, though. Again, TOML profiles probably will
never use this, but I can see other implementations wanting to do so?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
Re: [PR] feat(c/driver_manager): add connection profile interface [arrow-adbc]
zeroshade commented on code in PR #3876:
URL: https://github.com/apache/arrow-adbc/pull/3876#discussion_r2791227472
##
c/include/arrow-adbc/adbc_driver_manager.h:
##
@@ -175,6 +175,171 @@ AdbcStatusCode
AdbcDriverManagerDatabaseSetAdditionalSearchPathList(
ADBC_EXPORT
const char* AdbcStatusCodeMessage(AdbcStatusCode code);
+/// \defgroup adbc-driver-manager-connection-profile Connection Profiles
+/// Similar to odbc.ini, the ADBC driver manager can support "connection
profiles"
+/// that specify a driver and options to use when connecting. This allows
users to
+/// specify connection information in a file or environment variable, and have
the
+/// driver manager load the appropriate driver and set options accordingly.
+///
+/// This allows creating reusable connection configurations for sharing and
distribution
+/// without needing to hardcode driver names and options in application code.
Profiles
+/// will be loaded during DatabaseInit before attempting to initialize the
driver. Any
+/// options specified by the profile will be applied but will not override
options
+/// that have already been set using DatabaseSetOption.
+///
+/// To facilitate customization, we define an interface for implementing a
Connection
+/// Profile object along with a provider function definition which can be set
into
+/// the driver manager to allow for customized profile loading.
+///
+/// A profile can be specified to the Driver Manager in one of two ways,
+/// which will invoke the profile provider during the call to DatabaseInit:
+///
+/// 1. The "profile" option can be set using DatabaseSetOption with the name
of the
+/// profile to load.
+/// 2. The "uri" being used can have the form "profile://"
+///
+/// @{
+
+/// \brief Abstract interface for connection profile providers
+struct ADBC_EXPORT AdbcConnectionProfile {
+ /// \brief Opaque implementation-defined state.
+ /// This field is NULL if the profile is uninitialized/freed (but
+ /// it need not have a value even if the profile is initialized).
+ void* private_data;
+
+ /// \brief Release the profile and perform any cleanup.
+ void (*release)(struct AdbcConnectionProfile* profile);
+
+ /// \brief Get the driver to use as specified by this profile.
+ ///
+ /// It is not required that a profile specify a driver. If the options
+ // can be reusable across drivers, then the profile does not need to specify
+ /// a driver (if this provides an empty string or nullptr then the driver
+ /// must be defined by other means, e.g. by the driver / uri options).
+ ///
+ /// \param[in] profile The profile to query.
+ /// \param[out] driver_name The name of the driver to use, or NULL if not
specified.
+ /// \param[out] error An optional location to return an error message
+ AdbcStatusCode (*GetDriverName)(struct AdbcConnectionProfile* profile,
+ const char** driver_name, struct AdbcError*
error);
Review Comment:
init_func is for user customizability, so I didn't want to override or
prevent the ability for a user to provide their own init_func. Using init_func
would be instead of using the default implementation of profiles. More to the
point, if init_func is used, you don't get the default driver resolution, and
we're taking advantage of that to let the profile still use drivers by name
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
Re: [PR] feat(c/driver_manager): add connection profile interface [arrow-adbc]
lidavidm commented on code in PR #3876:
URL: https://github.com/apache/arrow-adbc/pull/3876#discussion_r2791218565
##
c/include/arrow-adbc/adbc_driver_manager.h:
##
@@ -175,6 +175,171 @@ AdbcStatusCode
AdbcDriverManagerDatabaseSetAdditionalSearchPathList(
ADBC_EXPORT
const char* AdbcStatusCodeMessage(AdbcStatusCode code);
+/// \defgroup adbc-driver-manager-connection-profile Connection Profiles
+/// Similar to odbc.ini, the ADBC driver manager can support "connection
profiles"
+/// that specify a driver and options to use when connecting. This allows
users to
+/// specify connection information in a file or environment variable, and have
the
+/// driver manager load the appropriate driver and set options accordingly.
+///
+/// This allows creating reusable connection configurations for sharing and
distribution
+/// without needing to hardcode driver names and options in application code.
Profiles
+/// will be loaded during DatabaseInit before attempting to initialize the
driver. Any
+/// options specified by the profile will be applied but will not override
options
+/// that have already been set using DatabaseSetOption.
+///
+/// To facilitate customization, we define an interface for implementing a
Connection
+/// Profile object along with a provider function definition which can be set
into
+/// the driver manager to allow for customized profile loading.
+///
+/// A profile can be specified to the Driver Manager in one of two ways,
+/// which will invoke the profile provider during the call to DatabaseInit:
+///
+/// 1. The "profile" option can be set using DatabaseSetOption with the name
of the
+/// profile to load.
+/// 2. The "uri" being used can have the form "profile://"
+///
+/// @{
+
+/// \brief Abstract interface for connection profile providers
+struct ADBC_EXPORT AdbcConnectionProfile {
+ /// \brief Opaque implementation-defined state.
+ /// This field is NULL if the profile is uninitialized/freed (but
+ /// it need not have a value even if the profile is initialized).
+ void* private_data;
+
+ /// \brief Release the profile and perform any cleanup.
+ void (*release)(struct AdbcConnectionProfile* profile);
+
+ /// \brief Get the driver to use as specified by this profile.
+ ///
+ /// It is not required that a profile specify a driver. If the options
+ // can be reusable across drivers, then the profile does not need to specify
+ /// a driver (if this provides an empty string or nullptr then the driver
+ /// must be defined by other means, e.g. by the driver / uri options).
+ ///
+ /// \param[in] profile The profile to query.
+ /// \param[out] driver_name The name of the driver to use, or NULL if not
specified.
+ /// \param[out] error An optional location to return an error message
+ AdbcStatusCode (*GetDriverName)(struct AdbcConnectionProfile* profile,
+ const char** driver_name, struct AdbcError*
error);
Review Comment:
No, I mean: if init_func is sufficient, then why not implement profiles in
terms of init_func?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
Re: [PR] feat(c/driver_manager): add connection profile interface [arrow-adbc]
zeroshade commented on code in PR #3876:
URL: https://github.com/apache/arrow-adbc/pull/3876#discussion_r2791159007
##
c/include/arrow-adbc/adbc_driver_manager.h:
##
@@ -175,6 +175,171 @@ AdbcStatusCode
AdbcDriverManagerDatabaseSetAdditionalSearchPathList(
ADBC_EXPORT
const char* AdbcStatusCodeMessage(AdbcStatusCode code);
+/// \defgroup adbc-driver-manager-connection-profile Connection Profiles
+/// Similar to odbc.ini, the ADBC driver manager can support "connection
profiles"
+/// that specify a driver and options to use when connecting. This allows
users to
+/// specify connection information in a file or environment variable, and have
the
+/// driver manager load the appropriate driver and set options accordingly.
+///
+/// This allows creating reusable connection configurations for sharing and
distribution
+/// without needing to hardcode driver names and options in application code.
Profiles
+/// will be loaded during DatabaseInit before attempting to initialize the
driver. Any
+/// options specified by the profile will be applied but will not override
options
+/// that have already been set using DatabaseSetOption.
+///
+/// To facilitate customization, we define an interface for implementing a
Connection
+/// Profile object along with a provider function definition which can be set
into
+/// the driver manager to allow for customized profile loading.
+///
+/// A profile can be specified to the Driver Manager in one of two ways,
+/// which will invoke the profile provider during the call to DatabaseInit:
+///
+/// 1. The "profile" option can be set using DatabaseSetOption with the name
of the
+/// profile to load.
+/// 2. The "uri" being used can have the form "profile://"
+///
+/// @{
+
+/// \brief Abstract interface for connection profile providers
+struct ADBC_EXPORT AdbcConnectionProfile {
+ /// \brief Opaque implementation-defined state.
+ /// This field is NULL if the profile is uninitialized/freed (but
+ /// it need not have a value even if the profile is initialized).
+ void* private_data;
+
+ /// \brief Release the profile and perform any cleanup.
+ void (*release)(struct AdbcConnectionProfile* profile);
+
+ /// \brief Get the driver to use as specified by this profile.
+ ///
+ /// It is not required that a profile specify a driver. If the options
+ // can be reusable across drivers, then the profile does not need to specify
+ /// a driver (if this provides an empty string or nullptr then the driver
+ /// must be defined by other means, e.g. by the driver / uri options).
+ ///
+ /// \param[in] profile The profile to query.
+ /// \param[out] driver_name The name of the driver to use, or NULL if not
specified.
+ /// \param[out] error An optional location to return an error message
+ AdbcStatusCode (*GetDriverName)(struct AdbcConnectionProfile* profile,
+ const char** driver_name, struct AdbcError*
error);
Review Comment:
> Conversely...if telling the user to just init_func is sufficient, then why
doesn't that also apply to profiles themselves?
That's my point, the `init_func` should apply to the profile itself.
I guess I'm not completely opposed to adding it to the profile
implementation, but I'd want a reason why just providing the `init_func` isn't
sufficient
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
Re: [PR] feat(c/driver_manager): add connection profile interface [arrow-adbc]
lidavidm commented on code in PR #3876:
URL: https://github.com/apache/arrow-adbc/pull/3876#discussion_r2791122120
##
c/driver_manager/adbc_driver_manager.cc:
##
@@ -1600,22 +1951,26 @@ std::optional
InternalAdbcParseDriverUri(std::string_view
std::string_view d = str.substr(0, pos);
if (str.size() <= pos + 1) {
-return ParseDriverUriResult{d, std::nullopt};
+return ParseDriverUriResult{d, std::nullopt, std::nullopt};
}
#ifdef _WIN32
if (std::filesystem::exists(std::filesystem::path(str))) {
// No scheme, just a path
-return ParseDriverUriResult{str, std::nullopt};
+return ParseDriverUriResult{str, std::nullopt, std::nullopt};
}
#endif
if (str[pos + 1] == '/') { // scheme is also driver
-return ParseDriverUriResult{d, str};
+if (d == "profile" && str.size() > pos + 2) {
+ // found a profile URI "profile://"
+ return ParseDriverUriResult{"", std::nullopt, str.substr(pos + 3)};
Review Comment:
not necessarily something we have to do in this PR, but I think this should
become a std::variant (driver+URI or profile)
##
c/include/arrow-adbc/adbc_driver_manager.h:
##
@@ -175,6 +175,183 @@ AdbcStatusCode
AdbcDriverManagerDatabaseSetAdditionalSearchPathList(
ADBC_EXPORT
const char* AdbcStatusCodeMessage(AdbcStatusCode code);
+/// \defgroup adbc-driver-manager-connection-profile Connection Profiles
+/// Similar to odbc.ini, the ADBC driver manager can support "connection
profiles"
Review Comment:
This should also be updated? (If anything I'd like to maybe not mention ODBC
at all in the actual header?)
##
docs/source/format/connection_profiles.rst:
##
@@ -0,0 +1,568 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements. See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership. The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License. You may obtain a copy of the License at
+..
+.. http://www.apache.org/licenses/LICENSE-2.0
+..
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied. See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+==
+Driver Manager Connection Profiles
+==
+
+Overview
+
+
+Similar to ODBC's ``odbc.ini``, the ADBC driver manager supports **connection
profiles**
+that specify a driver and connection options in a reusable configuration. This
allows users to:
+
+- Define connection information in files or environment variables
+- Share connection configurations across applications
+- Distribute standardized connection settings
+- Avoid hardcoding driver names and credentials in application code
+
+Profiles are loaded during ``AdbcDatabaseInit()`` before initializing the
driver. Options
+from the profile are applied automatically but do not override options already
set via ``AdbcDatabaseSetOption()``.
+
+Quick Start
+===
+
+Using a Profile via URI
+---
+
+The simplest way to use a profile is through a URI:
+
+.. code-block:: c
+
+ AdbcDatabase database;
+ AdbcDatabaseNew(&database, &error);
+ AdbcDatabaseSetOption(&database, "uri", "profile://my_snowflake_prod",
&error);
+ AdbcDatabaseInit(&database, &error);
+
+Using a Profile via Option
+---
+
+Alternatively, specify the profile name directly:
+
+.. code-block:: c
+
+ AdbcDatabase database;
+ AdbcDatabaseNew(&database, &error);
+ AdbcDatabaseSetOption(&database, "profile", "my_snowflake_prod", &error);
+ AdbcDatabaseInit(&database, &error);
+
+Profile File Format
+===
+
+Filesystem-based profiles use TOML format with the following structure:
+
+.. code-block:: toml
+
+ version = 1
+ driver = "snowflake"
+
+ [options]
+ # String options
+ adbc.snowflake.sql.account = "mycompany"
+ adbc.snowflake.sql.warehouse = "COMPUTE_WH"
+ adbc.snowflake.sql.database = "PRODUCTION"
+ adbc.snowflake.sql.schema = "PUBLIC"
+
+ # Integer options
+ adbc.snowflake.sql.client_session_keep_alive_heartbeat_frequency = 3600
+
+ # Double options
+ adbc.snowflake.sql.client_timeout = 30.5
+
+ # Boolean options (converted to "true" or "false" strings)
+ adbc.snowflake.sql.client_session_keep_alive = true
+
+version
+---
+
+- **Required**: Yes
+- **Type**: Integer
+- **Supported values**: ``1``
+
+The ``version`` field specifies the profile format version. Currently, only
version 1 is supported.
+This will enable future changes while maintaining backward compatibility.
+
+driver
+--
+
+- **Required**: No
+-
Re: [PR] feat(c/driver_manager): add connection profile interface [arrow-adbc]
zeroshade commented on code in PR #3876:
URL: https://github.com/apache/arrow-adbc/pull/3876#discussion_r2790832468
##
c/driver_manager/adbc_driver_manager.cc:
##
@@ -398,6 +416,81 @@ AdbcStatusCode LoadDriverFromRegistry(HKEY root, const
std::wstring& driver_name
}
#endif // _WIN32
+#define CHECK_STATUS(EXPR)\
+ if (auto _status = (EXPR); _status != ADBC_STATUS_OK) { \
+return _status; \
+ }
+
+AdbcStatusCode ProcessProfileValue(std::string_view value, std::string& out,
+ struct AdbcError* error) {
+ if (value.empty()) {
+SetError(error, "Profile value is null");
+return ADBC_STATUS_INVALID_ARGUMENT;
+ }
+
+ size_t pos = 0;
+ size_t prev_pos = 0;
+ out.resize(0);
+ while ((pos = value.find("env_var(", prev_pos)) != std::string_view::npos) {
+const auto closing_paren = value.find_first_of(')', pos);
+if (closing_paren == std::string_view::npos) {
+ SetError(error, "Malformed env_var() profile value: missing closing
parenthesis");
+ return ADBC_STATUS_INVALID_ARGUMENT;
+}
+
+const auto env_var_start = pos + 8;
+const auto env_var_len = closing_paren - env_var_start;
+if (env_var_len == 0) {
+ SetError(error,
+ "Malformed env_var() profile value: missing environment
variable name");
+ return ADBC_STATUS_INVALID_ARGUMENT;
+}
+
+out.append(value.substr(prev_pos, pos - prev_pos));
+prev_pos = closing_paren + 1;
+
+// Extract the environment variable name from the value
+// which should be formatted as env_var(VAR_NAME) as we confirmed
+// above.
+const auto env_var_name = value.substr(env_var_start, env_var_len);
+#ifdef _WIN32
+auto local_env_var = Utf8Decode(std::string(env_var_name));
+DWORD required_size = GetEnvironmentVariableW(local_env_var.c_str(), NULL,
0);
+if (required_size == 0) {
+ out = "";
+ return ADBC_STATUS_OK;
+}
Review Comment:
I can agree with this, let's follow up in a separate PR for this though as
you suggest.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
Re: [PR] feat(c/driver_manager): add connection profile interface [arrow-adbc]
zeroshade commented on code in PR #3876:
URL: https://github.com/apache/arrow-adbc/pull/3876#discussion_r2790830410
##
c/driver_manager/adbc_driver_manager.cc:
##
@@ -1042,6 +1137,261 @@ struct ManagedLibrary {
#endif // defined(_WIN32)
};
+struct FilesystemProfile {
+ std::filesystem::path path;
+ std::string driver;
+ std::unordered_map options;
+ std::unordered_map int_options;
+ std::unordered_map double_options;
+
+ std::vector options_keys;
+ std::vector options_values;
+
+ std::vector int_option_keys;
+ std::vector int_option_values;
+
+ std::vector double_option_keys;
+ std::vector double_option_values;
+
+ void PopulateConnectionProfile(struct AdbcConnectionProfile* out) {
+options_keys.reserve(options.size());
+options_values.reserve(options.size());
+for (const auto& [key, value] : options) {
+ options_keys.push_back(key.c_str());
+ options_values.push_back(value.c_str());
+}
+
+int_option_keys.reserve(int_options.size());
+int_option_values.reserve(int_options.size());
+for (const auto& [key, value] : int_options) {
+ int_option_keys.push_back(key.c_str());
+ int_option_values.push_back(value);
+}
+
+double_option_keys.reserve(double_options.size());
+double_option_values.reserve(double_options.size());
+for (const auto& [key, value] : double_options) {
+ double_option_keys.push_back(key.c_str());
+ double_option_values.push_back(value);
+}
+
+out->private_data = new FilesystemProfile(std::move(*this));
+out->release = [](AdbcConnectionProfile* profile) {
+ if (!profile || !profile->private_data) {
+return;
+ }
+
+ delete static_cast(profile->private_data);
+ profile->private_data = nullptr;
+ profile->release = nullptr;
+};
+
+out->GetDriverName = [](AdbcConnectionProfile* profile, const char** out,
+struct AdbcError* error) -> AdbcStatusCode {
+ if (!profile || !profile->private_data) {
+SetError(error, "Invalid connection profile");
+return ADBC_STATUS_INVALID_ARGUMENT;
+ }
+
+ auto* fs_profile =
static_cast(profile->private_data);
+ *out = fs_profile->driver.c_str();
+ return ADBC_STATUS_OK;
+};
+
+out->GetOptions = [](AdbcConnectionProfile* profile, const char*** keys,
+ const char*** values, size_t* num_options,
+ struct AdbcError* error) -> AdbcStatusCode {
+ if (!profile || !profile->private_data) {
+SetError(error, "Invalid connection profile");
+return ADBC_STATUS_INVALID_ARGUMENT;
+ }
+
+ if (!keys || !values || !num_options) {
+SetError(error, "Output parameters cannot be null");
+return ADBC_STATUS_INVALID_ARGUMENT;
+ }
+
+ auto* fs_profile =
static_cast(profile->private_data);
+ *num_options = fs_profile->options.size();
+ *keys = fs_profile->options_keys.data();
+ *values = fs_profile->options_values.data();
+ return ADBC_STATUS_OK;
+};
+
+out->GetIntOptions = [](AdbcConnectionProfile* profile, const char*** keys,
+const int64_t** values, size_t* num_options,
+struct AdbcError* error) -> AdbcStatusCode {
+ if (!profile || !profile->private_data) {
+SetError(error, "Invalid connection profile");
+return ADBC_STATUS_INVALID_ARGUMENT;
+ }
+
+ if (!keys || !values || !num_options) {
+SetError(error, "Output parameters cannot be null");
+return ADBC_STATUS_INVALID_ARGUMENT;
+ }
+
+ auto* fs_profile =
static_cast(profile->private_data);
+ *num_options = fs_profile->int_options.size();
+ *keys = fs_profile->int_option_keys.data();
+ *values = fs_profile->int_option_values.data();
+ return ADBC_STATUS_OK;
+};
+
+out->GetDoubleOptions = [](AdbcConnectionProfile* profile, const char***
keys,
+ const double** values, size_t* num_options,
+ struct AdbcError* error) -> AdbcStatusCode {
+ if (!profile || !profile->private_data) {
+SetError(error, "Invalid connection profile");
+return ADBC_STATUS_INVALID_ARGUMENT;
+ }
+
+ if (!keys || !values || !num_options) {
+SetError(error, "Output parameters cannot be null");
+return ADBC_STATUS_INVALID_ARGUMENT;
+ }
+
+ auto* fs_profile =
static_cast(profile->private_data);
+ *num_options = fs_profile->double_options.size();
+ *keys = fs_profile->double_option_keys.data();
+ *values = fs_profile->double_option_values.data();
+ return ADBC_STATUS_OK;
+};
+ }
+};
+
+struct ProfileVisitor {
+ FilesystemProfile& profile;
+ const std::filesystem::path& profile_path;
+ struct AdbcError* error;
+
+ bool VisitTable(const std::string& prefix, toml::table& table) {
+for (const auto& [key, value] : table) {
Re: [PR] feat(c/driver_manager): add connection profile interface [arrow-adbc]
zeroshade commented on code in PR #3876:
URL: https://github.com/apache/arrow-adbc/pull/3876#discussion_r2790826808
##
docs/source/format/connection_profiles.rst:
##
@@ -0,0 +1,568 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements. See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership. The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License. You may obtain a copy of the License at
+..
+.. http://www.apache.org/licenses/LICENSE-2.0
+..
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied. See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+==
+Driver Manager Connection Profiles
+==
+
+Overview
+
+
+Similar to ODBC's ``odbc.ini``, the ADBC driver manager supports **connection
profiles**
+that specify a driver and connection options in a reusable configuration. This
allows users to:
+
+- Define connection information in files or environment variables
+- Share connection configurations across applications
+- Distribute standardized connection settings
+- Avoid hardcoding driver names and credentials in application code
+
+Profiles are loaded during ``AdbcDatabaseInit()`` before initializing the
driver. Options
+from the profile are applied automatically but do not override options already
set via ``AdbcDatabaseSetOption()``.
+
+Quick Start
+===
+
+Using a Profile via URI
+---
+
+The simplest way to use a profile is through a URI:
+
+.. code-block:: c
+
+ AdbcDatabase database;
+ AdbcDatabaseNew(&database, &error);
+ AdbcDatabaseSetOption(&database, "uri", "profile://my_snowflake_prod",
&error);
+ AdbcDatabaseInit(&database, &error);
+
+Using a Profile via Option
+---
+
+Alternatively, specify the profile name directly:
+
+.. code-block:: c
+
+ AdbcDatabase database;
+ AdbcDatabaseNew(&database, &error);
+ AdbcDatabaseSetOption(&database, "profile", "my_snowflake_prod", &error);
+ AdbcDatabaseInit(&database, &error);
+
+Profile File Format
+===
+
+Filesystem-based profiles use TOML format with the following structure:
+
+.. code-block:: toml
+
+ version = 1
+ driver = "snowflake"
+
+ [options]
+ # String options
+ adbc.snowflake.sql.account = "mycompany"
+ adbc.snowflake.sql.warehouse = "COMPUTE_WH"
+ adbc.snowflake.sql.database = "PRODUCTION"
+ adbc.snowflake.sql.schema = "PUBLIC"
+
+ # Integer options
+ adbc.snowflake.sql.client_session_keep_alive_heartbeat_frequency = 3600
+
+ # Double options
+ adbc.snowflake.sql.client_timeout = 30.5
+
+ # Boolean options (converted to "true" or "false" strings)
+ adbc.snowflake.sql.client_session_keep_alive = true
+
+version
+---
+
+- **Required**: Yes
+- **Type**: Integer
+- **Supported values**: ``1``
+
+The ``version`` field specifies the profile format version. Currently, only
version 1 is supported.
+This will enable future changes while maintaining backward compatibility.
+
+driver
+--
+
+- **Required**: No
+- **Type**: String
+
+The ``driver`` field specifies which ADBC driver to load. This can be:
+
+- A driver name (e.g., ``"snowflake"``)
+- A path to a shared library (e.g.,
``"/usr/local/lib/libadbc_driver_snowflake.so"``)
+- A path to a driver manifest (e.g., ``"/etc/adbc/drivers/snowflake.toml"``)
+
+If omitted, the driver must be specified through other means (e.g., the
``driver`` option or ``uri`` parameter).
+The driver will be loaded identically to if it was specified via
``AdbcDatabaseSetOption("driver", "")``.
+For more detils, see :doc:`driver_manifests`.
+
+Options Section
+---
+
+The ``[options]`` section contains driver-specific configuration options.
Options can be of the following types:
+
+**String values**
+ Applied using ``AdbcDatabaseSetOption()``
+
+ .. code-block:: toml
+
+ adbc.snowflake.sql.account = "mycompany"
+ adbc.snowflake.sql.warehouse = "COMPUTE_WH"
+
+**Integer values**
+ Applied using ``AdbcDatabaseSetOptionInt()``
+
+ .. code-block:: toml
+
+ adbc.snowflake.sql.client_session_keep_alive_heartbeat_frequency = 3600
+
+**Double values**
+ Applied using ``AdbcDatabaseSetOptionDouble()``
+
+ .. code-block:: toml
+
+ adbc.snowflake.sql.client_timeout = 30.5
+
+**Boolean values**
+ Converted to strings ``"true"`` or ``"false"`` and applied using
``AdbcDatabaseSetOption()``
+
+ .. code-block:: toml
+
+ adbc.snowflake.sql.client_session_keep_alive = true
+
+Environment Variable Substitution
+--
Re: [PR] feat(c/driver_manager): add connection profile interface [arrow-adbc]
zeroshade commented on code in PR #3876:
URL: https://github.com/apache/arrow-adbc/pull/3876#discussion_r2790822449
##
c/include/arrow-adbc/adbc_driver_manager.h:
##
@@ -175,6 +175,171 @@ AdbcStatusCode
AdbcDriverManagerDatabaseSetAdditionalSearchPathList(
ADBC_EXPORT
const char* AdbcStatusCodeMessage(AdbcStatusCode code);
+/// \defgroup adbc-driver-manager-connection-profile Connection Profiles
+/// Similar to odbc.ini, the ADBC driver manager can support "connection
profiles"
+/// that specify a driver and options to use when connecting. This allows
users to
+/// specify connection information in a file or environment variable, and have
the
+/// driver manager load the appropriate driver and set options accordingly.
+///
+/// This allows creating reusable connection configurations for sharing and
distribution
+/// without needing to hardcode driver names and options in application code.
Profiles
+/// will be loaded during DatabaseInit before attempting to initialize the
driver. Any
+/// options specified by the profile will be applied but will not override
options
+/// that have already been set using DatabaseSetOption.
+///
+/// To facilitate customization, we define an interface for implementing a
Connection
+/// Profile object along with a provider function definition which can be set
into
+/// the driver manager to allow for customized profile loading.
+///
+/// A profile can be specified to the Driver Manager in one of two ways,
+/// which will invoke the profile provider during the call to DatabaseInit:
+///
+/// 1. The "profile" option can be set using DatabaseSetOption with the name
of the
+/// profile to load.
+/// 2. The "uri" being used can have the form "profile://"
+///
+/// @{
+
+/// \brief Abstract interface for connection profile providers
+struct ADBC_EXPORT AdbcConnectionProfile {
+ /// \brief Opaque implementation-defined state.
+ /// This field is NULL if the profile is uninitialized/freed (but
+ /// it need not have a value even if the profile is initialized).
+ void* private_data;
+
+ /// \brief Release the profile and perform any cleanup.
+ void (*release)(struct AdbcConnectionProfile* profile);
+
+ /// \brief Get the driver to use as specified by this profile.
+ ///
+ /// It is not required that a profile specify a driver. If the options
+ // can be reusable across drivers, then the profile does not need to specify
+ /// a driver (if this provides an empty string or nullptr then the driver
+ /// must be defined by other means, e.g. by the driver / uri options).
+ ///
+ /// \param[in] profile The profile to query.
+ /// \param[out] driver_name The name of the driver to use, or NULL if not
specified.
+ /// \param[out] error An optional location to return an error message
+ AdbcStatusCode (*GetDriverName)(struct AdbcConnectionProfile* profile,
+ const char** driver_name, struct AdbcError*
error);
Review Comment:
Nothing currently prevents a user from setting the `init_func` already and
handle profiles in their own way though. I'd prefer for the Profile object be
pretty well tied to what a profile could actually contain and there isn't a way
to define/specify the `init_func` from inside of the toml file currently. So
I'm not sure about this (or maybe I'm misunderstanding you)
@lidavidm thoughts?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
Re: [PR] feat(c/driver_manager): add connection profile interface [arrow-adbc]
paleolimbot commented on code in PR #3876:
URL: https://github.com/apache/arrow-adbc/pull/3876#discussion_r2774963821
##
c/driver_manager/adbc_driver_manager.cc:
##
@@ -398,6 +416,81 @@ AdbcStatusCode LoadDriverFromRegistry(HKEY root, const
std::wstring& driver_name
}
#endif // _WIN32
+#define CHECK_STATUS(EXPR)\
+ if (auto _status = (EXPR); _status != ADBC_STATUS_OK) { \
+return _status; \
+ }
+
+AdbcStatusCode ProcessProfileValue(std::string_view value, std::string& out,
+ struct AdbcError* error) {
+ if (value.empty()) {
+SetError(error, "Profile value is null");
+return ADBC_STATUS_INVALID_ARGUMENT;
+ }
+
+ size_t pos = 0;
+ size_t prev_pos = 0;
+ out.resize(0);
+ while ((pos = value.find("env_var(", prev_pos)) != std::string_view::npos) {
+const auto closing_paren = value.find_first_of(')', pos);
+if (closing_paren == std::string_view::npos) {
+ SetError(error, "Malformed env_var() profile value: missing closing
parenthesis");
+ return ADBC_STATUS_INVALID_ARGUMENT;
+}
+
+const auto env_var_start = pos + 8;
+const auto env_var_len = closing_paren - env_var_start;
+if (env_var_len == 0) {
+ SetError(error,
+ "Malformed env_var() profile value: missing environment
variable name");
+ return ADBC_STATUS_INVALID_ARGUMENT;
+}
+
+out.append(value.substr(prev_pos, pos - prev_pos));
+prev_pos = closing_paren + 1;
+
+// Extract the environment variable name from the value
+// which should be formatted as env_var(VAR_NAME) as we confirmed
+// above.
+const auto env_var_name = value.substr(env_var_start, env_var_len);
+#ifdef _WIN32
+auto local_env_var = Utf8Decode(std::string(env_var_name));
+DWORD required_size = GetEnvironmentVariableW(local_env_var.c_str(), NULL,
0);
+if (required_size == 0) {
+ out = "";
+ return ADBC_STATUS_OK;
+}
Review Comment:
I see why this is important (to avoid putting secrets in .toml files), but
there is now quite a bit of environment variable state that is required to do
certain things (like prepend the driver search path with a directory of
preferred drivers if I remember the order correctly), which means the chance of
an application concurrently setting and the driver manager fetching an
environment variable is non-trivial in the presence of something like a
connection pool. Not in this PR, but it may be worth documenting how to create
connections in a thread-safe way and/or ensuring there is a thread safe
alternative to accomplish some of these things.
##
c/include/arrow-adbc/adbc_driver_manager.h:
##
@@ -175,6 +175,171 @@ AdbcStatusCode
AdbcDriverManagerDatabaseSetAdditionalSearchPathList(
ADBC_EXPORT
const char* AdbcStatusCodeMessage(AdbcStatusCode code);
+/// \defgroup adbc-driver-manager-connection-profile Connection Profiles
+/// Similar to odbc.ini, the ADBC driver manager can support "connection
profiles"
+/// that specify a driver and options to use when connecting. This allows
users to
+/// specify connection information in a file or environment variable, and have
the
+/// driver manager load the appropriate driver and set options accordingly.
+///
+/// This allows creating reusable connection configurations for sharing and
distribution
+/// without needing to hardcode driver names and options in application code.
Profiles
+/// will be loaded during DatabaseInit before attempting to initialize the
driver. Any
+/// options specified by the profile will be applied but will not override
options
+/// that have already been set using DatabaseSetOption.
+///
+/// To facilitate customization, we define an interface for implementing a
Connection
+/// Profile object along with a provider function definition which can be set
into
+/// the driver manager to allow for customized profile loading.
+///
+/// A profile can be specified to the Driver Manager in one of two ways,
+/// which will invoke the profile provider during the call to DatabaseInit:
+///
+/// 1. The "profile" option can be set using DatabaseSetOption with the name
of the
+/// profile to load.
+/// 2. The "uri" being used can have the form "profile://"
+///
+/// @{
+
+/// \brief Abstract interface for connection profile providers
+struct ADBC_EXPORT AdbcConnectionProfile {
+ /// \brief Opaque implementation-defined state.
+ /// This field is NULL if the profile is uninitialized/freed (but
+ /// it need not have a value even if the profile is initialized).
+ void* private_data;
+
+ /// \brief Release the profile and perform any cleanup.
+ void (*release)(struct AdbcConnectionProfile* profile);
+
+ /// \brief Get the driver to use as specified by this profile.
+ ///
+ /// It is not required that a profile specify a driver. If the options
+ // can be reusable across drivers, then the profile does not need
Re: [PR] feat(c/driver_manager): add connection profile interface [arrow-adbc]
zeroshade commented on code in PR #3876:
URL: https://github.com/apache/arrow-adbc/pull/3876#discussion_r2729193598
##
c/driver_manager/adbc_driver_manager.cc:
##
@@ -1857,20 +2204,188 @@ AdbcStatusCode
AdbcDriverManagerDatabaseSetInitFunc(struct AdbcDatabase* databas
return ADBC_STATUS_OK;
}
+AdbcStatusCode AdbcFilesystemProfileProvider(const char* profile_name,
+ const char*
additional_search_path_list,
+ struct AdbcConnectionProfile* out,
+ struct AdbcError* error) {
+ if (profile_name == nullptr || strlen(profile_name) == 0) {
+SetError(error, "Profile name is empty");
+return ADBC_STATUS_INVALID_ARGUMENT;
+ }
+
+ if (!out) {
+SetError(error, "Output profile is null");
+return ADBC_STATUS_INVALID_ARGUMENT;
+ }
+
+ std::memset(out, 0, sizeof(*out));
+ std::filesystem::path profile_path(profile_name);
+ if (profile_path.has_extension()) {
+if (HasExtension(profile_path, ".toml")) {
+ if (!std::filesystem::exists(profile_path)) {
+SetError(error, "Profile file does not exist: " +
profile_path.string());
+return ADBC_STATUS_NOT_FOUND;
+ }
+
+ FilesystemProfile profile;
+ CHECK_STATUS(LoadProfileFile(profile_path, profile, error));
+ FilesystemProfile::populate_connection_profile(std::move(profile), out);
+ return ADBC_STATUS_OK;
+}
+ }
+
+ if (profile_path.is_absolute()) {
+profile_path.replace_extension(".toml");
+
+FilesystemProfile profile;
+CHECK_STATUS(LoadProfileFile(profile_path, profile, error));
+FilesystemProfile::populate_connection_profile(std::move(profile), out);
+return ADBC_STATUS_OK;
+ }
+
+ SearchPaths search_paths =
GetProfileSearchPaths(additional_search_path_list);
+ SearchPaths extra_debug_info;
+ for (const auto& [source, search_path] : search_paths) {
+if (source == SearchPathSource::kRegistry || source ==
SearchPathSource::kUnset ||
+source == SearchPathSource::kDoesNotExist ||
+source == SearchPathSource::kDisabledAtCompileTime ||
+source == SearchPathSource::kDisabledAtRunTime ||
+source == SearchPathSource::kOtherError) {
+ continue;
+}
+
+std::filesystem::path full_path = search_path / profile_path;
+full_path.replace_extension(".toml");
+if (std::filesystem::exists(full_path)) {
+ OwnedError intermediate_error;
+
+ FilesystemProfile profile;
+ auto status = LoadProfileFile(full_path, profile,
&intermediate_error.error);
+ if (status == ADBC_STATUS_OK) {
+FilesystemProfile::populate_connection_profile(std::move(profile),
out);
+return ADBC_STATUS_OK;
+ } else if (status == ADBC_STATUS_INVALID_ARGUMENT) {
+search_paths.insert(search_paths.end(), extra_debug_info.begin(),
+extra_debug_info.end());
+if (intermediate_error.error.message) {
+ std::string error_message = intermediate_error.error.message;
+ AddSearchPathsToError(search_paths, SearchPathType::kProfile,
error_message);
+ SetError(error, std::move(error_message));
+}
+return status;
+ }
+
+ std::string message = "found ";
+ message += full_path.string();
+ message += " but: ";
+ if (intermediate_error.error.message) {
+message += intermediate_error.error.message;
+ } else {
+message += "could not load the profile";
+ }
+ extra_debug_info.emplace_back(SearchPathSource::kOtherError,
std::move(message));
+}
+ }
+
+ search_paths.insert(search_paths.end(), extra_debug_info.begin(),
+ extra_debug_info.end());
+ std::string error_message = "Profile not found: " +
std::string(profile_name);
+ AddSearchPathsToError(search_paths, SearchPathType::kProfile, error_message);
+ SetError(error, std::move(error_message));
+ return ADBC_STATUS_NOT_FOUND;
+}
+
+struct ProfileGuard {
Review Comment:
updated
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
Re: [PR] feat(c/driver_manager): add connection profile interface [arrow-adbc]
zeroshade commented on code in PR #3876:
URL: https://github.com/apache/arrow-adbc/pull/3876#discussion_r2729162809
##
c/driver_manager/adbc_driver_manager.cc:
##
@@ -398,6 +416,59 @@ AdbcStatusCode LoadDriverFromRegistry(HKEY root, const
std::wstring& driver_name
}
#endif // _WIN32
+#define CHECK_STATUS(EXPR)\
+ if (auto _status = (EXPR); _status != ADBC_STATUS_OK) { \
+return _status; \
+ }
+
+AdbcStatusCode ProcessProfileValue(std::string_view value, std::string& out,
+ struct AdbcError* error) {
+ if (value.empty()) {
+SetError(error, "Profile value is null");
+return ADBC_STATUS_INVALID_ARGUMENT;
+ }
+
+ const auto pos = value.find("env_var(");
+ if (pos == std::string_view::npos || pos != 0) {
Review Comment:
good point. fixing this
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
Re: [PR] feat(c/driver_manager): add connection profile interface [arrow-adbc]
lidavidm commented on code in PR #3876:
URL: https://github.com/apache/arrow-adbc/pull/3876#discussion_r2726409474
##
c/driver_manager/adbc_driver_manager.cc:
##
@@ -1042,6 +1115,262 @@ struct ManagedLibrary {
#endif // defined(_WIN32)
};
+struct FilesystemProfile {
+ std::filesystem::path path;
+ std::string driver;
+ std::unordered_map options;
+ std::unordered_map int_options;
+ std::unordered_map double_options;
+
+ std::vector options_keys;
+ std::vector options_values;
+
+ std::vector int_option_keys;
+ std::vector int_option_values;
+
+ std::vector double_option_keys;
+ std::vector double_option_values;
+
+ static void populate_connection_profile(FilesystemProfile&& profile,
+ struct AdbcConnectionProfile* out) {
+profile.options_keys.reserve(profile.options.size());
+profile.options_values.reserve(profile.options.size());
+for (const auto& [key, value] : profile.options) {
+ profile.options_keys.push_back(key.c_str());
+ profile.options_values.push_back(value.c_str());
+}
+
+profile.int_option_keys.reserve(profile.int_options.size());
+profile.int_option_values.reserve(profile.int_options.size());
+for (const auto& [key, value] : profile.int_options) {
+ profile.int_option_keys.push_back(key.c_str());
+ profile.int_option_values.push_back(value);
+}
+
+profile.double_option_keys.reserve(profile.double_options.size());
+profile.double_option_values.reserve(profile.double_options.size());
+for (const auto& [key, value] : profile.double_options) {
+ profile.double_option_keys.push_back(key.c_str());
+ profile.double_option_values.push_back(value);
+}
+
+out->private_data = new FilesystemProfile(std::move(profile));
+out->release = [](AdbcConnectionProfile* profile) {
+ if (!profile || !profile->private_data) {
+return;
+ }
+
+ delete static_cast(profile->private_data);
+ profile->private_data = nullptr;
+ profile->release = nullptr;
+};
+
+out->GetDriverName = [](AdbcConnectionProfile* profile, const char** out,
+struct AdbcError* error) -> AdbcStatusCode {
+ if (!profile || !profile->private_data) {
+SetError(error, "Invalid connection profile");
+return ADBC_STATUS_INVALID_ARGUMENT;
+ }
+
+ auto* fs_profile =
static_cast(profile->private_data);
+ *out = fs_profile->driver.c_str();
+ return ADBC_STATUS_OK;
+};
+
+out->GetOptions = [](AdbcConnectionProfile* profile, const char*** keys,
+ const char*** values, size_t* num_options,
+ struct AdbcError* error) -> AdbcStatusCode {
+ if (!profile || !profile->private_data) {
+SetError(error, "Invalid connection profile");
+return ADBC_STATUS_INVALID_ARGUMENT;
+ }
+
+ if (!keys || !values || !num_options) {
+SetError(error, "Output parameters cannot be null");
+return ADBC_STATUS_INVALID_ARGUMENT;
+ }
+
+ auto* fs_profile =
static_cast(profile->private_data);
+ *num_options = fs_profile->options.size();
+ *keys = fs_profile->options_keys.data();
+ *values = fs_profile->options_values.data();
+ return ADBC_STATUS_OK;
+};
+
+out->GetIntOptions = [](AdbcConnectionProfile* profile, const char*** keys,
+const int64_t** values, size_t* num_options,
+struct AdbcError* error) -> AdbcStatusCode {
+ if (!profile || !profile->private_data) {
+SetError(error, "Invalid connection profile");
+return ADBC_STATUS_INVALID_ARGUMENT;
+ }
+
+ if (!keys || !values || !num_options) {
+SetError(error, "Output parameters cannot be null");
+return ADBC_STATUS_INVALID_ARGUMENT;
+ }
+
+ auto* fs_profile =
static_cast(profile->private_data);
+ *num_options = fs_profile->int_options.size();
+ *keys = fs_profile->int_option_keys.data();
+ *values = fs_profile->int_option_values.data();
+ return ADBC_STATUS_OK;
+};
+
+out->GetDoubleOptions = [](AdbcConnectionProfile* profile, const char***
keys,
+ const double** values, size_t* num_options,
+ struct AdbcError* error) -> AdbcStatusCode {
+ if (!profile || !profile->private_data) {
+SetError(error, "Invalid connection profile");
+return ADBC_STATUS_INVALID_ARGUMENT;
+ }
+
+ if (!keys || !values || !num_options) {
+SetError(error, "Output parameters cannot be null");
+return ADBC_STATUS_INVALID_ARGUMENT;
+ }
+
+ auto* fs_profile =
static_cast(profile->private_data);
+ *num_options = fs_profile->double_options.size();
+ *keys = fs_profile->double_option_keys.data();
+ *values = fs_profile->double_option_values.data();
+ return ADBC_STATUS_OK;
+};
+ }
+}
Re: [PR] feat(c/driver_manager): add connection profile interface [arrow-adbc]
lidavidm commented on code in PR #3876:
URL: https://github.com/apache/arrow-adbc/pull/3876#discussion_r2715834850
##
c/include/arrow-adbc/adbc_driver_manager.h:
##
@@ -175,6 +175,183 @@ AdbcStatusCode
AdbcDriverManagerDatabaseSetAdditionalSearchPathList(
ADBC_EXPORT
const char* AdbcStatusCodeMessage(AdbcStatusCode code);
+/// \defgroup adbc-driver-manager-connection-profile Connection Profiles
+/// Similar to odbc.ini, the ADBC driver manager can support "connection
profiles"
+/// that specify a driver and options to use when connecting. This allows
users to
+/// specify connection information in a file or environment variable, and have
the
+/// driver manager load the appropriate driver and set options accordingly.
+///
+/// This allows creating reusable connection configurations for sharing and
distribution
+/// without needing to hardcode driver names and options in application code.
Profiles
+/// will be loaded during DatabaseInit before attempting to initialize the
driver. Any
+/// options specified by the profile will be applied but will not override
options
+/// that have already been set using DatabaseSetOption.
+///
+/// To facilitate customization, we define an interface for implementing a
Connection
+/// Profile object along with a provider function definition which can be set
into
+/// the driver manager to allow for customized profile loading.
+///
+/// A profile can be specified to the Driver Manager in one of two ways,
+/// which will invoke the profile provider during the call to DatabaseInit:
+///
+/// 1. The "profile" option can be set using DatabaseSetOption with the name
of the
+/// profile to load.
+/// 2. The "uri" being used can have the form "profile://"
+///
+/// @{
+
+/// \brief Abstract interface for connection profile providers
+struct ADBC_EXPORT AdbcConnectionProfile {
+ /// \brief Opaque implementation-defined state.
+ /// This field is NULL if the profile is uninitialized/freed (but
+ /// it need not have a value even if the profile is initialized).
+ void* private_data;
+
+ /// \brief Release the profile and perform any cleanup.
+ void (*release)(struct AdbcConnectionProfile* profile);
+
+ /// \brief Get the driver to use as specified by this profile.
+ ///
+ /// It is not required that a profile specify a driver. If the options
+ // can be reusable across drivers, then the profile does not need to specify
+ /// a driver (if this provides an empty string or nullptr then the driver
+ /// must be defined by other means, e.g. by the driver / uri options).
+ ///
+ /// \param[in] profile The profile to query.
+ /// \param[out] driver_name The name of the driver to use, or NULL if not
specified.
+ /// \param[out] error An optional location to return an error message
+ AdbcStatusCode (*GetDriverName)(struct AdbcConnectionProfile* profile,
+ const char** driver_name, struct AdbcError*
error);
+
+ /// \brief Get the string options specified by the profile
+ ///
+ /// The keys and values returned by this function are owned by the profile
+ /// object itself and do not need to be freed or managed by the caller.
+ /// They must not be accessed after calling release on the profile.
+ ///
+ /// The profile can also indicate that a value should be pulled from the
environment
+ /// by having a value in the form `env_var(ENV_VAR_NAME)`. If the driver
+ /// manager encounters a value of this form, it will replace it with the
actual value
+ /// of the environment variable `ENV_VAR_NAME` before setting the option.
This
+ /// is only valid for option *values* not *keys*.
+ ///
+ /// \param[in] profile The profile to query.
+ /// \param[out] keys The keys of the options specified by the profile.
+ /// \param[out] values The values of the options specified by the profile.
+ /// \param[out] num_options The number of options specified by the profile,
+ /// consumers must not access keys or values beyond this count.
+ /// \param[out] error An optional location to return an error message
+ AdbcStatusCode (*GetOptions)(struct AdbcConnectionProfile* profile, const
char*** keys,
+ const char*** values, size_t* num_options,
+ struct AdbcError* error);
+
+ /// \brief Get the integer options specified by the profile
+ ///
+ /// The keys and values returned by this function are owned by the profile
+ /// object itself and do not need to be freed or managed by the caller. They
must not be
+ /// accessed after calling release on the profile.
+ ///
+ /// Values returned by this function will be set using the
DatabaseSetOptionInt function
+ /// on the database object being initialized. If the driver does not support
the
+ /// DatabaseSetOptionInt function, then options should only be returned as
strings.
+ ///
+ /// \param[in] profile The profile to query.
+ /// \param[out] keys The keys of the options specified by the profil
Re: [PR] feat(c/driver_manager): add connection profile interface [arrow-adbc]
felipecrv commented on code in PR #3876:
URL: https://github.com/apache/arrow-adbc/pull/3876#discussion_r2715166689
##
docs/source/format/connection_profiles.rst:
##
@@ -0,0 +1,568 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements. See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership. The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License. You may obtain a copy of the License at
+..
+.. http://www.apache.org/licenses/LICENSE-2.0
+..
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied. See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+==
+Driver Manager Connection Profiles
+==
+
+Overview
+
+
+Similar to ODBC's ``odbc.ini``, the ADBC driver manager supports **connection
profiles**
+that specify a driver and connection options in a reusable configuration. This
allows users to:
+
+- Define connection information in files or environment variables
+- Share connection configurations across applications
+- Distribute standardized connection settings
+- Avoid hardcoding driver names and credentials in application code
+
+Profiles are loaded during ``AdbcDatabaseInit()`` before initializing the
driver. Options
+from the profile are applied automatically but do not override options already
set via ``AdbcDatabaseSetOption()``.
+
+Quick Start
+===
+
+Using a Profile via URI
+---
+
+The simplest way to use a profile is through a URI:
+
+.. code-block:: c
+
+ AdbcDatabase database;
+ AdbcDatabaseNew(&database, &error);
+ AdbcDatabaseSetOption(&database, "uri", "profile://my_snowflake_prod",
&error);
+ AdbcDatabaseInit(&database, &error);
+
+Using a Profile via Option
+---
+
+Alternatively, specify the profile name directly:
+
+.. code-block:: c
+
+ AdbcDatabase database;
+ AdbcDatabaseNew(&database, &error);
+ AdbcDatabaseSetOption(&database, "profile", "my_snowflake_prod", &error);
+ AdbcDatabaseInit(&database, &error);
+
+Profile File Format
+===
+
+Filesystem-based profiles use TOML format with the following structure:
+
+.. code-block:: toml
+
+ version = 1
+ driver = "snowflake"
+
+ [options]
+ # String options
+ adbc.snowflake.sql.account = "mycompany"
+ adbc.snowflake.sql.warehouse = "COMPUTE_WH"
+ adbc.snowflake.sql.database = "PRODUCTION"
+ adbc.snowflake.sql.schema = "PUBLIC"
+
+ # Integer options
+ adbc.snowflake.sql.client_session_keep_alive_heartbeat_frequency = 3600
+
+ # Double options
+ adbc.snowflake.sql.client_timeout = 30.5
+
+ # Boolean options (converted to "true" or "false" strings)
+ adbc.snowflake.sql.client_session_keep_alive = true
+
+version
+---
+
+- **Required**: Yes
+- **Type**: Integer
+- **Supported values**: ``1``
+
+The ``version`` field specifies the profile format version. Currently, only
version 1 is supported.
+This will enable future changes while maintaining backward compatibility.
+
+driver
+--
+
+- **Required**: No
+- **Type**: String
+
+The ``driver`` field specifies which ADBC driver to load. This can be:
+
+- A driver name (e.g., ``"snowflake"``)
+- A path to a shared library (e.g.,
``"/usr/local/lib/libadbc_driver_snowflake.so"``)
+- A path to a driver manifest (e.g., ``"/etc/adbc/drivers/snowflake.toml"``)
+
+If omitted, the driver must be specified through other means (e.g., the
``driver`` option or ``uri`` parameter).
+The driver will be loaded identically to if it was specified via
``AdbcDatabaseSetOption("driver", "")``.
+For more detils, see :doc:`driver_manifests`.
+
+Options Section
+---
+
+The ``[options]`` section contains driver-specific configuration options.
Options can be of the following types:
+
+**String values**
+ Applied using ``AdbcDatabaseSetOption()``
+
+ .. code-block:: toml
+
+ adbc.snowflake.sql.account = "mycompany"
+ adbc.snowflake.sql.warehouse = "COMPUTE_WH"
+
+**Integer values**
+ Applied using ``AdbcDatabaseSetOptionInt()``
+
+ .. code-block:: toml
+
+ adbc.snowflake.sql.client_session_keep_alive_heartbeat_frequency = 3600
+
+**Double values**
+ Applied using ``AdbcDatabaseSetOptionDouble()``
+
+ .. code-block:: toml
+
+ adbc.snowflake.sql.client_timeout = 30.5
+
+**Boolean values**
+ Converted to strings ``"true"`` or ``"false"`` and applied using
``AdbcDatabaseSetOption()``
+
+ .. code-block:: toml
+
+ adbc.snowflake.sql.client_session_keep_alive = true
+
+Environment Variable Substitution
+--
Re: [PR] feat(c/driver_manager): add connection profile interface [arrow-adbc]
davidhcoe commented on PR #3876: URL: https://github.com/apache/arrow-adbc/pull/3876#issuecomment-3775550779 fyi @jadewang-db and @eric-wang-1990 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [PR] feat(c/driver_manager): add connection profile interface [arrow-adbc]
zeroshade commented on PR #3876: URL: https://github.com/apache/arrow-adbc/pull/3876#issuecomment-3770088541 I've updated the comments and the rst document based on everyone's comments. Please re-review -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [PR] feat(c/driver_manager): add connection profile interface [arrow-adbc]
zeroshade commented on code in PR #3876: URL: https://github.com/apache/arrow-adbc/pull/3876#discussion_r2706046527 ## docs/source/format/connection_profiles.rst: ## @@ -0,0 +1,568 @@ +.. Licensed to the Apache Software Foundation (ASF) under one +.. or more contributor license agreements. See the NOTICE file +.. distributed with this work for additional information +.. regarding copyright ownership. The ASF licenses this file +.. to you under the Apache License, Version 2.0 (the +.. "License"); you may not use this file except in compliance +.. with the License. You may obtain a copy of the License at +.. +.. http://www.apache.org/licenses/LICENSE-2.0 +.. +.. Unless required by applicable law or agreed to in writing, +.. software distributed under the License is distributed on an +.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +.. KIND, either express or implied. See the License for the +.. specific language governing permissions and limitations +.. under the License. + +== +Driver Manager Connection Profiles +== + +Overview + + +Similar to ODBC's ``odbc.ini``, the ADBC driver manager supports **connection profiles** Review Comment: Updated the rst document. Let me know what you think -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [PR] feat(c/driver_manager): add connection profile interface [arrow-adbc]
zeroshade commented on code in PR #3876:
URL: https://github.com/apache/arrow-adbc/pull/3876#discussion_r2706044351
##
c/include/arrow-adbc/adbc_driver_manager.h:
##
@@ -175,6 +175,183 @@ AdbcStatusCode
AdbcDriverManagerDatabaseSetAdditionalSearchPathList(
ADBC_EXPORT
const char* AdbcStatusCodeMessage(AdbcStatusCode code);
+/// \defgroup adbc-driver-manager-connection-profile Connection Profiles
+/// Similar to odbc.ini, the ADBC driver manager can support "connection
profiles"
+/// that specify a driver and options to use when connecting. This allows
users to
+/// specify connection information in a file or environment variable, and have
the
+/// driver manager load the appropriate driver and set options accordingly.
+///
+/// This allows creating reusable connection configurations for sharing and
distribution
+/// without needing to hardcode driver names and options in application code.
Profiles
+/// will be loaded during DatabaseInit before attempting to initialize the
driver. Any
+/// options specified by the profile will be applied but will not override
options
+/// that have already been set using DatabaseSetOption.
+///
+/// To facilitate customization, we define an interface for implementing a
Connection
+/// Profile object along with a provider function definition which can be set
into
+/// the driver manager to allow for customized profile loading.
+///
+/// A profile can be specified to the Driver Manager in one of two ways,
+/// which will invoke the profile provider during the call to DatabaseInit:
+///
+/// 1. The "profile" option can be set using DatabaseSetOption with the name
of the
+/// profile to load.
+/// 2. The "uri" being used can have the form "profile://"
+///
+/// @{
+
+/// \brief Abstract interface for connection profile providers
+struct ADBC_EXPORT AdbcConnectionProfile {
+ /// \brief Opaque implementation-defined state.
+ /// This field is NULL if the profile is uninitialized/freed (but
+ /// it need not have a value even if the profile is initialized).
+ void* private_data;
+
+ /// \brief Release the profile and perform any cleanup.
+ void (*release)(struct AdbcConnectionProfile* profile);
Review Comment:
I've added a note to the docstring on this so that it's clear the driver
manager will call it
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
Re: [PR] feat(c/driver_manager): add connection profile interface [arrow-adbc]
lidavidm commented on code in PR #3876: URL: https://github.com/apache/arrow-adbc/pull/3876#discussion_r2702858420 ## docs/source/format/connection_profiles.rst: ## @@ -0,0 +1,568 @@ +.. Licensed to the Apache Software Foundation (ASF) under one +.. or more contributor license agreements. See the NOTICE file +.. distributed with this work for additional information +.. regarding copyright ownership. The ASF licenses this file +.. to you under the Apache License, Version 2.0 (the +.. "License"); you may not use this file except in compliance +.. with the License. You may obtain a copy of the License at +.. +.. http://www.apache.org/licenses/LICENSE-2.0 +.. +.. Unless required by applicable law or agreed to in writing, +.. software distributed under the License is distributed on an +.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +.. KIND, either express or implied. See the License for the +.. specific language governing permissions and limitations +.. under the License. + +== +Driver Manager Connection Profiles +== + +Overview + + +Similar to ODBC's ``odbc.ini``, the ADBC driver manager supports **connection profiles** +that specify a driver and connection options in a reusable configuration. This allows users to: + +- Define connection information in files or environment variables +- Share connection configurations across applications +- Distribute standardized connection settings +- Avoid hardcoding driver names and credentials in application code + +Profiles are loaded during ``AdbcDatabaseInit()`` before initializing the driver. Options +from the profile are applied automatically but do not override options already set via ``AdbcDatabaseSetOption()``. + +Quick Start +=== + +Using a Profile via URI +--- + +The simplest way to use a profile is through a URI: + +.. code-block:: c + + AdbcDatabase database; + AdbcDatabaseNew(&database, &error); + AdbcDatabaseSetOption(&database, "uri", "profile://my_snowflake_prod", &error); + AdbcDatabaseInit(&database, &error); + +Using a Profile via Option +--- + +Alternatively, specify the profile name directly: + +.. code-block:: c + + AdbcDatabase database; + AdbcDatabaseNew(&database, &error); + AdbcDatabaseSetOption(&database, "profile", "my_snowflake_prod", &error); Review Comment: I think this is OK, it's a doc clarification -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [PR] feat(c/driver_manager): add connection profile interface [arrow-adbc]
ianmcook commented on code in PR #3876: URL: https://github.com/apache/arrow-adbc/pull/3876#discussion_r2700263491 ## docs/source/format/connection_profiles.rst: ## @@ -0,0 +1,568 @@ +.. Licensed to the Apache Software Foundation (ASF) under one +.. or more contributor license agreements. See the NOTICE file +.. distributed with this work for additional information +.. regarding copyright ownership. The ASF licenses this file +.. to you under the Apache License, Version 2.0 (the +.. "License"); you may not use this file except in compliance +.. with the License. You may obtain a copy of the License at +.. +.. http://www.apache.org/licenses/LICENSE-2.0 +.. +.. Unless required by applicable law or agreed to in writing, +.. software distributed under the License is distributed on an +.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +.. KIND, either express or implied. See the License for the +.. specific language governing permissions and limitations +.. under the License. + +== +Driver Manager Connection Profiles +== + +Overview + + +Similar to ODBC's ``odbc.ini``, the ADBC driver manager supports **connection profiles** +that specify a driver and connection options in a reusable configuration. This allows users to: + +- Define connection information in files or environment variables +- Share connection configurations across applications +- Distribute standardized connection settings +- Avoid hardcoding driver names and credentials in application code + +Profiles are loaded during ``AdbcDatabaseInit()`` before initializing the driver. Options +from the profile are applied automatically but do not override options already set via ``AdbcDatabaseSetOption()``. + +Quick Start +=== + +Using a Profile via URI +--- + +The simplest way to use a profile is through a URI: + +.. code-block:: c + + AdbcDatabase database; + AdbcDatabaseNew(&database, &error); + AdbcDatabaseSetOption(&database, "uri", "profile://my_snowflake_prod", &error); + AdbcDatabaseInit(&database, &error); + +Using a Profile via Option +--- + +Alternatively, specify the profile name directly: + +.. code-block:: c + + AdbcDatabase database; + AdbcDatabaseNew(&database, &error); + AdbcDatabaseSetOption(&database, "profile", "my_snowflake_prod", &error); Review Comment: @lidavidm Does this comment qualify as a sufficiently small change to `adbc.h` that this would not be considered a change to the ADBC API spec? I hope so. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [PR] feat(c/driver_manager): add connection profile interface [arrow-adbc]
ianmcook commented on code in PR #3876: URL: https://github.com/apache/arrow-adbc/pull/3876#discussion_r2700203754 ## docs/source/format/connection_profiles.rst: ## @@ -0,0 +1,568 @@ +.. Licensed to the Apache Software Foundation (ASF) under one +.. or more contributor license agreements. See the NOTICE file +.. distributed with this work for additional information +.. regarding copyright ownership. The ASF licenses this file +.. to you under the Apache License, Version 2.0 (the +.. "License"); you may not use this file except in compliance +.. with the License. You may obtain a copy of the License at +.. +.. http://www.apache.org/licenses/LICENSE-2.0 +.. +.. Unless required by applicable law or agreed to in writing, +.. software distributed under the License is distributed on an +.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +.. KIND, either express or implied. See the License for the +.. specific language governing permissions and limitations +.. under the License. + +== +Driver Manager Connection Profiles +== + +Overview + + +Similar to ODBC's ``odbc.ini``, the ADBC driver manager supports **connection profiles** +that specify a driver and connection options in a reusable configuration. This allows users to: + +- Define connection information in files or environment variables +- Share connection configurations across applications +- Distribute standardized connection settings +- Avoid hardcoding driver names and credentials in application code + +Profiles are loaded during ``AdbcDatabaseInit()`` before initializing the driver. Options +from the profile are applied automatically but do not override options already set via ``AdbcDatabaseSetOption()``. + +Quick Start +=== + +Using a Profile via URI +--- + +The simplest way to use a profile is through a URI: + +.. code-block:: c + + AdbcDatabase database; + AdbcDatabaseNew(&database, &error); + AdbcDatabaseSetOption(&database, "uri", "profile://my_snowflake_prod", &error); + AdbcDatabaseInit(&database, &error); + +Using a Profile via Option +--- + +Alternatively, specify the profile name directly: + +.. code-block:: c + + AdbcDatabase database; + AdbcDatabaseNew(&database, &error); + AdbcDatabaseSetOption(&database, "profile", "my_snowflake_prod", &error); Review Comment: Added in 7b8d420. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [PR] feat(c/driver_manager): add connection profile interface [arrow-adbc]
lidavidm commented on code in PR #3876: URL: https://github.com/apache/arrow-adbc/pull/3876#discussion_r2696847478 ## docs/source/format/connection_profiles.rst: ## @@ -0,0 +1,568 @@ +.. Licensed to the Apache Software Foundation (ASF) under one +.. or more contributor license agreements. See the NOTICE file +.. distributed with this work for additional information +.. regarding copyright ownership. The ASF licenses this file +.. to you under the Apache License, Version 2.0 (the +.. "License"); you may not use this file except in compliance +.. with the License. You may obtain a copy of the License at +.. +.. http://www.apache.org/licenses/LICENSE-2.0 +.. +.. Unless required by applicable law or agreed to in writing, +.. software distributed under the License is distributed on an +.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +.. KIND, either express or implied. See the License for the +.. specific language governing permissions and limitations +.. under the License. + +== +Driver Manager Connection Profiles +== + +Overview + + +Similar to ODBC's ``odbc.ini``, the ADBC driver manager supports **connection profiles** +that specify a driver and connection options in a reusable configuration. This allows users to: + +- Define connection information in files or environment variables +- Share connection configurations across applications +- Distribute standardized connection settings +- Avoid hardcoding driver names and credentials in application code + +Profiles are loaded during ``AdbcDatabaseInit()`` before initializing the driver. Options +from the profile are applied automatically but do not override options already set via ``AdbcDatabaseSetOption()``. + +Quick Start +=== + +Using a Profile via URI +--- + +The simplest way to use a profile is through a URI: + +.. code-block:: c + + AdbcDatabase database; + AdbcDatabaseNew(&database, &error); + AdbcDatabaseSetOption(&database, "uri", "profile://my_snowflake_prod", &error); + AdbcDatabaseInit(&database, &error); + +Using a Profile via Option +--- + +Alternatively, specify the profile name directly: + +.. code-block:: c + + AdbcDatabase database; + AdbcDatabaseNew(&database, &error); + AdbcDatabaseSetOption(&database, "profile", "my_snowflake_prod", &error); Review Comment: Maybe the docstring for AdbcDatabaseSetOption in adbc.h is best. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [PR] feat(c/driver_manager): add connection profile interface [arrow-adbc]
ianmcook commented on code in PR #3876: URL: https://github.com/apache/arrow-adbc/pull/3876#discussion_r2694507859 ## docs/source/format/connection_profiles.rst: ## @@ -0,0 +1,568 @@ +.. Licensed to the Apache Software Foundation (ASF) under one +.. or more contributor license agreements. See the NOTICE file +.. distributed with this work for additional information +.. regarding copyright ownership. The ASF licenses this file +.. to you under the Apache License, Version 2.0 (the +.. "License"); you may not use this file except in compliance +.. with the License. You may obtain a copy of the License at +.. +.. http://www.apache.org/licenses/LICENSE-2.0 +.. +.. Unless required by applicable law or agreed to in writing, +.. software distributed under the License is distributed on an +.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +.. KIND, either express or implied. See the License for the +.. specific language governing permissions and limitations +.. under the License. + +== +Driver Manager Connection Profiles +== + +Overview + + +Similar to ODBC's ``odbc.ini``, the ADBC driver manager supports **connection profiles** +that specify a driver and connection options in a reusable configuration. This allows users to: + +- Define connection information in files or environment variables +- Share connection configurations across applications +- Distribute standardized connection settings +- Avoid hardcoding driver names and credentials in application code + +Profiles are loaded during ``AdbcDatabaseInit()`` before initializing the driver. Options +from the profile are applied automatically but do not override options already set via ``AdbcDatabaseSetOption()``. + +Quick Start +=== + +Using a Profile via URI +--- + +The simplest way to use a profile is through a URI: + +.. code-block:: c + + AdbcDatabase database; + AdbcDatabaseNew(&database, &error); + AdbcDatabaseSetOption(&database, "uri", "profile://my_snowflake_prod", &error); + AdbcDatabaseInit(&database, &error); + +Using a Profile via Option +--- + +Alternatively, specify the profile name directly: + +.. code-block:: c + + AdbcDatabase database; + AdbcDatabaseNew(&database, &error); + AdbcDatabaseSetOption(&database, "profile", "my_snowflake_prod", &error); Review Comment: Where's the best place to add this, @lidavidm ? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [PR] feat(c/driver_manager): add connection profile interface [arrow-adbc]
lidavidm commented on code in PR #3876: URL: https://github.com/apache/arrow-adbc/pull/3876#discussion_r2693203878 ## docs/source/format/connection_profiles.rst: ## @@ -0,0 +1,568 @@ +.. Licensed to the Apache Software Foundation (ASF) under one +.. or more contributor license agreements. See the NOTICE file +.. distributed with this work for additional information +.. regarding copyright ownership. The ASF licenses this file +.. to you under the Apache License, Version 2.0 (the +.. "License"); you may not use this file except in compliance +.. with the License. You may obtain a copy of the License at +.. +.. http://www.apache.org/licenses/LICENSE-2.0 +.. +.. Unless required by applicable law or agreed to in writing, +.. software distributed under the License is distributed on an +.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +.. KIND, either express or implied. See the License for the +.. specific language governing permissions and limitations +.. under the License. + +== +Driver Manager Connection Profiles +== + +Overview + + +Similar to ODBC's ``odbc.ini``, the ADBC driver manager supports **connection profiles** +that specify a driver and connection options in a reusable configuration. This allows users to: + +- Define connection information in files or environment variables +- Share connection configurations across applications +- Distribute standardized connection settings +- Avoid hardcoding driver names and credentials in application code + +Profiles are loaded during ``AdbcDatabaseInit()`` before initializing the driver. Options +from the profile are applied automatically but do not override options already set via ``AdbcDatabaseSetOption()``. + +Quick Start +=== + +Using a Profile via URI +--- + +The simplest way to use a profile is through a URI: + +.. code-block:: c + + AdbcDatabase database; + AdbcDatabaseNew(&database, &error); + AdbcDatabaseSetOption(&database, "uri", "profile://my_snowflake_prod", &error); + AdbcDatabaseInit(&database, &error); + +Using a Profile via Option +--- + +Alternatively, specify the profile name directly: + +.. code-block:: c + + AdbcDatabase database; + AdbcDatabaseNew(&database, &error); + AdbcDatabaseSetOption(&database, "profile", "my_snowflake_prod", &error); Review Comment: Yes, I think that's reasonable -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [PR] feat(c/driver_manager): add connection profile interface [arrow-adbc]
ianmcook commented on code in PR #3876: URL: https://github.com/apache/arrow-adbc/pull/3876#discussion_r2692999190 ## docs/source/format/connection_profiles.rst: ## @@ -0,0 +1,568 @@ +.. Licensed to the Apache Software Foundation (ASF) under one +.. or more contributor license agreements. See the NOTICE file +.. distributed with this work for additional information +.. regarding copyright ownership. The ASF licenses this file +.. to you under the Apache License, Version 2.0 (the +.. "License"); you may not use this file except in compliance +.. with the License. You may obtain a copy of the License at +.. +.. http://www.apache.org/licenses/LICENSE-2.0 +.. +.. Unless required by applicable law or agreed to in writing, +.. software distributed under the License is distributed on an +.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +.. KIND, either express or implied. See the License for the +.. specific language governing permissions and limitations +.. under the License. + +== +Driver Manager Connection Profiles +== + +Overview + + +Similar to ODBC's ``odbc.ini``, the ADBC driver manager supports **connection profiles** Review Comment: Agreed. How about we structure the beginning of this page similar to how the [ADBC Driver Manager and Manifests](https://arrow.apache.org/adbc/current/format/driver_manifests.html) docs page is structured? Like: > There are two ways to pass connection options to driver managers: > 1. Directly specifying connection options as arguments to driver manager functions in your application code. > 2. Referring to a connection profile which contains connection options. (or something like that) And we could add a section that briefly explains method 1, before continuing to the content here which covers method 2. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [PR] feat(c/driver_manager): add connection profile interface [arrow-adbc]
ianmcook commented on code in PR #3876:
URL: https://github.com/apache/arrow-adbc/pull/3876#discussion_r2692936253
##
docs/source/format/connection_profiles.rst:
##
@@ -0,0 +1,568 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements. See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership. The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License. You may obtain a copy of the License at
+..
+.. http://www.apache.org/licenses/LICENSE-2.0
+..
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied. See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+==
+Driver Manager Connection Profiles
+==
+
+Overview
+
+
+Similar to ODBC's ``odbc.ini``, the ADBC driver manager supports **connection
profiles**
+that specify a driver and connection options in a reusable configuration. This
allows users to:
+
+- Define connection information in files or environment variables
+- Share connection configurations across applications
+- Distribute standardized connection settings
+- Avoid hardcoding driver names and credentials in application code
+
+Profiles are loaded during ``AdbcDatabaseInit()`` before initializing the
driver. Options
+from the profile are applied automatically but do not override options already
set via ``AdbcDatabaseSetOption()``.
+
+Quick Start
+===
+
+Using a Profile via URI
+---
+
+The simplest way to use a profile is through a URI:
+
+.. code-block:: c
+
+ AdbcDatabase database;
+ AdbcDatabaseNew(&database, &error);
+ AdbcDatabaseSetOption(&database, "uri", "profile://my_snowflake_prod",
&error);
+ AdbcDatabaseInit(&database, &error);
+
+Using a Profile via Option
+---
+
+Alternatively, specify the profile name directly:
+
+.. code-block:: c
+
+ AdbcDatabase database;
+ AdbcDatabaseNew(&database, &error);
+ AdbcDatabaseSetOption(&database, "profile", "my_snowflake_prod", &error);
+ AdbcDatabaseInit(&database, &error);
+
+Profile File Format
+===
+
+Filesystem-based profiles use TOML format with the following structure:
+
+.. code-block:: toml
+
+ version = 1
+ driver = "snowflake"
+
+ [options]
+ # String options
+ adbc.snowflake.sql.account = "mycompany"
+ adbc.snowflake.sql.warehouse = "COMPUTE_WH"
+ adbc.snowflake.sql.database = "PRODUCTION"
+ adbc.snowflake.sql.schema = "PUBLIC"
+
+ # Integer options
+ adbc.snowflake.sql.client_session_keep_alive_heartbeat_frequency = 3600
+
+ # Double options
+ adbc.snowflake.sql.client_timeout = 30.5
+
+ # Boolean options (converted to "true" or "false" strings)
+ adbc.snowflake.sql.client_session_keep_alive = true
+
+version
+---
+
+- **Required**: Yes
+- **Type**: Integer
+- **Supported values**: ``1``
+
+The ``version`` field specifies the profile format version. Currently, only
version 1 is supported.
+This will enable future changes while maintaining backward compatibility.
+
+driver
+--
+
+- **Required**: No
+- **Type**: String
+
+The ``driver`` field specifies which ADBC driver to load. This can be:
+
+- A driver name (e.g., ``"snowflake"``)
+- A path to a shared library (e.g.,
``"/usr/local/lib/libadbc_driver_snowflake.so"``)
+- A path to a driver manifest (e.g., ``"/etc/adbc/drivers/snowflake.toml"``)
+
+If omitted, the driver must be specified through other means (e.g., the
``driver`` option or ``uri`` parameter).
+The driver will be loaded identically to if it was specified via
``AdbcDatabaseSetOption("driver", "")``.
+For more detils, see :doc:`driver_manifests`.
+
+Options Section
+---
+
+The ``[options]`` section contains driver-specific configuration options.
Options can be of the following types:
+
+**String values**
+ Applied using ``AdbcDatabaseSetOption()``
+
+ .. code-block:: toml
+
+ adbc.snowflake.sql.account = "mycompany"
+ adbc.snowflake.sql.warehouse = "COMPUTE_WH"
+
+**Integer values**
+ Applied using ``AdbcDatabaseSetOptionInt()``
+
+ .. code-block:: toml
+
+ adbc.snowflake.sql.client_session_keep_alive_heartbeat_frequency = 3600
+
+**Double values**
+ Applied using ``AdbcDatabaseSetOptionDouble()``
+
+ .. code-block:: toml
+
+ adbc.snowflake.sql.client_timeout = 30.5
+
+**Boolean values**
+ Converted to strings ``"true"`` or ``"false"`` and applied using
``AdbcDatabaseSetOption()``
+
+ .. code-block:: toml
+
+ adbc.snowflake.sql.client_session_keep_alive = true
+
+Environment Variable Substitution
+---
Re: [PR] feat(c/driver_manager): add connection profile interface [arrow-adbc]
ianmcook commented on code in PR #3876: URL: https://github.com/apache/arrow-adbc/pull/3876#discussion_r2693032999 ## docs/source/format/connection_profiles.rst: ## @@ -0,0 +1,568 @@ +.. Licensed to the Apache Software Foundation (ASF) under one +.. or more contributor license agreements. See the NOTICE file +.. distributed with this work for additional information +.. regarding copyright ownership. The ASF licenses this file +.. to you under the Apache License, Version 2.0 (the +.. "License"); you may not use this file except in compliance +.. with the License. You may obtain a copy of the License at +.. +.. http://www.apache.org/licenses/LICENSE-2.0 +.. +.. Unless required by applicable law or agreed to in writing, +.. software distributed under the License is distributed on an +.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +.. KIND, either express or implied. See the License for the +.. specific language governing permissions and limitations +.. under the License. + +== +Driver Manager Connection Profiles +== + +Overview + + +Similar to ODBC's ``odbc.ini``, the ADBC driver manager supports **connection profiles** +that specify a driver and connection options in a reusable configuration. This allows users to: + +- Define connection information in files or environment variables +- Share connection configurations across applications +- Distribute standardized connection settings +- Avoid hardcoding driver names and credentials in application code + +Profiles are loaded during ``AdbcDatabaseInit()`` before initializing the driver. Options +from the profile are applied automatically but do not override options already set via ``AdbcDatabaseSetOption()``. + +Quick Start +=== + +Using a Profile via URI +--- + +The simplest way to use a profile is through a URI: + +.. code-block:: c + + AdbcDatabase database; + AdbcDatabaseNew(&database, &error); + AdbcDatabaseSetOption(&database, "uri", "profile://my_snowflake_prod", &error); + AdbcDatabaseInit(&database, &error); + +Using a Profile via Option +--- + +Alternatively, specify the profile name directly: + +.. code-block:: c + + AdbcDatabase database; + AdbcDatabaseNew(&database, &error); + AdbcDatabaseSetOption(&database, "profile", "my_snowflake_prod", &error); Review Comment: @lidavidm Similar to what I added in af734d3, should we also add a warning like this somewhere in the docs? > Driver managers may treat some option keys as manager-reserved and handle them without forwarding them to the underlying driver. In particular, the option key `profile` is reserved for connection profiles and must not be implemented or interpreted by drivers. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [PR] feat(c/driver_manager): add connection profile interface [arrow-adbc]
zeroshade commented on code in PR #3876:
URL: https://github.com/apache/arrow-adbc/pull/3876#discussion_r2692557889
##
c/include/arrow-adbc/adbc_driver_manager.h:
##
@@ -175,6 +175,183 @@ AdbcStatusCode
AdbcDriverManagerDatabaseSetAdditionalSearchPathList(
ADBC_EXPORT
const char* AdbcStatusCodeMessage(AdbcStatusCode code);
+/// \defgroup adbc-driver-manager-connection-profile Connection Profiles
+/// Similar to odbc.ini, the ADBC driver manager can support "connection
profiles"
+/// that specify a driver and options to use when connecting. This allows
users to
+/// specify connection information in a file or environment variable, and have
the
+/// driver manager load the appropriate driver and set options accordingly.
+///
+/// This allows creating reusable connection configurations for sharing and
distribution
+/// without needing to hardcode driver names and options in application code.
Profiles
+/// will be loaded during DatabaseInit before attempting to initialize the
driver. Any
+/// options specified by the profile will be applied but will not override
options
+/// that have already been set using DatabaseSetOption.
+///
+/// To facilitate customization, we define an interface for implementing a
Connection
+/// Profile object along with a provider function definition which can be set
into
+/// the driver manager to allow for customized profile loading.
+///
+/// A profile can be specified to the Driver Manager in one of two ways,
+/// which will invoke the profile provider during the call to DatabaseInit:
+///
+/// 1. The "profile" option can be set using DatabaseSetOption with the name
of the
+/// profile to load.
+/// 2. The "uri" being used can have the form "profile://"
+///
+/// @{
+
+/// \brief Abstract interface for connection profile providers
+struct ADBC_EXPORT AdbcConnectionProfile {
+ /// \brief Opaque implementation-defined state.
+ /// This field is NULL if the profile is uninitialized/freed (but
+ /// it need not have a value even if the profile is initialized).
+ void* private_data;
+
+ /// \brief Release the profile and perform any cleanup.
+ void (*release)(struct AdbcConnectionProfile* profile);
Review Comment:
The manager calls release currently, in general the application will never
interact with profile objects unless they explicitly want to.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
Re: [PR] feat(c/driver_manager): add connection profile interface [arrow-adbc]
zeroshade commented on code in PR #3876:
URL: https://github.com/apache/arrow-adbc/pull/3876#discussion_r2692557889
##
c/include/arrow-adbc/adbc_driver_manager.h:
##
@@ -175,6 +175,183 @@ AdbcStatusCode
AdbcDriverManagerDatabaseSetAdditionalSearchPathList(
ADBC_EXPORT
const char* AdbcStatusCodeMessage(AdbcStatusCode code);
+/// \defgroup adbc-driver-manager-connection-profile Connection Profiles
+/// Similar to odbc.ini, the ADBC driver manager can support "connection
profiles"
+/// that specify a driver and options to use when connecting. This allows
users to
+/// specify connection information in a file or environment variable, and have
the
+/// driver manager load the appropriate driver and set options accordingly.
+///
+/// This allows creating reusable connection configurations for sharing and
distribution
+/// without needing to hardcode driver names and options in application code.
Profiles
+/// will be loaded during DatabaseInit before attempting to initialize the
driver. Any
+/// options specified by the profile will be applied but will not override
options
+/// that have already been set using DatabaseSetOption.
+///
+/// To facilitate customization, we define an interface for implementing a
Connection
+/// Profile object along with a provider function definition which can be set
into
+/// the driver manager to allow for customized profile loading.
+///
+/// A profile can be specified to the Driver Manager in one of two ways,
+/// which will invoke the profile provider during the call to DatabaseInit:
+///
+/// 1. The "profile" option can be set using DatabaseSetOption with the name
of the
+/// profile to load.
+/// 2. The "uri" being used can have the form "profile://"
+///
+/// @{
+
+/// \brief Abstract interface for connection profile providers
+struct ADBC_EXPORT AdbcConnectionProfile {
+ /// \brief Opaque implementation-defined state.
+ /// This field is NULL if the profile is uninitialized/freed (but
+ /// it need not have a value even if the profile is initialized).
+ void* private_data;
+
+ /// \brief Release the profile and perform any cleanup.
+ void (*release)(struct AdbcConnectionProfile* profile);
Review Comment:
The manager calls release currently.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
Re: [PR] feat(c/driver_manager): add connection profile interface [arrow-adbc]
zeroshade commented on code in PR #3876:
URL: https://github.com/apache/arrow-adbc/pull/3876#discussion_r2692557207
##
docs/source/format/connection_profiles.rst:
##
@@ -0,0 +1,568 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements. See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership. The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License. You may obtain a copy of the License at
+..
+.. http://www.apache.org/licenses/LICENSE-2.0
+..
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied. See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+==
+Driver Manager Connection Profiles
+==
+
+Overview
+
+
+Similar to ODBC's ``odbc.ini``, the ADBC driver manager supports **connection
profiles**
+that specify a driver and connection options in a reusable configuration. This
allows users to:
+
+- Define connection information in files or environment variables
+- Share connection configurations across applications
+- Distribute standardized connection settings
+- Avoid hardcoding driver names and credentials in application code
+
+Profiles are loaded during ``AdbcDatabaseInit()`` before initializing the
driver. Options
+from the profile are applied automatically but do not override options already
set via ``AdbcDatabaseSetOption()``.
+
+Quick Start
+===
+
+Using a Profile via URI
+---
+
+The simplest way to use a profile is through a URI:
+
+.. code-block:: c
+
+ AdbcDatabase database;
+ AdbcDatabaseNew(&database, &error);
+ AdbcDatabaseSetOption(&database, "uri", "profile://my_snowflake_prod",
&error);
+ AdbcDatabaseInit(&database, &error);
+
+Using a Profile via Option
+---
+
+Alternatively, specify the profile name directly:
+
+.. code-block:: c
+
+ AdbcDatabase database;
+ AdbcDatabaseNew(&database, &error);
+ AdbcDatabaseSetOption(&database, "profile", "my_snowflake_prod", &error);
+ AdbcDatabaseInit(&database, &error);
+
+Profile File Format
+===
+
+Filesystem-based profiles use TOML format with the following structure:
+
+.. code-block:: toml
+
+ version = 1
+ driver = "snowflake"
+
+ [options]
+ # String options
+ adbc.snowflake.sql.account = "mycompany"
+ adbc.snowflake.sql.warehouse = "COMPUTE_WH"
+ adbc.snowflake.sql.database = "PRODUCTION"
+ adbc.snowflake.sql.schema = "PUBLIC"
+
+ # Integer options
+ adbc.snowflake.sql.client_session_keep_alive_heartbeat_frequency = 3600
+
+ # Double options
+ adbc.snowflake.sql.client_timeout = 30.5
+
+ # Boolean options (converted to "true" or "false" strings)
+ adbc.snowflake.sql.client_session_keep_alive = true
+
+version
+---
+
+- **Required**: Yes
+- **Type**: Integer
+- **Supported values**: ``1``
+
+The ``version`` field specifies the profile format version. Currently, only
version 1 is supported.
+This will enable future changes while maintaining backward compatibility.
+
+driver
+--
+
+- **Required**: No
+- **Type**: String
+
+The ``driver`` field specifies which ADBC driver to load. This can be:
+
+- A driver name (e.g., ``"snowflake"``)
+- A path to a shared library (e.g.,
``"/usr/local/lib/libadbc_driver_snowflake.so"``)
+- A path to a driver manifest (e.g., ``"/etc/adbc/drivers/snowflake.toml"``)
+
+If omitted, the driver must be specified through other means (e.g., the
``driver`` option or ``uri`` parameter).
+The driver will be loaded identically to if it was specified via
``AdbcDatabaseSetOption("driver", "")``.
+For more detils, see :doc:`driver_manifests`.
+
+Options Section
+---
+
+The ``[options]`` section contains driver-specific configuration options.
Options can be of the following types:
+
+**String values**
+ Applied using ``AdbcDatabaseSetOption()``
+
+ .. code-block:: toml
+
+ adbc.snowflake.sql.account = "mycompany"
+ adbc.snowflake.sql.warehouse = "COMPUTE_WH"
+
+**Integer values**
+ Applied using ``AdbcDatabaseSetOptionInt()``
+
+ .. code-block:: toml
+
+ adbc.snowflake.sql.client_session_keep_alive_heartbeat_frequency = 3600
+
+**Double values**
+ Applied using ``AdbcDatabaseSetOptionDouble()``
+
+ .. code-block:: toml
+
+ adbc.snowflake.sql.client_timeout = 30.5
+
+**Boolean values**
+ Converted to strings ``"true"`` or ``"false"`` and applied using
``AdbcDatabaseSetOption()``
+
+ .. code-block:: toml
+
+ adbc.snowflake.sql.client_session_keep_alive = true
+
+Environment Variable Substitution
+--
Re: [PR] feat(c/driver_manager): add connection profile interface [arrow-adbc]
lidavidm commented on code in PR #3876:
URL: https://github.com/apache/arrow-adbc/pull/3876#discussion_r2692553068
##
docs/source/format/connection_profiles.rst:
##
@@ -0,0 +1,568 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements. See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership. The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License. You may obtain a copy of the License at
+..
+.. http://www.apache.org/licenses/LICENSE-2.0
+..
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied. See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+==
+Driver Manager Connection Profiles
+==
+
+Overview
+
+
+Similar to ODBC's ``odbc.ini``, the ADBC driver manager supports **connection
profiles**
+that specify a driver and connection options in a reusable configuration. This
allows users to:
+
+- Define connection information in files or environment variables
+- Share connection configurations across applications
+- Distribute standardized connection settings
+- Avoid hardcoding driver names and credentials in application code
+
+Profiles are loaded during ``AdbcDatabaseInit()`` before initializing the
driver. Options
+from the profile are applied automatically but do not override options already
set via ``AdbcDatabaseSetOption()``.
+
+Quick Start
+===
+
+Using a Profile via URI
+---
+
+The simplest way to use a profile is through a URI:
+
+.. code-block:: c
+
+ AdbcDatabase database;
+ AdbcDatabaseNew(&database, &error);
+ AdbcDatabaseSetOption(&database, "uri", "profile://my_snowflake_prod",
&error);
+ AdbcDatabaseInit(&database, &error);
+
+Using a Profile via Option
+---
+
+Alternatively, specify the profile name directly:
+
+.. code-block:: c
+
+ AdbcDatabase database;
+ AdbcDatabaseNew(&database, &error);
+ AdbcDatabaseSetOption(&database, "profile", "my_snowflake_prod", &error);
+ AdbcDatabaseInit(&database, &error);
+
+Profile File Format
+===
+
+Filesystem-based profiles use TOML format with the following structure:
+
+.. code-block:: toml
+
+ version = 1
+ driver = "snowflake"
+
+ [options]
+ # String options
+ adbc.snowflake.sql.account = "mycompany"
+ adbc.snowflake.sql.warehouse = "COMPUTE_WH"
+ adbc.snowflake.sql.database = "PRODUCTION"
+ adbc.snowflake.sql.schema = "PUBLIC"
+
+ # Integer options
+ adbc.snowflake.sql.client_session_keep_alive_heartbeat_frequency = 3600
+
+ # Double options
+ adbc.snowflake.sql.client_timeout = 30.5
+
+ # Boolean options (converted to "true" or "false" strings)
+ adbc.snowflake.sql.client_session_keep_alive = true
+
+version
+---
+
+- **Required**: Yes
+- **Type**: Integer
+- **Supported values**: ``1``
+
+The ``version`` field specifies the profile format version. Currently, only
version 1 is supported.
+This will enable future changes while maintaining backward compatibility.
+
+driver
+--
+
+- **Required**: No
+- **Type**: String
+
+The ``driver`` field specifies which ADBC driver to load. This can be:
+
+- A driver name (e.g., ``"snowflake"``)
+- A path to a shared library (e.g.,
``"/usr/local/lib/libadbc_driver_snowflake.so"``)
+- A path to a driver manifest (e.g., ``"/etc/adbc/drivers/snowflake.toml"``)
+
+If omitted, the driver must be specified through other means (e.g., the
``driver`` option or ``uri`` parameter).
+The driver will be loaded identically to if it was specified via
``AdbcDatabaseSetOption("driver", "")``.
+For more detils, see :doc:`driver_manifests`.
+
+Options Section
+---
+
+The ``[options]`` section contains driver-specific configuration options.
Options can be of the following types:
+
+**String values**
+ Applied using ``AdbcDatabaseSetOption()``
+
+ .. code-block:: toml
+
+ adbc.snowflake.sql.account = "mycompany"
+ adbc.snowflake.sql.warehouse = "COMPUTE_WH"
+
+**Integer values**
+ Applied using ``AdbcDatabaseSetOptionInt()``
+
+ .. code-block:: toml
+
+ adbc.snowflake.sql.client_session_keep_alive_heartbeat_frequency = 3600
+
+**Double values**
+ Applied using ``AdbcDatabaseSetOptionDouble()``
+
+ .. code-block:: toml
+
+ adbc.snowflake.sql.client_timeout = 30.5
+
+**Boolean values**
+ Converted to strings ``"true"`` or ``"false"`` and applied using
``AdbcDatabaseSetOption()``
+
+ .. code-block:: toml
+
+ adbc.snowflake.sql.client_session_keep_alive = true
+
+Environment Variable Substitution
+---
Re: [PR] feat(c/driver_manager): add connection profile interface [arrow-adbc]
zeroshade commented on PR #3876: URL: https://github.com/apache/arrow-adbc/pull/3876#issuecomment-3751131510 @CurtHagenlocher @davidhcoe @lidavidm @ianmcook any further comments? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [PR] feat(c/driver_manager): add connection profile interface [arrow-adbc]
ianmcook commented on code in PR #3876:
URL: https://github.com/apache/arrow-adbc/pull/3876#discussion_r2684562568
##
c/include/arrow-adbc/adbc_driver_manager.h:
##
@@ -175,6 +175,171 @@ AdbcStatusCode
AdbcDriverManagerDatabaseSetAdditionalSearchPathList(
ADBC_EXPORT
const char* AdbcStatusCodeMessage(AdbcStatusCode code);
+/// \defgroup adbc-driver-manager-connection-profile Connection Profiles
+/// Similar to odbc.ini, the ADBC driver manager can support "connection
profiles"
+/// that specify a driver and options to use when connecting. This allows
users to
+/// specify connection information in a file or environment variable, and have
the
+/// driver manager load the appropriate driver and set options accordingly.
+///
+/// This allows creating reusable connection configurations for sharing and
distribution
+/// without needing to hardcode driver names and options in application code.
Profiles
+/// will be loaded during DatabaseInit before attempting to initialize the
driver. Any
+/// options specified by the profile will be applied but will not override
options
+/// that have already been set using DatabaseSetOption.
+///
+/// To facilitate customization, we define an interface for implementing a
Connection
+/// Profile object along with a provider function definition which can be set
into
+/// the driver manager to allow for customized profile loading.
+///
+/// A profile can be specified to the Driver Manager in one of two ways,
+/// which will invoke the profile provider during the call to DatabaseInit:
+///
+/// 1. The "profile" option can be set using DatabaseSetOption with the name
of the
+/// profile to load.
+/// 2. The "uri" being used can have the form "profile://"
+///
+/// @{
+
+/// \brief Abstract interface for connection profile providers
+struct ADBC_EXPORT AdbcConnectionProfile {
+ /// \brief Opaque implementation-defined state.
+ /// This field is NULL if the profile is uninitialized/freed (but
+ /// it need not have a value even if the profile is initialized).
+ void* private_data;
+
+ /// \brief Release the profile and perform any cleanup.
+ void (*release)(struct AdbcConnectionProfile* profile);
+
+ /// \brief Get the driver to use as specified by this profile.
+ ///
+ /// It is not required that a profile specify a driver. If the options
+ // can be reusable across drivers, then the profile does not need to specify
+ /// a driver (if this provides an empty string or nullptr then the driver
+ /// must be defined by other means, e.g. by the driver / uri options).
+ ///
+ /// \param[in] profile The profile to query.
+ /// \param[out] driver_name The name of the driver to use, or NULL if not
specified.
+ /// \param[out] error An optional location to return an error message
+ AdbcStatusCode (*GetDriverName)(struct AdbcConnectionProfile* profile,
+ const char** driver_name, struct AdbcError*
error);
+
+ /// \brief Get the string options specified by the profile
+ ///
+ /// The keys and values returned by this function are owned by the profile
+ /// object itself and do not need to be freed or managed by the caller.
+ /// They must not be accessed after calling release on the profile.
+ ///
+ /// The profile can also indicate that a value should be pulled from the
environment
+ /// by having a value in the form `env_var(ENV_VAR_NAME)`. If the driver
+ /// manager encounters a value of this form, it will replace it with the
actual value
+ /// of the environment variable `ENV_VAR_NAME` before setting the option.
This
+ /// is only valid for option *values* not *keys*.
+ ///
+ /// \param[in] profile The profile to query.
+ /// \param[out] keys The keys of the options specified by the profile.
+ /// \param[out] values The values of the options specified by the profile.
+ /// \param[out] num_options The number of options specified by the profile,
+ /// consumers must not access keys or values beyond this count.
+ /// \param[out] error An optional location to return an error message
+ AdbcStatusCode (*GetOptions)(struct AdbcConnectionProfile* profile, const
char*** keys,
+ const char*** values, size_t* num_options,
+ struct AdbcError* error);
+
+ /// \brief Get the integer options specified by the profile
+ ///
+ /// The keys and values returned by this function are owned by the profile
+ /// object itself and do not need to be freed or managed by the caller. They
must not be
+ /// accessed after calling release on the profile.
+ ///
+ /// Values returned by this function will be set using the
DatabaseSetOptionInt function
+ /// on the database object being initialized. If the driver does not support
the
+ /// DatabaseSetOptionInt function, then options should only be returned as
strings.
+ ///
+ /// \param[in] profile The profile to query.
+ /// \param[out] keys The keys of the options specified by the profil
Re: [PR] feat(c/driver_manager): add connection profile interface [arrow-adbc]
ianmcook commented on code in PR #3876:
URL: https://github.com/apache/arrow-adbc/pull/3876#discussion_r2684553629
##
docs/source/format/connection_profiles.rst:
##
@@ -0,0 +1,568 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements. See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership. The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License. You may obtain a copy of the License at
+..
+.. http://www.apache.org/licenses/LICENSE-2.0
+..
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied. See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+==
+Driver Manager Connection Profiles
+==
+
+Overview
+
+
+Similar to ODBC's ``odbc.ini``, the ADBC driver manager supports **connection
profiles**
+that specify a driver and connection options in a reusable configuration. This
allows users to:
+
+- Define connection information in files or environment variables
+- Share connection configurations across applications
+- Distribute standardized connection settings
+- Avoid hardcoding driver names and credentials in application code
+
+Profiles are loaded during ``AdbcDatabaseInit()`` before initializing the
driver. Options
+from the profile are applied automatically but do not override options already
set via ``AdbcDatabaseSetOption()``.
+
+Quick Start
+===
+
+Using a Profile via URI
+---
+
+The simplest way to use a profile is through a URI:
+
+.. code-block:: c
+
+ AdbcDatabase database;
+ AdbcDatabaseNew(&database, &error);
+ AdbcDatabaseSetOption(&database, "uri", "profile://my_snowflake_prod",
&error);
+ AdbcDatabaseInit(&database, &error);
+
+Using a Profile via Option
+---
+
+Alternatively, specify the profile name directly:
+
+.. code-block:: c
+
+ AdbcDatabase database;
+ AdbcDatabaseNew(&database, &error);
+ AdbcDatabaseSetOption(&database, "profile", "my_snowflake_prod", &error);
+ AdbcDatabaseInit(&database, &error);
+
+Profile File Format
+===
+
+Filesystem-based profiles use TOML format with the following structure:
+
+.. code-block:: toml
+
+ version = 1
+ driver = "snowflake"
+
+ [options]
+ # String options
+ adbc.snowflake.sql.account = "mycompany"
+ adbc.snowflake.sql.warehouse = "COMPUTE_WH"
+ adbc.snowflake.sql.database = "PRODUCTION"
+ adbc.snowflake.sql.schema = "PUBLIC"
+
+ # Integer options
+ adbc.snowflake.sql.client_session_keep_alive_heartbeat_frequency = 3600
+
+ # Double options
+ adbc.snowflake.sql.client_timeout = 30.5
+
+ # Boolean options (converted to "true" or "false" strings)
+ adbc.snowflake.sql.client_session_keep_alive = true
+
+version
+---
+
+- **Required**: Yes
+- **Type**: Integer
+- **Supported values**: ``1``
+
+The ``version`` field specifies the profile format version. Currently, only
version 1 is supported.
+This will enable future changes while maintaining backward compatibility.
+
+driver
+--
+
+- **Required**: No
+- **Type**: String
+
+The ``driver`` field specifies which ADBC driver to load. This can be:
+
+- A driver name (e.g., ``"snowflake"``)
+- A path to a shared library (e.g.,
``"/usr/local/lib/libadbc_driver_snowflake.so"``)
+- A path to a driver manifest (e.g., ``"/etc/adbc/drivers/snowflake.toml"``)
+
+If omitted, the driver must be specified through other means (e.g., the
``driver`` option or ``uri`` parameter).
+The driver will be loaded identically to if it was specified via
``AdbcDatabaseSetOption("driver", "")``.
+For more detils, see :doc:`driver_manifests`.
+
+Options Section
+---
+
+The ``[options]`` section contains driver-specific configuration options.
Options can be of the following types:
+
+**String values**
+ Applied using ``AdbcDatabaseSetOption()``
+
+ .. code-block:: toml
+
+ adbc.snowflake.sql.account = "mycompany"
+ adbc.snowflake.sql.warehouse = "COMPUTE_WH"
+
+**Integer values**
+ Applied using ``AdbcDatabaseSetOptionInt()``
+
+ .. code-block:: toml
+
+ adbc.snowflake.sql.client_session_keep_alive_heartbeat_frequency = 3600
+
+**Double values**
+ Applied using ``AdbcDatabaseSetOptionDouble()``
+
+ .. code-block:: toml
+
+ adbc.snowflake.sql.client_timeout = 30.5
+
+**Boolean values**
+ Converted to strings ``"true"`` or ``"false"`` and applied using
``AdbcDatabaseSetOption()``
+
+ .. code-block:: toml
+
+ adbc.snowflake.sql.client_session_keep_alive = true
+
+Environment Variable Substitution
+---
Re: [PR] feat(c/driver_manager): add connection profile interface [arrow-adbc]
ianmcook commented on code in PR #3876:
URL: https://github.com/apache/arrow-adbc/pull/3876#discussion_r2684549600
##
docs/source/format/connection_profiles.rst:
##
@@ -0,0 +1,568 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements. See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership. The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License. You may obtain a copy of the License at
+..
+.. http://www.apache.org/licenses/LICENSE-2.0
+..
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied. See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+==
+Driver Manager Connection Profiles
+==
+
+Overview
+
+
+Similar to ODBC's ``odbc.ini``, the ADBC driver manager supports **connection
profiles**
+that specify a driver and connection options in a reusable configuration. This
allows users to:
+
+- Define connection information in files or environment variables
+- Share connection configurations across applications
+- Distribute standardized connection settings
+- Avoid hardcoding driver names and credentials in application code
+
+Profiles are loaded during ``AdbcDatabaseInit()`` before initializing the
driver. Options
+from the profile are applied automatically but do not override options already
set via ``AdbcDatabaseSetOption()``.
+
+Quick Start
+===
+
+Using a Profile via URI
+---
+
+The simplest way to use a profile is through a URI:
+
+.. code-block:: c
+
+ AdbcDatabase database;
+ AdbcDatabaseNew(&database, &error);
+ AdbcDatabaseSetOption(&database, "uri", "profile://my_snowflake_prod",
&error);
+ AdbcDatabaseInit(&database, &error);
+
+Using a Profile via Option
+---
+
+Alternatively, specify the profile name directly:
+
+.. code-block:: c
+
+ AdbcDatabase database;
+ AdbcDatabaseNew(&database, &error);
+ AdbcDatabaseSetOption(&database, "profile", "my_snowflake_prod", &error);
+ AdbcDatabaseInit(&database, &error);
+
+Profile File Format
+===
+
+Filesystem-based profiles use TOML format with the following structure:
+
+.. code-block:: toml
+
+ version = 1
+ driver = "snowflake"
+
+ [options]
+ # String options
+ adbc.snowflake.sql.account = "mycompany"
+ adbc.snowflake.sql.warehouse = "COMPUTE_WH"
+ adbc.snowflake.sql.database = "PRODUCTION"
+ adbc.snowflake.sql.schema = "PUBLIC"
+
+ # Integer options
+ adbc.snowflake.sql.client_session_keep_alive_heartbeat_frequency = 3600
+
+ # Double options
+ adbc.snowflake.sql.client_timeout = 30.5
+
+ # Boolean options (converted to "true" or "false" strings)
+ adbc.snowflake.sql.client_session_keep_alive = true
+
+version
+---
+
+- **Required**: Yes
+- **Type**: Integer
+- **Supported values**: ``1``
+
+The ``version`` field specifies the profile format version. Currently, only
version 1 is supported.
+This will enable future changes while maintaining backward compatibility.
+
+driver
+--
+
+- **Required**: No
+- **Type**: String
+
+The ``driver`` field specifies which ADBC driver to load. This can be:
+
+- A driver name (e.g., ``"snowflake"``)
+- A path to a shared library (e.g.,
``"/usr/local/lib/libadbc_driver_snowflake.so"``)
+- A path to a driver manifest (e.g., ``"/etc/adbc/drivers/snowflake.toml"``)
+
+If omitted, the driver must be specified through other means (e.g., the
``driver`` option or ``uri`` parameter).
+The driver will be loaded identically to if it was specified via
``AdbcDatabaseSetOption("driver", "")``.
+For more detils, see :doc:`driver_manifests`.
+
+Options Section
+---
+
+The ``[options]`` section contains driver-specific configuration options.
Options can be of the following types:
+
+**String values**
+ Applied using ``AdbcDatabaseSetOption()``
+
+ .. code-block:: toml
+
+ adbc.snowflake.sql.account = "mycompany"
+ adbc.snowflake.sql.warehouse = "COMPUTE_WH"
+
+**Integer values**
+ Applied using ``AdbcDatabaseSetOptionInt()``
+
+ .. code-block:: toml
+
+ adbc.snowflake.sql.client_session_keep_alive_heartbeat_frequency = 3600
+
+**Double values**
+ Applied using ``AdbcDatabaseSetOptionDouble()``
+
+ .. code-block:: toml
+
+ adbc.snowflake.sql.client_timeout = 30.5
+
+**Boolean values**
+ Converted to strings ``"true"`` or ``"false"`` and applied using
``AdbcDatabaseSetOption()``
+
+ .. code-block:: toml
+
+ adbc.snowflake.sql.client_session_keep_alive = true
+
+Environment Variable Substitution
+---
Re: [PR] feat(c/driver_manager): add connection profile interface [arrow-adbc]
lidavidm commented on code in PR #3876: URL: https://github.com/apache/arrow-adbc/pull/3876#discussion_r2680695464 ## docs/source/format/connection_profiles.rst: ## @@ -0,0 +1,568 @@ +.. Licensed to the Apache Software Foundation (ASF) under one +.. or more contributor license agreements. See the NOTICE file +.. distributed with this work for additional information +.. regarding copyright ownership. The ASF licenses this file +.. to you under the Apache License, Version 2.0 (the +.. "License"); you may not use this file except in compliance +.. with the License. You may obtain a copy of the License at +.. +.. http://www.apache.org/licenses/LICENSE-2.0 +.. +.. Unless required by applicable law or agreed to in writing, +.. software distributed under the License is distributed on an +.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +.. KIND, either express or implied. See the License for the +.. specific language governing permissions and limitations +.. under the License. + +== +Driver Manager Connection Profiles +== + +Overview + + +Similar to ODBC's ``odbc.ini``, the ADBC driver manager supports **connection profiles** Review Comment: nit, but I think we should explain this as its own thing, and cite ODBC as an inspiration further down ## docs/source/format/connection_profiles.rst: ## @@ -0,0 +1,568 @@ +.. Licensed to the Apache Software Foundation (ASF) under one +.. or more contributor license agreements. See the NOTICE file +.. distributed with this work for additional information +.. regarding copyright ownership. The ASF licenses this file +.. to you under the Apache License, Version 2.0 (the +.. "License"); you may not use this file except in compliance +.. with the License. You may obtain a copy of the License at +.. +.. http://www.apache.org/licenses/LICENSE-2.0 +.. +.. Unless required by applicable law or agreed to in writing, +.. software distributed under the License is distributed on an +.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +.. KIND, either express or implied. See the License for the +.. specific language governing permissions and limitations +.. under the License. + +== +Driver Manager Connection Profiles +== + +Overview + + +Similar to ODBC's ``odbc.ini``, the ADBC driver manager supports **connection profiles** +that specify a driver and connection options in a reusable configuration. This allows users to: + +- Define connection information in files or environment variables +- Share connection configurations across applications +- Distribute standardized connection settings +- Avoid hardcoding driver names and credentials in application code + +Profiles are loaded during ``AdbcDatabaseInit()`` before initializing the driver. Options +from the profile are applied automatically but do not override options already set via ``AdbcDatabaseSetOption()``. + +Quick Start +=== + +Using a Profile via URI +--- + +The simplest way to use a profile is through a URI: + +.. code-block:: c + + AdbcDatabase database; + AdbcDatabaseNew(&database, &error); + AdbcDatabaseSetOption(&database, "uri", "profile://my_snowflake_prod", &error); + AdbcDatabaseInit(&database, &error); + +Using a Profile via Option +--- + +Alternatively, specify the profile name directly: + +.. code-block:: c + + AdbcDatabase database; + AdbcDatabaseNew(&database, &error); + AdbcDatabaseSetOption(&database, "profile", "my_snowflake_prod", &error); + AdbcDatabaseInit(&database, &error); + +Profile File Format +=== + +Filesystem-based profiles use TOML format with the following structure: + +.. code-block:: toml + + version = 1 + driver = "snowflake" + + [options] + # String options + adbc.snowflake.sql.account = "mycompany" + adbc.snowflake.sql.warehouse = "COMPUTE_WH" + adbc.snowflake.sql.database = "PRODUCTION" + adbc.snowflake.sql.schema = "PUBLIC" + + # Integer options + adbc.snowflake.sql.client_session_keep_alive_heartbeat_frequency = 3600 + + # Double options + adbc.snowflake.sql.client_timeout = 30.5 + + # Boolean options (converted to "true" or "false" strings) + adbc.snowflake.sql.client_session_keep_alive = true + +version +--- + +- **Required**: Yes +- **Type**: Integer +- **Supported values**: ``1`` + +The ``version`` field specifies the profile format version. Currently, only version 1 is supported. +This will enable future changes while maintaining backward compatibility. + +driver +-- + +- **Required**: No +- **Type**: String + +The ``driver`` field specifies which ADBC driver to load. This can be: + +- A driver name (e.g., ``"snowflake"``) +- A path to a shared library (e.g., ``"/usr/local/lib/libadbc_driver_snowflake.so"``) +- A path to a driver manifest (e.g., ``"/etc/a
Re: [PR] feat(c/driver_manager): add connection profile interface [arrow-adbc]
zeroshade commented on PR #3876: URL: https://github.com/apache/arrow-adbc/pull/3876#issuecomment-3733769265 *sigh* I'm not quite sure why the env_var processing isn't working on Windows :frowning: if anyone has an idea, please feel free to help out. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [PR] feat(c/driver_manager): add connection profile interface [arrow-adbc]
zeroshade commented on code in PR #3876:
URL: https://github.com/apache/arrow-adbc/pull/3876#discussion_r2678905076
##
c/include/arrow-adbc/adbc_driver_manager.h:
##
@@ -175,6 +175,171 @@ AdbcStatusCode
AdbcDriverManagerDatabaseSetAdditionalSearchPathList(
ADBC_EXPORT
const char* AdbcStatusCodeMessage(AdbcStatusCode code);
+/// \defgroup adbc-driver-manager-connection-profile Connection Profiles
+/// Similar to odbc.ini, the ADBC driver manager can support "connection
profiles"
+/// that specify a driver and options to use when connecting. This allows
users to
+/// specify connection information in a file or environment variable, and have
the
+/// driver manager load the appropriate driver and set options accordingly.
+///
+/// This allows creating reusable connection configurations for sharing and
distribution
+/// without needing to hardcode driver names and options in application code.
Profiles
+/// will be loaded during DatabaseInit before attempting to initialize the
driver. Any
+/// options specified by the profile will be applied but will not override
options
+/// that have already been set using DatabaseSetOption.
+///
+/// To facilitate customization, we define an interface for implementing a
Connection
+/// Profile object along with a provider function definition which can be set
into
+/// the driver manager to allow for customized profile loading.
+///
+/// A profile can be specified to the Driver Manager in one of two ways,
+/// which will invoke the profile provider during the call to DatabaseInit:
+///
+/// 1. The "profile" option can be set using DatabaseSetOption with the name
of the
+/// profile to load.
+/// 2. The "uri" being used can have the form "profile://"
+///
+/// @{
+
+/// \brief Abstract interface for connection profile providers
+struct ADBC_EXPORT AdbcConnectionProfile {
+ /// \brief Opaque implementation-defined state.
+ /// This field is NULL if the profile is uninitialized/freed (but
+ /// it need not have a value even if the profile is initialized).
+ void* private_data;
+
+ /// \brief Release the profile and perform any cleanup.
+ void (*release)(struct AdbcConnectionProfile* profile);
+
+ /// \brief Get the driver to use as specified by this profile.
+ ///
+ /// It is not required that a profile specify a driver. If the options
+ // can be reusable across drivers, then the profile does not need to specify
+ /// a driver (if this provides an empty string or nullptr then the driver
+ /// must be defined by other means, e.g. by the driver / uri options).
+ ///
+ /// \param[in] profile The profile to query.
+ /// \param[out] driver_name The name of the driver to use, or NULL if not
specified.
+ /// \param[out] error An optional location to return an error message
+ AdbcStatusCode (*GetDriverName)(struct AdbcConnectionProfile* profile,
+ const char** driver_name, struct AdbcError*
error);
Review Comment:
I personally prefer not, but I'm open to it if others think it's worthwhile.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
Re: [PR] feat(c/driver_manager): add connection profile interface [arrow-adbc]
zeroshade commented on code in PR #3876:
URL: https://github.com/apache/arrow-adbc/pull/3876#discussion_r2677965868
##
c/include/arrow-adbc/adbc_driver_manager.h:
##
@@ -175,6 +175,171 @@ AdbcStatusCode
AdbcDriverManagerDatabaseSetAdditionalSearchPathList(
ADBC_EXPORT
const char* AdbcStatusCodeMessage(AdbcStatusCode code);
+/// \defgroup adbc-driver-manager-connection-profile Connection Profiles
+/// Similar to odbc.ini, the ADBC driver manager can support "connection
profiles"
+/// that specify a driver and options to use when connecting. This allows
users to
+/// specify connection information in a file or environment variable, and have
the
+/// driver manager load the appropriate driver and set options accordingly.
+///
+/// This allows creating reusable connection configurations for sharing and
distribution
+/// without needing to hardcode driver names and options in application code.
Profiles
+/// will be loaded during DatabaseInit before attempting to initialize the
driver. Any
+/// options specified by the profile will be applied but will not override
options
+/// that have already been set using DatabaseSetOption.
+///
+/// To facilitate customization, we define an interface for implementing a
Connection
+/// Profile object along with a provider function definition which can be set
into
+/// the driver manager to allow for customized profile loading.
+///
+/// A profile can be specified to the Driver Manager in one of two ways,
+/// which will invoke the profile provider during the call to DatabaseInit:
+///
+/// 1. The "profile" option can be set using DatabaseSetOption with the name
of the
+/// profile to load.
+/// 2. The "uri" being used can have the form "profile://"
+///
+/// @{
+
+/// \brief Abstract interface for connection profile providers
+struct ADBC_EXPORT AdbcConnectionProfile {
+ /// \brief Opaque implementation-defined state.
+ /// This field is NULL if the profile is uninitialized/freed (but
+ /// it need not have a value even if the profile is initialized).
+ void* private_data;
+
+ /// \brief Release the profile and perform any cleanup.
+ void (*release)(struct AdbcConnectionProfile* profile);
+
+ /// \brief Get the driver to use as specified by this profile.
+ ///
+ /// It is not required that a profile specify a driver. If the options
+ // can be reusable across drivers, then the profile does not need to specify
+ /// a driver (if this provides an empty string or nullptr then the driver
+ /// must be defined by other means, e.g. by the driver / uri options).
+ ///
+ /// \param[in] profile The profile to query.
+ /// \param[out] driver_name The name of the driver to use, or NULL if not
specified.
+ /// \param[out] error An optional location to return an error message
+ AdbcStatusCode (*GetDriverName)(struct AdbcConnectionProfile* profile,
+ const char** driver_name, struct AdbcError*
error);
+
+ /// \brief Get the string options specified by the profile
+ ///
+ /// The keys and values returned by this function are owned by the profile
+ /// object itself and do not need to be freed or managed by the caller.
+ /// They must not be accessed after calling release on the profile.
+ ///
+ /// The profile can also indicate that a value should be pulled from the
environment
+ /// by having a value in the form `env_var(ENV_VAR_NAME)`. If the driver
+ /// manager encounters a value of this form, it will replace it with the
actual value
+ /// of the environment variable `ENV_VAR_NAME` before setting the option.
This
+ /// is only valid for option *values* not *keys*.
+ ///
+ /// \param[in] profile The profile to query.
+ /// \param[out] keys The keys of the options specified by the profile.
+ /// \param[out] values The values of the options specified by the profile.
+ /// \param[out] num_options The number of options specified by the profile,
+ /// consumers must not access keys or values beyond this count.
+ /// \param[out] error An optional location to return an error message
+ AdbcStatusCode (*GetOptions)(struct AdbcConnectionProfile* profile, const
char*** keys,
+ const char*** values, size_t* num_options,
+ struct AdbcError* error);
+
+ /// \brief Get the integer options specified by the profile
+ ///
+ /// The keys and values returned by this function are owned by the profile
+ /// object itself and do not need to be freed or managed by the caller. They
must not be
+ /// accessed after calling release on the profile.
+ ///
+ /// Values returned by this function will be set using the
DatabaseSetOptionInt function
+ /// on the database object being initialized. If the driver does not support
the
+ /// DatabaseSetOptionInt function, then options should only be returned as
strings.
+ ///
+ /// \param[in] profile The profile to query.
+ /// \param[out] keys The keys of the options specified by the profi
Re: [PR] feat(c/driver_manager): add connection profile interface [arrow-adbc]
davidhcoe commented on PR #3876: URL: https://github.com/apache/arrow-adbc/pull/3876#issuecomment-3730963399 @CurtHagenlocher will be interested in this as well. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [PR] feat(c/driver_manager): add connection profile interface [arrow-adbc]
zeroshade commented on code in PR #3876:
URL: https://github.com/apache/arrow-adbc/pull/3876#discussion_r2677896193
##
go/adbc/drivermgr/adbc_driver_manager.cc:
##
@@ -1857,20 +2218,188 @@ AdbcStatusCode
AdbcDriverManagerDatabaseSetInitFunc(struct AdbcDatabase* databas
return ADBC_STATUS_OK;
}
+AdbcStatusCode AdbcFilesystemProfileProvider(const char* profile_name,
+ const char*
additional_search_path_list,
+ struct AdbcConnectionProfile* out,
+ struct AdbcError* error) {
+ if (profile_name == nullptr || strlen(profile_name) == 0) {
+SetError(error, "Profile name is empty");
+return ADBC_STATUS_INVALID_ARGUMENT;
+ }
+
+ if (!out) {
+SetError(error, "Output profile is null");
+return ADBC_STATUS_INVALID_ARGUMENT;
+ }
+
+ std::memset(out, 0, sizeof(*out));
+ std::filesystem::path profile_path(profile_name);
+ if (profile_path.has_extension()) {
+if (HasExtension(profile_path, ".toml")) {
+ if (!std::filesystem::exists(profile_path)) {
+SetError(error, "Profile file does not exist: " +
profile_path.string());
+return ADBC_STATUS_NOT_FOUND;
+ }
+
+ FilesystemProfile profile;
+ CHECK_STATUS(LoadProfileFile(profile_path, profile, error));
+ FilesystemProfile::populate_connection_profile(std::move(profile), out);
+ return ADBC_STATUS_OK;
+}
+ }
+
+ if (profile_path.is_absolute()) {
+profile_path.replace_extension(".toml");
+
+FilesystemProfile profile;
+CHECK_STATUS(LoadProfileFile(profile_path, profile, error));
+FilesystemProfile::populate_connection_profile(std::move(profile), out);
+return ADBC_STATUS_OK;
+ }
+
+ SearchPaths search_paths =
GetProfileSearchPaths(additional_search_path_list);
+ SearchPaths extra_debug_info;
+ for (const auto& [source, search_path] : search_paths) {
+if (source == SearchPathSource::kRegistry || source ==
SearchPathSource::kUnset ||
+source == SearchPathSource::kDoesNotExist ||
+source == SearchPathSource::kDisabledAtCompileTime ||
+source == SearchPathSource::kDisabledAtRunTime ||
+source == SearchPathSource::kOtherError) {
+ continue;
+}
+
+std::filesystem::path full_path = search_path / profile_path;
+full_path.replace_extension(".toml");
+if (std::filesystem::exists(full_path)) {
+ OwnedError intermediate_error;
+
+ FilesystemProfile profile;
+ auto status = LoadProfileFile(full_path, profile,
&intermediate_error.error);
+ if (status == ADBC_STATUS_OK) {
+FilesystemProfile::populate_connection_profile(std::move(profile),
out);
+return ADBC_STATUS_OK;
+ } else if (status == ADBC_STATUS_INVALID_ARGUMENT) {
+search_paths.insert(search_paths.end(), extra_debug_info.begin(),
+extra_debug_info.end());
+if (intermediate_error.error.message) {
+ std::string error_message = intermediate_error.error.message;
+ AddSearchPathsToError(search_paths, SearchPathType::kProfile,
error_message);
+ SetError(error, std::move(error_message));
+}
+return status;
+ }
+
+ std::string message = "found ";
+ message += full_path.string();
+ message += " but: ";
+ if (intermediate_error.error.message) {
+message += intermediate_error.error.message;
+ } else {
+message += "could not load the profile";
+ }
+ extra_debug_info.emplace_back(SearchPathSource::kOtherError,
std::move(message));
+}
+ }
+
+ search_paths.insert(search_paths.end(), extra_debug_info.begin(),
+ extra_debug_info.end());
+ std::string error_message = "Profile not found: " +
std::string(profile_name);
+ AddSearchPathsToError(search_paths, SearchPathType::kProfile, error_message);
+ SetError(error, std::move(error_message));
+ return ADBC_STATUS_NOT_FOUND;
+}
+
+struct ProfileGuard {
+ AdbcConnectionProfile& profile;
+ ProfileGuard(AdbcConnectionProfile& profile) : profile(profile) {}
+ ~ProfileGuard() {
+if (profile.release) {
+ profile.release(&profile);
+}
+ }
+};
+
+AdbcStatusCode InternalInitializeProfile(TempDatabase* args,
+ const std::string_view profile,
+ struct AdbcError* error) {
+ if (!args->profile_provider) {
+args->profile_provider = AdbcFilesystemProfileProvider;
+ }
+
+ AdbcConnectionProfile connection_profile;
Review Comment:
possibly, though the default does do a memset but it's better to zero it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastruct
Re: [PR] feat(c/driver_manager): add connection profile interface [arrow-adbc]
paleolimbot commented on code in PR #3876:
URL: https://github.com/apache/arrow-adbc/pull/3876#discussion_r2677892547
##
go/adbc/drivermgr/adbc_driver_manager.cc:
##
@@ -1857,20 +2218,188 @@ AdbcStatusCode
AdbcDriverManagerDatabaseSetInitFunc(struct AdbcDatabase* databas
return ADBC_STATUS_OK;
}
+AdbcStatusCode AdbcFilesystemProfileProvider(const char* profile_name,
+ const char*
additional_search_path_list,
+ struct AdbcConnectionProfile* out,
+ struct AdbcError* error) {
+ if (profile_name == nullptr || strlen(profile_name) == 0) {
+SetError(error, "Profile name is empty");
+return ADBC_STATUS_INVALID_ARGUMENT;
+ }
+
+ if (!out) {
+SetError(error, "Output profile is null");
+return ADBC_STATUS_INVALID_ARGUMENT;
+ }
+
+ std::memset(out, 0, sizeof(*out));
+ std::filesystem::path profile_path(profile_name);
+ if (profile_path.has_extension()) {
+if (HasExtension(profile_path, ".toml")) {
+ if (!std::filesystem::exists(profile_path)) {
+SetError(error, "Profile file does not exist: " +
profile_path.string());
+return ADBC_STATUS_NOT_FOUND;
+ }
+
+ FilesystemProfile profile;
+ CHECK_STATUS(LoadProfileFile(profile_path, profile, error));
+ FilesystemProfile::populate_connection_profile(std::move(profile), out);
+ return ADBC_STATUS_OK;
+}
+ }
+
+ if (profile_path.is_absolute()) {
+profile_path.replace_extension(".toml");
+
+FilesystemProfile profile;
+CHECK_STATUS(LoadProfileFile(profile_path, profile, error));
+FilesystemProfile::populate_connection_profile(std::move(profile), out);
+return ADBC_STATUS_OK;
+ }
+
+ SearchPaths search_paths =
GetProfileSearchPaths(additional_search_path_list);
+ SearchPaths extra_debug_info;
+ for (const auto& [source, search_path] : search_paths) {
+if (source == SearchPathSource::kRegistry || source ==
SearchPathSource::kUnset ||
+source == SearchPathSource::kDoesNotExist ||
+source == SearchPathSource::kDisabledAtCompileTime ||
+source == SearchPathSource::kDisabledAtRunTime ||
+source == SearchPathSource::kOtherError) {
+ continue;
+}
+
+std::filesystem::path full_path = search_path / profile_path;
+full_path.replace_extension(".toml");
+if (std::filesystem::exists(full_path)) {
+ OwnedError intermediate_error;
+
+ FilesystemProfile profile;
+ auto status = LoadProfileFile(full_path, profile,
&intermediate_error.error);
+ if (status == ADBC_STATUS_OK) {
+FilesystemProfile::populate_connection_profile(std::move(profile),
out);
+return ADBC_STATUS_OK;
+ } else if (status == ADBC_STATUS_INVALID_ARGUMENT) {
+search_paths.insert(search_paths.end(), extra_debug_info.begin(),
+extra_debug_info.end());
+if (intermediate_error.error.message) {
+ std::string error_message = intermediate_error.error.message;
+ AddSearchPathsToError(search_paths, SearchPathType::kProfile,
error_message);
+ SetError(error, std::move(error_message));
+}
+return status;
+ }
+
+ std::string message = "found ";
+ message += full_path.string();
+ message += " but: ";
+ if (intermediate_error.error.message) {
+message += intermediate_error.error.message;
+ } else {
+message += "could not load the profile";
+ }
+ extra_debug_info.emplace_back(SearchPathSource::kOtherError,
std::move(message));
+}
+ }
+
+ search_paths.insert(search_paths.end(), extra_debug_info.begin(),
+ extra_debug_info.end());
+ std::string error_message = "Profile not found: " +
std::string(profile_name);
+ AddSearchPathsToError(search_paths, SearchPathType::kProfile, error_message);
+ SetError(error, std::move(error_message));
+ return ADBC_STATUS_NOT_FOUND;
+}
+
+struct ProfileGuard {
+ AdbcConnectionProfile& profile;
+ ProfileGuard(AdbcConnectionProfile& profile) : profile(profile) {}
+ ~ProfileGuard() {
+if (profile.release) {
+ profile.release(&profile);
+}
+ }
+};
+
+AdbcStatusCode InternalInitializeProfile(TempDatabase* args,
+ const std::string_view profile,
+ struct AdbcError* error) {
+ if (!args->profile_provider) {
+args->profile_provider = AdbcFilesystemProfileProvider;
+ }
+
+ AdbcConnectionProfile connection_profile;
Review Comment:
```suggestion
AdbcConnectionProfile connection_profile{};
```
Should this be zeroed?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service
Re: [PR] feat(c/driver_manager): add connection profile interface [arrow-adbc]
paleolimbot commented on code in PR #3876:
URL: https://github.com/apache/arrow-adbc/pull/3876#discussion_r2677858803
##
c/include/arrow-adbc/adbc_driver_manager.h:
##
@@ -175,6 +175,171 @@ AdbcStatusCode
AdbcDriverManagerDatabaseSetAdditionalSearchPathList(
ADBC_EXPORT
const char* AdbcStatusCodeMessage(AdbcStatusCode code);
+/// \defgroup adbc-driver-manager-connection-profile Connection Profiles
+/// Similar to odbc.ini, the ADBC driver manager can support "connection
profiles"
+/// that specify a driver and options to use when connecting. This allows
users to
+/// specify connection information in a file or environment variable, and have
the
+/// driver manager load the appropriate driver and set options accordingly.
+///
+/// This allows creating reusable connection configurations for sharing and
distribution
+/// without needing to hardcode driver names and options in application code.
Profiles
+/// will be loaded during DatabaseInit before attempting to initialize the
driver. Any
+/// options specified by the profile will be applied but will not override
options
+/// that have already been set using DatabaseSetOption.
+///
+/// To facilitate customization, we define an interface for implementing a
Connection
+/// Profile object along with a provider function definition which can be set
into
+/// the driver manager to allow for customized profile loading.
+///
+/// A profile can be specified to the Driver Manager in one of two ways,
+/// which will invoke the profile provider during the call to DatabaseInit:
+///
+/// 1. The "profile" option can be set using DatabaseSetOption with the name
of the
+/// profile to load.
+/// 2. The "uri" being used can have the form "profile://"
+///
+/// @{
+
+/// \brief Abstract interface for connection profile providers
+struct ADBC_EXPORT AdbcConnectionProfile {
+ /// \brief Opaque implementation-defined state.
+ /// This field is NULL if the profile is uninitialized/freed (but
+ /// it need not have a value even if the profile is initialized).
+ void* private_data;
+
+ /// \brief Release the profile and perform any cleanup.
+ void (*release)(struct AdbcConnectionProfile* profile);
+
+ /// \brief Get the driver to use as specified by this profile.
+ ///
+ /// It is not required that a profile specify a driver. If the options
+ // can be reusable across drivers, then the profile does not need to specify
+ /// a driver (if this provides an empty string or nullptr then the driver
+ /// must be defined by other means, e.g. by the driver / uri options).
+ ///
+ /// \param[in] profile The profile to query.
+ /// \param[out] driver_name The name of the driver to use, or NULL if not
specified.
+ /// \param[out] error An optional location to return an error message
+ AdbcStatusCode (*GetDriverName)(struct AdbcConnectionProfile* profile,
+ const char** driver_name, struct AdbcError*
error);
+
+ /// \brief Get the string options specified by the profile
+ ///
+ /// The keys and values returned by this function are owned by the profile
+ /// object itself and do not need to be freed or managed by the caller.
+ /// They must not be accessed after calling release on the profile.
+ ///
+ /// The profile can also indicate that a value should be pulled from the
environment
+ /// by having a value in the form `env_var(ENV_VAR_NAME)`. If the driver
+ /// manager encounters a value of this form, it will replace it with the
actual value
+ /// of the environment variable `ENV_VAR_NAME` before setting the option.
This
+ /// is only valid for option *values* not *keys*.
+ ///
+ /// \param[in] profile The profile to query.
+ /// \param[out] keys The keys of the options specified by the profile.
+ /// \param[out] values The values of the options specified by the profile.
+ /// \param[out] num_options The number of options specified by the profile,
+ /// consumers must not access keys or values beyond this count.
+ /// \param[out] error An optional location to return an error message
+ AdbcStatusCode (*GetOptions)(struct AdbcConnectionProfile* profile, const
char*** keys,
+ const char*** values, size_t* num_options,
+ struct AdbcError* error);
+
+ /// \brief Get the integer options specified by the profile
+ ///
+ /// The keys and values returned by this function are owned by the profile
+ /// object itself and do not need to be freed or managed by the caller. They
must not be
+ /// accessed after calling release on the profile.
+ ///
+ /// Values returned by this function will be set using the
DatabaseSetOptionInt function
+ /// on the database object being initialized. If the driver does not support
the
+ /// DatabaseSetOptionInt function, then options should only be returned as
strings.
+ ///
+ /// \param[in] profile The profile to query.
+ /// \param[out] keys The keys of the options specified by the pro
Re: [PR] feat(c/driver_manager): add connection profile interface [arrow-adbc]
paleolimbot commented on code in PR #3876:
URL: https://github.com/apache/arrow-adbc/pull/3876#discussion_r2677854878
##
c/include/arrow-adbc/adbc_driver_manager.h:
##
@@ -175,6 +175,171 @@ AdbcStatusCode
AdbcDriverManagerDatabaseSetAdditionalSearchPathList(
ADBC_EXPORT
const char* AdbcStatusCodeMessage(AdbcStatusCode code);
+/// \defgroup adbc-driver-manager-connection-profile Connection Profiles
+/// Similar to odbc.ini, the ADBC driver manager can support "connection
profiles"
+/// that specify a driver and options to use when connecting. This allows
users to
+/// specify connection information in a file or environment variable, and have
the
+/// driver manager load the appropriate driver and set options accordingly.
+///
+/// This allows creating reusable connection configurations for sharing and
distribution
+/// without needing to hardcode driver names and options in application code.
Profiles
+/// will be loaded during DatabaseInit before attempting to initialize the
driver. Any
+/// options specified by the profile will be applied but will not override
options
+/// that have already been set using DatabaseSetOption.
+///
+/// To facilitate customization, we define an interface for implementing a
Connection
+/// Profile object along with a provider function definition which can be set
into
+/// the driver manager to allow for customized profile loading.
+///
+/// A profile can be specified to the Driver Manager in one of two ways,
+/// which will invoke the profile provider during the call to DatabaseInit:
+///
+/// 1. The "profile" option can be set using DatabaseSetOption with the name
of the
+/// profile to load.
+/// 2. The "uri" being used can have the form "profile://"
+///
+/// @{
+
+/// \brief Abstract interface for connection profile providers
+struct ADBC_EXPORT AdbcConnectionProfile {
+ /// \brief Opaque implementation-defined state.
+ /// This field is NULL if the profile is uninitialized/freed (but
+ /// it need not have a value even if the profile is initialized).
+ void* private_data;
+
+ /// \brief Release the profile and perform any cleanup.
+ void (*release)(struct AdbcConnectionProfile* profile);
+
+ /// \brief Get the driver to use as specified by this profile.
+ ///
+ /// It is not required that a profile specify a driver. If the options
+ // can be reusable across drivers, then the profile does not need to specify
+ /// a driver (if this provides an empty string or nullptr then the driver
+ /// must be defined by other means, e.g. by the driver / uri options).
+ ///
+ /// \param[in] profile The profile to query.
+ /// \param[out] driver_name The name of the driver to use, or NULL if not
specified.
+ /// \param[out] error An optional location to return an error message
+ AdbcStatusCode (*GetDriverName)(struct AdbcConnectionProfile* profile,
+ const char** driver_name, struct AdbcError*
error);
Review Comment:
I think you'd just set `args->init_func` instead of `args->driver`
(understood if you'd prefer not to)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
Re: [PR] feat(c/driver_manager): add connection profile interface [arrow-adbc]
zeroshade commented on code in PR #3876:
URL: https://github.com/apache/arrow-adbc/pull/3876#discussion_r2677835327
##
c/include/arrow-adbc/adbc_driver_manager.h:
##
@@ -175,6 +175,171 @@ AdbcStatusCode
AdbcDriverManagerDatabaseSetAdditionalSearchPathList(
ADBC_EXPORT
const char* AdbcStatusCodeMessage(AdbcStatusCode code);
+/// \defgroup adbc-driver-manager-connection-profile Connection Profiles
+/// Similar to odbc.ini, the ADBC driver manager can support "connection
profiles"
+/// that specify a driver and options to use when connecting. This allows
users to
+/// specify connection information in a file or environment variable, and have
the
+/// driver manager load the appropriate driver and set options accordingly.
+///
+/// This allows creating reusable connection configurations for sharing and
distribution
+/// without needing to hardcode driver names and options in application code.
Profiles
+/// will be loaded during DatabaseInit before attempting to initialize the
driver. Any
+/// options specified by the profile will be applied but will not override
options
+/// that have already been set using DatabaseSetOption.
+///
+/// To facilitate customization, we define an interface for implementing a
Connection
+/// Profile object along with a provider function definition which can be set
into
+/// the driver manager to allow for customized profile loading.
+///
+/// A profile can be specified to the Driver Manager in one of two ways,
+/// which will invoke the profile provider during the call to DatabaseInit:
+///
+/// 1. The "profile" option can be set using DatabaseSetOption with the name
of the
+/// profile to load.
+/// 2. The "uri" being used can have the form "profile://"
+///
+/// @{
+
+/// \brief Abstract interface for connection profile providers
+struct ADBC_EXPORT AdbcConnectionProfile {
+ /// \brief Opaque implementation-defined state.
+ /// This field is NULL if the profile is uninitialized/freed (but
+ /// it need not have a value even if the profile is initialized).
+ void* private_data;
+
+ /// \brief Release the profile and perform any cleanup.
+ void (*release)(struct AdbcConnectionProfile* profile);
+
+ /// \brief Get the driver to use as specified by this profile.
+ ///
+ /// It is not required that a profile specify a driver. If the options
+ // can be reusable across drivers, then the profile does not need to specify
+ /// a driver (if this provides an empty string or nullptr then the driver
+ /// must be defined by other means, e.g. by the driver / uri options).
+ ///
+ /// \param[in] profile The profile to query.
+ /// \param[out] driver_name The name of the driver to use, or NULL if not
specified.
+ /// \param[out] error An optional location to return an error message
+ AdbcStatusCode (*GetDriverName)(struct AdbcConnectionProfile* profile,
+ const char** driver_name, struct AdbcError*
error);
+
+ /// \brief Get the string options specified by the profile
+ ///
+ /// The keys and values returned by this function are owned by the profile
+ /// object itself and do not need to be freed or managed by the caller.
+ /// They must not be accessed after calling release on the profile.
+ ///
+ /// The profile can also indicate that a value should be pulled from the
environment
+ /// by having a value in the form `env_var(ENV_VAR_NAME)`. If the driver
+ /// manager encounters a value of this form, it will replace it with the
actual value
+ /// of the environment variable `ENV_VAR_NAME` before setting the option.
This
+ /// is only valid for option *values* not *keys*.
+ ///
+ /// \param[in] profile The profile to query.
+ /// \param[out] keys The keys of the options specified by the profile.
+ /// \param[out] values The values of the options specified by the profile.
+ /// \param[out] num_options The number of options specified by the profile,
+ /// consumers must not access keys or values beyond this count.
+ /// \param[out] error An optional location to return an error message
+ AdbcStatusCode (*GetOptions)(struct AdbcConnectionProfile* profile, const
char*** keys,
+ const char*** values, size_t* num_options,
+ struct AdbcError* error);
+
+ /// \brief Get the integer options specified by the profile
+ ///
+ /// The keys and values returned by this function are owned by the profile
+ /// object itself and do not need to be freed or managed by the caller. They
must not be
+ /// accessed after calling release on the profile.
+ ///
+ /// Values returned by this function will be set using the
DatabaseSetOptionInt function
+ /// on the database object being initialized. If the driver does not support
the
+ /// DatabaseSetOptionInt function, then options should only be returned as
strings.
+ ///
+ /// \param[in] profile The profile to query.
+ /// \param[out] keys The keys of the options specified by the profi
Re: [PR] feat(c/driver_manager): add connection profile interface [arrow-adbc]
zeroshade commented on code in PR #3876:
URL: https://github.com/apache/arrow-adbc/pull/3876#discussion_r2677817138
##
c/include/arrow-adbc/adbc_driver_manager.h:
##
@@ -175,6 +175,171 @@ AdbcStatusCode
AdbcDriverManagerDatabaseSetAdditionalSearchPathList(
ADBC_EXPORT
const char* AdbcStatusCodeMessage(AdbcStatusCode code);
+/// \defgroup adbc-driver-manager-connection-profile Connection Profiles
+/// Similar to odbc.ini, the ADBC driver manager can support "connection
profiles"
+/// that specify a driver and options to use when connecting. This allows
users to
+/// specify connection information in a file or environment variable, and have
the
+/// driver manager load the appropriate driver and set options accordingly.
+///
+/// This allows creating reusable connection configurations for sharing and
distribution
+/// without needing to hardcode driver names and options in application code.
Profiles
+/// will be loaded during DatabaseInit before attempting to initialize the
driver. Any
+/// options specified by the profile will be applied but will not override
options
+/// that have already been set using DatabaseSetOption.
+///
+/// To facilitate customization, we define an interface for implementing a
Connection
+/// Profile object along with a provider function definition which can be set
into
+/// the driver manager to allow for customized profile loading.
+///
+/// A profile can be specified to the Driver Manager in one of two ways,
+/// which will invoke the profile provider during the call to DatabaseInit:
+///
+/// 1. The "profile" option can be set using DatabaseSetOption with the name
of the
+/// profile to load.
+/// 2. The "uri" being used can have the form "profile://"
+///
+/// @{
+
+/// \brief Abstract interface for connection profile providers
+struct ADBC_EXPORT AdbcConnectionProfile {
+ /// \brief Opaque implementation-defined state.
+ /// This field is NULL if the profile is uninitialized/freed (but
+ /// it need not have a value even if the profile is initialized).
+ void* private_data;
+
+ /// \brief Release the profile and perform any cleanup.
+ void (*release)(struct AdbcConnectionProfile* profile);
+
+ /// \brief Get the driver to use as specified by this profile.
+ ///
+ /// It is not required that a profile specify a driver. If the options
+ // can be reusable across drivers, then the profile does not need to specify
+ /// a driver (if this provides an empty string or nullptr then the driver
+ /// must be defined by other means, e.g. by the driver / uri options).
+ ///
+ /// \param[in] profile The profile to query.
+ /// \param[out] driver_name The name of the driver to use, or NULL if not
specified.
+ /// \param[out] error An optional location to return an error message
+ AdbcStatusCode (*GetDriverName)(struct AdbcConnectionProfile* profile,
+ const char** driver_name, struct AdbcError*
error);
+
+ /// \brief Get the string options specified by the profile
+ ///
+ /// The keys and values returned by this function are owned by the profile
+ /// object itself and do not need to be freed or managed by the caller.
+ /// They must not be accessed after calling release on the profile.
+ ///
+ /// The profile can also indicate that a value should be pulled from the
environment
+ /// by having a value in the form `env_var(ENV_VAR_NAME)`. If the driver
+ /// manager encounters a value of this form, it will replace it with the
actual value
+ /// of the environment variable `ENV_VAR_NAME` before setting the option.
This
+ /// is only valid for option *values* not *keys*.
+ ///
+ /// \param[in] profile The profile to query.
+ /// \param[out] keys The keys of the options specified by the profile.
+ /// \param[out] values The values of the options specified by the profile.
+ /// \param[out] num_options The number of options specified by the profile,
+ /// consumers must not access keys or values beyond this count.
+ /// \param[out] error An optional location to return an error message
+ AdbcStatusCode (*GetOptions)(struct AdbcConnectionProfile* profile, const
char*** keys,
+ const char*** values, size_t* num_options,
+ struct AdbcError* error);
+
+ /// \brief Get the integer options specified by the profile
+ ///
+ /// The keys and values returned by this function are owned by the profile
+ /// object itself and do not need to be freed or managed by the caller. They
must not be
+ /// accessed after calling release on the profile.
+ ///
+ /// Values returned by this function will be set using the
DatabaseSetOptionInt function
+ /// on the database object being initialized. If the driver does not support
the
+ /// DatabaseSetOptionInt function, then options should only be returned as
strings.
+ ///
+ /// \param[in] profile The profile to query.
+ /// \param[out] keys The keys of the options specified by the profi
Re: [PR] feat(c/driver_manager): add connection profile interface [arrow-adbc]
zeroshade commented on code in PR #3876:
URL: https://github.com/apache/arrow-adbc/pull/3876#discussion_r2677816292
##
go/adbc/drivermgr/adbc_driver_manager.cc:
##
@@ -1857,20 +2278,96 @@ AdbcStatusCode
AdbcDriverManagerDatabaseSetInitFunc(struct AdbcDatabase* databas
return ADBC_STATUS_OK;
}
+struct ProfileGuard {
+ AdbcConnectionProfile* profile;
+ ~ProfileGuard() {
+if (profile) {
+ profile->release(profile);
+}
Review Comment:
good point, i'll fix this
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
Re: [PR] feat(c/driver_manager): add connection profile interface [arrow-adbc]
zeroshade commented on code in PR #3876:
URL: https://github.com/apache/arrow-adbc/pull/3876#discussion_r2677808417
##
c/include/arrow-adbc/adbc_driver_manager.h:
##
@@ -175,6 +175,171 @@ AdbcStatusCode
AdbcDriverManagerDatabaseSetAdditionalSearchPathList(
ADBC_EXPORT
const char* AdbcStatusCodeMessage(AdbcStatusCode code);
+/// \defgroup adbc-driver-manager-connection-profile Connection Profiles
+/// Similar to odbc.ini, the ADBC driver manager can support "connection
profiles"
+/// that specify a driver and options to use when connecting. This allows
users to
+/// specify connection information in a file or environment variable, and have
the
+/// driver manager load the appropriate driver and set options accordingly.
+///
+/// This allows creating reusable connection configurations for sharing and
distribution
+/// without needing to hardcode driver names and options in application code.
Profiles
+/// will be loaded during DatabaseInit before attempting to initialize the
driver. Any
+/// options specified by the profile will be applied but will not override
options
+/// that have already been set using DatabaseSetOption.
+///
+/// To facilitate customization, we define an interface for implementing a
Connection
+/// Profile object along with a provider function definition which can be set
into
+/// the driver manager to allow for customized profile loading.
+///
+/// A profile can be specified to the Driver Manager in one of two ways,
+/// which will invoke the profile provider during the call to DatabaseInit:
+///
+/// 1. The "profile" option can be set using DatabaseSetOption with the name
of the
+/// profile to load.
+/// 2. The "uri" being used can have the form "profile://"
+///
+/// @{
+
+/// \brief Abstract interface for connection profile providers
+struct ADBC_EXPORT AdbcConnectionProfile {
+ /// \brief Opaque implementation-defined state.
+ /// This field is NULL if the profile is uninitialized/freed (but
+ /// it need not have a value even if the profile is initialized).
+ void* private_data;
+
+ /// \brief Release the profile and perform any cleanup.
+ void (*release)(struct AdbcConnectionProfile* profile);
+
+ /// \brief Get the driver to use as specified by this profile.
+ ///
+ /// It is not required that a profile specify a driver. If the options
+ // can be reusable across drivers, then the profile does not need to specify
+ /// a driver (if this provides an empty string or nullptr then the driver
+ /// must be defined by other means, e.g. by the driver / uri options).
+ ///
+ /// \param[in] profile The profile to query.
+ /// \param[out] driver_name The name of the driver to use, or NULL if not
specified.
+ /// \param[out] error An optional location to return an error message
+ AdbcStatusCode (*GetDriverName)(struct AdbcConnectionProfile* profile,
+ const char** driver_name, struct AdbcError*
error);
Review Comment:
The way the code is set up doesn't allow for this. If there is an init func
set then the profile handling is currently entirely skipped and it's up to the
init_func to handle everything. Given the way we envision profiles working, I
don't think that it makes sense for the profile to set or specify the init
function.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
Re: [PR] feat(c/driver_manager): add connection profile interface [arrow-adbc]
zeroshade commented on code in PR #3876:
URL: https://github.com/apache/arrow-adbc/pull/3876#discussion_r2677804209
##
c/include/arrow-adbc/adbc_driver_manager.h:
##
@@ -175,6 +175,171 @@ AdbcStatusCode
AdbcDriverManagerDatabaseSetAdditionalSearchPathList(
ADBC_EXPORT
const char* AdbcStatusCodeMessage(AdbcStatusCode code);
+/// \defgroup adbc-driver-manager-connection-profile Connection Profiles
+/// Similar to odbc.ini, the ADBC driver manager can support "connection
profiles"
+/// that specify a driver and options to use when connecting. This allows
users to
+/// specify connection information in a file or environment variable, and have
the
+/// driver manager load the appropriate driver and set options accordingly.
+///
+/// This allows creating reusable connection configurations for sharing and
distribution
+/// without needing to hardcode driver names and options in application code.
Profiles
+/// will be loaded during DatabaseInit before attempting to initialize the
driver. Any
+/// options specified by the profile will be applied but will not override
options
+/// that have already been set using DatabaseSetOption.
+///
+/// To facilitate customization, we define an interface for implementing a
Connection
+/// Profile object along with a provider function definition which can be set
into
+/// the driver manager to allow for customized profile loading.
+///
+/// A profile can be specified to the Driver Manager in one of two ways,
+/// which will invoke the profile provider during the call to DatabaseInit:
+///
+/// 1. The "profile" option can be set using DatabaseSetOption with the name
of the
+/// profile to load.
+/// 2. The "uri" being used can have the form "profile://"
+///
+/// @{
+
+/// \brief Abstract interface for connection profile providers
+struct ADBC_EXPORT AdbcConnectionProfile {
+ /// \brief Opaque implementation-defined state.
+ /// This field is NULL if the profile is uninitialized/freed (but
+ /// it need not have a value even if the profile is initialized).
+ void* private_data;
+
+ /// \brief Release the profile and perform any cleanup.
+ void (*release)(struct AdbcConnectionProfile* profile);
+
+ /// \brief Get the driver to use as specified by this profile.
+ ///
+ /// It is not required that a profile specify a driver. If the options
+ // can be reusable across drivers, then the profile does not need to specify
+ /// a driver (if this provides an empty string or nullptr then the driver
+ /// must be defined by other means, e.g. by the driver / uri options).
+ ///
+ /// \param[in] profile The profile to query.
+ /// \param[out] driver_name The name of the driver to use, or NULL if not
specified.
+ /// \param[out] error An optional location to return an error message
+ AdbcStatusCode (*GetDriverName)(struct AdbcConnectionProfile* profile,
+ const char** driver_name, struct AdbcError*
error);
+
+ /// \brief Get the string options specified by the profile
+ ///
+ /// The keys and values returned by this function are owned by the profile
+ /// object itself and do not need to be freed or managed by the caller.
+ /// They must not be accessed after calling release on the profile.
+ ///
+ /// The profile can also indicate that a value should be pulled from the
environment
+ /// by having a value in the form `env_var(ENV_VAR_NAME)`. If the driver
+ /// manager encounters a value of this form, it will replace it with the
actual value
+ /// of the environment variable `ENV_VAR_NAME` before setting the option.
This
+ /// is only valid for option *values* not *keys*.
+ ///
+ /// \param[in] profile The profile to query.
+ /// \param[out] keys The keys of the options specified by the profile.
+ /// \param[out] values The values of the options specified by the profile.
+ /// \param[out] num_options The number of options specified by the profile,
+ /// consumers must not access keys or values beyond this count.
+ /// \param[out] error An optional location to return an error message
+ AdbcStatusCode (*GetOptions)(struct AdbcConnectionProfile* profile, const
char*** keys,
+ const char*** values, size_t* num_options,
+ struct AdbcError* error);
+
+ /// \brief Get the integer options specified by the profile
+ ///
+ /// The keys and values returned by this function are owned by the profile
+ /// object itself and do not need to be freed or managed by the caller. They
must not be
+ /// accessed after calling release on the profile.
+ ///
+ /// Values returned by this function will be set using the
DatabaseSetOptionInt function
+ /// on the database object being initialized. If the driver does not support
the
+ /// DatabaseSetOptionInt function, then options should only be returned as
strings.
+ ///
+ /// \param[in] profile The profile to query.
+ /// \param[out] keys The keys of the options specified by the profi
Re: [PR] feat(c/driver_manager): add connection profile interface [arrow-adbc]
paleolimbot commented on code in PR #3876:
URL: https://github.com/apache/arrow-adbc/pull/3876#discussion_r266946
##
c/include/arrow-adbc/adbc_driver_manager.h:
##
@@ -175,6 +175,171 @@ AdbcStatusCode
AdbcDriverManagerDatabaseSetAdditionalSearchPathList(
ADBC_EXPORT
const char* AdbcStatusCodeMessage(AdbcStatusCode code);
+/// \defgroup adbc-driver-manager-connection-profile Connection Profiles
+/// Similar to odbc.ini, the ADBC driver manager can support "connection
profiles"
+/// that specify a driver and options to use when connecting. This allows
users to
+/// specify connection information in a file or environment variable, and have
the
+/// driver manager load the appropriate driver and set options accordingly.
+///
+/// This allows creating reusable connection configurations for sharing and
distribution
+/// without needing to hardcode driver names and options in application code.
Profiles
+/// will be loaded during DatabaseInit before attempting to initialize the
driver. Any
+/// options specified by the profile will be applied but will not override
options
+/// that have already been set using DatabaseSetOption.
+///
+/// To facilitate customization, we define an interface for implementing a
Connection
+/// Profile object along with a provider function definition which can be set
into
+/// the driver manager to allow for customized profile loading.
+///
+/// A profile can be specified to the Driver Manager in one of two ways,
+/// which will invoke the profile provider during the call to DatabaseInit:
+///
+/// 1. The "profile" option can be set using DatabaseSetOption with the name
of the
+/// profile to load.
+/// 2. The "uri" being used can have the form "profile://"
+///
+/// @{
+
+/// \brief Abstract interface for connection profile providers
+struct ADBC_EXPORT AdbcConnectionProfile {
+ /// \brief Opaque implementation-defined state.
+ /// This field is NULL if the profile is uninitialized/freed (but
+ /// it need not have a value even if the profile is initialized).
+ void* private_data;
+
+ /// \brief Release the profile and perform any cleanup.
+ void (*release)(struct AdbcConnectionProfile* profile);
+
+ /// \brief Get the driver to use as specified by this profile.
+ ///
+ /// It is not required that a profile specify a driver. If the options
+ // can be reusable across drivers, then the profile does not need to specify
+ /// a driver (if this provides an empty string or nullptr then the driver
+ /// must be defined by other means, e.g. by the driver / uri options).
+ ///
+ /// \param[in] profile The profile to query.
+ /// \param[out] driver_name The name of the driver to use, or NULL if not
specified.
+ /// \param[out] error An optional location to return an error message
+ AdbcStatusCode (*GetDriverName)(struct AdbcConnectionProfile* profile,
+ const char** driver_name, struct AdbcError*
error);
Review Comment:
```suggestion
AdbcStatusCode (*GetDriverName)(struct AdbcConnectionProfile* profile,
const char** driver_name,
AdbcDriverInitFunc* init_func, struct AdbcError* error);
```
Is there any value to allowing a profile to optionally specify the init
function directly? (R could maybe use this to find R package versions of
drivers)
##
c/include/arrow-adbc/adbc_driver_manager.h:
##
@@ -175,6 +175,171 @@ AdbcStatusCode
AdbcDriverManagerDatabaseSetAdditionalSearchPathList(
ADBC_EXPORT
const char* AdbcStatusCodeMessage(AdbcStatusCode code);
+/// \defgroup adbc-driver-manager-connection-profile Connection Profiles
+/// Similar to odbc.ini, the ADBC driver manager can support "connection
profiles"
+/// that specify a driver and options to use when connecting. This allows
users to
+/// specify connection information in a file or environment variable, and have
the
+/// driver manager load the appropriate driver and set options accordingly.
+///
+/// This allows creating reusable connection configurations for sharing and
distribution
+/// without needing to hardcode driver names and options in application code.
Profiles
+/// will be loaded during DatabaseInit before attempting to initialize the
driver. Any
+/// options specified by the profile will be applied but will not override
options
+/// that have already been set using DatabaseSetOption.
+///
+/// To facilitate customization, we define an interface for implementing a
Connection
+/// Profile object along with a provider function definition which can be set
into
+/// the driver manager to allow for customized profile loading.
+///
+/// A profile can be specified to the Driver Manager in one of two ways,
+/// which will invoke the profile provider during the call to DatabaseInit:
+///
+/// 1. The "profile" option can be set using DatabaseSetOption with the name
of the
+/// profile to load.
+/// 2. The "uri" being used can have the form "profile://"
+///
+/// @
Re: [PR] feat(c/driver_manager): add connection profile interface [arrow-adbc]
zeroshade commented on PR #3876: URL: https://github.com/apache/arrow-adbc/pull/3876#issuecomment-3730406124 CC @davidhcoe if there's anyone I missed please feel free to tag others that might be interested in looking this over and reviewing it. Once everyone is onboard with the design and ideas I'll implement the handling of these profiles for the other language bindings. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
