srilman commented on code in PR #35701: URL: https://github.com/apache/arrow/pull/35701#discussion_r1248948216
########## cpp/src/arrow/filesystem/azurefs.h: ########## @@ -0,0 +1,152 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements. See the NOTICE file +// distributed with this work for additional information +// regarding copyright ownership. The ASF licenses this file +// to you under the Apache License, Version 2.0 (the +// "License"); you may not use this file except in compliance +// with the License. You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, +// software distributed under the License is distributed on an +// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +// KIND, either express or implied. See the License for the +// specific language governing permissions and limitations +// under the License. + +#pragma once + +#include <memory> +#include <string> +#include <vector> + +#include "arrow/filesystem/filesystem.h" +#include "arrow/util/macros.h" +#include "arrow/util/uri.h" + +namespace Azure { +namespace Core { +namespace Credentials { + +class TokenCredential; + +} // namespace Credentials +} // namespace Core +namespace Storage { + +class StorageSharedKeyCredential; + +} // namespace Storage +} // namespace Azure + +namespace arrow { +namespace fs { + +enum class AzureCredentialsKind : int8_t { + /// Anonymous access (no credentials used), public + Anonymous, + /// Use explicitly-provided access key pair + StorageCredentials, + /// Use ServicePrincipleCredentials + ServicePrincipleCredentials, + /// Use Sas Token to authenticate + Sas, + /// Use Connection String + ConnectionString +}; + +/// Options for the AzureFileSystem implementation. +struct ARROW_EXPORT AzureOptions { + std::string scheme; + std::string account_dfs_url; + std::string account_blob_url; + bool is_azurite = false; Review Comment: 1. There are a couple of areas with some special behavior. In particular, - Azurite does not support hierarchical namespaces, then `AzureFileSystem::Impl::is_hierarchical_namespace_enabled_` needs to always be set to false - In the original PR, when `is_azurite` is set, then it will configure the following with account key credentials: ``` account_blob_url = "http://127.0.0.1:10000/"; + account_name + "/"; account_dfs_url = "http://127.0.0.1:10000/"; + account_name + "/"; ``` This is instead of the official URLs. I'm don't like the built-in localhost URLs, so I'll probably modify it a bit. I'm not aware of any other backends, unless ADLS or ADLS Gen1 happens to differ significantly from ABFS. 2. I'm not aware if it can be detected from response headers or not. With the hierarchical namespaces issue, it seems like this might be necessary even for setting up the connection. 3. Good point, will use a enum instead. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
