dabla commented on code in PR #68344:
URL: https://github.com/apache/airflow/pull/68344#discussion_r4597106716


##########
providers/microsoft/azure/docs/filesystems/adls.rst:
##########
@@ -0,0 +1,77 @@
+ .. Licensed to the Apache Software Foundation (ASF) under one
+    or more contributor license agreements.  See the NOTICE file
+    distributed with this work for additional information
+    regarding copyright ownership.  The ASF licenses this file
+    to you under the Apache License, Version 2.0 (the
+    "License"); you may not use this file except in compliance
+    with the License.  You may obtain a copy of the License at
+
+ ..   http://www.apache.org/licenses/LICENSE-2.0
+
+ .. Unless required by applicable law or agreed to in writing,
+    software distributed under the License is distributed on an
+    "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+    KIND, either express or implied.  See the License for the
+    specific language governing permissions and limitations
+    under the License.
+
+Azure Data Lake Storage Gen2 Filesystem
+=======================================
+
+The Azure Data Lake Storage Gen2 filesystem provides access to ADLS Gen2
+through Airflow's ObjectStoragePath interface.
+
+Supported URL formats:
+
+* ``abfs://connection_id@container/path/to/file``
+* ``abfss://connection_id@container/path/to/file``
+* ``adl://connection_id@container/path/to/file``
+
+Connection Configuration
+------------------------
+
+Create an ADLS Gen2 connection in Airflow with the following parameters:
+
+* **Connection Type**: adls
+* **Host**: Storage account name, for example ``mystorageaccount``. For a 
private endpoint or
+  custom domain, set the ``account_host`` field below rather than a full URL 
here.
+* **Login**: Client ID for Active Directory authentication
+* **Password**: Client secret for Active Directory authentication, or account 
key for shared key authentication
+
+The connection form provides additional configuration fields:

Review Comment:
   **[nit]** `account_name` and `account_host` are not fields of the connection 
form.
   
   The `adls` form in 
[`provider.yaml`](https://github.com/apache/airflow/blob/ff3dbfc69cdd53bb644f90f04019558465b05831/providers/microsoft/azure/provider.yaml#L983)
 has `connection_string`, `tenant_id`, `managed_identity_client_id` and 
`workload_identity_tenant_id`. `account_name` and `account_host` are keys of 
the Extra JSON, which is how the new test passes them. A user who looks for 
them in the form will not find them. Suggest splitting the list: the form 
fields (the two identity fields included, since `get_fs()` reads them) and 
"supported in Extra": `account_name`, `account_host`.



##########
providers/microsoft/azure/docs/connections/adls_v2.rst:
##########
@@ -58,7 +58,8 @@ Password (optional)
     It can be left out to fall back on DefaultAzureCredential_.
 
 Host (optional)
-    Specify the account url for anonymous public read, Active Directory, 
shared access key authentication.
+    Specify the storage account name or account URL for anonymous public read, 
Active Directory,

Review Comment:
   **[warning]** "or account URL" describes a host value that works for neither 
consumer of this connection.
   
   The new `filesystems/adls.rst` tells users to put the bare account name in 
Host and to use `account_host` for a custom endpoint "rather than a full URL 
here". This line now says the opposite, and a full URL in Host fails in both 
places that read it:
   
   - `AzureDataLakeStorageV2Hook.get_conn()` builds 
[`f"https://{conn.host}.dfs.core.windows.net"`](https://github.com/apache/airflow/blob/ff3dbfc69cdd53bb644f90f04019558465b05831/providers/microsoft/azure/src/airflow/providers/microsoft/azure/hooks/data_lake.py#L375),
 so a URL in Host gives `https://https://...dfs.core.windows.net`.
   - `get_fs()` sends a non-bare host down the `account_url` path, which 
`AzureBlobFileSystem` does not read; without an `account_name` adlfs then 
raises ["Must provide either a connection_string or account_name with 
credentials!!"](https://github.com/fsspec/adlfs/blob/2026.5.0/adlfs/spec.py#L566).
   
   The connection form already labels the field "ADLS Gen2 Account Name". 
Suggest: "Specify the storage account name. For a private endpoint or custom 
domain, set `account_host` in Extra." That also closes the docs half of the 
thread on the dropped `account_url`, which was fixed in `adls.rst` only.



##########
providers/microsoft/azure/src/airflow/providers/microsoft/azure/fs/adls.py:
##########
@@ -47,12 +68,20 @@ def get_fs(conn_id: str | None, storage_options: dict[str, 
Any] | None = None) -
     if connection_string:
         return AzureBlobFileSystem(connection_string=connection_string)
 
-    options: dict[str, Any] = {
-        "account_url": parse_blob_account_url(conn.host, conn.login),
-    }
-
     # mirror handling of custom field "client_secret_auth_config" from extras. 
Ignore if missing as AzureBlobFileSystem can handle.
     tenant_id = get_field(conn_id=conn_id, conn_type=conn_type, extras=extras, 
field_name="tenant_id")
+    options: dict[str, Any] = {}
+    account_name = _bare_account_name_from_host(conn.host)
+    if conn_type == "adls" and account_name:
+        # An ADLS Active Directory connection uses ``login`` as the client_id, 
so a bare host
+        # is the storage account name rather than part of an account URL.
+        options["account_name"] = account_name
+    elif conn_type != "adls" or conn.host or not tenant_id:
+        # WASB, full-URL, and custom-endpoint connections keep the historical 
account_url path.
+        options["account_url"] = parse_blob_account_url(conn.host, conn.login)
+    # Otherwise (adls + no host + a tenant_id) neither account_name nor 
account_url is set, so
+    # adlfs falls back to DefaultAzureCredential using the client_id/secret 
assembled below.

Review Comment:
   **[nit]** The comment on the fall-through case states behaviour adlfs does 
not have.
   
   For `adls` + no host + `tenant_id`, the comment says adlfs "falls back to 
DefaultAzureCredential using the client_id/secret". Reading adlfs at the 
provider's floor (2026.5.0):
   
   - with a `client_id` set, adlfs builds a service principal credential 
([`_get_credential_from_service_principal()`](https://github.com/fsspec/adlfs/blob/2026.5.0/adlfs/spec.py#L362-L368));
 `DefaultAzureCredential` is only used when there is no client id, key or SAS 
token;
   - whichever credential it gets, `do_connect()` needs an account name 
([spec.py#L533](https://github.com/fsspec/adlfs/blob/2026.5.0/adlfs/spec.py#L533))
 and otherwise raises the `ValueError` linked in the comment above.
   
   So this combination only connects when the account name comes from 
`extra.account_name` (the last test case, which works) or from 
`AZURE_STORAGE_ACCOUNT_NAME`. The parametrize case with 
`login`/`password`/`tenant_id` and no host passes against the mocked 
`AzureBlobFileSystem`, while the real class rejects those arguments.
   
   Suggest rewording the comment to what happens ("no account is set here; it 
has to come from `account_name` in Extra or `AZURE_STORAGE_ACCOUNT_NAME`") and 
saying the same next to that test case, so nobody reads it as a supported 
configuration.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to