Manish Bhatt created HADOOP-19941:
-------------------------------------
Summary: ABFS: Support Photon (Apache Arrow) based ListBlobs on
Blob endpoint with XML fallback
Key: HADOOP-19941
URL: https://issues.apache.org/jira/browse/HADOOP-19941
Project: Hadoop Common
Issue Type: New Feature
Components: fs/azure
Reporter: Manish Bhatt
Assignee: Manish Bhatt
Add config-gated support (fs.azure.photon.enabled, default off) for consuming
ListBlobs responses in the Apache Arrow (Photon) format on the ABFS Blob
endpoint. When enabled, ABFS advertises Arrow via an Accept header; the
response Content-Type selects an Arrow or the existing XML parser, both
producing identical FileStatus results, so downstream behaviour is unchanged.
Includes automatic, transparent fallback to XML, full parsing parity (metadata,
directory markers, implicit directories, copy properties, native
timestamp/length vectors), interrupt-safe Arrow parsing, and Photon telemetry
(request, response, fallback, parse-failure counts and listing latency). No
public API changes. Covered by unit and integration tests.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
