[
https://issues.apache.org/jira/browse/BEAM-7937?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16910490#comment-16910490
]
Reenu Saluja commented on BEAM-7937:
------------------------------------
To add more details about the problem statement, We are already using
hadoop-azure 2.7 version. with this API we are able to download data from ADLS
Gen 2 on a shared storage. Than with Beam api transformation of data is
happening.
But we are looking for api through with Beam code can directly connect with
ADLS Gen 2. For e.g. to read data from kafka, there is option for KafkaIO
read(). ----p.apply(KafkaIO.<String, String>read()
Do we have similar function for Hadoop also(ADLS Gen2 ) ?
> Support Hadoop 3.x on Hadoop File System
> ----------------------------------------
>
> Key: BEAM-7937
> URL: https://issues.apache.org/jira/browse/BEAM-7937
> Project: Beam
> Issue Type: Improvement
> Components: io-java-hadoop-file-system
> Reporter: Reenu Saluja
> Priority: Major
>
> I try to get a Beam pipeline to have input from Azure Data Lake Storage Gen
> 2. ADLS Gen 2 Support Hadoop 3.2+. I tried using Apache Beam 2.8.1 later on
> 2.14.0. I am getting below error:
> Error: Caused by: org.apache.hadoop.fs.UnsupportedFileSystemException: No
> FileSystem for scheme "wasbs"
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
