hudi-bot opened a new issue, #15541:
URL: https://github.com/apache/hudi/issues/15541
Standalone install not working with AWS S3 tables
# Install Java JDK 1.8
# Reference JAVA_HOME
# Install Maven 3.8.6
# Install Spark 3.2.2
{code:java}
wget https://dlcdn.apache.org/spark/spark-3.2.2/spark-3.2.2-bin-hadoop3.2.tgz
{code}
# Reference SPARK_HOME
# Download Hudi Sources 0.12.1
# Build Hudi
# Download Client jars for S3 and reference CLIENT_JAR
/!\ Note that documentation needs to precise where
'/lib/spark-3.2.0-bin-hadoop3.2/jars/...' should apply
# Launch Hudi CLI
{code:java}
hudi->connect --path
s3://stef-raw-euwest1-435665164035-prt/dms/infoquai/hudi_iq_log_quai_mor/
16388 [main] INFO org.apache.hudi.common.table.HoodieTableMetaClient [] -
Loading HoodieTableMetaClient from
s3://stef-raw-euwest1-435665164035-prt/dms/infoquai/hudi_iq_log_quai_mor/
16961 [main] WARN org.apache.hadoop.util.NativeCodeLoader [] - Unable to
load native-hadoop library for your platform... using builtin-java classes
where applicable
Failed to get instance of org.apache.hadoop.fs.FileSystem
Details of the error have been omitted. You can use the stacktrace command
to print the full stacktrace.
hudi->stacktrace
org.apache.hudi.exception.HoodieIOException: Failed to get instance of
org.apache.hadoop.fs.FileSystem
at org.apache.hudi.common.fs.FSUtils.getFs(FSUtils.java:112)
at
org.apache.hudi.common.table.HoodieTableMetaClient.getFs(HoodieTableMetaClient.java:295)
at
org.apache.hudi.common.table.HoodieTableMetaClient.<init>(HoodieTableMetaClient.java:127)
at
org.apache.hudi.common.table.HoodieTableMetaClient.newMetaClient(HoodieTableMetaClient.java:642)
at
org.apache.hudi.common.table.HoodieTableMetaClient.access$000(HoodieTableMetaClient.java:80)
at
org.apache.hudi.common.table.HoodieTableMetaClient$Builder.build(HoodieTableMetaClient.java:711)
at
org.apache.hudi.cli.HoodieCLI.refreshTableMetadata(HoodieCLI.java:89)
at org.apache.hudi.cli.HoodieCLI.connectTo(HoodieCLI.java:95)
at
org.apache.hudi.cli.commands.TableCommand.connect(TableCommand.java:86)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.springframework.shell.command.invocation.InvocableShellMethod.doInvoke(InvocableShellMethod.java:306)
at
org.springframework.shell.command.invocation.InvocableShellMethod.invoke(InvocableShellMethod.java:232)
at
org.springframework.shell.command.CommandExecution$DefaultCommandExecution.evaluate(CommandExecution.java:158)
at org.springframework.shell.Shell.evaluate(Shell.java:208)
at org.springframework.shell.Shell.run(Shell.java:140)
at
org.springframework.shell.jline.InteractiveShellRunner.run(InteractiveShellRunner.java:73)
at
org.springframework.shell.DefaultShellApplicationRunner.run(DefaultShellApplicationRunner.java:65)
at
org.springframework.boot.SpringApplication.callRunner(SpringApplication.java:762)
at
org.springframework.boot.SpringApplication.callRunners(SpringApplication.java:752)
at
org.springframework.boot.SpringApplication.run(SpringApplication.java:315)
at
org.springframework.boot.SpringApplication.run(SpringApplication.java:1306)
at
org.springframework.boot.SpringApplication.run(SpringApplication.java:1295)
at org.apache.hudi.cli.Main.main(Main.java:34)
Caused by: org.apache.hadoop.fs.UnsupportedFileSystemException: No
FileSystem for scheme "s3"
at
org.apache.hadoop.fs.FileSystem.getFileSystemClass(FileSystem.java:3225)
at
org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:3245)
at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:121)
at
org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:3296)
at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:3264)
at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:475)
at org.apache.hadoop.fs.Path.getFileSystem(Path.java:356)
at org.apache.hudi.common.fs.FSUtils.getFs(FSUtils.java:110)
{code}
See Hudi CLI launch output [^hudi-cli_trace.txt]
## JIRA info
- Link: https://issues.apache.org/jira/browse/HUDI-5156
- Type: Bug
- Attachment(s):
- 03/Nov/22
18:24;benoit.colas;hudi-cli_trace.txt;https://issues.apache.org/jira/secure/attachment/13051761/hudi-cli_trace.txt
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
