iwasakims commented on pull request #736:
URL: https://github.com/apache/bigtop/pull/736#issuecomment-779708795
I tested this on docker provisioned CentOS 8 env. hadoop-aws is loaded only
if HADOOP_OPTIONAL_TOOLS contains the entry. HADOOP_OPTIONAL_TOOLS can be
specified via command line or configuration files such as hadoop-env.sh.
```
# hadoop fs -ls s3a://iwasakims-test/
-ls: Fatal internal error
java.lang.RuntimeException: java.lang.ClassNotFoundException: Class
org.apache.hadoop.fs.s3a.S3AFileSystem not found
at
org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2638)
at
org.apache.hadoop.fs.FileSystem.getFileSystemClass(FileSystem.java:3341)
# HADOOP_OPTIONAL_TOOLS=hadoop-aws hadoop fs -ls s3a://iwasakims-test/
2021-02-16 08:45:07,177 INFO impl.MetricsConfig: Loaded properties from
hadoop-metrics2.properties
2021-02-16 08:45:07,239 INFO impl.MetricsSystemImpl: Scheduled Metric
snapshot period at 10 second(s).
2021-02-16 08:45:07,239 INFO impl.MetricsSystemImpl: s3a-file-system metrics
system started
Found 4 items
-rw-rw-rw- 1 root root 15841 2020-10-21 23:35
s3a://iwasakims-test/NOTICE.txt
-rw-rw-rw- 1 root root 1366 2020-10-21 23:35
s3a://iwasakims-test/README.txt
drwxrwxrwx - root root 0 2021-02-16 08:45 s3a://iwasakims-test/bar
drwxrwxrwx - root root 0 2021-02-16 08:45 s3a://iwasakims-test/foo
2021-02-16 08:45:08,157 INFO impl.MetricsSystemImpl: Stopping
s3a-file-system metrics system...
2021-02-16 08:45:08,157 INFO impl.MetricsSystemImpl: s3a-file-system metrics
system stopped.
2021-02-16 08:45:08,157 INFO impl.MetricsSystemImpl: s3a-file-system metrics
system shutdown complete.
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
