Hello all,
Newbie here! We are running in Amazon EMR with the following installed in the EMR Software Configuration Hadoop 2.8.5 JupyterHub 1.0.0 Ganglia 3.7.2 Hive 2.3.6 Flink 1.9.0 I am trying to get a Streaming job from one S3 bucket into an another S3 bucket using the StreamingFileSink I got the infamous exception: Caused by: java.lang.UnsupportedOperationException: Recoverable writers on Hadoop are only supported for HDFS and for Hadoop version 2.7 or newer According to this, I needed to install flink-s3-fs-hadoop-1.9.0.jar in /usr/lib/flink/lib https://stackoverflow.com/questions/55517566/amazon-emr-while-submitting-job-for-apache-flink-getting-error-with-hadoop-recov That did not work. Further googling, revealed for Flink 1.9.0 and above: (according to this) https://ci.apache.org/projects/flink/flink-docs-stable/ops/filesystems/ it seems that I need to install the jar file in the plugins directory (/usr/lib/flink/plugins/s3-fs-hadoop) That did not work either. At this point, I am not sure what to do and would appreciate some help! Cheers Kumar
