Hi,
I’m using Flink 1.5.6 and Hadoop 2.7.1.
*My requirement is to read hdfs sequence file (SequenceFileInputFormat),
then write it back to hdfs (SequenceFileAsBinaryOutputFormat with
compression).*
Below code won’t work until I copy the flink-hadoop-compatibility jar to
FLINK_HOME/lib. I f
Hi,
I’d like to sink my data into hdfs using SequenceFileAsBinaryOutputFormat
with compression, and I find a way from the link
https://ci.apache.org/projects/flink/flink-docs-stable/dev/batch/hadoop_compatibility.html,
the code works, but I’m curious to know, since it creates a mapreduce Job
ins
Hi Fabian,
Thank you for the clarification.
Best,
Morven Huang
On Wed, Apr 10, 2019 at 9:57 PM Fabian Hueske wrote:
> Hi,
>
> Flink's Hadoop compatibility functions just wrap functions that were
> implemented against Hadoop's interfaces in wrapper functions that are
org.apache.flink
flink-streaming-java_${scala.binary.version}
${flink.version}
compile
On Wed, Apr 10, 2019 at 9:53 PM Fabian Hueske wrote:
> Hi,
>
> Packaging the flink-hadoop-compatibility dependency with your code into a
> "fat" job jar should work as well.
>
> Best,
&g
Hi,
I’m using Flink 1.5.6 and Hadoop 2.7.1.
My requirement is to read hdfs sequence file (SequenceFileInputFormat), then
write it back to hdfs (SequenceFileAsBinaryOutputFormat with compression).
Below code won’t work until I copy the flink-hadoop-compatibility jar to
FLINK_HOME/lib. I find
Hi,
I’d like to sink my data into hdfs using SequenceFileAsBinaryOutputFormat with
compression, and I find a way from the link
https://ci.apache.org/projects/flink/flink-docs-stable/dev/batch/hadoop_compatibility.html,
the code works, but I’m curious to know, since it creates a mapreduce Job
