Dear Hawin,
You can pass a hdfs path to DataStream's and DataSet's writeAsText and
writeAsCsv methods.
I assume that you are running a Streaming topology, because your source is
Kafka, so it would look like the following:
StreamExecutionEnvironment env =
StreamExecutionEnvironment.getExecutionEnvironment();
env.addSource(PerisitentKafkaSource(..))
.map(/* do you operations*/)
.wirteAsText("hdfs://<namenode_name>:<namenode_port>/path/to/your/file");
Check out the relevant section of the streaming docs for more info. [1]
[1]
http://ci.apache.org/projects/flink/flink-docs-master/apis/streaming_guide.html#connecting-to-the-outside-world
Best,
Marton
On Wed, Jun 10, 2015 at 10:22 AM, Hawin Jiang <[email protected]> wrote:
> Hi All
>
>
>
> Can someone tell me what is the best way to write data to HDFS when Flink
> received data from Kafka?
>
> Big thanks for your example.
>
>
>
>
>
>
>
>
>
> Best regards
>
> Hawin
>
>
>
