Re: Writing stream to Hadoop

OMG i missed it ... Thanks, MIki On Tue, Jun 5, 2018 at 1:30 PM Chesnay Schepler wrote: > This particular version of the method is deprecated, use > enableCheckpointing(long > checkpointingInterval) instead. > > On 05.06.2018 12:19, miki haiat wrote: > > I saw the option of enabling checkpoin

Re: Writing stream to Hadoop

This particular version of the method is deprecated, use enableCheckpointing(long checkpointingInterval) instead. On 05.06.2018 12:19, miki haiat wrote: I saw the option of enabling checkpoint enabling-and-configuring-checkpointing

Re: Writing stream to Hadoop

I saw the option of enabling checkpoint enabling-and-configuring-checkpointing But on 1.5 it said that the method is deprecated so im a bit confused . /** @deprecated

Re: Writing stream to Hadoop

Hi Miki, Have you enabled checkpointing? Kostas > On Jun 5, 2018, at 11:14 AM, miki haiat wrote: > > Im trying to write some data to Hadoop by using this code > > The state backend is set without time > StateBackend sb = new > FsStateBackend("hdfs://***:9000/flink/my_city/checkpoints");

Re: Writing stream to Hadoop

I think you can look at this comment, thanks. * Part files can be in one of three states: {@code in-progress}, {@code pending} or {@code finished}. * The reason for this is how the sink works together with the checkpointing mechanism to provide exactly-once * semantics and fault-tolerance. The pa

Writing stream to Hadoop

Im trying to write some data to Hadoop by using this code The state backend is set without time StateBackend sb = new FsStateBackend("hdfs://***:9000/flink/my_city/checkpoints"); env.setStateBackend(sb); BucketingSink> sink = new BucketingSink<>("hdfs://:9000/mycity/raw"); sink.setBu