ling-list-archive.2336050.n4.nabble.com/file/n14522/eventFilter.zip>
--
View this message in context:
http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Fink-KafkaProducer-Data-Loss-tp11413p14522.html
Sent from the Apache Flink User Mailing List archive. mailing
or so.
Thanks for all your help.
eventFilter.zip
<http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/file/n14522/eventFilter.zip>
--
View this message in context:
http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Fink-KafkaProducer-Data-Loss-t
link 1.3.2 and 1.4 releases.
>>
>> Piotrek
>>
>>> On Jul 12, 2017, at 7:32 PM, ninad <nni...@gmail.com
>>> <mailto:nni...@gmail.com>> wrote:
>>>
>>> Hey guys, any update on this? If needed I can attach our code.
>>>
>>>
--
View this message in context:
http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Fink-KafkaProducer-Data-Loss-tp11413p14224.html
Sent from the Apache Flink User Mailing List archive. mailing list archive at
Nabble.com.
message in context:
> http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Fink-KafkaProducer-Data-Loss-tp11413p14224.html
> Sent from the Apache Flink User Mailing List archive. mailing list archive at
> Nabble.com.
Hey guys, any update on this? If needed I can attach our code.
--
View this message in context:
http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Fink-KafkaProducer-Data-Loss-tp11413p14224.html
Sent from the Apache Flink User Mailing List archive. mailing list archive
ge in context:
http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Fink-KafkaProducer-Data-Loss-tp11413p13805.html
Sent from the Apache Flink User Mailing List archive. mailing list archive at
Nabble.com.
few messages were not replayed, and the kafka sink didn't receive all the
>> messages.
>>
>> Attaching the files here.
>>
>> jmCloudera583.log
>> <http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/file/n13597/jmCloudera583.log>
>&
nk-user-mailing-list-archive.2336050.
> n4.nabble.com/file/n13597/tmOneCloudera583.log>
> tmTwoCloudera583.log
> <http://apache-flink-user-mailing-list-archive.2336050.
> n4.nabble.com/file/n13597/tmTwoCloudera583.log>
>
> BTW, I po
that.
Thanks for your patience guys.
--
View this message in context:
http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Fink-KafkaProducer-Data-Loss-tp11413p13597.html
Sent from the Apache Flink User Mailing List archive. mailing list archive at
Nabble.com.
. Hope you guys
saw that.
Thanks for your patience guys.
--
View this message in context:
http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Fink-KafkaProducer-Data-Loss-tp11413p13597.html
Sent from the Apache Flink User Mailing List archive. mailing list archive at
Nabble.com.
View this message in context:
http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Fink-KafkaProducer-Data-Loss-tp11413p13549.html
Sent from the Apache Flink User Mailing List archive. mailing list archive at
Nabble.com.
Not yet. Planning to do that today.
--
View this message in context:
http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Fink-KafkaProducer-Data-Loss-tp11413p13544.html
Sent from the Apache Flink User Mailing List archive. mailing list archive at
Nabble.com.
13527/tmTwo.log>
--
View this message in context:
http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Fink-KafkaProducer-Data-Loss-tp11413p13527.html
Sent from the Apache Flink User Mailing List archive. mailing list archive at
Nabble.com.
wo.log
<http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/file/n13527/tmTwo.log>
--
View this message in context:
http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Fink-KafkaProducer-Data-Loss-tp11413p13527.html
Sent from the Apache Flink User Ma
inad ([hidden email]
>> <http:///user/SendEmail.jtp?type=node=13479=0>) wrote:
>>
>> I tested this with the standalone cluster, and I don't see this problem.
>> So,
>> the problem could be that we haven't built Flink against cloudera Hadoop?
>> I
>> will test it
sted this with the standalone cluster, and I don't see this problem.
> So,
> the problem could be that we haven't built Flink against cloudera Hadoop?
> I
> will test it out.
>
>
>
> --
> View this message in context: http://apache-flink-user-
> mailing-list-archive.2
? I
will test it out.
--
View this message in context:
http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Fink-KafkaProducer-Data-Loss-tp11413p13477.html
Sent from the Apache Flink User Mailing List archive. mailing list archive at
Nabble.com.
I tested this with the standalone cluster, and I don't see this problem. So,
the problem could be that we haven't built Flink against cloudera Hadoop? I
will test it out.
--
View this message in context:
http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Fink-KafkaProducer
aLoss.log>
tmTwoNoDataLoss.log
<http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/file/n13467/tmTwoNoDataLoss.log>
--
View this message in context:
http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Fink-KafkaProducer-Data-Loss-tp11413p13467.html
Sent fro
Thanks Till. The log files I have attached are the complete logs. They are
DEBUG level. There are three files:
jobManger.log, tmOne.log and tmTwo.log.
--
View this message in context:
http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Fink-KafkaProducer-Data-Loss-tp11413p13463
e7f2caa0b75.*
>
> Now, if I bring up all Kafka brokers, I am expecting that the messages
> which
> didn't make it to the Kafka sink should be sent.
>
> But that's not happening.
>
> All these logs are present in the files that I attached.
>
> I am going to try this on
uld be sent.
But that's not happening.
All these logs are present in the files that I attached.
I am going to try this on the standalone cluster today.
--
View this message in context:
http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Fink-KafkaProducer-Data-Loss-tp11413p13
er=LEADER_NOT_AVAILABLE}
--
View this message in context:
http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Fink-KafkaProducer-Data-Loss-tp11413p13443.html
Sent from the Apache Flink User Mailing List archive. mailing list archive at
Nabble.com.
gt; at
> org.apache.flink.streaming.connectors.kafka.FlinkKafkaProducerBase.snapshotState(FlinkKafkaProducerBase.java:335)
>
>
>
>
>
>
>
> --
> View this message in context:
> http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Fink-KafkaProducer-Data-Loss-tp11413p13412.html
>
> Sent from the Apache Flink User Mailing List archive. mailing list archive at
> Nabble.com.
e
> Caused by: java.lang.Exception: Failed to send data to Kafka: Batch Expired
> at
> org.apache.flink.streaming.connectors.kafka.FlinkKafkaProducerBase.checkErroneous(FlinkKafkaProducerBase.java:366)
> at
> org.apache.flink.streaming.connectors.kafka.FlinkKafkaProducerBase.snapshotStat
Thanks, Gordon and Till.
--
View this message in context:
http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Fink-KafkaProducer-Data-Loss-tp11413p11431.html
Sent from the Apache Flink User Mailing List archive. mailing list archive at
Nabble.com.
ka settings.
So, is our understanding correct? Or is there a way we can avoid this data
loss by modifying some Flink settings?
Thanks.
--
View this message in context:
http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Fink-KafkaProducer-Data-Loss-tp11413.html
Sent from the A
g the possible side effects of changing the above
> kafka settings.
>
> So, is our understanding correct? Or is there a way we can avoid this data
> loss by modifying some Flink settings?
>
> Thanks.
>
>
>
> --
> View this message in context: http://apache-flink-user-
> mailing-list-archive.2336050.n4.nabble.com/Fink-KafkaProducer-Data-Loss-
> tp11413.html
> Sent from the Apache Flink User Mailing List archive. mailing list archive
> at Nabble.com.
>
an avoid this data
loss by modifying some Flink settings?
Thanks.
--
View this message in context:
http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Fink-KafkaProducer-Data-Loss-tp11413.html
Sent from the Apache Flink User Mailing List archive. mailing list archive at
Nabble.com.
30 matches
Mail list logo