[
https://issues.apache.org/jira/browse/FLINK-7343?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16112654#comment-16112654
]
ASF GitHub Bot commented on FLINK-7343:
---------------------------------------
GitHub user pnowojski opened a pull request:
https://github.com/apache/flink/pull/4470
[FLINK-7343] Simulate network failures in kafka at-least-once test
We shouldn't fail KafkaServers directly, because they might not be able to
flush the data (`log.flush.interval.***` properties). Since we don't want to
test how well Kafka implements at-least-once/exactly-once semantic, it is a
better idea (and hopefully more reliable) to just simulate network failure
between Flink and Kafka in our at-least-once tests. To achieve that I have
implemented `NetworkFailuresProxy` class.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/pnowojski/flink network-failures
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/flink/pull/4470.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #4470
----
commit 0e28327619893cfbf793fa842be3d965f649516c
Author: Piotr Nowojski <piotr.nowoj...@gmail.com>
Date: 2017-08-01T14:05:49Z
[FLINK-7343][kafka] Increase Xmx for tests
Sometimes 1000m was not enough memory to run at-least-once tests with
broker failures on Travis
commit 8d820c3d0e77624a945e074f4a1bc476b5fd0f75
Author: Piotr Nowojski <piotr.nowoj...@gmail.com>
Date: 2017-08-01T16:11:27Z
[FLINK-7343] Add network proxy utility to simulate network failures
commit 967e1dfc87846b4011652bbaefab696900abc8dd
Author: Piotr Nowojski <piotr.nowoj...@gmail.com>
Date: 2017-08-03T07:25:04Z
fixup! [FLINK-7343][kafka] Increase Xmx for tests
commit 27b20f2ec3770231d95c3c7918c9313ce58b5e18
Author: Piotr Nowojski <piotr.nowoj...@gmail.com>
Date: 2017-08-03T09:27:12Z
[FLINK-7343] Use NetworkFailureProxy in kafka tests
We shouldn't fail KafkaServers directly, because they might not be able
to flush the data. Since we don't want to test how well Kafka implements
at-least-once/exactly-once semantic, we just simulate network failure
between Flink and Kafka in our at-least-once tests.
commit 692b5944f16b98aafe716ca1d18a04fa8a033798
Author: Piotr Nowojski <piotr.nowoj...@gmail.com>
Date: 2017-08-03T09:35:26Z
[hotfix][Kafka] Clean up getKafkaServer method
----
> Kafka010ProducerITCase instability
> ----------------------------------
>
> Key: FLINK-7343
> URL: https://issues.apache.org/jira/browse/FLINK-7343
> Project: Flink
> Issue Type: Bug
> Components: Kafka Connector
> Reporter: Piotr Nowojski
> Assignee: Piotr Nowojski
> Labels: test-stability
>
> As reported by [~till.rohrmann] in
> https://issues.apache.org/jira/browse/FLINK-6996 there seems to be a test
> instability with
> `Kafka010ProducerITCase>KafkaProducerTestBase.testOneToOneAtLeastOnceRegularSink`
> https://travis-ci.org/tillrohrmann/flink/jobs/258538641
> It is probably related to log.flush intervals in Kafka, which delay flushing
> the data to files and potentially causing data loses on killing Kafka brokers
> in the tests.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
