GitHub user ZoneMayor reopened a pull request:
https://github.com/apache/kafka/pull/671
KAFKA-2977: Transient Failure in
kafka.log.LogCleanerIntegrationTest.cleanerTest
Hi @guozhangwang
Code is as below:
val appends = writeDups(numKeys = 100, numDups = 3, log,
CompressionCodec.getCompressionCodec(compressionCodec))
cleaner.startup()
val firstDirty = log.activeSegment.baseOffset
cleaner.awaitCleaned("log", 0, firstDirty)
....
val appends2 = appends ++ writeDups(numKeys = 100, numDups = 3, log,
CompressionCodec.getCompressionCodec(compressionCodec))
val firstDirty2 = log.activeSegment.baseOffset
cleaner.awaitCleaned("log", 0, firstDirty2)
log cleaner and writeDups are two different threads;
log cleaner do cleaning every 15s, timeout in "cleaner.awaitCleaned" is 60s;
there is a filtering condition for a log to be chosen to become a cleaning
target: cleanableRatio> 0.5(configured log.cleaner.min.cleanable.ratio) by
default;
It may happen that, during "val appends2 = appends ++ writeDups(numKeys =
100, numDups = 3, log,
CompressionCodec.getCompressionCodec(compressionCodec))", log is also
undergoing a cleaning process;
Since the segment size configured in this test is quite small: 100, there
is possibility that before the end of 'writeDups', some 'dirty segment' of the
log is already cleaned;
With tiny dirty part left, cleanableRatio> 0.5 cannot be satisfied;
thus firstDirty2>lastCleaned2, which leads to this test failed;
Does it make sense?
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/ZoneMayor/kafka trunk-KAFKA-2977
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/kafka/pull/671.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #671
----
commit 95374147a28208d4850f6e73f714bf418935fc2d
Author: ZoneMayor <jinxing6...@126.com>
Date: 2015-11-27T03:49:34Z
Merge pull request #1 from apache/trunk
merge
commit cec5b48b651a7efd3900cfa3c1fd0ab1eeeaa3ec
Author: ZoneMayor <jinxing6...@126.com>
Date: 2015-12-01T10:44:02Z
Merge pull request #2 from apache/trunk
2015-12-1
commit a119d547bf1741625ce0627073c7909992a20f15
Author: ZoneMayor <jinxing6...@126.com>
Date: 2015-12-04T13:42:27Z
Merge pull request #3 from apache/trunk
2015-12-04#KAFKA-2893
commit b767a8dff85fc71c75d4cf5178c3f6f03ff81bfc
Author: ZoneMayor <jinxing6...@126.com>
Date: 2015-12-09T10:42:30Z
Merge pull request #5 from apache/trunk
2015-12-9
commit 0070c2d71d06ee8baa1cddb3451cd5af6c6b1d4a
Author: ZoneMayor <jinxing6...@126.com>
Date: 2015-12-11T14:50:30Z
Merge pull request #8 from apache/trunk
2015-12-11
commit 09908ac646d4c84f854dad63b8c99213b74a7063
Author: ZoneMayor <jinxing6...@126.com>
Date: 2015-12-13T14:17:19Z
Merge pull request #9 from apache/trunk
2015-12-13
commit ff1e68bb7101d12624c189174ef1dceb21ed9798
Author: jinxing <jinx...@fenbi.com>
Date: 2015-12-13T14:31:34Z
KAFKA-2054: Transient Failure in
kafka.log.LogCleanerIntegrationTest.cleanerTest
commit 6321ab6599cb7a981fac2a4eea64a5f2ea805dd6
Author: jinxing <jinx...@fenbi.com>
Date: 2015-12-13T14:36:11Z
removed unnecessary maven repo
commit 05cae52c72a02c0ed40fd4e3be03e1cb19f33f7a
Author: jinxing <jinx...@fenbi.com>
Date: 2015-12-17T12:21:12Z
removed the semicolon
commit 651de48663cf375ea714cdbeb34650d75f1f4d71
Author: jinxing <jinx...@fenbi.com>
Date: 2015-12-18T07:43:38Z
KAFKA-2977: Transient Failure in
kafka.log.LogCleanerIntegrationTest.cleanerTest
commit ba03eee18045dc4aabc56ff17907036c238b1f7d
Author: jinxing <jinx...@fenbi.com>
Date: 2015-12-18T07:50:57Z
KAFKA-2977: fix
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
