[
https://issues.apache.org/jira/browse/KAFKA-2489?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14721153#comment-14721153
]
ASF GitHub Bot commented on KAFKA-2489:
---------------------------------------
GitHub user granders opened a pull request:
https://github.com/apache/kafka/pull/179
KAFKA-2489: add benchmark for new consumer
@ewencp
The changes here are smaller than they look - mostly refactoring/cleanup.
- ConsumerPerformance.scala - corrected timeout inequality which had
prevented this from working with new consumer
- ConsumerPerformanceService: added new_consumer flag, and exposed more
command-line settings
- benchmark.py: refactored to use `@parametrize` and `@matrix` - this
reduced some amount of repeated code
- benchmark.py: added consumer performance tests with new consumer (using
`@parametrize`)
- benchmark.py: added more detailed test descriptions
- performance.py: broke into separate files
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/confluentinc/kafka
KAFKA-2489-benchmark-new-consumer
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/kafka/pull/179.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #179
----
commit 6525594e2c4803f85403f260ddcb18ff5ae6c0a0
Author: Geoff Anderson <ge...@confluent.io>
Date: 2015-08-29T05:24:29Z
Fixed typo in ProducerPerformance.java
commit fa5f81094215a69f00076cc2a07c2e3d19f9a34f
Author: Geoff Anderson <ge...@confluent.io>
Date: 2015-08-29T05:25:53Z
Fixed ConsumerPerformance.scala with new consumer: incorrect inequality
previously caused this to time out without consuming any messages with new
consumer.
commit 45b31f3d5f7a2ae6d4d27737012495cc8ad1c70d
Author: Geoff Anderson <ge...@confluent.io>
Date: 2015-08-29T05:28:57Z
Updated consumer throughput tests to run with new and old consumer.
Refactored with @parametrize and @matrix to reduce duplicated code.
commit f4d837330fdda2bc7918b97e740ba91a9d456462
Author: Geoff Anderson <ge...@confluent.io>
Date: 2015-08-29T05:30:37Z
Refactored performance.py - broke services into individual files. This does
not change the way external users would import and use the performance services
however.
----
> System tests: update benchmark tests to run with new and old consumer
> ---------------------------------------------------------------------
>
> Key: KAFKA-2489
> URL: https://issues.apache.org/jira/browse/KAFKA-2489
> Project: Kafka
> Issue Type: Bug
> Reporter: Geoff Anderson
> Assignee: Geoff Anderson
>
> Update benchmark tests to run w/new consumer to help catch performance
> regressions
> For context:
> https://www.mail-archive.com/dev@kafka.apache.org/msg33633.html
> The new consumer was previously reaching getting good performance. However, a
> recent report on the mailing list indicates it's dropped significantly. After
> evaluation, even with a local broker it seems to only be reaching a 2-10MB/s,
> compared to 600+MB/s previously. Before release, we should get the
> performance
> back on par.
> Some details about where the regression occurred from the mailing list
> http://mail-archives.apache.org/mod_mbox/kafka-dev/201508.mbox/%3CCAAdKFaE8bPSeWZf%2BF9RuA-xZazRpBrZG6vo454QLVHBAk_VOJg%40mail.gmail.com%3E
> :
> bq. At 49026f11781181c38e9d5edb634be9d27245c961 (May 14th), we went from good
> performance -> an error due to broker apparently not accepting the partition
> assignment strategy. Since this commit seems to add heartbeats and the server
> side code for partition assignment strategies, I assume we were missing
> something on the client side and by filling in the server side, things
> stopped
> working.
> bq. On either 84636272422b6379d57d4c5ef68b156edc1c67f8 or
> a5b11886df8c7aad0548efd2c7c3dbc579232f03 (July 17th), I am able to run the
> perf
> test again, but it's slow -- ~10MB/s for me vs the 2MB/s Jay was seeing, but
> that's still far less than the 600MB/s I saw on the earlier commits.
> Ideally we would also at least have a system test in place for the new
> consumer, even if regressions weren't automatically detected. It would at
> least
> allow for manually checking for regressions. This should not be difficult
> since
> there are already old consumer performance tests.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
