[
https://issues.apache.org/jira/browse/KAFKA-2467?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14710829#comment-14710829
]
Ewen Cheslack-Postava commented on KAFKA-2467:
----------------------------------------------
[~ijuma] Sort of. We still need refinements since the tests would have hung and
had to be killed by Jenkins (we're missing a timeout in a pretty critical
place).
As for running system tests on every PR, that would be nice but at the moment
isn't practical because of how long they can take to run. It'd be great to get
there eventually, but we need to continue refining the tests, ducktape (e.g.,
adding the parallel test runner), and even the jenkins config (reuse clusters
to avoid the startup costs for each test run that we currently pay). For now,
we're working on making it easy for people to run the system tests manually on
specific PRs when they think it would be useful, which I think is a reasonable
intermediate step before doing builds on every PR.
> ConsoleConsumer regressions
> ---------------------------
>
> Key: KAFKA-2467
> URL: https://issues.apache.org/jira/browse/KAFKA-2467
> Project: Kafka
> Issue Type: Bug
> Components: tools
> Reporter: Ewen Cheslack-Postava
> Assignee: Ewen Cheslack-Postava
>
> It seems that the patch for KAFKA-2015 caused a few changes in the behavior
> of the console consumer. I picked this up because it caused the new mirror
> maker sanity system test to hang. We need a separate fix for ducktape to
> address the lack of a timeout where it got stuck, but I'd also like to get
> this fixed ASAP since it affects pretty much all system test efforts since
> they commonly use console consumer to validate data produced to Kafka.
> I've tracked down a couple of changes so far:
> 1. The --consumer.config option handling was changed entirely. I think the
> new approach was trying to parse it as key=value parameters, but it's
> supposed to be a properties file *containing* key=value pairs.
> 2. A few different exceptions during message processing are not handled the
> same way. The skipMessageOnErrorOpt is not longer being used at all (it's
> parsed, but that option is never checked anymore). Also, exceptions during
> iteration are not caught. After fixing the consumer.config issue, which was
> keeping the consumer.timeout.ms setting from making it into the consumer
> config, this also caused the process to hang. It killed the main thread, but
> there must be another non-daemon thread still running (presumably the
> consumer threads?)
> 3. The "consumed X messages" message changed from stderr to stdout.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
