[GitHub] metron issue #1039: METRON-1588 Migrate storm-kafka-client to 1.2.1
Github user HeartSaVioR commented on the issue: https://github.com/apache/metron/pull/1039 @cestella Thanks for merging! Would you mind to mark Apache issue as resolved (with assigning issue to me) too? Thanks in advance! ---
[GitHub] metron issue #1039: METRON-1588 Migrate storm-kafka-client to 1.2.1
Github user cestella commented on the issue: https://github.com/apache/metron/pull/1039 Ok, the 1.2.2 change was run through some performance tests and I'm +1 on this; thanks for the contribution! ---
[GitHub] metron issue #1039: METRON-1588 Migrate storm-kafka-client to 1.2.1
Github user HeartSaVioR commented on the issue: https://github.com/apache/metron/pull/1039 Just updated the version. There's no change in pom.xml between 1.2.1 and 1.2.2, hence no change in transitive dependencies. Two files (KafkaSpoutRetryExponentialBackoff.java and Timer.java) are only referring Time.nanoTime() which require hack. So this is unchanged while bumping up the version. ---
[GitHub] metron issue #1039: METRON-1588 Migrate storm-kafka-client to 1.2.1
Github user HeartSaVioR commented on the issue: https://github.com/apache/metron/pull/1039 @anandsubbu Thanks for doing sanity & performance tests. @cestella Thanks for reviewing. STORM-2994: Before the patch, KafkaSpout cannot commit `null tuple`, which means it can still consume messages but doesn't commit offsets, and probably eventually stuck. I'll update the PR to include storm-kafka-client 1.2.2. Thanks again! ---
[GitHub] metron issue #1039: METRON-1588 Migrate storm-kafka-client to 1.2.1
Github user cestella commented on the issue: https://github.com/apache/metron/pull/1039 I think I'm ready to +1 this. I wouldn't mind going to 1.2.2. Of those, I'm most concerned about STORM-2994. Can you characterize the risk of that happening? ---
[GitHub] metron issue #1039: METRON-1588 Migrate storm-kafka-client to 1.2.1
Github user anandsubbu commented on the issue: https://github.com/apache/metron/pull/1039 I ran a few performance tests comparing ver 1.1.0 and 1.2.1 on the enrichment topology using the [load_tool](https://github.com/apache/metron/blob/master/metron-contrib/metron-performance/README.md). The EPS numbers for 1.2.1 are seen to be on-par or better than 1.1.0. However, I noticed that the Avg CPU usage is higher on 1.2.1 (about 70%) as compared to 1.1.0 (about 40%). ---
[GitHub] metron issue #1039: METRON-1588 Migrate storm-kafka-client to 1.2.1
Github user HeartSaVioR commented on the issue: https://github.com/apache/metron/pull/1039 Would you mind if I propose upgrading storm-kafka-client version to 1.2.2? There's no feature addition on storm-kafka-client but three issues were fixed. * [STORM-2994](https://issues.apache.org/jira/browse/STORM-2994) - KafkaSpout consumes messages but doesn't commit offsets * [STORM-3043](https://issues.apache.org/jira/browse/STORM-3043) - NullPointerException thrown in SimpleRecordTranslator.apply() * [STORM-3059](https://issues.apache.org/jira/browse/STORM-3059) - KafkaSpout throws NPE when hitting a null tuple if the processing guarantee is not AT_LEAST_ONCE They're neither blocker nor critical, so the storm-kafka-client module is on the way to stabilize. ---
[GitHub] metron issue #1039: METRON-1588 Migrate storm-kafka-client to 1.2.1
Github user cestella commented on the issue: https://github.com/apache/metron/pull/1039 I'm holding a +1 on this until we've gotten a chance to performance test this to ensure there's no unexpected issues. I'll keep everyone informed here when that's done. ---
[GitHub] metron issue #1039: METRON-1588 Migrate storm-kafka-client to 1.2.1
Github user cestella commented on the issue: https://github.com/apache/metron/pull/1039 I ran this up in full-dev and data flowed through just fine with no errors in the storm logs for any of the topologies. ---
[GitHub] metron issue #1039: METRON-1588 Migrate storm-kafka-client to 1.2.1
Github user cestella commented on the issue: https://github.com/apache/metron/pull/1039 So, this is awesome! I'll spin it up in full-dev and test it out for you today. ---
[GitHub] metron issue #1039: METRON-1588 Migrate storm-kafka-client to 1.2.1
Github user HeartSaVioR commented on the issue: https://github.com/apache/metron/pull/1039 I've rebased to change commit titles slightly, so the build is re-triggered. The build was succeed before rebasing. Travis build in my folk: https://travis-ci.org/HeartSaVioR/metron/builds/385935022 ---
[GitHub] metron issue #1039: METRON-1588 Migrate storm-kafka-client to 1.2.1
Github user HeartSaVioR commented on the issue: https://github.com/apache/metron/pull/1039 I've seen Travis CI succeeded in my fork, but unfortunately I don't know how to do manual test with full dev. Much appreciated if reviewers could do the manual validation phase. ---
