Thanks for running this all down Joe/Aldrin/Oleg. I think Joe's suggestion of having a test which checks for this sort of environment condition and fails with a nice explanation would be really nice.
It looks like Oleg fixed the copyright concerns by removing the lines which appear to have been unused. Will get that merged and kick of an RC2 later tonight or early tomorrow. On Sat, Apr 9, 2016 at 6:02 PM, Joe Skora <jsk...@gmail.com> wrote: > PROBLEM SOLVED. The problem was a mixture of DNS and routing. > > For a couple of reasons, including possibly bad router config, each had a > mixture of a bad nameserver configuration or bad routing to the > nameserver. So, they couldn't resolve their own hostnames to an IP > address. That's not unique to NiFi or Kafka as a failure mode, but it > might be good to add an upfront test for that to make it clear that other > things will fail if it isn't fixed. > > All 3 systems have successfully run the Kafka tests now while connected to > the new router. OSX ran with JDK1.8.0_77 and the Ubuntu boxes with > JDK1.7.0_80. I'll run JDK1.8.0_77 on the Ubuntu boxes later. > > (FYI, the CentOS box is on a completely different network which is why it > wasn't affected.) > > Thank you Oleg and Aldrin for helping track it down!!! > > > On Sat, Apr 9, 2016 at 5:02 PM, Aldrin Piri <aldrinp...@gmail.com> wrote: > >> Well that's great to hear, Joe. Thanks for gathering a bit more info. >> >> We should have tests that are a bit more resilient to varying network >> conditions. I know we got nabbed by a few when the project first started >> incubation To that end, if you know of anything specific between the two >> routers so that we can address in the tests and/or potentially in the code >> itself would be supremely helpful. As another point of data, just ran a >> full build completely network disconnected and things seemed fine with >> Kafka although a few others had issues. Will create a ticket for those. >> >> A bit bewildering as to how OS X and Ubuntu failed but CentOS was fine. >> >> On Sat, Apr 9, 2016 at 4:32 PM, Joe Skora <jsk...@gmail.com> wrote: >> >> > Aldrin, >> > >> > Wow, great catch on it being a network issue, the OSX system has >> > successfully finished a "mvn -Pcontrib-check clean install" run! >> > >> > I haven't had any other problems on this router, but something must be >> up. >> > Now the "fun" starts now to figure out what's wrong? ;-) >> > >> > I'll let you know if it's something interesting or just a PEBKAC error. >> > >> > Joe >> > >> > >> > >> > >> > On Sat, Apr 9, 2016 at 3:39 PM, Joe Skora <jsk...@gmail.com> wrote: >> > >> > > Aldrin, >> > > >> > > Yes same network, nothing novel that I know of, they all have internet >> > > access and I've disabled firewalls for comparison with no affect. This >> > is >> > > a new router, so I will move one to the old router and see if that >> makes >> > a >> > > difference. >> > > >> > > The validate/valudate mismatch is a fat fingering error on my part, I >> > > copied and pasted the stacktrace but I typed the details manually into >> > the >> > > breakdown by system and JVM. >> > > >> > > The source start point is the "nifi-0.6.1-source-release.zip" file, >> after >> > > checksum validation and unzipping. >> > > >> > > I let you know if network makes a difference. >> > > >> > > Joe >> > > >> > > >> > > >> > > On Sat, Apr 9, 2016 at 3:26 PM, Aldrin Piri <aldrinp...@gmail.com> >> > wrote: >> > > >> > >> Joe, >> > >> >> > >> Are these all running on the same network? If so, anything novel or >> > >> different about it? >> > >> >> > >> Also, a I assume these are remnants of debugging, but noticed some >> > >> differences in terms of the source at the tagged RC and your >> > stacktraces. >> > >> >> > >> TestGetHTTP, for instance, only has 431 lines and the test listed >> > >> "KafkaPublisherTest.valudateSuccessfulSendAsDelimited2" also does not >> > seem >> > >> to exist, but KafkaPublisherTest.validateSuccessfulSendAsDelimited2 >> > does. >> > >> >> > >> Thanks for the detailed report and matrix of JDKs and operating >> systems. >> > >> >> > >> On Sat, Apr 9, 2016 at 2:47 PM, Joe Skora <jsk...@gmail.com> wrote: >> > >> >> > >> > -1 (non-binding) >> > >> > >> > >> > Builds and checkouts out ok, but 3 of 4 systems I tried fail in >> > >> > KafkaPublisherTest unit tests with Java 7 and Java 8. Details and >> > >> sample >> > >> > results are included below. >> > >> > >> > >> > The tests appear to handle ConsumerTimeoutException when it is >> > expected, >> > >> > but the very first call to ConsumerIterator.next() appears to fail >> and >> > >> that >> > >> > is unchecked. >> > >> > >> > >> > Note: I also think KafkaPublisherTest should use sample text that is >> > not >> > >> > copyrighted or possibly considered offensive. >> > >> > >> > >> > * builds and tests ok on Centos 6.7 with Java 7 >> > >> > >> > >> > * tests fail on 3 systems in KafkaPublisherTest >> > >> > >> > >> > * OSX 10.11.4 (El Capitan), i7-2635QM @ 2.0 GHz, 8GB >> > >> > * Maven 3.3.9 >> > >> > * JDK1.7.0_51 TestGetHTTP.testExpressionLanguage():433 and >> > >> others >> > >> > fail with "java.net.SocketException: Invalid argument" >> > >> > in Apache HTTP >> > SessionInputBufferImpl.streamRead():139. >> > >> > * JDK1.7.0_80 >> > >> > KafkaPublisherTest.valudateSuccessfulSendAsDelimited2():140 and >> others >> > >> > error out with "kafka.consumer.ConsumerTimeoutException". >> > >> > * JDK1.8.0_77 >> > >> > KafkaPublisherTest.valudateSuccessfulSendAsDelimited2():140 and >> others >> > >> > error out with "kafka.consumer.ConsumerTimeoutException". >> > >> > >> > >> > * Ubuntu 15.10 (Wily Werewolf), i7-4720HQ @ 2.6 GHz, 16GB >> > >> > * Maven 3.3.3 >> > >> > * JDK1.7.0_80 >> > >> > KafkaPublisherTest.valudateSuccessfulSendAsDelimited2():140 and >> other >> > >> test >> > >> > error out with "kafka.consumer.ConsumerTimeoutException". >> > >> > >> > >> > * Ubuntu 15.10 (Wily Werewolf), i7-4770K @ 3.5 GHz, 16GB >> > >> > * JDK1.7.0_80 >> > >> > KafkaPublisherTest.valudateSuccessfulSendAsDelimited2():140 and >> other >> > >> test >> > >> > error out with "kafka.consumer.ConsumerTimeoutException". >> > >> > * JDK1.8.0_77 >> > >> > KafkaPublisherTest.valudateSuccessfulSendAsDelimited2():140 and >> other >> > >> test >> > >> > error out with "kafka.consumer.ConsumerTimeoutException". >> > >> > >> > >> > >> > >> > ------------------------------------------------------- >> > >> > T E S T S >> > >> > ------------------------------------------------------- >> > >> > Running org.apache.nifi.processors.kafka.KafkaPublisherTest >> > >> > Tests run: 5, Failures: 0, Errors: 5, Skipped: 0, Time elapsed: >> > 701.885 >> > >> sec >> > >> > <<< FAILURE! - in >> org.apache.nifi.processors.kafka.KafkaPublisherTest >> > >> > >> > >> > >> > >> >> > >> validateSuccessfulSendAsDelimited2(org.apache.nifi.processors.kafka.KafkaPublisherTest) >> > >> > Time elapsed: 185.158 sec <<< ERROR! >> > >> > kafka.consumer.ConsumerTimeoutException: null >> > >> > at >> > >> > kafka.consumer.ConsumerIterator.makeNext(ConsumerIterator.scala:69) >> > >> > at >> > >> > kafka.consumer.ConsumerIterator.makeNext(ConsumerIterator.scala:33) >> > >> > at >> > >> > >> > kafka.utils.IteratorTemplate.maybeComputeNext(IteratorTemplate.scala:66) >> > >> > at >> > >> kafka.utils.IteratorTemplate.hasNext(IteratorTemplate.scala:58) >> > >> > at >> > kafka.utils.IteratorTemplate.next(IteratorTemplate.scala:38) >> > >> > at >> > >> kafka.consumer.ConsumerIterator.next(ConsumerIterator.scala:46) >> > >> > at >> > >> > >> > >> > >> > >> >> > >> org.apache.nifi.processors.kafka.KafkaPublisherTest.validateSuccessfulSendAsDelimited2(KafkaPublisherTest.java:140) >> > >> > >> > >> > >> > >> > >> > >> >> > >> validateSuccessfulSendAsDelimited(org.apache.nifi.processors.kafka.KafkaPublisherTest) >> > >> > Time elapsed: 245.077 sec <<< ERROR! >> > >> > kafka.consumer.ConsumerTimeoutException: null >> > >> > at >> > >> > kafka.consumer.ConsumerIterator.makeNext(ConsumerIterator.scala:69) >> > >> > at >> > >> > kafka.consumer.ConsumerIterator.makeNext(ConsumerIterator.scala:33) >> > >> > at >> > >> > >> > kafka.utils.IteratorTemplate.maybeComputeNext(IteratorTemplate.scala:66) >> > >> > at >> > >> kafka.utils.IteratorTemplate.hasNext(IteratorTemplate.scala:58) >> > >> > at >> > kafka.utils.IteratorTemplate.next(IteratorTemplate.scala:38) >> > >> > at >> > >> kafka.consumer.ConsumerIterator.next(ConsumerIterator.scala:46) >> > >> > at >> > >> > >> > >> > >> > >> >> > >> org.apache.nifi.processors.kafka.KafkaPublisherTest.validateSuccessfulSendAsDelimited(KafkaPublisherTest.java:114) >> > >> > >> > >> > >> > >> > >> > >> >> > >> validateSuccessfulReSendOfFailedSegments(org.apache.nifi.processors.kafka.KafkaPublisherTest) >> > >> > Time elapsed: 125.07 sec <<< ERROR! >> > >> > kafka.consumer.ConsumerTimeoutException: null >> > >> > at >> > >> > kafka.consumer.ConsumerIterator.makeNext(ConsumerIterator.scala:69) >> > >> > at >> > >> > kafka.consumer.ConsumerIterator.makeNext(ConsumerIterator.scala:33) >> > >> > at >> > >> > >> > kafka.utils.IteratorTemplate.maybeComputeNext(IteratorTemplate.scala:66) >> > >> > at >> > >> kafka.utils.IteratorTemplate.hasNext(IteratorTemplate.scala:58) >> > >> > at >> > kafka.utils.IteratorTemplate.next(IteratorTemplate.scala:38) >> > >> > at >> > >> kafka.consumer.ConsumerIterator.next(ConsumerIterator.scala:46) >> > >> > at >> > >> > >> > >> > >> > >> >> > >> org.apache.nifi.processors.kafka.KafkaPublisherTest.validateSuccessfulReSendOfFailedSegments(KafkaPublisherTest.java:167) >> > >> > >> > >> > >> > >> > >> > >> >> > >> validateWithMultiByteCharacters(org.apache.nifi.processors.kafka.KafkaPublisherTest) >> > >> > Time elapsed: 65.09 sec <<< ERROR! >> > >> > kafka.consumer.ConsumerTimeoutException: null >> > >> > at >> > >> > kafka.consumer.ConsumerIterator.makeNext(ConsumerIterator.scala:69) >> > >> > at >> > >> > kafka.consumer.ConsumerIterator.makeNext(ConsumerIterator.scala:33) >> > >> > at >> > >> > >> > kafka.utils.IteratorTemplate.maybeComputeNext(IteratorTemplate.scala:66) >> > >> > at >> > >> kafka.utils.IteratorTemplate.hasNext(IteratorTemplate.scala:58) >> > >> > at >> > kafka.utils.IteratorTemplate.next(IteratorTemplate.scala:38) >> > >> > at >> > >> kafka.consumer.ConsumerIterator.next(ConsumerIterator.scala:46) >> > >> > at >> > >> > >> > >> > >> > >> >> > >> org.apache.nifi.processors.kafka.KafkaPublisherTest.validateWithMultiByteCharacters(KafkaPublisherTest.java:195) >> > >> > >> > >> > >> > >> > >> > >> >> > >> validateSuccessfulSendAsWhole(org.apache.nifi.processors.kafka.KafkaPublisherTest) >> > >> > Time elapsed: 65.076 sec <<< ERROR! >> > >> > kafka.consumer.ConsumerTimeoutException: null >> > >> > at >> > >> > kafka.consumer.ConsumerIterator.makeNext(ConsumerIterator.scala:69) >> > >> > at >> > >> > kafka.consumer.ConsumerIterator.makeNext(ConsumerIterator.scala:33) >> > >> > at >> > >> > >> > kafka.utils.IteratorTemplate.maybeComputeNext(IteratorTemplate.scala:66) >> > >> > at >> > >> kafka.utils.IteratorTemplate.hasNext(IteratorTemplate.scala:58) >> > >> > at >> > kafka.utils.IteratorTemplate.next(IteratorTemplate.scala:38) >> > >> > at >> > >> kafka.consumer.ConsumerIterator.next(ConsumerIterator.scala:46) >> > >> > at >> > >> > >> > >> > >> > >> >> > >> org.apache.nifi.processors.kafka.KafkaPublisherTest.validateSuccessfulSendAsWhole(KafkaPublisherTest.java:92) >> > >> > >> > >> > Running org.apache.nifi.processors.kafka.TestPutKafka >> > >> > Tests run: 6, Failures: 4, Errors: 0, Skipped: 2, Time elapsed: >> 288.48 >> > >> sec >> > >> > <<< FAILURE! - in org.apache.nifi.processors.kafka.TestPutKafka >> > >> > >> > >> > >> > >> >> > >> testComplexRightPartialDelimitedMessages(org.apache.nifi.processors.kafka.TestPutKafka) >> > >> > Time elapsed: 63.071 sec <<< FAILURE! >> > >> > java.lang.AssertionError: Expected all Transferred FlowFiles to go >> to >> > >> > success but 1 were routed to failure >> > >> > at org.junit.Assert.fail(Assert.java:88) >> > >> > at >> > >> > >> > >> > >> > >> >> > >> org.apache.nifi.util.MockProcessSession.assertAllFlowFilesTransferred(MockProcessSession.java:985) >> > >> > at >> > >> > >> > >> > >> > >> >> > >> org.apache.nifi.util.StandardProcessorTestRunner.assertAllFlowFilesTransferred(StandardProcessorTestRunner.java:312) >> > >> > at >> > >> > >> > >> > >> > >> >> > >> org.apache.nifi.util.StandardProcessorTestRunner.assertAllFlowFilesTransferred(StandardProcessorTestRunner.java:324) >> > >> > at >> > >> > >> > >> > >> > >> >> > >> org.apache.nifi.processors.kafka.TestPutKafka.testComplexRightPartialDelimitedMessages(TestPutKafka.java:204) >> > >> > >> > >> > >> > >> > >> > >> >> > >> testComplexPartialMatchDelimitedMessages(org.apache.nifi.processors.kafka.TestPutKafka) >> > >> > Time elapsed: 60.005 sec <<< FAILURE! >> > >> > java.lang.AssertionError: Expected all Transferred FlowFiles to go >> to >> > >> > success but 1 were routed to failure >> > >> > at org.junit.Assert.fail(Assert.java:88) >> > >> > at >> > >> > >> > >> > >> > >> >> > >> org.apache.nifi.util.MockProcessSession.assertAllFlowFilesTransferred(MockProcessSession.java:985) >> > >> > at >> > >> > >> > >> > >> > >> >> > >> org.apache.nifi.util.StandardProcessorTestRunner.assertAllFlowFilesTransferred(StandardProcessorTestRunner.java:312) >> > >> > at >> > >> > >> > >> > >> > >> >> > >> org.apache.nifi.util.StandardProcessorTestRunner.assertAllFlowFilesTransferred(StandardProcessorTestRunner.java:324) >> > >> > at >> > >> > >> > >> > >> > >> >> > >> org.apache.nifi.processors.kafka.TestPutKafka.testComplexPartialMatchDelimitedMessages(TestPutKafka.java:248) >> > >> > >> > >> > testWithEmptyMessages(org.apache.nifi.processors.kafka.TestPutKafka) >> > >> Time >> > >> > elapsed: 30.004 sec <<< FAILURE! >> > >> > java.lang.AssertionError: >> > >> org.apache.kafka.common.errors.TimeoutException: >> > >> > Failed to update metadata after 30000 ms. >> > >> > at >> > >> > >> > >> > >> > >> >> > >> org.apache.nifi.util.StandardProcessorTestRunner.run(StandardProcessorTestRunner.java:238) >> > >> > at >> > >> > >> > >> > >> > >> >> > >> org.apache.nifi.util.StandardProcessorTestRunner.run(StandardProcessorTestRunner.java:197) >> > >> > at >> > >> > >> > >> > >> > >> >> > >> org.apache.nifi.util.StandardProcessorTestRunner.run(StandardProcessorTestRunner.java:192) >> > >> > at >> > >> > >> > >> > >> > >> >> > >> org.apache.nifi.util.StandardProcessorTestRunner.run(StandardProcessorTestRunner.java:187) >> > >> > at >> > >> > >> > >> > >> > >> >> > >> org.apache.nifi.processors.kafka.TestPutKafka.testWithEmptyMessages(TestPutKafka.java:174) >> > >> > Caused by: org.apache.kafka.common.errors.TimeoutException: Failed >> to >> > >> > update metadata after 30000 ms. >> > >> > >> > >> > >> > >> > >> > >> >> > >> testComplexLeftPartialDelimitedMessages(org.apache.nifi.processors.kafka.TestPutKafka) >> > >> > Time elapsed: 120.009 sec <<< FAILURE! >> > >> > java.lang.AssertionError: Expected all Transferred FlowFiles to go >> to >> > >> > success but 1 were routed to failure >> > >> > at org.junit.Assert.fail(Assert.java:88) >> > >> > at >> > >> > >> > >> > >> > >> >> > >> org.apache.nifi.util.MockProcessSession.assertAllFlowFilesTransferred(MockProcessSession.java:985) >> > >> > at >> > >> > >> > >> > >> > >> >> > >> org.apache.nifi.util.StandardProcessorTestRunner.assertAllFlowFilesTransferred(StandardProcessorTestRunner.java:312) >> > >> > at >> > >> > >> > >> > >> > >> >> > >> org.apache.nifi.util.StandardProcessorTestRunner.assertAllFlowFilesTransferred(StandardProcessorTestRunner.java:324) >> > >> > at >> > >> > >> > >> > >> > >> >> > >> org.apache.nifi.processors.kafka.TestPutKafka.testComplexLeftPartialDelimitedMessages(TestPutKafka.java:225) >> > >> > >> > >> > Running org.apache.nifi.processors.kafka.TestGetKafka >> > >> > Tests run: 3, Failures: 0, Errors: 0, Skipped: 1, Time elapsed: 0 >> sec >> > - >> > >> in >> > >> > org.apache.nifi.processors.kafka.TestGetKafka >> > >> > Running >> org.apache.nifi.processors.kafka.SplittableMessageContextTest >> > >> > Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0 >> sec >> > - >> > >> in >> > >> > org.apache.nifi.processors.kafka.SplittableMessageContextTest >> > >> > >> > >> > Results : >> > >> > >> > >> > Failed tests: >> > >> > TestPutKafka.testComplexLeftPartialDelimitedMessages:225 Expected >> > all >> > >> > Transferred FlowFiles to go to success but 1 were routed to failure >> > >> > TestPutKafka.testComplexPartialMatchDelimitedMessages:248 Expected >> > all >> > >> > Transferred FlowFiles to go to success but 1 were routed to failure >> > >> > TestPutKafka.testComplexRightPartialDelimitedMessages:204 Expected >> > all >> > >> > Transferred FlowFiles to go to success but 1 were routed to failure >> > >> > TestPutKafka.testWithEmptyMessages:174 >> > >> > org.apache.kafka.common.errors.TimeoutException: Failed to update >> > >> metadata >> > >> > after 30000 ms. >> > >> > >> > >> > Tests in error: >> > >> > KafkaPublisherTest.validateSuccessfulReSendOfFailedSegments:167 » >> > >> > ConsumerTimeout >> > >> > KafkaPublisherTest.validateSuccessfulSendAsDelimited:114 » >> > >> > ConsumerTimeout >> > >> > KafkaPublisherTest.validateSuccessfulSendAsDelimited2:140 » >> > >> > ConsumerTimeout >> > >> > KafkaPublisherTest.validateSuccessfulSendAsWhole:92 » >> > ConsumerTimeout >> > >> > KafkaPublisherTest.validateWithMultiByteCharacters:195 » >> > >> ConsumerTimeout >> > >> > >> > >> > >> > >> > Tests run: 18, Failures: 4, Errors: 5, Skipped: 3 >> > >> > >> > >> > >> > >> > >> > >> > On Wed, Apr 6, 2016 at 5:33 PM, Joe Witt <joew...@apache.org> >> wrote: >> > >> > >> > >> > > Hello Apache NiFi Community, >> > >> > > >> > >> > > I am pleased to be calling this vote for the source release of >> > Apache >> > >> > > NiFi 0.6.1. >> > >> > > >> > >> > > The source zip, including signatures, digests, etc. can be found >> at: >> > >> > > https://dist.apache.org/repos/dist/dev/nifi/nifi-0.6.1/ >> > >> > > >> > >> > > The Git tag is nifi-0.6.1-RC1 >> > >> > > The Git commit hash is d51b24e18356b76ce649c8285af0806acd9071d0 >> > >> > > * >> > >> > > >> > >> > >> > >> >> > >> https://git-wip-us.apache.org/repos/asf?p=nifi.git;a=commit;h=d51b24e18356b76ce649c8285af0806acd9071d0 >> > >> > > * >> > >> > > >> > >> > >> > >> >> > >> https://github.com/apache/nifi/commit/d51b24e18356b76ce649c8285af0806acd9071d0 >> > >> > > >> > >> > > Checksums of nifi-0.6.1-source-release.zip: >> > >> > > MD5: b82a7fe60e03b679d7c06f57319fa396 >> > >> > > SHA1: 32c45d51d1e1858eaba1df889711c69c17b44781 >> > >> > > >> > >> > > Release artifacts are signed with the following key: >> > >> > > https://people.apache.org/keys/committer/joewitt.asc >> > >> > > >> > >> > > KEYS file available here: >> > >> > > https://dist.apache.org/repos/dist/release/nifi/KEYS >> > >> > > >> > >> > > 11 issues were closed/resolved for this release: >> > >> > > >> > >> > > >> > >> > >> > >> >> > >> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12316020&version=12335496 >> > >> > > Release note highlights can be found here: >> > >> > > >> > >> > > >> > >> > >> > >> >> > >> https://cwiki.apache.org/confluence/display/NIFI/Release+Notes#ReleaseNotes-Version0.6.1 >> > >> > > >> > >> > > The vote will be open for 72 hours. >> > >> > > Please download the release candidate and evaluate the necessary >> > items >> > >> > > including checking hashes, signatures, build from source, and >> test. >> > >> Then >> > >> > > please vote: >> > >> > > >> > >> > > [ ] +1 Release this package as nifi-0.6.1 >> > >> > > [ ] +0 no opinion >> > >> > > [ ] -1 Do not release this package because... >> > >> > > >> > >> > > Thanks! >> > >> > > >> > >> > >> > >> >> > > >> > > >> > >> > On Sat, Apr 9, 2016 at 3:39 PM, Joe Skora <jsk...@gmail.com> wrote: >> > >> > > Aldrin, >> > > >> > > Yes same network, nothing novel that I know of, they all have internet >> > > access and I've disabled firewalls for comparison with no affect. This >> > is >> > > a new router, so I will move one to the old router and see if that >> makes >> > a >> > > difference. >> > > >> > > The validate/valudate mismatch is a fat fingering error on my part, I >> > > copied and pasted the stacktrace but I typed the details manually into >> > the >> > > breakdown by system and JVM. >> > > >> > > The source start point is the "nifi-0.6.1-source-release.zip" file, >> after >> > > checksum validation and unzipping. >> > > >> > > I let you know if network makes a difference. >> > > >> > > Joe >> > > >> > > >> > > >> > > On Sat, Apr 9, 2016 at 3:26 PM, Aldrin Piri <aldrinp...@gmail.com> >> > wrote: >> > > >> > >> Joe, >> > >> >> > >> Are these all running on the same network? If so, anything novel or >> > >> different about it? >> > >> >> > >> Also, a I assume these are remnants of debugging, but noticed some >> > >> differences in terms of the source at the tagged RC and your >> > stacktraces. >> > >> >> > >> TestGetHTTP, for instance, only has 431 lines and the test listed >> > >> "KafkaPublisherTest.valudateSuccessfulSendAsDelimited2" also does not >> > seem >> > >> to exist, but KafkaPublisherTest.validateSuccessfulSendAsDelimited2 >> > does. >> > >> >> > >> Thanks for the detailed report and matrix of JDKs and operating >> systems. >> > >> >> > >> On Sat, Apr 9, 2016 at 2:47 PM, Joe Skora <jsk...@gmail.com> wrote: >> > >> >> > >> > -1 (non-binding) >> > >> > >> > >> > Builds and checkouts out ok, but 3 of 4 systems I tried fail in >> > >> > KafkaPublisherTest unit tests with Java 7 and Java 8. Details and >> > >> sample >> > >> > results are included below. >> > >> > >> > >> > The tests appear to handle ConsumerTimeoutException when it is >> > expected, >> > >> > but the very first call to ConsumerIterator.next() appears to fail >> and >> > >> that >> > >> > is unchecked. >> > >> > >> > >> > Note: I also think KafkaPublisherTest should use sample text that is >> > not >> > >> > copyrighted or possibly considered offensive. >> > >> > >> > >> > * builds and tests ok on Centos 6.7 with Java 7 >> > >> > >> > >> > * tests fail on 3 systems in KafkaPublisherTest >> > >> > >> > >> > * OSX 10.11.4 (El Capitan), i7-2635QM @ 2.0 GHz, 8GB >> > >> > * Maven 3.3.9 >> > >> > * JDK1.7.0_51 TestGetHTTP.testExpressionLanguage():433 and >> > >> others >> > >> > fail with "java.net.SocketException: Invalid argument" >> > >> > in Apache HTTP >> > SessionInputBufferImpl.streamRead():139. >> > >> > * JDK1.7.0_80 >> > >> > KafkaPublisherTest.valudateSuccessfulSendAsDelimited2():140 and >> others >> > >> > error out with "kafka.consumer.ConsumerTimeoutException". >> > >> > * JDK1.8.0_77 >> > >> > KafkaPublisherTest.valudateSuccessfulSendAsDelimited2():140 and >> others >> > >> > error out with "kafka.consumer.ConsumerTimeoutException". >> > >> > >> > >> > * Ubuntu 15.10 (Wily Werewolf), i7-4720HQ @ 2.6 GHz, 16GB >> > >> > * Maven 3.3.3 >> > >> > * JDK1.7.0_80 >> > >> > KafkaPublisherTest.valudateSuccessfulSendAsDelimited2():140 and >> other >> > >> test >> > >> > error out with "kafka.consumer.ConsumerTimeoutException". >> > >> > >> > >> > * Ubuntu 15.10 (Wily Werewolf), i7-4770K @ 3.5 GHz, 16GB >> > >> > * JDK1.7.0_80 >> > >> > KafkaPublisherTest.valudateSuccessfulSendAsDelimited2():140 and >> other >> > >> test >> > >> > error out with "kafka.consumer.ConsumerTimeoutException". >> > >> > * JDK1.8.0_77 >> > >> > KafkaPublisherTest.valudateSuccessfulSendAsDelimited2():140 and >> other >> > >> test >> > >> > error out with "kafka.consumer.ConsumerTimeoutException". >> > >> > >> > >> > >> > >> > ------------------------------------------------------- >> > >> > T E S T S >> > >> > ------------------------------------------------------- >> > >> > Running org.apache.nifi.processors.kafka.KafkaPublisherTest >> > >> > Tests run: 5, Failures: 0, Errors: 5, Skipped: 0, Time elapsed: >> > 701.885 >> > >> sec >> > >> > <<< FAILURE! - in >> org.apache.nifi.processors.kafka.KafkaPublisherTest >> > >> > >> > >> > >> > >> >> > >> validateSuccessfulSendAsDelimited2(org.apache.nifi.processors.kafka.KafkaPublisherTest) >> > >> > Time elapsed: 185.158 sec <<< ERROR! >> > >> > kafka.consumer.ConsumerTimeoutException: null >> > >> > at >> > >> > kafka.consumer.ConsumerIterator.makeNext(ConsumerIterator.scala:69) >> > >> > at >> > >> > kafka.consumer.ConsumerIterator.makeNext(ConsumerIterator.scala:33) >> > >> > at >> > >> > >> > kafka.utils.IteratorTemplate.maybeComputeNext(IteratorTemplate.scala:66) >> > >> > at >> > >> kafka.utils.IteratorTemplate.hasNext(IteratorTemplate.scala:58) >> > >> > at >> > kafka.utils.IteratorTemplate.next(IteratorTemplate.scala:38) >> > >> > at >> > >> kafka.consumer.ConsumerIterator.next(ConsumerIterator.scala:46) >> > >> > at >> > >> > >> > >> > >> > >> >> > >> org.apache.nifi.processors.kafka.KafkaPublisherTest.validateSuccessfulSendAsDelimited2(KafkaPublisherTest.java:140) >> > >> > >> > >> > >> > >> > >> > >> >> > >> validateSuccessfulSendAsDelimited(org.apache.nifi.processors.kafka.KafkaPublisherTest) >> > >> > Time elapsed: 245.077 sec <<< ERROR! >> > >> > kafka.consumer.ConsumerTimeoutException: null >> > >> > at >> > >> > kafka.consumer.ConsumerIterator.makeNext(ConsumerIterator.scala:69) >> > >> > at >> > >> > kafka.consumer.ConsumerIterator.makeNext(ConsumerIterator.scala:33) >> > >> > at >> > >> > >> > kafka.utils.IteratorTemplate.maybeComputeNext(IteratorTemplate.scala:66) >> > >> > at >> > >> kafka.utils.IteratorTemplate.hasNext(IteratorTemplate.scala:58) >> > >> > at >> > kafka.utils.IteratorTemplate.next(IteratorTemplate.scala:38) >> > >> > at >> > >> kafka.consumer.ConsumerIterator.next(ConsumerIterator.scala:46) >> > >> > at >> > >> > >> > >> > >> > >> >> > >> org.apache.nifi.processors.kafka.KafkaPublisherTest.validateSuccessfulSendAsDelimited(KafkaPublisherTest.java:114) >> > >> > >> > >> > >> > >> > >> > >> >> > >> validateSuccessfulReSendOfFailedSegments(org.apache.nifi.processors.kafka.KafkaPublisherTest) >> > >> > Time elapsed: 125.07 sec <<< ERROR! >> > >> > kafka.consumer.ConsumerTimeoutException: null >> > >> > at >> > >> > kafka.consumer.ConsumerIterator.makeNext(ConsumerIterator.scala:69) >> > >> > at >> > >> > kafka.consumer.ConsumerIterator.makeNext(ConsumerIterator.scala:33) >> > >> > at >> > >> > >> > kafka.utils.IteratorTemplate.maybeComputeNext(IteratorTemplate.scala:66) >> > >> > at >> > >> kafka.utils.IteratorTemplate.hasNext(IteratorTemplate.scala:58) >> > >> > at >> > kafka.utils.IteratorTemplate.next(IteratorTemplate.scala:38) >> > >> > at >> > >> kafka.consumer.ConsumerIterator.next(ConsumerIterator.scala:46) >> > >> > at >> > >> > >> > >> > >> > >> >> > >> org.apache.nifi.processors.kafka.KafkaPublisherTest.validateSuccessfulReSendOfFailedSegments(KafkaPublisherTest.java:167) >> > >> > >> > >> > >> > >> > >> > >> >> > >> validateWithMultiByteCharacters(org.apache.nifi.processors.kafka.KafkaPublisherTest) >> > >> > Time elapsed: 65.09 sec <<< ERROR! >> > >> > kafka.consumer.ConsumerTimeoutException: null >> > >> > at >> > >> > kafka.consumer.ConsumerIterator.makeNext(ConsumerIterator.scala:69) >> > >> > at >> > >> > kafka.consumer.ConsumerIterator.makeNext(ConsumerIterator.scala:33) >> > >> > at >> > >> > >> > kafka.utils.IteratorTemplate.maybeComputeNext(IteratorTemplate.scala:66) >> > >> > at >> > >> kafka.utils.IteratorTemplate.hasNext(IteratorTemplate.scala:58) >> > >> > at >> > kafka.utils.IteratorTemplate.next(IteratorTemplate.scala:38) >> > >> > at >> > >> kafka.consumer.ConsumerIterator.next(ConsumerIterator.scala:46) >> > >> > at >> > >> > >> > >> > >> > >> >> > >> org.apache.nifi.processors.kafka.KafkaPublisherTest.validateWithMultiByteCharacters(KafkaPublisherTest.java:195) >> > >> > >> > >> > >> > >> > >> > >> >> > >> validateSuccessfulSendAsWhole(org.apache.nifi.processors.kafka.KafkaPublisherTest) >> > >> > Time elapsed: 65.076 sec <<< ERROR! >> > >> > kafka.consumer.ConsumerTimeoutException: null >> > >> > at >> > >> > kafka.consumer.ConsumerIterator.makeNext(ConsumerIterator.scala:69) >> > >> > at >> > >> > kafka.consumer.ConsumerIterator.makeNext(ConsumerIterator.scala:33) >> > >> > at >> > >> > >> > kafka.utils.IteratorTemplate.maybeComputeNext(IteratorTemplate.scala:66) >> > >> > at >> > >> kafka.utils.IteratorTemplate.hasNext(IteratorTemplate.scala:58) >> > >> > at >> > kafka.utils.IteratorTemplate.next(IteratorTemplate.scala:38) >> > >> > at >> > >> kafka.consumer.ConsumerIterator.next(ConsumerIterator.scala:46) >> > >> > at >> > >> > >> > >> > >> > >> >> > >> org.apache.nifi.processors.kafka.KafkaPublisherTest.validateSuccessfulSendAsWhole(KafkaPublisherTest.java:92) >> > >> > >> > >> > Running org.apache.nifi.processors.kafka.TestPutKafka >> > >> > Tests run: 6, Failures: 4, Errors: 0, Skipped: 2, Time elapsed: >> 288.48 >> > >> sec >> > >> > <<< FAILURE! - in org.apache.nifi.processors.kafka.TestPutKafka >> > >> > >> > >> > >> > >> >> > >> testComplexRightPartialDelimitedMessages(org.apache.nifi.processors.kafka.TestPutKafka) >> > >> > Time elapsed: 63.071 sec <<< FAILURE! >> > >> > java.lang.AssertionError: Expected all Transferred FlowFiles to go >> to >> > >> > success but 1 were routed to failure >> > >> > at org.junit.Assert.fail(Assert.java:88) >> > >> > at >> > >> > >> > >> > >> > >> >> > >> org.apache.nifi.util.MockProcessSession.assertAllFlowFilesTransferred(MockProcessSession.java:985) >> > >> > at >> > >> > >> > >> > >> > >> >> > >> org.apache.nifi.util.StandardProcessorTestRunner.assertAllFlowFilesTransferred(StandardProcessorTestRunner.java:312) >> > >> > at >> > >> > >> > >> > >> > >> >> > >> org.apache.nifi.util.StandardProcessorTestRunner.assertAllFlowFilesTransferred(StandardProcessorTestRunner.java:324) >> > >> > at >> > >> > >> > >> > >> > >> >> > >> org.apache.nifi.processors.kafka.TestPutKafka.testComplexRightPartialDelimitedMessages(TestPutKafka.java:204) >> > >> > >> > >> > >> > >> > >> > >> >> > >> testComplexPartialMatchDelimitedMessages(org.apache.nifi.processors.kafka.TestPutKafka) >> > >> > Time elapsed: 60.005 sec <<< FAILURE! >> > >> > java.lang.AssertionError: Expected all Transferred FlowFiles to go >> to >> > >> > success but 1 were routed to failure >> > >> > at org.junit.Assert.fail(Assert.java:88) >> > >> > at >> > >> > >> > >> > >> > >> >> > >> org.apache.nifi.util.MockProcessSession.assertAllFlowFilesTransferred(MockProcessSession.java:985) >> > >> > at >> > >> > >> > >> > >> > >> >> > >> org.apache.nifi.util.StandardProcessorTestRunner.assertAllFlowFilesTransferred(StandardProcessorTestRunner.java:312) >> > >> > at >> > >> > >> > >> > >> > >> >> > >> org.apache.nifi.util.StandardProcessorTestRunner.assertAllFlowFilesTransferred(StandardProcessorTestRunner.java:324) >> > >> > at >> > >> > >> > >> > >> > >> >> > >> org.apache.nifi.processors.kafka.TestPutKafka.testComplexPartialMatchDelimitedMessages(TestPutKafka.java:248) >> > >> > >> > >> > testWithEmptyMessages(org.apache.nifi.processors.kafka.TestPutKafka) >> > >> Time >> > >> > elapsed: 30.004 sec <<< FAILURE! >> > >> > java.lang.AssertionError: >> > >> org.apache.kafka.common.errors.TimeoutException: >> > >> > Failed to update metadata after 30000 ms. >> > >> > at >> > >> > >> > >> > >> > >> >> > >> org.apache.nifi.util.StandardProcessorTestRunner.run(StandardProcessorTestRunner.java:238) >> > >> > at >> > >> > >> > >> > >> > >> >> > >> org.apache.nifi.util.StandardProcessorTestRunner.run(StandardProcessorTestRunner.java:197) >> > >> > at >> > >> > >> > >> > >> > >> >> > >> org.apache.nifi.util.StandardProcessorTestRunner.run(StandardProcessorTestRunner.java:192) >> > >> > at >> > >> > >> > >> > >> > >> >> > >> org.apache.nifi.util.StandardProcessorTestRunner.run(StandardProcessorTestRunner.java:187) >> > >> > at >> > >> > >> > >> > >> > >> >> > >> org.apache.nifi.processors.kafka.TestPutKafka.testWithEmptyMessages(TestPutKafka.java:174) >> > >> > Caused by: org.apache.kafka.common.errors.TimeoutException: Failed >> to >> > >> > update metadata after 30000 ms. >> > >> > >> > >> > >> > >> > >> > >> >> > >> testComplexLeftPartialDelimitedMessages(org.apache.nifi.processors.kafka.TestPutKafka) >> > >> > Time elapsed: 120.009 sec <<< FAILURE! >> > >> > java.lang.AssertionError: Expected all Transferred FlowFiles to go >> to >> > >> > success but 1 were routed to failure >> > >> > at org.junit.Assert.fail(Assert.java:88) >> > >> > at >> > >> > >> > >> > >> > >> >> > >> org.apache.nifi.util.MockProcessSession.assertAllFlowFilesTransferred(MockProcessSession.java:985) >> > >> > at >> > >> > >> > >> > >> > >> >> > >> org.apache.nifi.util.StandardProcessorTestRunner.assertAllFlowFilesTransferred(StandardProcessorTestRunner.java:312) >> > >> > at >> > >> > >> > >> > >> > >> >> > >> org.apache.nifi.util.StandardProcessorTestRunner.assertAllFlowFilesTransferred(StandardProcessorTestRunner.java:324) >> > >> > at >> > >> > >> > >> > >> > >> >> > >> org.apache.nifi.processors.kafka.TestPutKafka.testComplexLeftPartialDelimitedMessages(TestPutKafka.java:225) >> > >> > >> > >> > Running org.apache.nifi.processors.kafka.TestGetKafka >> > >> > Tests run: 3, Failures: 0, Errors: 0, Skipped: 1, Time elapsed: 0 >> sec >> > - >> > >> in >> > >> > org.apache.nifi.processors.kafka.TestGetKafka >> > >> > Running >> org.apache.nifi.processors.kafka.SplittableMessageContextTest >> > >> > Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0 >> sec >> > - >> > >> in >> > >> > org.apache.nifi.processors.kafka.SplittableMessageContextTest >> > >> > >> > >> > Results : >> > >> > >> > >> > Failed tests: >> > >> > TestPutKafka.testComplexLeftPartialDelimitedMessages:225 Expected >> > all >> > >> > Transferred FlowFiles to go to success but 1 were routed to failure >> > >> > TestPutKafka.testComplexPartialMatchDelimitedMessages:248 Expected >> > all >> > >> > Transferred FlowFiles to go to success but 1 were routed to failure >> > >> > TestPutKafka.testComplexRightPartialDelimitedMessages:204 Expected >> > all >> > >> > Transferred FlowFiles to go to success but 1 were routed to failure >> > >> > TestPutKafka.testWithEmptyMessages:174 >> > >> > org.apache.kafka.common.errors.TimeoutException: Failed to update >> > >> metadata >> > >> > after 30000 ms. >> > >> > >> > >> > Tests in error: >> > >> > KafkaPublisherTest.validateSuccessfulReSendOfFailedSegments:167 » >> > >> > ConsumerTimeout >> > >> > KafkaPublisherTest.validateSuccessfulSendAsDelimited:114 » >> > >> > ConsumerTimeout >> > >> > KafkaPublisherTest.validateSuccessfulSendAsDelimited2:140 » >> > >> > ConsumerTimeout >> > >> > KafkaPublisherTest.validateSuccessfulSendAsWhole:92 » >> > ConsumerTimeout >> > >> > KafkaPublisherTest.validateWithMultiByteCharacters:195 » >> > >> ConsumerTimeout >> > >> > >> > >> > >> > >> > Tests run: 18, Failures: 4, Errors: 5, Skipped: 3 >> > >> > >> > >> > >> > >> > >> > >> > On Wed, Apr 6, 2016 at 5:33 PM, Joe Witt <joew...@apache.org> >> wrote: >> > >> > >> > >> > > Hello Apache NiFi Community, >> > >> > > >> > >> > > I am pleased to be calling this vote for the source release of >> > Apache >> > >> > > NiFi 0.6.1. >> > >> > > >> > >> > > The source zip, including signatures, digests, etc. can be found >> at: >> > >> > > https://dist.apache.org/repos/dist/dev/nifi/nifi-0.6.1/ >> > >> > > >> > >> > > The Git tag is nifi-0.6.1-RC1 >> > >> > > The Git commit hash is d51b24e18356b76ce649c8285af0806acd9071d0 >> > >> > > * >> > >> > > >> > >> > >> > >> >> > >> https://git-wip-us.apache.org/repos/asf?p=nifi.git;a=commit;h=d51b24e18356b76ce649c8285af0806acd9071d0 >> > >> > > * >> > >> > > >> > >> > >> > >> >> > >> https://github.com/apache/nifi/commit/d51b24e18356b76ce649c8285af0806acd9071d0 >> > >> > > >> > >> > > Checksums of nifi-0.6.1-source-release.zip: >> > >> > > MD5: b82a7fe60e03b679d7c06f57319fa396 >> > >> > > SHA1: 32c45d51d1e1858eaba1df889711c69c17b44781 >> > >> > > >> > >> > > Release artifacts are signed with the following key: >> > >> > > https://people.apache.org/keys/committer/joewitt.asc >> > >> > > >> > >> > > KEYS file available here: >> > >> > > https://dist.apache.org/repos/dist/release/nifi/KEYS >> > >> > > >> > >> > > 11 issues were closed/resolved for this release: >> > >> > > >> > >> > > >> > >> > >> > >> >> > >> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12316020&version=12335496 >> > >> > > Release note highlights can be found here: >> > >> > > >> > >> > > >> > >> > >> > >> >> > >> https://cwiki.apache.org/confluence/display/NIFI/Release+Notes#ReleaseNotes-Version0.6.1 >> > >> > > >> > >> > > The vote will be open for 72 hours. >> > >> > > Please download the release candidate and evaluate the necessary >> > items >> > >> > > including checking hashes, signatures, build from source, and >> test. >> > >> Then >> > >> > > please vote: >> > >> > > >> > >> > > [ ] +1 Release this package as nifi-0.6.1 >> > >> > > [ ] +0 no opinion >> > >> > > [ ] -1 Do not release this package because... >> > >> > > >> > >> > > Thanks! >> > >> > > >> > >> > >> > >> >> > > >> > > >> > >>
