The OSX system passed the tests switching to a different network. Putting
it back on the problem network and checking KafkaPublisherTest-output.txt I
see repeated occurrences of the following. (fat fingered, ignore any
obvious typos)
17:01:59,225 WARN kafka-producer-network-thread | producer-2
network.Selector:276 - Error in I/O with /0.0.0.0
java.net.SocketException: Network is unreachable
Crazier still, the Ubuntu system I moved to the old network still fails
with messages like this.
17:01:59,225 WARN kafka-producer-network-thread | producer-2
network.Selector:276 - Error in I/O with /0.0.0.0
java.net.NoRouteToHostException: No route to host
There's obviously some network issue on the new router, possibly routing or
DNS, but I've never had these kind of problems on the old router and one
system passes and one fails on the new router.
I'm trying to track it further but unless binding to 0.0.0.0 is new, this
looks like a problem on my end.
On Sat, Apr 9, 2016 at 4:49 PM, Oleg Zhurakousky <
ozhurakou...@hortonworks.com> wrote:
> Perfect
>
> I was just replying too you again ;). Indeed ‘test’ was not used and I
> already commented to that effect in JIRA and removed.
> Yes, I was wondering why they fail as this is the first time someone
> reported KafkaPublisherTest failures, so kind of curious, so please feed
> all the info you can get if they fail again.
>
> Cheers
> Oleg
>
> > On Apr 9, 2016, at 4:45 PM, Joe Skora <jsk...@gmail.com> wrote:
> >
> > Oleg,
> >
> > I'm not familiar enough with Kafka to know what the tests need, but
> looking
> > at the test source now I don't think the "String test" on line 70 is used
> > anywhere, so I think you can just remove it.
> >
> > The failures where pretty much the same on all 3 systems. But based on
> > Aldrin's question about the network I moved the OS X to a different
> network
> > and it passed. I'm running one of the Ubuntu boxes now.
> >
> > Thanks,
> > Joe
> >
> > On Sat, Apr 9, 2016 at 4:30 PM, Oleg Zhurakousky <
> > ozhurakou...@hortonworks.com> wrote:
> >
> >> Joe
> >>
> >> Thanks for pointing that out. I’ll remove the quote from CNN article,
> just
> >> used it as sample text. Do you see any other issue with sample data?
> >> Also, which systems did you observe failures described below?
> >>
> >> Cheers
> >> Oleg
> >>
> >>> On Apr 9, 2016, at 2:47 PM, Joe Skora <jsk...@gmail.com> wrote:
> >>>
> >>> -1 (non-binding)
> >>>
> >>> Builds and checkouts out ok, but 3 of 4 systems I tried fail in
> >>> KafkaPublisherTest unit tests with Java 7 and Java 8. Details and
> sample
> >>> results are included below.
> >>>
> >>> The tests appear to handle ConsumerTimeoutException when it is
> expected,
> >>> but the very first call to ConsumerIterator.next() appears to fail and
> >> that
> >>> is unchecked.
> >>>
> >>> Note: I also think KafkaPublisherTest should use sample text that is
> not
> >>> copyrighted or possibly considered offensive.
> >>>
> >>> * builds and tests ok on Centos 6.7 with Java 7
> >>>
> >>> * tests fail on 3 systems in KafkaPublisherTest
> >>>
> >>> * OSX 10.11.4 (El Capitan), i7-2635QM @ 2.0 GHz, 8GB
> >>> * Maven 3.3.9
> >>> * JDK1.7.0_51 TestGetHTTP.testExpressionLanguage():433 and
> >> others
> >>> fail with "java.net.SocketException: Invalid argument"
> >>> in Apache HTTP SessionInputBufferImpl.streamRead():139.
> >>> * JDK1.7.0_80
> >>> KafkaPublisherTest.valudateSuccessfulSendAsDelimited2():140 and others
> >>> error out with "kafka.consumer.ConsumerTimeoutException".
> >>> * JDK1.8.0_77
> >>> KafkaPublisherTest.valudateSuccessfulSendAsDelimited2():140 and others
> >>> error out with "kafka.consumer.ConsumerTimeoutException".
> >>>
> >>> * Ubuntu 15.10 (Wily Werewolf), i7-4720HQ @ 2.6 GHz, 16GB
> >>> * Maven 3.3.3
> >>> * JDK1.7.0_80
> >>> KafkaPublisherTest.valudateSuccessfulSendAsDelimited2():140 and other
> >> test
> >>> error out with "kafka.consumer.ConsumerTimeoutException".
> >>>
> >>> * Ubuntu 15.10 (Wily Werewolf), i7-4770K @ 3.5 GHz, 16GB
> >>> * JDK1.7.0_80
> >>> KafkaPublisherTest.valudateSuccessfulSendAsDelimited2():140 and other
> >> test
> >>> error out with "kafka.consumer.ConsumerTimeoutException".
> >>> * JDK1.8.0_77
> >>> KafkaPublisherTest.valudateSuccessfulSendAsDelimited2():140 and other
> >> test
> >>> error out with "kafka.consumer.ConsumerTimeoutException".
> >>>
> >>>
> >>> -------------------------------------------------------
> >>> T E S T S
> >>> -------------------------------------------------------
> >>> Running org.apache.nifi.processors.kafka.KafkaPublisherTest
> >>> Tests run: 5, Failures: 0, Errors: 5, Skipped: 0, Time elapsed: 701.885
> >> sec
> >>> <<< FAILURE! - in org.apache.nifi.processors.kafka.KafkaPublisherTest
> >>>
> >>
> validateSuccessfulSendAsDelimited2(org.apache.nifi.processors.kafka.KafkaPublisherTest)
> >>> Time elapsed: 185.158 sec <<< ERROR!
> >>> kafka.consumer.ConsumerTimeoutException: null
> >>> at
> >>> kafka.consumer.ConsumerIterator.makeNext(ConsumerIterator.scala:69)
> >>> at
> >>> kafka.consumer.ConsumerIterator.makeNext(ConsumerIterator.scala:33)
> >>> at
> >>>
> kafka.utils.IteratorTemplate.maybeComputeNext(IteratorTemplate.scala:66)
> >>> at
> kafka.utils.IteratorTemplate.hasNext(IteratorTemplate.scala:58)
> >>> at kafka.utils.IteratorTemplate.next(IteratorTemplate.scala:38)
> >>> at
> kafka.consumer.ConsumerIterator.next(ConsumerIterator.scala:46)
> >>> at
> >>>
> >>
> org.apache.nifi.processors.kafka.KafkaPublisherTest.validateSuccessfulSendAsDelimited2(KafkaPublisherTest.java:140)
> >>>
> >>>
> >>
> validateSuccessfulSendAsDelimited(org.apache.nifi.processors.kafka.KafkaPublisherTest)
> >>> Time elapsed: 245.077 sec <<< ERROR!
> >>> kafka.consumer.ConsumerTimeoutException: null
> >>> at
> >>> kafka.consumer.ConsumerIterator.makeNext(ConsumerIterator.scala:69)
> >>> at
> >>> kafka.consumer.ConsumerIterator.makeNext(ConsumerIterator.scala:33)
> >>> at
> >>>
> kafka.utils.IteratorTemplate.maybeComputeNext(IteratorTemplate.scala:66)
> >>> at
> kafka.utils.IteratorTemplate.hasNext(IteratorTemplate.scala:58)
> >>> at kafka.utils.IteratorTemplate.next(IteratorTemplate.scala:38)
> >>> at
> kafka.consumer.ConsumerIterator.next(ConsumerIterator.scala:46)
> >>> at
> >>>
> >>
> org.apache.nifi.processors.kafka.KafkaPublisherTest.validateSuccessfulSendAsDelimited(KafkaPublisherTest.java:114)
> >>>
> >>>
> >>
> validateSuccessfulReSendOfFailedSegments(org.apache.nifi.processors.kafka.KafkaPublisherTest)
> >>> Time elapsed: 125.07 sec <<< ERROR!
> >>> kafka.consumer.ConsumerTimeoutException: null
> >>> at
> >>> kafka.consumer.ConsumerIterator.makeNext(ConsumerIterator.scala:69)
> >>> at
> >>> kafka.consumer.ConsumerIterator.makeNext(ConsumerIterator.scala:33)
> >>> at
> >>>
> kafka.utils.IteratorTemplate.maybeComputeNext(IteratorTemplate.scala:66)
> >>> at
> kafka.utils.IteratorTemplate.hasNext(IteratorTemplate.scala:58)
> >>> at kafka.utils.IteratorTemplate.next(IteratorTemplate.scala:38)
> >>> at
> kafka.consumer.ConsumerIterator.next(ConsumerIterator.scala:46)
> >>> at
> >>>
> >>
> org.apache.nifi.processors.kafka.KafkaPublisherTest.validateSuccessfulReSendOfFailedSegments(KafkaPublisherTest.java:167)
> >>>
> >>>
> >>
> validateWithMultiByteCharacters(org.apache.nifi.processors.kafka.KafkaPublisherTest)
> >>> Time elapsed: 65.09 sec <<< ERROR!
> >>> kafka.consumer.ConsumerTimeoutException: null
> >>> at
> >>> kafka.consumer.ConsumerIterator.makeNext(ConsumerIterator.scala:69)
> >>> at
> >>> kafka.consumer.ConsumerIterator.makeNext(ConsumerIterator.scala:33)
> >>> at
> >>>
> kafka.utils.IteratorTemplate.maybeComputeNext(IteratorTemplate.scala:66)
> >>> at
> kafka.utils.IteratorTemplate.hasNext(IteratorTemplate.scala:58)
> >>> at kafka.utils.IteratorTemplate.next(IteratorTemplate.scala:38)
> >>> at
> kafka.consumer.ConsumerIterator.next(ConsumerIterator.scala:46)
> >>> at
> >>>
> >>
> org.apache.nifi.processors.kafka.KafkaPublisherTest.validateWithMultiByteCharacters(KafkaPublisherTest.java:195)
> >>>
> >>>
> >>
> validateSuccessfulSendAsWhole(org.apache.nifi.processors.kafka.KafkaPublisherTest)
> >>> Time elapsed: 65.076 sec <<< ERROR!
> >>> kafka.consumer.ConsumerTimeoutException: null
> >>> at
> >>> kafka.consumer.ConsumerIterator.makeNext(ConsumerIterator.scala:69)
> >>> at
> >>> kafka.consumer.ConsumerIterator.makeNext(ConsumerIterator.scala:33)
> >>> at
> >>>
> kafka.utils.IteratorTemplate.maybeComputeNext(IteratorTemplate.scala:66)
> >>> at
> kafka.utils.IteratorTemplate.hasNext(IteratorTemplate.scala:58)
> >>> at kafka.utils.IteratorTemplate.next(IteratorTemplate.scala:38)
> >>> at
> kafka.consumer.ConsumerIterator.next(ConsumerIterator.scala:46)
> >>> at
> >>>
> >>
> org.apache.nifi.processors.kafka.KafkaPublisherTest.validateSuccessfulSendAsWhole(KafkaPublisherTest.java:92)
> >>>
> >>> Running org.apache.nifi.processors.kafka.TestPutKafka
> >>> Tests run: 6, Failures: 4, Errors: 0, Skipped: 2, Time elapsed: 288.48
> >> sec
> >>> <<< FAILURE! - in org.apache.nifi.processors.kafka.TestPutKafka
> >>>
> >>
> testComplexRightPartialDelimitedMessages(org.apache.nifi.processors.kafka.TestPutKafka)
> >>> Time elapsed: 63.071 sec <<< FAILURE!
> >>> java.lang.AssertionError: Expected all Transferred FlowFiles to go to
> >>> success but 1 were routed to failure
> >>> at org.junit.Assert.fail(Assert.java:88)
> >>> at
> >>>
> >>
> org.apache.nifi.util.MockProcessSession.assertAllFlowFilesTransferred(MockProcessSession.java:985)
> >>> at
> >>>
> >>
> org.apache.nifi.util.StandardProcessorTestRunner.assertAllFlowFilesTransferred(StandardProcessorTestRunner.java:312)
> >>> at
> >>>
> >>
> org.apache.nifi.util.StandardProcessorTestRunner.assertAllFlowFilesTransferred(StandardProcessorTestRunner.java:324)
> >>> at
> >>>
> >>
> org.apache.nifi.processors.kafka.TestPutKafka.testComplexRightPartialDelimitedMessages(TestPutKafka.java:204)
> >>>
> >>>
> >>
> testComplexPartialMatchDelimitedMessages(org.apache.nifi.processors.kafka.TestPutKafka)
> >>> Time elapsed: 60.005 sec <<< FAILURE!
> >>> java.lang.AssertionError: Expected all Transferred FlowFiles to go to
> >>> success but 1 were routed to failure
> >>> at org.junit.Assert.fail(Assert.java:88)
> >>> at
> >>>
> >>
> org.apache.nifi.util.MockProcessSession.assertAllFlowFilesTransferred(MockProcessSession.java:985)
> >>> at
> >>>
> >>
> org.apache.nifi.util.StandardProcessorTestRunner.assertAllFlowFilesTransferred(StandardProcessorTestRunner.java:312)
> >>> at
> >>>
> >>
> org.apache.nifi.util.StandardProcessorTestRunner.assertAllFlowFilesTransferred(StandardProcessorTestRunner.java:324)
> >>> at
> >>>
> >>
> org.apache.nifi.processors.kafka.TestPutKafka.testComplexPartialMatchDelimitedMessages(TestPutKafka.java:248)
> >>>
> >>> testWithEmptyMessages(org.apache.nifi.processors.kafka.TestPutKafka)
> >> Time
> >>> elapsed: 30.004 sec <<< FAILURE!
> >>> java.lang.AssertionError:
> >> org.apache.kafka.common.errors.TimeoutException:
> >>> Failed to update metadata after 30000 ms.
> >>> at
> >>>
> >>
> org.apache.nifi.util.StandardProcessorTestRunner.run(StandardProcessorTestRunner.java:238)
> >>> at
> >>>
> >>
> org.apache.nifi.util.StandardProcessorTestRunner.run(StandardProcessorTestRunner.java:197)
> >>> at
> >>>
> >>
> org.apache.nifi.util.StandardProcessorTestRunner.run(StandardProcessorTestRunner.java:192)
> >>> at
> >>>
> >>
> org.apache.nifi.util.StandardProcessorTestRunner.run(StandardProcessorTestRunner.java:187)
> >>> at
> >>>
> >>
> org.apache.nifi.processors.kafka.TestPutKafka.testWithEmptyMessages(TestPutKafka.java:174)
> >>> Caused by: org.apache.kafka.common.errors.TimeoutException: Failed to
> >>> update metadata after 30000 ms.
> >>>
> >>>
> >>
> testComplexLeftPartialDelimitedMessages(org.apache.nifi.processors.kafka.TestPutKafka)
> >>> Time elapsed: 120.009 sec <<< FAILURE!
> >>> java.lang.AssertionError: Expected all Transferred FlowFiles to go to
> >>> success but 1 were routed to failure
> >>> at org.junit.Assert.fail(Assert.java:88)
> >>> at
> >>>
> >>
> org.apache.nifi.util.MockProcessSession.assertAllFlowFilesTransferred(MockProcessSession.java:985)
> >>> at
> >>>
> >>
> org.apache.nifi.util.StandardProcessorTestRunner.assertAllFlowFilesTransferred(StandardProcessorTestRunner.java:312)
> >>> at
> >>>
> >>
> org.apache.nifi.util.StandardProcessorTestRunner.assertAllFlowFilesTransferred(StandardProcessorTestRunner.java:324)
> >>> at
> >>>
> >>
> org.apache.nifi.processors.kafka.TestPutKafka.testComplexLeftPartialDelimitedMessages(TestPutKafka.java:225)
> >>>
> >>> Running org.apache.nifi.processors.kafka.TestGetKafka
> >>> Tests run: 3, Failures: 0, Errors: 0, Skipped: 1, Time elapsed: 0 sec -
> >> in
> >>> org.apache.nifi.processors.kafka.TestGetKafka
> >>> Running org.apache.nifi.processors.kafka.SplittableMessageContextTest
> >>> Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0 sec -
> >> in
> >>> org.apache.nifi.processors.kafka.SplittableMessageContextTest
> >>>
> >>> Results :
> >>>
> >>> Failed tests:
> >>> TestPutKafka.testComplexLeftPartialDelimitedMessages:225 Expected all
> >>> Transferred FlowFiles to go to success but 1 were routed to failure
> >>> TestPutKafka.testComplexPartialMatchDelimitedMessages:248 Expected all
> >>> Transferred FlowFiles to go to success but 1 were routed to failure
> >>> TestPutKafka.testComplexRightPartialDelimitedMessages:204 Expected all
> >>> Transferred FlowFiles to go to success but 1 were routed to failure
> >>> TestPutKafka.testWithEmptyMessages:174
> >>> org.apache.kafka.common.errors.TimeoutException: Failed to update
> >> metadata
> >>> after 30000 ms.
> >>>
> >>> Tests in error:
> >>> KafkaPublisherTest.validateSuccessfulReSendOfFailedSegments:167 »
> >>> ConsumerTimeout
> >>> KafkaPublisherTest.validateSuccessfulSendAsDelimited:114 »
> >> ConsumerTimeout
> >>> KafkaPublisherTest.validateSuccessfulSendAsDelimited2:140 »
> >>> ConsumerTimeout
> >>> KafkaPublisherTest.validateSuccessfulSendAsWhole:92 » ConsumerTimeout
> >>> KafkaPublisherTest.validateWithMultiByteCharacters:195 »
> ConsumerTimeout
> >>>
> >>>
> >>> Tests run: 18, Failures: 4, Errors: 5, Skipped: 3
> >>>
> >>>
> >>>
> >>> On Wed, Apr 6, 2016 at 5:33 PM, Joe Witt <joew...@apache.org> wrote:
> >>>
> >>>> Hello Apache NiFi Community,
> >>>>
> >>>> I am pleased to be calling this vote for the source release of Apache
> >>>> NiFi 0.6.1.
> >>>>
> >>>> The source zip, including signatures, digests, etc. can be found at:
> >>>> https://dist.apache.org/repos/dist/dev/nifi/nifi-0.6.1/
> >>>>
> >>>> The Git tag is nifi-0.6.1-RC1
> >>>> The Git commit hash is d51b24e18356b76ce649c8285af0806acd9071d0
> >>>> *
> >>>>
> >>
> https://git-wip-us.apache.org/repos/asf?p=nifi.git;a=commit;h=d51b24e18356b76ce649c8285af0806acd9071d0
> >>>> *
> >>>>
> >>
> https://github.com/apache/nifi/commit/d51b24e18356b76ce649c8285af0806acd9071d0
> >>>>
> >>>> Checksums of nifi-0.6.1-source-release.zip:
> >>>> MD5: b82a7fe60e03b679d7c06f57319fa396
> >>>> SHA1: 32c45d51d1e1858eaba1df889711c69c17b44781
> >>>>
> >>>> Release artifacts are signed with the following key:
> >>>> https://people.apache.org/keys/committer/joewitt.asc
> >>>>
> >>>> KEYS file available here:
> >>>> https://dist.apache.org/repos/dist/release/nifi/KEYS
> >>>>
> >>>> 11 issues were closed/resolved for this release:
> >>>>
> >>>>
> >>
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12316020&version=12335496
> >>>> Release note highlights can be found here:
> >>>>
> >>>>
> >>
> https://cwiki.apache.org/confluence/display/NIFI/Release+Notes#ReleaseNotes-Version0.6.1
> >>>>
> >>>> The vote will be open for 72 hours.
> >>>> Please download the release candidate and evaluate the necessary items
> >>>> including checking hashes, signatures, build from source, and test.
> Then
> >>>> please vote:
> >>>>
> >>>> [ ] +1 Release this package as nifi-0.6.1
> >>>> [ ] +0 no opinion
> >>>> [ ] -1 Do not release this package because...
> >>>>
> >>>> Thanks!
> >>>>
> >>
> >>
>
>
