[
https://issues.apache.org/jira/browse/SAMZA-1836?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yi Pan (Data Infrastructure) resolved SAMZA-1836.
-------------------------------------------------
Resolution: Fixed
Assignee: Sanil Jain
> StreamApplicationIntegrationTest and TestLocalTableWithSideInput fails due to
> wrong system configured
> -----------------------------------------------------------------------------------------------------
>
> Key: SAMZA-1836
> URL: https://issues.apache.org/jira/browse/SAMZA-1836
> Project: Samza
> Issue Type: Improvement
> Reporter: Yi Pan (Data Infrastructure)
> Assignee: Sanil Jain
> Priority: Major
> Fix For: 1.0
>
>
> The integration tests StreamApplicationIntegrationTest and
> TestLocalTableWithSideInput are using the new test framework and supposed to
> use InMemorySystemFactory. The recent check-in with PR 603 uses
> KafkaSystemDescriptor in the test, which will cause the following failure, if
> the generated configure is actually used in AbstractApplicationRunner:
> {code}
> org.apache.samza.SamzaException: Failed to start application:
> org.apache.samza.test.framework.StreamApplicationIntegrationTest$$Lambda$2/1844169442@4b0b0854.
> at
> org.apache.samza.runtime.LocalApplicationRunner.run(LocalApplicationRunner.java:201)
> at org.apache.samza.test.framework.TestRunner.run(TestRunner.java:293)
> at
> org.apache.samza.test.framework.StreamApplicationIntegrationTest.testHighLevelApi(StreamApplicationIntegrationTest.java:94)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
> at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
> at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
> at
> com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:68)
> at
> com.intellij.rt.execution.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:51)
> at
> com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:242)
> at
> com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:70)
> Caused by: org.apache.samza.SamzaException: No bootstrap servers defined in
> config for test.
> at
> org.apache.samza.config.KafkaProducerConfig$$anonfun$4.apply(KafkaConfig.scala:379)
> at
> org.apache.samza.config.KafkaProducerConfig$$anonfun$4.apply(KafkaConfig.scala:379)
> at scala.Option.getOrElse(Option.scala:121)
> at
> org.apache.samza.config.KafkaProducerConfig.<init>(KafkaConfig.scala:379)
> at
> org.apache.samza.config.KafkaConfig.getKafkaSystemProducerConfig(KafkaConfig.scala:316)
> at
> org.apache.samza.system.kafka.KafkaSystemFactory.getAdmin(KafkaSystemFactory.scala:108)
> at
> org.apache.samza.config.JavaSystemConfig.lambda$getSystemAdmins$1(JavaSystemConfig.java:83)
> at java.util.stream.Collectors.lambda$toMap$58(Collectors.java:1321)
> at java.util.stream.ReduceOps$3ReducingSink.accept(ReduceOps.java:169)
> at
> java.util.HashMap$EntrySpliterator.forEachRemaining(HashMap.java:1691)
> at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481)
> at
> java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
> at
> java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708)
> at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
> at
> java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:499)
> at
> org.apache.samza.config.JavaSystemConfig.getSystemAdmins(JavaSystemConfig.java:81)
> at org.apache.samza.system.SystemAdmins.<init>(SystemAdmins.java:38)
> at
> org.apache.samza.execution.StreamManager.<init>(StreamManager.java:55)
> at
> org.apache.samza.runtime.AbstractApplicationRunner.buildAndStartStreamManager(AbstractApplicationRunner.java:130)
> at
> org.apache.samza.runtime.AbstractApplicationRunner.getExecutionPlan(AbstractApplicationRunner.java:98)
> at
> org.apache.samza.runtime.AbstractApplicationRunner.getExecutionPlan(AbstractApplicationRunner.java:63)
> at
> org.apache.samza.runtime.LocalApplicationRunner.run(LocalApplicationRunner.java:164)
> ... 24 more
> {code}
> The tests in master were passing due to the following double issues:
> # in AbstractApplicationRunner, the expanded systemStreamConfigs are not used
> to create the StreamManager:
> {code}
> Map<String, String> appConfigs = new HashMap<>(cfg);
> appConfigs.putAll(systemStreamConfigs);
> // create the physical execution plan
> Config generatedConfig = new MapConfig(cfg);
> StreamManager streamManager = buildAndStartStreamManager(generatedConfig);
> {code}
> # In integration tests, we used KafkaSystemDescriptor for the test system,
> which should be using InMemorySystemFactory. Example in
> StreamApplicationIntegrationTest:
> {code}
> final StreamApplication pageViewFilter = (streamGraph, cfg) -> {
> KafkaSystemDescriptor ksd = new KafkaSystemDescriptor("test");
> KafkaInputDescriptor<KV<String, PageView>> isd =
> ksd.getInputDescriptor("PageView", KVSerde.of(new NoOpSerde<>(), new
> NoOpSerde<>()));
> MessageStream<KV<String, TestData.PageView>> inputStream =
> streamGraph.getInputStream(isd);
>
> inputStream.map(StreamApplicationIntegrationTest.Values.create()).filter(pv
> -> pv.getPageKey().equals("inbox"));
> };
> {code}
> The test failure was not triggered because it is masked by the first bug.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
