[jira] [Commented] (BEAM-1213) WordCount example failure on Apex Runner
[ https://issues.apache.org/jira/browse/BEAM-1213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16001057#comment-16001057 ] Davor Bonaci commented on BEAM-1213: Let's resolve this one, then. We'll anyways manually re-confirm scenarios like this in the acceptance criteria. > WordCount example failure on Apex Runner > > > Key: BEAM-1213 > URL: https://issues.apache.org/jira/browse/BEAM-1213 > Project: Beam > Issue Type: Bug > Components: runner-apex >Reporter: Sandeep Deshmukh >Assignee: Thomas Weise > Fix For: 2.0.0 > > > Tried running WordCount example using apex runner (on YARN) with > https://github.com/apache/incubator-beam/pull/1693 , fix for httpclient > version issue fixed. > Create /tmp/input-beam/ dir on all nodes and pushed a test data in that > directory. Ran the following command in examples/java/ > ~/bin/apache-maven-3.3.9/bin/mvn compile exec:java > -Dexec.mainClass=org.apache.beam.examples.WordCount > -Dexec.args="--inputFile=/tmp/input-beam/java.java --output=/tmp/output-beam/ > --runner=ApexRunner --embeddedExecution=false" -Papex-runner > INFO: using > /disk2/devhdfs/nn_20161013/usercache/sandeep/appcache/application_1482319446115_4336/container_1482319446115_4336_01_14/tmp/chkp5198889890309038810 > as the basepath for checkpointing. > Dec 23, 2016 7:17:44 AM com.datatorrent.stram.engine.StreamingContainer main > SEVERE: Fatal exception in container! > java.lang.IllegalArgumentException: Invalid slice: offset=1426, length=0 > array.length=67108864 > at com.datatorrent.netlet.util.Slice.(Slice.java:43) > at > com.datatorrent.bufferserver.util.SerializedData.(SerializedData.java:40) > at > com.datatorrent.bufferserver.internal.DataList$Block.purge(DataList.java:712) > at > com.datatorrent.bufferserver.internal.DataList.purge(DataList.java:192) > at com.datatorrent.bufferserver.server.Server.purge(Server.java:204) > at > com.datatorrent.stram.engine.StreamingContainer.processHeartbeatResponse(StreamingContainer.java:776) > at > com.datatorrent.stram.engine.StreamingContainer.heartbeatLoop(StreamingContainer.java:705) > at > com.datatorrent.stram.engine.StreamingContainer.main(StreamingContainer.java:310) > Dec 23, 2016 7:17:44 AM com.datatorrent.bufferserver.server.Server > unregistered > INFO: Server stopped listening at /0:0:0:0:0:0:0:0:40904 > Dec 23, 2016 7:17:44 AM com.datatorrent.stram.engine.StreamingContainer main > INFO: Exit status for container: 1 -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (BEAM-1213) WordCount example failure on Apex Runner
[ https://issues.apache.org/jira/browse/BEAM-1213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15997802#comment-15997802 ] Thomas Weise commented on BEAM-1213: I have not tested this with the RC but I expect it to be fixed, it looks related to what we saw due to APEXCORE-703. > WordCount example failure on Apex Runner > > > Key: BEAM-1213 > URL: https://issues.apache.org/jira/browse/BEAM-1213 > Project: Beam > Issue Type: Bug > Components: runner-apex >Reporter: Sandeep Deshmukh >Assignee: Thomas Weise > Fix For: First stable release > > > Tried running WordCount example using apex runner (on YARN) with > https://github.com/apache/incubator-beam/pull/1693 , fix for httpclient > version issue fixed. > Create /tmp/input-beam/ dir on all nodes and pushed a test data in that > directory. Ran the following command in examples/java/ > ~/bin/apache-maven-3.3.9/bin/mvn compile exec:java > -Dexec.mainClass=org.apache.beam.examples.WordCount > -Dexec.args="--inputFile=/tmp/input-beam/java.java --output=/tmp/output-beam/ > --runner=ApexRunner --embeddedExecution=false" -Papex-runner > INFO: using > /disk2/devhdfs/nn_20161013/usercache/sandeep/appcache/application_1482319446115_4336/container_1482319446115_4336_01_14/tmp/chkp5198889890309038810 > as the basepath for checkpointing. > Dec 23, 2016 7:17:44 AM com.datatorrent.stram.engine.StreamingContainer main > SEVERE: Fatal exception in container! > java.lang.IllegalArgumentException: Invalid slice: offset=1426, length=0 > array.length=67108864 > at com.datatorrent.netlet.util.Slice.(Slice.java:43) > at > com.datatorrent.bufferserver.util.SerializedData.(SerializedData.java:40) > at > com.datatorrent.bufferserver.internal.DataList$Block.purge(DataList.java:712) > at > com.datatorrent.bufferserver.internal.DataList.purge(DataList.java:192) > at com.datatorrent.bufferserver.server.Server.purge(Server.java:204) > at > com.datatorrent.stram.engine.StreamingContainer.processHeartbeatResponse(StreamingContainer.java:776) > at > com.datatorrent.stram.engine.StreamingContainer.heartbeatLoop(StreamingContainer.java:705) > at > com.datatorrent.stram.engine.StreamingContainer.main(StreamingContainer.java:310) > Dec 23, 2016 7:17:44 AM com.datatorrent.bufferserver.server.Server > unregistered > INFO: Server stopped listening at /0:0:0:0:0:0:0:0:40904 > Dec 23, 2016 7:17:44 AM com.datatorrent.stram.engine.StreamingContainer main > INFO: Exit status for container: 1 -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (BEAM-1213) WordCount example failure on Apex Runner
[ https://issues.apache.org/jira/browse/BEAM-1213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15997796#comment-15997796 ] Kenneth Knowles commented on BEAM-1213: --- [~thw] I noticed in the Apex release candidate vote that you had tested on Beam. Does that address this issue? Did you test on YARN or embedded? I've gotten lost in the weeds for a little bit provisioning an appropriate YARN cluster and properly bundling the right versions of dependencies and whatnot. > WordCount example failure on Apex Runner > > > Key: BEAM-1213 > URL: https://issues.apache.org/jira/browse/BEAM-1213 > Project: Beam > Issue Type: Bug > Components: runner-apex >Reporter: Sandeep Deshmukh >Assignee: Thomas Weise > Fix For: First stable release > > > Tried running WordCount example using apex runner (on YARN) with > https://github.com/apache/incubator-beam/pull/1693 , fix for httpclient > version issue fixed. > Create /tmp/input-beam/ dir on all nodes and pushed a test data in that > directory. Ran the following command in examples/java/ > ~/bin/apache-maven-3.3.9/bin/mvn compile exec:java > -Dexec.mainClass=org.apache.beam.examples.WordCount > -Dexec.args="--inputFile=/tmp/input-beam/java.java --output=/tmp/output-beam/ > --runner=ApexRunner --embeddedExecution=false" -Papex-runner > INFO: using > /disk2/devhdfs/nn_20161013/usercache/sandeep/appcache/application_1482319446115_4336/container_1482319446115_4336_01_14/tmp/chkp5198889890309038810 > as the basepath for checkpointing. > Dec 23, 2016 7:17:44 AM com.datatorrent.stram.engine.StreamingContainer main > SEVERE: Fatal exception in container! > java.lang.IllegalArgumentException: Invalid slice: offset=1426, length=0 > array.length=67108864 > at com.datatorrent.netlet.util.Slice.(Slice.java:43) > at > com.datatorrent.bufferserver.util.SerializedData.(SerializedData.java:40) > at > com.datatorrent.bufferserver.internal.DataList$Block.purge(DataList.java:712) > at > com.datatorrent.bufferserver.internal.DataList.purge(DataList.java:192) > at com.datatorrent.bufferserver.server.Server.purge(Server.java:204) > at > com.datatorrent.stram.engine.StreamingContainer.processHeartbeatResponse(StreamingContainer.java:776) > at > com.datatorrent.stram.engine.StreamingContainer.heartbeatLoop(StreamingContainer.java:705) > at > com.datatorrent.stram.engine.StreamingContainer.main(StreamingContainer.java:310) > Dec 23, 2016 7:17:44 AM com.datatorrent.bufferserver.server.Server > unregistered > INFO: Server stopped listening at /0:0:0:0:0:0:0:0:40904 > Dec 23, 2016 7:17:44 AM com.datatorrent.stram.engine.StreamingContainer main > INFO: Exit status for container: 1 -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (BEAM-1213) WordCount example failure on Apex Runner
[ https://issues.apache.org/jira/browse/BEAM-1213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15997044#comment-15997044 ] Davor Bonaci commented on BEAM-1213: [~kenn], any updates perhaps? > WordCount example failure on Apex Runner > > > Key: BEAM-1213 > URL: https://issues.apache.org/jira/browse/BEAM-1213 > Project: Beam > Issue Type: Bug > Components: runner-apex >Reporter: Sandeep Deshmukh >Assignee: Kenneth Knowles > Fix For: First stable release > > > Tried running WordCount example using apex runner (on YARN) with > https://github.com/apache/incubator-beam/pull/1693 , fix for httpclient > version issue fixed. > Create /tmp/input-beam/ dir on all nodes and pushed a test data in that > directory. Ran the following command in examples/java/ > ~/bin/apache-maven-3.3.9/bin/mvn compile exec:java > -Dexec.mainClass=org.apache.beam.examples.WordCount > -Dexec.args="--inputFile=/tmp/input-beam/java.java --output=/tmp/output-beam/ > --runner=ApexRunner --embeddedExecution=false" -Papex-runner > INFO: using > /disk2/devhdfs/nn_20161013/usercache/sandeep/appcache/application_1482319446115_4336/container_1482319446115_4336_01_14/tmp/chkp5198889890309038810 > as the basepath for checkpointing. > Dec 23, 2016 7:17:44 AM com.datatorrent.stram.engine.StreamingContainer main > SEVERE: Fatal exception in container! > java.lang.IllegalArgumentException: Invalid slice: offset=1426, length=0 > array.length=67108864 > at com.datatorrent.netlet.util.Slice.(Slice.java:43) > at > com.datatorrent.bufferserver.util.SerializedData.(SerializedData.java:40) > at > com.datatorrent.bufferserver.internal.DataList$Block.purge(DataList.java:712) > at > com.datatorrent.bufferserver.internal.DataList.purge(DataList.java:192) > at com.datatorrent.bufferserver.server.Server.purge(Server.java:204) > at > com.datatorrent.stram.engine.StreamingContainer.processHeartbeatResponse(StreamingContainer.java:776) > at > com.datatorrent.stram.engine.StreamingContainer.heartbeatLoop(StreamingContainer.java:705) > at > com.datatorrent.stram.engine.StreamingContainer.main(StreamingContainer.java:310) > Dec 23, 2016 7:17:44 AM com.datatorrent.bufferserver.server.Server > unregistered > INFO: Server stopped listening at /0:0:0:0:0:0:0:0:40904 > Dec 23, 2016 7:17:44 AM com.datatorrent.stram.engine.StreamingContainer main > INFO: Exit status for container: 1 -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (BEAM-1213) WordCount example failure on Apex Runner
[ https://issues.apache.org/jira/browse/BEAM-1213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15985653#comment-15985653 ] Kenneth Knowles commented on BEAM-1213: --- Seems ripe for a re-triage before stable release. > WordCount example failure on Apex Runner > > > Key: BEAM-1213 > URL: https://issues.apache.org/jira/browse/BEAM-1213 > Project: Beam > Issue Type: Bug > Components: runner-apex >Reporter: Sandeep Deshmukh > Fix For: First stable release > > > Tried running WordCount example using apex runner (on YARN) with > https://github.com/apache/incubator-beam/pull/1693 , fix for httpclient > version issue fixed. > Create /tmp/input-beam/ dir on all nodes and pushed a test data in that > directory. Ran the following command in examples/java/ > ~/bin/apache-maven-3.3.9/bin/mvn compile exec:java > -Dexec.mainClass=org.apache.beam.examples.WordCount > -Dexec.args="--inputFile=/tmp/input-beam/java.java --output=/tmp/output-beam/ > --runner=ApexRunner --embeddedExecution=false" -Papex-runner > INFO: using > /disk2/devhdfs/nn_20161013/usercache/sandeep/appcache/application_1482319446115_4336/container_1482319446115_4336_01_14/tmp/chkp5198889890309038810 > as the basepath for checkpointing. > Dec 23, 2016 7:17:44 AM com.datatorrent.stram.engine.StreamingContainer main > SEVERE: Fatal exception in container! > java.lang.IllegalArgumentException: Invalid slice: offset=1426, length=0 > array.length=67108864 > at com.datatorrent.netlet.util.Slice.(Slice.java:43) > at > com.datatorrent.bufferserver.util.SerializedData.(SerializedData.java:40) > at > com.datatorrent.bufferserver.internal.DataList$Block.purge(DataList.java:712) > at > com.datatorrent.bufferserver.internal.DataList.purge(DataList.java:192) > at com.datatorrent.bufferserver.server.Server.purge(Server.java:204) > at > com.datatorrent.stram.engine.StreamingContainer.processHeartbeatResponse(StreamingContainer.java:776) > at > com.datatorrent.stram.engine.StreamingContainer.heartbeatLoop(StreamingContainer.java:705) > at > com.datatorrent.stram.engine.StreamingContainer.main(StreamingContainer.java:310) > Dec 23, 2016 7:17:44 AM com.datatorrent.bufferserver.server.Server > unregistered > INFO: Server stopped listening at /0:0:0:0:0:0:0:0:40904 > Dec 23, 2016 7:17:44 AM com.datatorrent.stram.engine.StreamingContainer main > INFO: Exit status for container: 1 -- This message was sent by Atlassian JIRA (v6.3.15#6346)
