[jira] [Updated] (BEAM-3799) Nexmark Query 10 breaks with direct runner

2018-03-09 Thread JIRA 

 [ 
https://issues.apache.org/jira/browse/BEAM-3799?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ismaël Mejía updated BEAM-3799:
---
Labels: nexmark  (was: )

> Nexmark Query 10 breaks with direct runner
> --
>
> Key: BEAM-3799
> URL: https://issues.apache.org/jira/browse/BEAM-3799
> Project: Beam
>  Issue Type: Bug
>  Components: runner-direct
>Affects Versions: 2.4.0, 2.5.0
>Reporter: Ismaël Mejía
>Assignee: Thomas Groh
>Priority: Major
>  Labels: nexmark
> Fix For: 2.4.0
>
>  Time Spent: 40m
>  Remaining Estimate: 0h
>
> While running query 10 with the direct runner like this:
> {quote}mvn exec:java -Dexec.mainClass=org.apache.beam.sdk.nexmark.Main 
> -Pdirect-runner -Dexec.args="--runner=DirectRunner --query=10 
> --streaming=false --manageResources=false --monitorJobs=true 
> --enforceEncodability=true --enforceImmutability=true" -pl 'sdks/java/nexmark'
> {quote}
> I found that it breaks with the direct runner with  following exception (it 
> works ok with the other runners):
> {quote}[WARNING] 
> java.lang.reflect.InvocationTargetException
>     at sun.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)
>     at sun.reflect.NativeMethodAccessorImpl.invoke 
> (NativeMethodAccessorImpl.java:62)
>     at sun.reflect.DelegatingMethodAccessorImpl.invoke 
> (DelegatingMethodAccessorImpl.java:43)
>     at java.lang.reflect.Method.invoke (Method.java:498)
>     at org.codehaus.mojo.exec.ExecJavaMojo$1.run (ExecJavaMojo.java:294)
>     at java.lang.Thread.run (Thread.java:748)
> Caused by: org.apache.beam.sdk.util.IllegalMutationException: PTransform 
> Query10/Query10.UploadEvents/ParMultiDo(Anonymous) mutated value KV{null, 
> 2015-07-15T00:00:09.999Z shard-3-of-00025 0 ON_TIME null
> } after it was output (new value was KV{null, 2015-07-15T00:00:09.999Z 
> shard-3-of-00025 0 ON_TIME null
> }). Values must not be mutated in any way after being output.
>     at 
> org.apache.beam.runners.direct.ImmutabilityCheckingBundleFactory$ImmutabilityEnforcingBundle.commit
>  (ImmutabilityCheckingBundleFactory.java:134)
>     at org.apache.beam.runners.direct.EvaluationContext.commitBundles 
> (EvaluationContext.java:212)
>     at org.apache.beam.runners.direct.EvaluationContext.handleResult 
> (EvaluationContext.java:152)
>     at 
> org.apache.beam.runners.direct.QuiescenceDriver$TimerIterableCompletionCallback.handleResult
>  (QuiescenceDriver.java:258)
>     at org.apache.beam.runners.direct.DirectTransformExecutor.finishBundle 
> (DirectTransformExecutor.java:190)
>     at org.apache.beam.runners.direct.DirectTransformExecutor.run 
> (DirectTransformExecutor.java:127)
>     at java.util.concurrent.Executors$RunnableAdapter.call 
> (Executors.java:511)
>     at java.util.concurrent.FutureTask.run (FutureTask.java:266)
>     at java.util.concurrent.ThreadPoolExecutor.runWorker 
> (ThreadPoolExecutor.java:1149)
>     at java.util.concurrent.ThreadPoolExecutor$Worker.run 
> (ThreadPoolExecutor.java:624)
>     at java.lang.Thread.run (Thread.java:748)
> Caused by: org.apache.beam.sdk.util.IllegalMutationException: Value KV{null, 
> 2015-07-15T00:00:09.999Z shard-3-of-00025 0 ON_TIME null
> } mutated illegally, new value was KV{null, 2015-07-15T00:00:09.999Z 
> shard-3-of-00025 0 ON_TIME null
> }. Encoding was 
> rO0ABXNyADZvcmcuYXBhY2hlLmJlYW0uc2RrLm5leG1hcmsucXVlcmllcy5RdWVyeTEwJE91dHB1dEZpbGUWUg9rZM1SvgIABUoABWluZGV4TAAIZmlsZW5hbWV0ABJMamF2YS9sYW5nL1N0cmluZztMAAxtYXhUaW1lc3RhbXB0ABdMb3JnL2pvZGEvdGltZS9JbnN0YW50O0wABXNoYXJkcQB-AAFMAAZ0aW1pbmd0ADpMb3JnL2FwYWNoZS9iZWFtL3Nkay90cmFuc2Zvcm1zL3dpbmRvd2luZy9QYW5lSW5mbyRUaW1pbmc7eHAAAHBzcgAVb3JnLmpvZGEudGltZS5JbnN0YW50Lci-0MYOnM0CAAFKAAdpTWlsbGlzeHFOjwLrD3QAFHNoYXJkLTAwMDAzLW9mLTAwMDI1fnIAOG9yZy5hcGFjaGUuYmVhbS5zZGsudHJhbnNmb3Jtcy53aW5kb3dpbmcuUGFuZUluZm8kVGltaW5nAAASAAB4cgAOamF2YS5sYW5nLkVudW0AABIAAHhwdAAHT05fVElNRQ,
>  now 
> rO0ABXNyADZvcmcuYXBhY2hlLmJlYW0uc2RrLm5leG1hcmsucXVlcmllcy5RdWVyeTEwJE91dHB1dEZpbGUWUg9rZM1SvgIABUoABWluZGV4TAAIZmlsZW5hbWV0ABJMamF2YS9sYW5nL1N0cmluZztMAAxtYXhUaW1lc3RhbXB0ABdMb3JnL2pvZGEvdGltZS9JbnN0YW50O0wABXNoYXJkcQB-AAFMAAZ0aW1pbmd0ADpMb3JnL2FwYWNoZS9iZWFtL3Nkay90cmFuc2Zvcm1zL3dpbmRvd2luZy9QYW5lSW5mbyRUaW1pbmc7eHAAAHBzcgAVb3JnLmpvZGEudGltZS5JbnN0YW50Lci-0MYOnM0CAAFKAAdpTWlsbGlzeHFOjwLrD3QAFHNoYXJkLTAwMDAzLW9mLTAwMDI1fnIAOG9yZy5hcGFjaGUuYmVhbS5zZGsudHJhbnNmb3Jtcy53aW5kb3dpbmcuUGFuZUluZm8kVGltaW5nAAASAAB4cgAOamF2YS5sYW5nLkVudW0AABIAAHhwdAAHT05fVElNRQ.
>     at 
> org.apache.beam.sdk.util.MutationDetectors$CodedValueMutationDetector.illegalMutation
>  (MutationDetectors.java:144)
>     at 
> org.apache.beam.sdk.util.MutationDetectors$CodedValueMutationDetector.verifyUnmodifiedThrowingCheckedExceptions
>  (MutationDetectors.java:139)
>     at 
> o

[jira] [Updated] (BEAM-3799) Nexmark Query 10 breaks with direct runner

2018-03-07 Thread JIRA 

 [ 
https://issues.apache.org/jira/browse/BEAM-3799?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ismaël Mejía updated BEAM-3799:
---
Fix Version/s: 2.4.0

> Nexmark Query 10 breaks with direct runner
> --
>
> Key: BEAM-3799
> URL: https://issues.apache.org/jira/browse/BEAM-3799
> Project: Beam
>  Issue Type: Bug
>  Components: runner-direct
>Affects Versions: 2.4.0, 2.5.0
>Reporter: Ismaël Mejía
>Assignee: Thomas Groh
>Priority: Major
> Fix For: 2.4.0
>
>
> While running query 10 with the direct runner like this:
> {quote}mvn exec:java -Dexec.mainClass=org.apache.beam.sdk.nexmark.Main 
> -Pdirect-runner -Dexec.args="--runner=DirectRunner --query=10 
> --streaming=false --manageResources=false --monitorJobs=true 
> --enforceEncodability=true --enforceImmutability=true" -pl 'sdks/java/nexmark'
> {quote}
> I found that it breaks with the direct runner with  following exception (it 
> works ok with the other runners):
> {quote}[WARNING] 
> java.lang.reflect.InvocationTargetException
>     at sun.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)
>     at sun.reflect.NativeMethodAccessorImpl.invoke 
> (NativeMethodAccessorImpl.java:62)
>     at sun.reflect.DelegatingMethodAccessorImpl.invoke 
> (DelegatingMethodAccessorImpl.java:43)
>     at java.lang.reflect.Method.invoke (Method.java:498)
>     at org.codehaus.mojo.exec.ExecJavaMojo$1.run (ExecJavaMojo.java:294)
>     at java.lang.Thread.run (Thread.java:748)
> Caused by: org.apache.beam.sdk.util.IllegalMutationException: PTransform 
> Query10/Query10.UploadEvents/ParMultiDo(Anonymous) mutated value KV{null, 
> 2015-07-15T00:00:09.999Z shard-3-of-00025 0 ON_TIME null
> } after it was output (new value was KV{null, 2015-07-15T00:00:09.999Z 
> shard-3-of-00025 0 ON_TIME null
> }). Values must not be mutated in any way after being output.
>     at 
> org.apache.beam.runners.direct.ImmutabilityCheckingBundleFactory$ImmutabilityEnforcingBundle.commit
>  (ImmutabilityCheckingBundleFactory.java:134)
>     at org.apache.beam.runners.direct.EvaluationContext.commitBundles 
> (EvaluationContext.java:212)
>     at org.apache.beam.runners.direct.EvaluationContext.handleResult 
> (EvaluationContext.java:152)
>     at 
> org.apache.beam.runners.direct.QuiescenceDriver$TimerIterableCompletionCallback.handleResult
>  (QuiescenceDriver.java:258)
>     at org.apache.beam.runners.direct.DirectTransformExecutor.finishBundle 
> (DirectTransformExecutor.java:190)
>     at org.apache.beam.runners.direct.DirectTransformExecutor.run 
> (DirectTransformExecutor.java:127)
>     at java.util.concurrent.Executors$RunnableAdapter.call 
> (Executors.java:511)
>     at java.util.concurrent.FutureTask.run (FutureTask.java:266)
>     at java.util.concurrent.ThreadPoolExecutor.runWorker 
> (ThreadPoolExecutor.java:1149)
>     at java.util.concurrent.ThreadPoolExecutor$Worker.run 
> (ThreadPoolExecutor.java:624)
>     at java.lang.Thread.run (Thread.java:748)
> Caused by: org.apache.beam.sdk.util.IllegalMutationException: Value KV{null, 
> 2015-07-15T00:00:09.999Z shard-3-of-00025 0 ON_TIME null
> } mutated illegally, new value was KV{null, 2015-07-15T00:00:09.999Z 
> shard-3-of-00025 0 ON_TIME null
> }. Encoding was 
> rO0ABXNyADZvcmcuYXBhY2hlLmJlYW0uc2RrLm5leG1hcmsucXVlcmllcy5RdWVyeTEwJE91dHB1dEZpbGUWUg9rZM1SvgIABUoABWluZGV4TAAIZmlsZW5hbWV0ABJMamF2YS9sYW5nL1N0cmluZztMAAxtYXhUaW1lc3RhbXB0ABdMb3JnL2pvZGEvdGltZS9JbnN0YW50O0wABXNoYXJkcQB-AAFMAAZ0aW1pbmd0ADpMb3JnL2FwYWNoZS9iZWFtL3Nkay90cmFuc2Zvcm1zL3dpbmRvd2luZy9QYW5lSW5mbyRUaW1pbmc7eHAAAHBzcgAVb3JnLmpvZGEudGltZS5JbnN0YW50Lci-0MYOnM0CAAFKAAdpTWlsbGlzeHFOjwLrD3QAFHNoYXJkLTAwMDAzLW9mLTAwMDI1fnIAOG9yZy5hcGFjaGUuYmVhbS5zZGsudHJhbnNmb3Jtcy53aW5kb3dpbmcuUGFuZUluZm8kVGltaW5nAAASAAB4cgAOamF2YS5sYW5nLkVudW0AABIAAHhwdAAHT05fVElNRQ,
>  now 
> rO0ABXNyADZvcmcuYXBhY2hlLmJlYW0uc2RrLm5leG1hcmsucXVlcmllcy5RdWVyeTEwJE91dHB1dEZpbGUWUg9rZM1SvgIABUoABWluZGV4TAAIZmlsZW5hbWV0ABJMamF2YS9sYW5nL1N0cmluZztMAAxtYXhUaW1lc3RhbXB0ABdMb3JnL2pvZGEvdGltZS9JbnN0YW50O0wABXNoYXJkcQB-AAFMAAZ0aW1pbmd0ADpMb3JnL2FwYWNoZS9iZWFtL3Nkay90cmFuc2Zvcm1zL3dpbmRvd2luZy9QYW5lSW5mbyRUaW1pbmc7eHAAAHBzcgAVb3JnLmpvZGEudGltZS5JbnN0YW50Lci-0MYOnM0CAAFKAAdpTWlsbGlzeHFOjwLrD3QAFHNoYXJkLTAwMDAzLW9mLTAwMDI1fnIAOG9yZy5hcGFjaGUuYmVhbS5zZGsudHJhbnNmb3Jtcy53aW5kb3dpbmcuUGFuZUluZm8kVGltaW5nAAASAAB4cgAOamF2YS5sYW5nLkVudW0AABIAAHhwdAAHT05fVElNRQ.
>     at 
> org.apache.beam.sdk.util.MutationDetectors$CodedValueMutationDetector.illegalMutation
>  (MutationDetectors.java:144)
>     at 
> org.apache.beam.sdk.util.MutationDetectors$CodedValueMutationDetector.verifyUnmodifiedThrowingCheckedExceptions
>  (MutationDetectors.java:139)
>     at 
> org.apache.beam.sdk.util.MutationDetectors$CodedValueMutationDetector.verifyUnmodified
>