[jira] [Assigned] (BEAM-826) Dynamically create subscriptions if unspecified in PubSubIO
[ https://issues.apache.org/jira/browse/BEAM-826?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Groh reassigned BEAM-826: Assignee: (was: Thomas Groh) > Dynamically create subscriptions if unspecified in PubSubIO > --- > > Key: BEAM-826 > URL: https://issues.apache.org/jira/browse/BEAM-826 > Project: Beam > Issue Type: Sub-task > Components: sdk-java-gcp >Reporter: Thomas Groh > Labels: backward-incompatible > > PubSubIO uses the options to create a random subscription when one is not > specified. Move this code to run as part of creating a reader or splitting > the source, so it does not use Pipeline-construction-time PipelineOptions. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (BEAM-847) Make DelegatingAggregator creation nonblocking
[
https://issues.apache.org/jira/browse/BEAM-847?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Thomas Groh resolved BEAM-847.
--
Resolution: Fixed
Fix Version/s: 0.4.0-incubating
> Make DelegatingAggregator creation nonblocking
> --
>
> Key: BEAM-847
> URL: https://issues.apache.org/jira/browse/BEAM-847
> Project: Beam
> Issue Type: Improvement
> Components: runner-core, runner-dataflow, runner-direct,
> runner-flink, sdk-java-core
>Reporter: Thomas Groh
>Assignee: Thomas Groh
> Fix For: 0.4.0-incubating
>
>
> This normally would not be a problem, as users are expected to create
> Aggregators in a non-performance-sensitive fashion.
> {{GroupAlsoByWindowViaWindowSet}} also creates an {{Aggregator}}, and this is
> a runner-provided {{DoFn}} (and may be created some arbitrary number of times
> on workers); this call should not use a shared resource.
> As the IDs of an {{Aggregator}} are used only to ensure that an
> {{Aggregator}} has a consistent identity across a serialization boundary,
> used in the {{AggregatorPipelineExtractor}}, these can be less random so long
> as they remain mostly unique.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (BEAM-310) Exercise splitIntoBundles/generateInitialSplits in the Direct Runner
[ https://issues.apache.org/jira/browse/BEAM-310?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Groh resolved BEAM-310. -- Resolution: Fixed Fix Version/s: 0.3.0-incubating > Exercise splitIntoBundles/generateInitialSplits in the Direct Runner > > > Key: BEAM-310 > URL: https://issues.apache.org/jira/browse/BEAM-310 > Project: Beam > Issue Type: Improvement > Components: runner-direct >Reporter: Thomas Groh >Assignee: Thomas Groh > Fix For: 0.3.0-incubating > > > BoundedSource#splitIntoBundles and UnboundedSource#generateInitialSplits are > the methods by which sources can be accessed in parallel. Exercising these > methods allows reads (and all transforms downstream) to be executed in > parallel both pre and post a GroupByKey -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (BEAM-885) Move PipelineOptions from Pipeline.create() to Pipeline.run()
Thomas Groh created BEAM-885: Summary: Move PipelineOptions from Pipeline.create() to Pipeline.run() Key: BEAM-885 URL: https://issues.apache.org/jira/browse/BEAM-885 Project: Beam Issue Type: New Feature Components: sdk-java-core Reporter: Thomas Groh Assignee: Thomas Groh The specification of a Pipeline should be independent of its PipelineOptions. This delays specification of the options, including choices like Pipeline Runner. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (BEAM-885) Move PipelineOptions from Pipeline.create() to Pipeline.run()
[ https://issues.apache.org/jira/browse/BEAM-885?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Groh updated BEAM-885: - Labels: backward-incompatible (was: ) > Move PipelineOptions from Pipeline.create() to Pipeline.run() > - > > Key: BEAM-885 > URL: https://issues.apache.org/jira/browse/BEAM-885 > Project: Beam > Issue Type: New Feature > Components: sdk-java-core >Reporter: Thomas Groh >Assignee: Thomas Groh > Labels: backward-incompatible > > The specification of a Pipeline should be independent of its PipelineOptions. > This delays specification of the options, including choices like Pipeline > Runner. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (BEAM-903) Flatten Timer Deliveries in WatermarkManager
Thomas Groh created BEAM-903:
Summary: Flatten Timer Deliveries in WatermarkManager
Key: BEAM-903
URL: https://issues.apache.org/jira/browse/BEAM-903
Project: Beam
Issue Type: Improvement
Components: runner-direct
Reporter: Thomas Groh
Assignee: Thomas Groh
Most of the data associated with timers is already a property of the Timer, or
functions regardless of segregation into time domains. Reduce the complexity of
the {{fireTimers}} method in the monitor runnable by returning a flat list of
(Transform, Key, Timers) instead of a triply nested map.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (BEAM-903) Flatten Timer Deliveries in WatermarkManager
[
https://issues.apache.org/jira/browse/BEAM-903?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Thomas Groh resolved BEAM-903.
--
Resolution: Fixed
Fix Version/s: 0.4.0-incubating
> Flatten Timer Deliveries in WatermarkManager
>
>
> Key: BEAM-903
> URL: https://issues.apache.org/jira/browse/BEAM-903
> Project: Beam
> Issue Type: Improvement
> Components: runner-direct
>Reporter: Thomas Groh
>Assignee: Thomas Groh
> Fix For: 0.4.0-incubating
>
>
> Most of the data associated with timers is already a property of the Timer,
> or functions regardless of segregation into time domains. Reduce the
> complexity of the {{fireTimers}} method in the monitor runnable by returning
> a flat list of (Transform, Key, Timers) instead of a triply nested map.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (BEAM-796) Watermarks should be updated after all other changes have been committed
[ https://issues.apache.org/jira/browse/BEAM-796?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Groh resolved BEAM-796. -- Resolution: Fixed Fix Version/s: 0.4.0-incubating > Watermarks should be updated after all other changes have been committed > > > Key: BEAM-796 > URL: https://issues.apache.org/jira/browse/BEAM-796 > Project: Beam > Issue Type: Bug > Components: runner-direct >Reporter: Thomas Groh >Assignee: Thomas Groh > Fix For: 0.4.0-incubating > > > Otherwise, a transform may be executed based on state changes that are not > visible to it yet. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (BEAM-937) The DirectRunner does not advance watermarks if reading from an empty reader
Thomas Groh created BEAM-937: Summary: The DirectRunner does not advance watermarks if reading from an empty reader Key: BEAM-937 URL: https://issues.apache.org/jira/browse/BEAM-937 Project: Beam Issue Type: Bug Components: runner-direct Reporter: Thomas Groh Assignee: Thomas Groh -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (BEAM-938) Break in direct.UnboundedReadEvaluatorFactoryTest.noElementsAvailableReaderIncludedInResidual
[ https://issues.apache.org/jira/browse/BEAM-938?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Groh resolved BEAM-938. -- Resolution: Fixed Fix Version/s: 0.4.0-incubating > Break in > direct.UnboundedReadEvaluatorFactoryTest.noElementsAvailableReaderIncludedInResidual > - > > Key: BEAM-938 > URL: https://issues.apache.org/jira/browse/BEAM-938 > Project: Beam > Issue Type: Bug >Reporter: Jason Kuster >Assignee: Thomas Groh > Fix For: 0.4.0-incubating > > > Likely culprit: > https://github.com/apache/incubator-beam/commit/ff7fe07be96de393b763e7b3d213734040aa3795 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (BEAM-938) Break in direct.UnboundedReadEvaluatorFactoryTest.noElementsAvailableReaderIncludedInResidual
[
https://issues.apache.org/jira/browse/BEAM-938?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15648883#comment-15648883
]
Thomas Groh commented on BEAM-938:
--
This was due to an incompletely updated test;
Specifically, the TestUnboundedSource was updated to provide a watermark that
always advanced, but {{noElementsAvailableReaderIncludedInResidual}} used an
{{UnboundedCountingSource}}
https://github.com/apache/incubator-beam/pull/1303 reverts the test, and
https://github.com/apache/incubator-beam/pull/1313 is the complete fix.
> Break in
> direct.UnboundedReadEvaluatorFactoryTest.noElementsAvailableReaderIncludedInResidual
> -
>
> Key: BEAM-938
> URL: https://issues.apache.org/jira/browse/BEAM-938
> Project: Beam
> Issue Type: Bug
>Reporter: Jason Kuster
>Assignee: Thomas Groh
> Fix For: 0.4.0-incubating
>
>
> Likely culprit:
> https://github.com/apache/incubator-beam/commit/ff7fe07be96de393b763e7b3d213734040aa3795
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (BEAM-937) The DirectRunner does not advance watermarks if reading from an empty reader
[ https://issues.apache.org/jira/browse/BEAM-937?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Groh resolved BEAM-937. -- Resolution: Fixed Fix Version/s: 0.4.0-incubating > The DirectRunner does not advance watermarks if reading from an empty reader > > > Key: BEAM-937 > URL: https://issues.apache.org/jira/browse/BEAM-937 > Project: Beam > Issue Type: Bug > Components: runner-direct >Reporter: Thomas Groh >Assignee: Thomas Groh > Fix For: 0.4.0-incubating > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (BEAM-826) Dynamically create subscriptions if unspecified in PubSubIO
[ https://issues.apache.org/jira/browse/BEAM-826?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Groh resolved BEAM-826. -- Resolution: Fixed Fix Version/s: 0.4.0-incubating > Dynamically create subscriptions if unspecified in PubSubIO > --- > > Key: BEAM-826 > URL: https://issues.apache.org/jira/browse/BEAM-826 > Project: Beam > Issue Type: Sub-task > Components: sdk-java-gcp >Reporter: Thomas Groh >Assignee: Thomas Groh > Labels: backward-incompatible > Fix For: 0.4.0-incubating > > > PubSubIO uses the options to create a random subscription when one is not > specified. Move this code to run as part of creating a reader or splitting > the source, so it does not use Pipeline-construction-time PipelineOptions. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (BEAM-950) DoFn Setup and Teardown methods should have access to PipelineOptions
Thomas Groh created BEAM-950:
Summary: DoFn Setup and Teardown methods should have access to
PipelineOptions
Key: BEAM-950
URL: https://issues.apache.org/jira/browse/BEAM-950
Project: Beam
Issue Type: Bug
Components: sdk-java-core
Reporter: Thomas Groh
Assignee: Davor Bonaci
This enables any options-relevant decisions to be made once per DoFn, without
having to lazily initialize in {{startBundle}}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Assigned] (BEAM-950) DoFn Setup and Teardown methods should have access to PipelineOptions
[
https://issues.apache.org/jira/browse/BEAM-950?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Thomas Groh reassigned BEAM-950:
Assignee: Thomas Groh (was: Davor Bonaci)
> DoFn Setup and Teardown methods should have access to PipelineOptions
> -
>
> Key: BEAM-950
> URL: https://issues.apache.org/jira/browse/BEAM-950
> Project: Beam
> Issue Type: Bug
> Components: sdk-java-core
>Reporter: Thomas Groh
>Assignee: Thomas Groh
>
> This enables any options-relevant decisions to be made once per DoFn, without
> having to lazily initialize in {{startBundle}}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (BEAM-924) Findbugs doesn't pass in Direct Runner
[
https://issues.apache.org/jira/browse/BEAM-924?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Thomas Groh resolved BEAM-924.
--
Resolution: Fixed
Fix Version/s: 0.4.0-incubating
> Findbugs doesn't pass in Direct Runner
> --
>
> Key: BEAM-924
> URL: https://issues.apache.org/jira/browse/BEAM-924
> Project: Beam
> Issue Type: Bug
> Components: runner-direct
>Reporter: Daniel Halperin
>Assignee: Thomas Groh
> Fix For: 0.4.0-incubating
>
>
> {code}
> [INFO] AggregatorContainer$AggregatorInfo.accumulator not guarded against
> concurrent access; locked 71% of time
> [org.apache.beam.runners.direct.AggregatorContainer$AggregatorInfo,
> org.apache.beam.runners.direct.AggregatorContainer$AggregatorInfo,
> org.apache.beam.runners.direct.AggregatorContainer$AggregatorInfo,
> org.apache.beam.runners.direct.AggregatorContainer$AggregatorInfo,
> org.apache.beam.runners.direct.AggregatorContainer$AggregatorInfo,
> org.apache.beam.runners.direct.AggregatorContainer$AggregatorInfo,
> org.apache.beam.runners.direct.AggregatorContainer$AggregatorInfo,
> org.apache.beam.runners.direct.AggregatorContainer$AggregatorInfo,
> org.apache.beam.runners.direct.AggregatorContainer$AggregatorInfo,
> org.apache.beam.runners.direct.AggregatorContainer$AggregatorInfo,
> org.apache.beam.runners.direct.AggregatorContainer$AggregatorInfo,
> org.apache.beam.runners.direct.AggregatorContainer$AggregatorInfo,
> org.apache.beam.runners.direct.AggregatorContainer$AggregatorInfo]
> Unsynchronized access at AggregatorContainer.java:[line 80]Unsynchronized
> access at AggregatorContainer.java:[line 79]Unsynchronized access at
> AggregatorContainer.java:[line 81]Synchronized access at
> AggregatorContainer.java:[line 78]Synchronized access at
> AggregatorContainer.java:[line 79]Synchronized access at
> AggregatorContainer.java:[line 81]Synchronized access at
> AggregatorContainer.java:[line 81]Synchronized access at
> AggregatorContainer.java:[line 59]Synchronized access at
> AggregatorContainer.java:[line 62]Synchronized access at
> AggregatorContainer.java:[line 62]Synchronized access at
> AggregatorContainer.java:[line 60]Synchronized access at
> AggregatorContainer.java:[line 66]Synchronized access at
> AggregatorContainer.java:[line 66]
> [INFO] bundle must be non-null but is marked as nullable
> [org.apache.beam.runners.direct.ExecutorServiceParallelExecutor] At
> ExecutorServiceParallelExecutor.java:[lines 185-186]
> [INFO] Exceptional return value of
> java.util.concurrent.BlockingQueue.offer(Object) ignored in
> org.apache.beam.runners.direct.ExecutorServiceParallelExecutor$MonitorRunnable.run()
>
> [org.apache.beam.runners.direct.ExecutorServiceParallelExecutor$MonitorRunnable]
> At ExecutorServiceParallelExecutor.java:[line 412]
> [INFO]
> org.apache.beam.runners.direct.WatermarkCallbackExecutor$CallbackOrdering
> implements Comparator but not Serializable
> [org.apache.beam.runners.direct.WatermarkCallbackExecutor$CallbackOrdering]
> At WatermarkCallbackExecutor.java:[lines 132-138]
> [INFO]
> org.apache.beam.runners.direct.WatermarkManager$WindowedValueByTimestampComparator
> implements Comparator but not Serializable
> [org.apache.beam.runners.direct.WatermarkManager$WindowedValueByTimestampComparator]
> At WatermarkManager.java:[lines 1380-1385]
> [INFO]
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (BEAM-957) Flake in FlattenTest under DirectRunner
[ https://issues.apache.org/jira/browse/BEAM-957?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Groh resolved BEAM-957. -- Resolution: Fixed Fix Version/s: 0.4.0-incubating > Flake in FlattenTest under DirectRunner > --- > > Key: BEAM-957 > URL: https://issues.apache.org/jira/browse/BEAM-957 > Project: Beam > Issue Type: Bug > Components: runner-direct >Reporter: Kenneth Knowles >Assignee: Thomas Groh > Fix For: 0.4.0-incubating > > > Example occurrence: > https://builds.apache.org/job/beam_PreCommit_MavenVerify/4843/consoleFull -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (BEAM-839) The DirectRunner slows down significantly as the number of keys increases
[
https://issues.apache.org/jira/browse/BEAM-839?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Thomas Groh resolved BEAM-839.
--
Resolution: Fixed
Fix Version/s: 0.4.0-incubating
This was linear time (with regards to the number of keys) update in
{{WatermarkManager}}. It has been replaced with an O(log n) update.
> The DirectRunner slows down significantly as the number of keys increases
> -
>
> Key: BEAM-839
> URL: https://issues.apache.org/jira/browse/BEAM-839
> Project: Beam
> Issue Type: Bug
>Reporter: Thomas Groh
>Assignee: Thomas Groh
> Fix For: 0.4.0-incubating
>
>
> For example, running WordCount on KingLear takes approximately 10 seconds,
> while running WordCount on all of Shakespeare takes approximately 5 minutes.
> The primary cost is maintaining a PriorityQueue of Watermark Holds, which
> takes {{O(n**2)}} time, where {{n}} is the number of keys at a step.
> Additionally, there are two other things that cause slowness. The first is
> use of UUID.randomUUID in the constructor of DelegatingAggregator, which uses
> a shared SecureRandom, which synchronizes on call to {{nextBytes(byte[])}}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (BEAM-833) Exercise Dynamic Splitting in the DirectRunner
[
https://issues.apache.org/jira/browse/BEAM-833?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Thomas Groh resolved BEAM-833.
--
Resolution: Fixed
Fix Version/s: 0.4.0-incubating
> Exercise Dynamic Splitting in the DirectRunner
> --
>
> Key: BEAM-833
> URL: https://issues.apache.org/jira/browse/BEAM-833
> Project: Beam
> Issue Type: Improvement
> Components: runner-direct
>Reporter: Thomas Groh
>Assignee: Thomas Groh
> Fix For: 0.4.0-incubating
>
>
> This primarily means calling the {{BoundedSource#splitAtFraction}} API during
> the evaluation of a BoundedReadEvaluatorFactory - this can return the work
> via addUnprocessedElements.
> At a later point, the {{MonitorRunnable}} within
> {{ExecutorServiceParallelExecutor}} can examine currently evaluating work and
> invoke a {{TransformExecutor#split}} method, which can return to the runner
> some amount of input, which can be scheduled to use more of the available
> threads.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (BEAM-999) DirectRunner flake related to splitting
[
https://issues.apache.org/jira/browse/BEAM-999?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Thomas Groh resolved BEAM-999.
--
Resolution: Fixed
> DirectRunner flake related to splitting
> ---
>
> Key: BEAM-999
> URL: https://issues.apache.org/jira/browse/BEAM-999
> Project: Beam
> Issue Type: Bug
> Components: runner-direct
>Affects Versions: Not applicable
>Reporter: Daniel Halperin
>Assignee: Thomas Groh
> Fix For: Not applicable
>
>
> https://builds.apache.org/job/beam_PostCommit_MavenVerify/1838/
> {code}
> Error Message
> Expected: a value greater than <1>
> but: <1> was equal to <1>
> Stacktrace
> java.lang.AssertionError:
> Expected: a value greater than <1>
> but: <1> was equal to <1>
> at org.hamcrest.MatcherAssert.assertThat(MatcherAssert.java:20)
> at org.junit.Assert.assertThat(Assert.java:865)
> at org.junit.Assert.assertThat(Assert.java:832)
> at
> org.apache.beam.runners.direct.BoundedReadEvaluatorFactoryTest.boundedSourceEvaluatorProducesDynamicSplits(BoundedReadEvaluatorFactoryTest.java:183)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50)
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63)
> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236)
> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53)
> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229)
> at org.junit.runners.ParentRunner.run(ParentRunner.java:309)
> at org.junit.runners.Suite.runChild(Suite.java:127)
> at org.junit.runners.Suite.runChild(Suite.java:26)
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63)
> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236)
> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53)
> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229)
> at org.junit.runners.ParentRunner.run(ParentRunner.java:309)
> at org.apache.maven.surefire.junitcore.JUnitCore.run(JUnitCore.java:55)
> at
> org.apache.maven.surefire.junitcore.JUnitCoreWrapper.createRequestAndRun(JUnitCoreWrapper.java:137)
> at
> org.apache.maven.surefire.junitcore.JUnitCoreWrapper.executeEager(JUnitCoreWrapper.java:107)
> at
> org.apache.maven.surefire.junitcore.JUnitCoreWrapper.execute(JUnitCoreWrapper.java:83)
> at
> org.apache.maven.surefire.junitcore.JUnitCoreWrapper.execute(JUnitCoreWrapper.java:75)
> at
> org.apache.maven.surefire.junitcore.JUnitCoreProvider.invoke(JUnitCoreProvider.java:161)
> at
> org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:290)
> at
> org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:242)
> at
> org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:121)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (BEAM-1056) Reference beam version only once in Archetypes
Thomas Groh created BEAM-1056: - Summary: Reference beam version only once in Archetypes Key: BEAM-1056 URL: https://issues.apache.org/jira/browse/BEAM-1056 Project: Beam Issue Type: Improvement Components: examples-java Reporter: Thomas Groh Priority: Trivial Instead of referring to the version of the SDK inline, reference it once in a beam.version property and use that to determine sdk and runner versions. This reduces maintenance burden for updating archetypes. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (BEAM-1063) WordCount on DirectRunner is hanging for ~1 minute after job runs
[
https://issues.apache.org/jira/browse/BEAM-1063?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Thomas Groh resolved BEAM-1063.
---
Resolution: Fixed
Fix Version/s: 0.4.0-incubating
> WordCount on DirectRunner is hanging for ~1 minute after job runs
> -
>
> Key: BEAM-1063
> URL: https://issues.apache.org/jira/browse/BEAM-1063
> Project: Beam
> Issue Type: Bug
> Components: runner-direct
>Affects Versions: Not applicable
>Reporter: Daniel Halperin
>Assignee: Thomas Groh
> Fix For: 0.4.0-incubating
>
>
> generate example archetype
> {code}
> mvn compile exec:java -Dexec.mainClass=org.apache.beam.examples.WordCount \
>
> -Dexec.args="--inputFile=pom.xml --output=direct/counts"
> {code}
> In the demo I gave 2 weeks ago (sorry, not sure what commit this was at) this
> finished quickly.
> In the demo Ben gave yesterday, it hung. But only in the Direct runner. Some
> regression in that time?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (BEAM-1123) Migrate AppliedPTransform to use AutoValue
Thomas Groh created BEAM-1123: - Summary: Migrate AppliedPTransform to use AutoValue Key: BEAM-1123 URL: https://issues.apache.org/jira/browse/BEAM-1123 Project: Beam Issue Type: Improvement Components: sdk-java-core Reporter: Thomas Groh Assignee: Thomas Groh Priority: Minor -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (BEAM-1123) Migrate AppliedPTransform to use AutoValue
[ https://issues.apache.org/jira/browse/BEAM-1123?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Groh resolved BEAM-1123. --- Resolution: Fixed Fix Version/s: Not applicable > Migrate AppliedPTransform to use AutoValue > -- > > Key: BEAM-1123 > URL: https://issues.apache.org/jira/browse/BEAM-1123 > Project: Beam > Issue Type: Improvement > Components: sdk-java-core >Reporter: Thomas Groh >Assignee: Thomas Groh >Priority: Minor > Fix For: Not applicable > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (BEAM-1199) Condense recordAsOutput, finishSpecifyingOutput from POutput
Thomas Groh created BEAM-1199:
-
Summary: Condense recordAsOutput, finishSpecifyingOutput from
POutput
Key: BEAM-1199
URL: https://issues.apache.org/jira/browse/BEAM-1199
Project: Beam
Issue Type: Improvement
Components: sdk-java-core
Reporter: Thomas Groh
Assignee: Davor Bonaci
Priority: Minor
{{recordAsOutput}} and {{finishSpecifyingOutput}} are both methods which are
called after an output has been attached to a PTransform application. They can
be combined to only have one method that does any after-production work (such
as the initial run of Coder inference)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (BEAM-1202) Coders should have meaningful equals methods
[
https://issues.apache.org/jira/browse/BEAM-1202?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Thomas Groh updated BEAM-1202:
--
Assignee: (was: Davor Bonaci)
> Coders should have meaningful equals methods
>
>
> Key: BEAM-1202
> URL: https://issues.apache.org/jira/browse/BEAM-1202
> Project: Beam
> Issue Type: Bug
> Components: sdk-java-core
>Reporter: Thomas Groh
>
> {{StandardCoder}} implements an equality check based on the component coders
> and equal classes. Any coder that is configured, or that does not extend
> {{StandardCoder}}, should have meaningful implementations of {{equals}} and
> {{hashCode}}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (BEAM-1202) Coders should have meaningful equals methods
Thomas Groh created BEAM-1202:
-
Summary: Coders should have meaningful equals methods
Key: BEAM-1202
URL: https://issues.apache.org/jira/browse/BEAM-1202
Project: Beam
Issue Type: Bug
Components: sdk-java-core
Reporter: Thomas Groh
Assignee: Davor Bonaci
{{StandardCoder}} implements an equality check based on the component coders
and equal classes. Any coder that is configured, or that does not extend
{{StandardCoder}}, should have meaningful implementations of {{equals}} and
{{hashCode}}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
