[jira] [Commented] (BEAM-4295) Need ValueProvider that executes exactly once at pipeline runtime.

2018-06-29 Thread Thomas Groh (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-4295?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16528190#comment-16528190 ] Thomas Groh commented on BEAM-4295: --- ValueProvider isn't really designed for this usage,

[jira] [Assigned] (BEAM-4295) Need ValueProvider that executes exactly once at pipeline runtime.

2018-06-29 Thread Thomas Groh (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-4295?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Groh reassigned BEAM-4295: - Assignee: (was: Thomas Groh) > Need ValueProvider that executes exactly once at pipeline runt

[jira] [Assigned] (BEAM-2886) Portable pipeline submission proxy

2018-06-29 Thread Thomas Groh (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2886?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Groh reassigned BEAM-2886: - Assignee: Axel Magnuson (was: Thomas Groh) > Portable pipeline submission proxy > --

[jira] [Assigned] (BEAM-4380) Impossible to submit Google Dataflow job from Windows machine

2018-06-29 Thread Thomas Groh (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-4380?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Groh reassigned BEAM-4380: - Assignee: (was: Thomas Groh) > Impossible to submit Google Dataflow job from Windows machine

[jira] [Assigned] (BEAM-2470) Inconsistent behavior on the functioning of the dataflow templates?

2018-06-29 Thread Thomas Groh (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2470?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Groh reassigned BEAM-2470: - Assignee: (was: Thomas Groh) > Inconsistent behavior on the functioning of the dataflow templ

[jira] [Assigned] (BEAM-2885) Support job+artifact APIs locally

2018-06-29 Thread Thomas Groh (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2885?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Groh reassigned BEAM-2885: - Assignee: Luke Cwik (was: Thomas Groh) > Support job+artifact APIs locally > ---

[jira] [Commented] (BEAM-2885) Support job+artifact APIs locally

2018-06-29 Thread Thomas Groh (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2885?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16528185#comment-16528185 ] Thomas Groh commented on BEAM-2885: --- Luke for re-triage.   I think the missing compone

[jira] [Resolved] (BEAM-2899) Universal Local Runner

2018-06-29 Thread Thomas Groh (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2899?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Groh resolved BEAM-2899. --- Resolution: Fixed Fix Version/s: 2.6.0 I believe the initial java-based implementation to be ef

[jira] [Commented] (BEAM-2886) Portable pipeline submission proxy

2018-06-29 Thread Thomas Groh (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2886?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16528183#comment-16528183 ] Thomas Groh commented on BEAM-2886: --- I think this is done; Axel, do you know any more?

[jira] [Assigned] (BEAM-2886) Portable pipeline submission proxy

2018-06-29 Thread Thomas Groh (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2886?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Groh reassigned BEAM-2886: - Assignee: Thomas Groh (was: Axel Magnuson) > Portable pipeline submission proxy > --

[jira] [Assigned] (BEAM-2886) Portable pipeline submission proxy

2018-06-29 Thread Thomas Groh (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2886?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Groh reassigned BEAM-2886: - Assignee: Axel Magnuson (was: Thomas Groh) > Portable pipeline submission proxy > --

[jira] [Assigned] (BEAM-3334) NullPointerException in Direct Runner

2018-06-29 Thread Thomas Groh (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-3334?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Groh reassigned BEAM-3334: - Assignee: (was: Thomas Groh) > NullPointerException in Direct Runner > --

[jira] [Assigned] (BEAM-3017) DropInputs on BigQueryIO.writeTableRows() sink

2018-06-29 Thread Thomas Groh (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-3017?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Groh reassigned BEAM-3017: - Assignee: (was: Thomas Groh) > DropInputs on BigQueryIO.writeTableRows() sink > -

[jira] [Assigned] (BEAM-3487) GroupByKey stalls with GroupingShuffleReader split refusals

2018-06-29 Thread Thomas Groh (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-3487?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Groh reassigned BEAM-3487: - Assignee: (was: Thomas Groh) > GroupByKey stalls with GroupingShuffleReader split refusals >

[jira] [Assigned] (BEAM-2715) Expose PubsubSource to create UnboundedSource and utilize withMaxNumRecords from BoundedReadFromUnboundedSource

2018-06-29 Thread Thomas Groh (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2715?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Groh reassigned BEAM-2715: - Assignee: (was: Thomas Groh) > Expose PubsubSource to create UnboundedSource and utilize with

[jira] [Assigned] (BEAM-3495) core-construction-java should not depend on testing

2018-06-29 Thread Thomas Groh (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-3495?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Groh reassigned BEAM-3495: - Assignee: (was: Thomas Groh) > core-construction-java should not depend on testing >

[jira] [Assigned] (BEAM-3372) Duplicated 'zone' PipelineOption has inconsistent documentation

2018-06-29 Thread Thomas Groh (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-3372?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Groh reassigned BEAM-3372: - Assignee: (was: Thomas Groh) > Duplicated 'zone' PipelineOption has inconsistent documentatio

[jira] [Assigned] (BEAM-3652) Port WriteWithShardingFactoryTest off DoFnTester

2018-06-29 Thread Thomas Groh (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-3652?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Groh reassigned BEAM-3652: - Assignee: (was: Thomas Groh) > Port WriteWithShardingFactoryTest off DoFnTester > ---

[jira] [Assigned] (BEAM-3529) Side Input callbacks should fire after a window has expired in addition to when the trigger would fire

2018-06-29 Thread Thomas Groh (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-3529?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Groh reassigned BEAM-3529: - Assignee: (was: Thomas Groh) > Side Input callbacks should fire after a window has expired in

[jira] [Commented] (BEAM-3636) DirectRunner proto roundtrip masks stable name errors

2018-06-29 Thread Thomas Groh (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-3636?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16528179#comment-16528179 ] Thomas Groh commented on BEAM-3636: --- Generally that shouldn't be the case - the java {{P

[jira] [Assigned] (BEAM-2699) AppliedPTransform is used as a key in hashmaps but PTransform is not hashable/equality-comparable

2018-06-29 Thread Thomas Groh (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2699?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Groh reassigned BEAM-2699: - Assignee: (was: Thomas Groh) > AppliedPTransform is used as a key in hashmaps but PTransform

[jira] [Assigned] (BEAM-849) Redesign PipelineResult API

2018-06-29 Thread Thomas Groh (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-849?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Groh reassigned BEAM-849: Assignee: (was: Thomas Groh) > Redesign PipelineResult API > --- > >

[jira] [Assigned] (BEAM-3245) Dataflow runner does not respect ParDo's lifecycle on case of exceptions

2018-06-29 Thread Thomas Groh (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-3245?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Groh reassigned BEAM-3245: - Assignee: Batkhuyag Batsaikhan (was: Thomas Groh) > Dataflow runner does not respect ParDo's lif

[jira] [Assigned] (BEAM-3636) DirectRunner proto roundtrip masks stable name errors

2018-06-29 Thread Thomas Groh (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-3636?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Groh reassigned BEAM-3636: - Assignee: Batkhuyag Batsaikhan (was: Thomas Groh) > DirectRunner proto roundtrip masks stable na

[jira] [Resolved] (BEAM-3974) Rebuild the Dataflow worker to use new MetricQueryResults API

2018-06-29 Thread Thomas Groh (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-3974?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Groh resolved BEAM-3974. --- Resolution: Fixed Fix Version/s: Not applicable Parent is resolved   > Rebuild the Dataflow w

[jira] [Assigned] (BEAM-3877) Performance tests flaky due to NoClassDefFoundError

2018-06-29 Thread Thomas Groh (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-3877?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Groh reassigned BEAM-3877: - Assignee: (was: Thomas Groh) > Performance tests flaky due to NoClassDefFoundError >

[jira] [Assigned] (BEAM-3957) Support MetricsPusher in Direct Runner

2018-06-29 Thread Thomas Groh (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-3957?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Groh reassigned BEAM-3957: - Assignee: (was: Thomas Groh) > Support MetricsPusher in Direct Runner > -

[jira] [Assigned] (BEAM-3996) Invalid test util ResourceIdTester#ValidateFailureResolvingIds

2018-06-29 Thread Thomas Groh (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-3996?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Groh reassigned BEAM-3996: - Assignee: Batkhuyag Batsaikhan (was: Thomas Groh) > Invalid test util ResourceIdTester#ValidateF

[jira] [Assigned] (BEAM-4039) ExecutorServiceParallelExecutor should have a unified shutdown collection

2018-06-29 Thread Thomas Groh (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-4039?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Groh reassigned BEAM-4039: - Assignee: (was: Thomas Groh) > ExecutorServiceParallelExecutor should have a unified shutdown

[jira] [Assigned] (BEAM-4275) Pubsub: add DirectRunner support for id_label and timestamp_attribute in Python SDK

2018-06-29 Thread Thomas Groh (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-4275?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Groh reassigned BEAM-4275: - Assignee: Ahmet Altay (was: Thomas Groh) > Pubsub: add DirectRunner support for id_label and tim

[jira] [Assigned] (BEAM-3808) Dataflow Jobs meet Cloud Label Spec

2018-06-29 Thread Thomas Groh (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-3808?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Groh reassigned BEAM-3808: - Assignee: Ahmet Altay (was: Thomas Groh) > Dataflow Jobs meet Cloud Label Spec > ---

[jira] [Assigned] (BEAM-4247) Set region when BigqueryIO.write using DirectRunner

2018-06-29 Thread Thomas Groh (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-4247?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Groh reassigned BEAM-4247: - Assignee: (was: Thomas Groh) > Set region when BigqueryIO.write using DirectRunner >

[jira] [Assigned] (BEAM-3968) Fix flaky WordCount integration tests on dataflow runner

2018-06-29 Thread Thomas Groh (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-3968?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Groh reassigned BEAM-3968: - Assignee: (was: Thomas Groh) > Fix flaky WordCount integration tests on dataflow runner > ---

[jira] [Assigned] (BEAM-4016) Direct runner incorrect lifecycle, @SplitRestriction should execute after @Setup on SplittableDoFn

2018-06-29 Thread Thomas Groh (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-4016?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Groh reassigned BEAM-4016: - Assignee: (was: Thomas Groh) > Direct runner incorrect lifecycle, @SplitRestriction should ex

[jira] [Assigned] (BEAM-4301) NullPointerException with ValueState of custom class

2018-06-29 Thread Thomas Groh (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-4301?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Groh reassigned BEAM-4301: - Assignee: Batkhuyag Batsaikhan (was: Thomas Groh) > NullPointerException with ValueState of cust

[jira] [Assigned] (BEAM-4378) Update javadoc for set/getRegion in DataflowPipelineOptions.

2018-06-29 Thread Thomas Groh (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-4378?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Groh reassigned BEAM-4378: - Assignee: Batkhuyag Batsaikhan (was: Thomas Groh) > Update javadoc for set/getRegion in Dataflow

[jira] [Assigned] (BEAM-4563) Fix FutureReturnValueIgnored violations in direct runner

2018-06-29 Thread Thomas Groh (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-4563?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Groh reassigned BEAM-4563: - Assignee: (was: Thomas Groh) > Fix FutureReturnValueIgnored violations in direct runner > ---

[jira] [Assigned] (BEAM-4578) Cleanup paths due to obsolete worker/sdk upgrades.

2018-06-29 Thread Thomas Groh (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-4578?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Groh reassigned BEAM-4578: - Assignee: (was: Thomas Groh) > Cleanup paths due to obsolete worker/sdk upgrades. > -

[jira] [Assigned] (BEAM-4191) WatermarkManagerTest has had a masked failure (both copies)

2018-06-29 Thread Thomas Groh (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-4191?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Groh reassigned BEAM-4191: - Assignee: (was: Thomas Groh) > WatermarkManagerTest has had a masked failure (both copies) >

[jira] [Assigned] (BEAM-4232) ValueProvider with default value fails using DirectRunner

2018-06-29 Thread Thomas Groh (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-4232?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Groh reassigned BEAM-4232: - Assignee: Batkhuyag Batsaikhan (was: Thomas Groh) > ValueProvider with default value fails using

[jira] [Commented] (BEAM-4243) Consolidate Python Label Flag Handling

2018-06-29 Thread Thomas Groh (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-4243?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16528061#comment-16528061 ] Thomas Groh commented on BEAM-4243: --- ahmet for triage > Consolidate Python Label Flag H

[jira] [Resolved] (BEAM-4240) Implement the limited set of portable primitives in the Java DirectRunner

2018-06-29 Thread Thomas Groh (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-4240?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Groh resolved BEAM-4240. --- Resolution: Fixed Fix Version/s: Not applicable > Implement the limited set of portable primiti

[jira] [Resolved] (BEAM-4259) Instantiate a Pipeline (EvaluationContext, Executor, etc) via Proto format (only) in the DirectRunner

2018-06-29 Thread Thomas Groh (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-4259?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Groh resolved BEAM-4259. --- Resolution: Fixed Fix Version/s: Not applicable > Instantiate a Pipeline (EvaluationContext, Ex

[jira] [Assigned] (BEAM-4243) Consolidate Python Label Flag Handling

2018-06-29 Thread Thomas Groh (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-4243?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Groh reassigned BEAM-4243: - Assignee: Ahmet Altay (was: Thomas Groh) > Consolidate Python Label Flag Handling >

[jira] [Resolved] (BEAM-4258) Integrate Docker Environment Management in the ReferenceRunner

2018-06-29 Thread Thomas Groh (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-4258?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Groh resolved BEAM-4258. --- Resolution: Fixed Fix Version/s: 2.6.0 > Integrate Docker Environment Management in the Referen

[jira] [Resolved] (BEAM-3326) Execute a Stage via the portability framework in the ReferenceRunner

2018-06-29 Thread Thomas Groh (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-3326?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Groh resolved BEAM-3326. --- Resolution: Fixed Fix Version/s: 2.6.0 > Execute a Stage via the portability framework in the R

[jira] [Assigned] (BEAM-4440) When filesToStage is empty, the DataflowRunner should fail.

2018-06-01 Thread Thomas Groh (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-4440?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Groh reassigned BEAM-4440: - Assignee: Batkhuyag Batsaikhan (was: Thomas Groh) > When filesToStage is empty, the DataflowRunn

[jira] [Resolved] (BEAM-4404) Fix docker run arguments

2018-06-01 Thread Thomas Groh (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-4404?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Groh resolved BEAM-4404. --- Resolution: Fixed Fix Version/s: 2.5.0 > Fix docker run arguments > >

[jira] [Updated] (BEAM-4404) Fix docker run arguments

2018-06-01 Thread Thomas Groh (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-4404?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Groh updated BEAM-4404: -- Fix Version/s: (was: 2.5.0) Not applicable > Fix docker run arguments > -

[jira] [Resolved] (BEAM-3329) Add an in-process 'container' manager for testing in the Universal Local Runner

2018-06-01 Thread Thomas Groh (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-3329?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Groh resolved BEAM-3329. --- Resolution: Fixed Assignee: Thomas Groh Fix Version/s: Not applicable > Add an in-proc

[jira] [Resolved] (BEAM-3328) Create and relate portability service servers in the Universal Local Runner

2018-06-01 Thread Thomas Groh (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-3328?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Groh resolved BEAM-3328. --- Resolution: Fixed Fix Version/s: Not applicable > Create and relate portability service servers

[jira] [Resolved] (BEAM-4135) Remove Use of Java SDK Types in the DirectRunner "engine"

2018-06-01 Thread Thomas Groh (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-4135?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Groh resolved BEAM-4135. --- Resolution: Fixed Fix Version/s: Not applicable > Remove Use of Java SDK Types in the DirectRun

[jira] [Resolved] (BEAM-4073) The DirectRunner should interact with a Pipeline via an abstraction of the Graph rather than SDK types

2018-06-01 Thread Thomas Groh (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-4073?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Groh resolved BEAM-4073. --- Resolution: Fixed Fix Version/s: Not applicable > The DirectRunner should interact with a Pipel

[jira] [Resolved] (BEAM-4090) Fork/Update Primitive Implementations for the ReferenceRunner

2018-06-01 Thread Thomas Groh (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-4090?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Groh resolved BEAM-4090. --- Resolution: Fixed Fix Version/s: Not applicable > Fork/Update Primitive Implementations for the

[jira] [Created] (BEAM-4440) When filesToStage is empty, the DataflowRunner should fail.

2018-05-31 Thread Thomas Groh (JIRA)
Thomas Groh created BEAM-4440: - Summary: When filesToStage is empty, the DataflowRunner should fail. Key: BEAM-4440 URL: https://issues.apache.org/jira/browse/BEAM-4440 Project: Beam Issue Type:

[jira] [Assigned] (BEAM-4145) Java SDK Harness populates control request headers with worker id

2018-05-23 Thread Thomas Groh (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-4145?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Groh reassigned BEAM-4145: - Assignee: Thomas Groh (was: Luke Cwik) > Java SDK Harness populates control request headers with

[jira] [Assigned] (BEAM-4279) The Fn Harness doesn't properly handle Flattens which consume a single input multiple times

2018-05-23 Thread Thomas Groh (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-4279?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Groh reassigned BEAM-4279: - Assignee: Thomas Groh > The Fn Harness doesn't properly handle Flattens which consume a single inp

[jira] [Assigned] (BEAM-4279) The Fn Harness doesn't properly handle Flattens which consume a single input multiple times

2018-05-23 Thread Thomas Groh (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-4279?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Groh reassigned BEAM-4279: - Assignee: (was: Thomas Groh) > The Fn Harness doesn't properly handle Flattens which consume a

[jira] [Created] (BEAM-4384) Load in-process FnHarness instances with a restricted classpath

2018-05-22 Thread Thomas Groh (JIRA)
Thomas Groh created BEAM-4384: - Summary: Load in-process FnHarness instances with a restricted classpath Key: BEAM-4384 URL: https://issues.apache.org/jira/browse/BEAM-4384 Project: Beam Issue T

[jira] [Created] (BEAM-4279) The Fn Harness doesn't properly handle Flattens which consume a single input multiple times

2018-05-11 Thread Thomas Groh (JIRA)
Thomas Groh created BEAM-4279: - Summary: The Fn Harness doesn't properly handle Flattens which consume a single input multiple times Key: BEAM-4279 URL: https://issues.apache.org/jira/browse/BEAM-4279 Pro

[jira] [Created] (BEAM-4272) Consider allocating AssignContext only once in the Java SDK Harness

2018-05-10 Thread Thomas Groh (JIRA)
Thomas Groh created BEAM-4272: - Summary: Consider allocating AssignContext only once in the Java SDK Harness Key: BEAM-4272 URL: https://issues.apache.org/jira/browse/BEAM-4272 Project: Beam Iss

[jira] [Created] (BEAM-4269) Implement Assign Windows in the Java SDK Harness

2018-05-10 Thread Thomas Groh (JIRA)
Thomas Groh created BEAM-4269: - Summary: Implement Assign Windows in the Java SDK Harness Key: BEAM-4269 URL: https://issues.apache.org/jira/browse/BEAM-4269 Project: Beam Issue Type: New Feature

[jira] [Created] (BEAM-4259) Instantiate a Pipeline (EvaluationContext, Executor, etc) via Proto format (only) in the DirectRunner

2018-05-08 Thread Thomas Groh (JIRA)
Thomas Groh created BEAM-4259: - Summary: Instantiate a Pipeline (EvaluationContext, Executor, etc) via Proto format (only) in the DirectRunner Key: BEAM-4259 URL: https://issues.apache.org/jira/browse/BEAM-4259

[jira] [Created] (BEAM-4258) Integrate Docker Environment Management in the ReferenceRunner

2018-05-08 Thread Thomas Groh (JIRA)
Thomas Groh created BEAM-4258: - Summary: Integrate Docker Environment Management in the ReferenceRunner Key: BEAM-4258 URL: https://issues.apache.org/jira/browse/BEAM-4258 Project: Beam Issue Ty

[jira] [Resolved] (BEAM-3104) Implement a BeamFnState Service

2018-05-08 Thread Thomas Groh (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-3104?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Groh resolved BEAM-3104. --- Resolution: Fixed Assignee: Kenneth Knowles Fix Version/s: Not applicable > Implement a

[jira] [Resolved] (BEAM-3103) Implement a BeamFnControl Service

2018-05-08 Thread Thomas Groh (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-3103?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Groh resolved BEAM-3103. --- Resolution: Fixed Assignee: Thomas Groh Fix Version/s: Not applicable > Implement a Bea

[jira] [Commented] (BEAM-4088) ExecutorServiceParallelExecutorTest#ensureMetricsThreadDoesntLeak in PR #4965 does not pass in gradle

2018-05-07 Thread Thomas Groh (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-4088?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16466691#comment-16466691 ] Thomas Groh commented on BEAM-4088: --- This test has failed in 9 of the past 10 builds, and

[jira] [Commented] (BEAM-4088) ExecutorServiceParallelExecutorTest#ensureMetricsThreadDoesntLeak in PR #4965 does not pass in gradle

2018-05-07 Thread Thomas Groh (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-4088?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16466139#comment-16466139 ] Thomas Groh commented on BEAM-4088: --- If this is really just a timeout thing, I'm pretty h

[jira] [Resolved] (BEAM-3102) Implement a BeamFnData Service

2018-05-04 Thread Thomas Groh (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-3102?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Groh resolved BEAM-3102. --- Resolution: Fixed Assignee: Thomas Groh Fix Version/s: Not applicable > Implement a Bea

[jira] [Resolved] (BEAM-3101) Implement a Logging Service

2018-05-04 Thread Thomas Groh (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-3101?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Groh resolved BEAM-3101. --- Resolution: Fixed Assignee: Thomas Groh Fix Version/s: Not applicable > Implement a Log

[jira] [Created] (BEAM-4241) Implement AssignWindows for known WindowFns in the ReferenceRunner

2018-05-04 Thread Thomas Groh (JIRA)
Thomas Groh created BEAM-4241: - Summary: Implement AssignWindows for known WindowFns in the ReferenceRunner Key: BEAM-4241 URL: https://issues.apache.org/jira/browse/BEAM-4241 Project: Beam Issu

[jira] [Created] (BEAM-4240) Implement the limited set of portable primitives in the Java DirectRunner

2018-05-04 Thread Thomas Groh (JIRA)
Thomas Groh created BEAM-4240: - Summary: Implement the limited set of portable primitives in the Java DirectRunner Key: BEAM-4240 URL: https://issues.apache.org/jira/browse/BEAM-4240 Project: Beam

[jira] [Created] (BEAM-4239) Implement the native Java DirectRunner via the ReferenceRunner and an InProcessServerFactory/Channel

2018-05-04 Thread Thomas Groh (JIRA)
Thomas Groh created BEAM-4239: - Summary: Implement the native Java DirectRunner via the ReferenceRunner and an InProcessServerFactory/Channel Key: BEAM-4239 URL: https://issues.apache.org/jira/browse/BEAM-4239

[jira] [Created] (BEAM-4238) Join the local-java and direct-java packages

2018-05-04 Thread Thomas Groh (JIRA)
Thomas Groh created BEAM-4238: - Summary: Join the local-java and direct-java packages Key: BEAM-4238 URL: https://issues.apache.org/jira/browse/BEAM-4238 Project: Beam Issue Type: New Feature

[jira] [Created] (BEAM-4237) Migrate WatermarkManager to the local-java package

2018-05-04 Thread Thomas Groh (JIRA)
Thomas Groh created BEAM-4237: - Summary: Migrate WatermarkManager to the local-java package Key: BEAM-4237 URL: https://issues.apache.org/jira/browse/BEAM-4237 Project: Beam Issue Type: New Featu

[jira] [Resolved] (BEAM-4125) Add a library to manipulate the proto representation of a pipeline

2018-05-03 Thread Thomas Groh (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-4125?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Groh resolved BEAM-4125. --- Resolution: Fixed Fix Version/s: Not applicable > Add a library to manipulate the proto represen

[jira] [Created] (BEAM-4228) The FlinkRunner shouldn't require all of the values for a key to fit in memory

2018-05-02 Thread Thomas Groh (JIRA)
Thomas Groh created BEAM-4228: - Summary: The FlinkRunner shouldn't require all of the values for a key to fit in memory Key: BEAM-4228 URL: https://issues.apache.org/jira/browse/BEAM-4228 Project: Beam

[jira] [Commented] (BEAM-4191) WatermarkManagerTest has had a masked failure (both copies)

2018-05-01 Thread Thomas Groh (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-4191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16459811#comment-16459811 ] Thomas Groh commented on BEAM-4191: ---   Can you provide any more context? the linked PR d

[jira] [Created] (BEAM-4135) Remove Use of Java SDK Types in the DirectRunner "engine"

2018-04-18 Thread Thomas Groh (JIRA)
Thomas Groh created BEAM-4135: - Summary: Remove Use of Java SDK Types in the DirectRunner "engine" Key: BEAM-4135 URL: https://issues.apache.org/jira/browse/BEAM-4135 Project: Beam Issue Type: Ne

[jira] [Created] (BEAM-4125) Add a library to manipulate the proto representation of a pipeline

2018-04-18 Thread Thomas Groh (JIRA)
Thomas Groh created BEAM-4125: - Summary: Add a library to manipulate the proto representation of a pipeline Key: BEAM-4125 URL: https://issues.apache.org/jira/browse/BEAM-4125 Project: Beam Issu

[jira] [Created] (BEAM-4123) SDK Harness Environments should be able to report health.

2018-04-18 Thread Thomas Groh (JIRA)
Thomas Groh created BEAM-4123: - Summary: SDK Harness Environments should be able to report health. Key: BEAM-4123 URL: https://issues.apache.org/jira/browse/BEAM-4123 Project: Beam Issue Type: Ne

[jira] [Created] (BEAM-4090) Fork/Update Primitive Implementations for the ReferenceRunner

2018-04-16 Thread Thomas Groh (JIRA)
Thomas Groh created BEAM-4090: - Summary: Fork/Update Primitive Implementations for the ReferenceRunner Key: BEAM-4090 URL: https://issues.apache.org/jira/browse/BEAM-4090 Project: Beam Issue Typ

[jira] [Resolved] (BEAM-3988) QueryablePipeline should consider any PTransform with no subtransforms to be a primtiive

2018-04-13 Thread Thomas Groh (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-3988?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Groh resolved BEAM-3988. --- Resolution: Fixed Fix Version/s: 2.5.0 > QueryablePipeline should consider any PTransform with n

[jira] [Resolved] (BEAM-3952) GreedyStageFuserTest broken

2018-04-13 Thread Thomas Groh (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-3952?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Groh resolved BEAM-3952. --- Resolution: Fixed Fix Version/s: Not applicable > GreedyStageFuserTest broken >

[jira] [Created] (BEAM-4073) The DirectRunner should interact with a Pipeline via an abstraction of the Graph rather than SDK types

2018-04-13 Thread Thomas Groh (JIRA)
Thomas Groh created BEAM-4073: - Summary: The DirectRunner should interact with a Pipeline via an abstraction of the Graph rather than SDK types Key: BEAM-4073 URL: https://issues.apache.org/jira/browse/BEAM-4073

[jira] [Assigned] (BEAM-3119) direct-metrics-counter-committer threads are leaking

2018-04-11 Thread Thomas Groh (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-3119?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Groh reassigned BEAM-3119: - Assignee: Romain Manni-Bucau (was: Thomas Groh) > direct-metrics-counter-committer threads are le

[jira] [Commented] (BEAM-4014) Migrate MavenInstall Jenkins PostCommits to Gradle

2018-04-05 Thread Thomas Groh (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-4014?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16427807#comment-16427807 ] Thomas Groh commented on BEAM-4014: --- There are a couple of difficulties here - the primar

[jira] [Assigned] (BEAM-4014) Migrate MavenInstall Jenkins PostCommits to Gradle

2018-04-05 Thread Thomas Groh (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-4014?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Groh reassigned BEAM-4014: - Assignee: Thomas Groh > Migrate MavenInstall Jenkins PostCommits to Gradle > -

[jira] [Created] (BEAM-3988) QueryablePipeline should consider any PTransform with no subtransforms to be a primtiive

2018-04-02 Thread Thomas Groh (JIRA)
Thomas Groh created BEAM-3988: - Summary: QueryablePipeline should consider any PTransform with no subtransforms to be a primtiive Key: BEAM-3988 URL: https://issues.apache.org/jira/browse/BEAM-3988 Projec

[jira] [Resolved] (BEAM-3913) Support custom runner primitives in Fusion

2018-04-02 Thread Thomas Groh (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-3913?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Groh resolved BEAM-3913. --- Resolution: Fixed Fix Version/s: Not applicable > Support custom runner primitives in Fusion > -

[jira] [Resolved] (BEAM-3963) Dataflow PostCommits have not built since Mar 26, 2018

2018-04-02 Thread Thomas Groh (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-3963?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Groh resolved BEAM-3963. --- Resolution: Fixed Fix Version/s: Not applicable > Dataflow PostCommits have not built since Mar

[jira] [Resolved] (BEAM-3964) Java Github HEAD doesn't run on Dataflow

2018-04-02 Thread Thomas Groh (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-3964?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Groh resolved BEAM-3964. --- Resolution: Fixed Fix Version/s: Not applicable > Java Github HEAD doesn't run on Dataflow > ---

[jira] [Commented] (BEAM-3964) Java Github HEAD doesn't run on Dataflow

2018-03-28 Thread Thomas Groh (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-3964?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16417874#comment-16417874 ] Thomas Groh commented on BEAM-3964: --- [https://github.com/apache/beam/commit/a767342371247

[jira] [Updated] (BEAM-3964) Java Github HEAD doesn't run on Dataflow

2018-03-28 Thread Thomas Groh (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-3964?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Groh updated BEAM-3964: -- Description: Rather: it runs, but cannot make progress, seemingly due to an Error with metric names:  

[jira] [Created] (BEAM-3964) Java Github HEAD doesn't run on Dataflow

2018-03-28 Thread Thomas Groh (JIRA)
Thomas Groh created BEAM-3964: - Summary: Java Github HEAD doesn't run on Dataflow Key: BEAM-3964 URL: https://issues.apache.org/jira/browse/BEAM-3964 Project: Beam Issue Type: Bug Compo

[jira] [Comment Edited] (BEAM-3963) Dataflow PostCommits have not built since Mar 26, 2018

2018-03-28 Thread Thomas Groh (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-3963?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16417821#comment-16417821 ] Thomas Groh edited comment on BEAM-3963 at 3/28/18 5:47 PM: Mos

[jira] [Commented] (BEAM-3963) Dataflow PostCommits have not built since Mar 26, 2018

2018-03-28 Thread Thomas Groh (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-3963?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16417821#comment-16417821 ] Thomas Groh commented on BEAM-3963: --- Most of the 'failed to build' are actually timeouts

[jira] [Created] (BEAM-3963) Dataflow PostCommits have not built since Mar 26, 2018 and

2018-03-28 Thread Thomas Groh (JIRA)
Thomas Groh created BEAM-3963: - Summary: Dataflow PostCommits have not built since Mar 26, 2018 and Key: BEAM-3963 URL: https://issues.apache.org/jira/browse/BEAM-3963 Project: Beam Issue Type:

[jira] [Updated] (BEAM-3963) Dataflow PostCommits have not built since Mar 26, 2018

2018-03-28 Thread Thomas Groh (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-3963?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Groh updated BEAM-3963: -- Summary: Dataflow PostCommits have not built since Mar 26, 2018 (was: Dataflow PostCommits have not bui

[jira] [Resolved] (BEAM-3895) Side Inputs should be available on ExecutableStage

2018-03-23 Thread Thomas Groh (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-3895?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Groh resolved BEAM-3895. --- Resolution: Fixed Fix Version/s: Not applicable > Side Inputs should be available on ExecutableS

  1   2   3   4   5   6   >