+1, 3 is a good option. On Wed, Jul 19, 2017, 12:40 PM Kenneth Knowles <k...@google.com.invalid> wrote:
> I think (3) sounds good for BEAM-2644. I think keep them both open since > one is to develop the capability and the other is to use it. > > On Wed, Jul 19, 2017 at 12:32 PM, Ben Chambers > <bchamb...@google.com.invalid > > wrote: > > > I also reported something similar to this as > > https://issues.apache.org/jira/browse/BEAM-2577. That issue was reported > > because we don't have any tests that use a runner and attempt to pass > > ValueProviders in. This means that we've found bugs such as > > NestedValueProviders used with non-serializable anonymous functions. > > > > One solution seems be to use that pattern: > > > > 1. Create an extension of PipelineOptions with some ValueProviders > > 2. Use that in your test pipeline > > 3. allow TestPipeline.run() to take additional arguments to provide at > > template execution time to populate those value providers > > > > On Wed, Jul 19, 2017 at 12:03 PM Eugene Kirpichov > > <kirpic...@google.com.invalid> wrote: > > > > > Hi, > > > > > > Just filed JIRA https://issues.apache.org/jira/browse/BEAM-2644 > > > > > > Many transforms that take ValueProvider's have different codepaths for > > when > > > the provider is accessible or not. However, as far as I can tell, there > > is > > > no good way to construct a pipeline with PipelineOptions containing an > > > inaccessible ValueProvider, and then test how it would run as a > template > > > invocation with an actual value supplied. > > > > > > The only way I could come up with is mimicking > > > > > > https://github.com/apache/beam/blob/master/sdks/java/ > > > core/src/test/java/org/apache/beam/sdk/options/ValueProviderTest.java#L202 > > > , which is very ugly. > > > > > > Am I missing something? Is there already a good way to do this? > > > > > >
