I sent a PR for review with something that I think is a still better option:
https://github.com/apache/beam/pull/3753 +Ben Chambers
<bchamb...@google.com>
Example usage:
p.apply("Read", AvroIO.read(GenericClass.class)
.from(*p.newProvider*(outputFile.getAbsolutePath()))))
TestPipeline.newProvider() returns a ValueProvider that is inaccessible at
construction time, but will have the given value at runtime.
The advantage of this approach is that it makes it very easy to have
testing on inaccessible ValueProvider's be the default, with no boilerplate
required to write such tests: this is even fewer code than writing
StaticValueProvider.of().
On Wed, Jul 19, 2017 at 12:46 PM Eugene Kirpichov <kirpic...@google.com>
wrote:
> +1, 3 is a good option.
>
> On Wed, Jul 19, 2017, 12:40 PM Kenneth Knowles <k...@google.com.invalid>
> wrote:
>
>> I think (3) sounds good for BEAM-2644. I think keep them both open since
>> one is to develop the capability and the other is to use it.
>>
>> On Wed, Jul 19, 2017 at 12:32 PM, Ben Chambers
>> <bchamb...@google.com.invalid
>> > wrote:
>>
>> > I also reported something similar to this as
>> > https://issues.apache.org/jira/browse/BEAM-2577. That issue was
>> reported
>> > because we don't have any tests that use a runner and attempt to pass
>> > ValueProviders in. This means that we've found bugs such as
>> > NestedValueProviders used with non-serializable anonymous functions.
>> >
>> > One solution seems be to use that pattern:
>> >
>> > 1. Create an extension of PipelineOptions with some ValueProviders
>> > 2. Use that in your test pipeline
>> > 3. allow TestPipeline.run() to take additional arguments to provide at
>> > template execution time to populate those value providers
>> >
>> > On Wed, Jul 19, 2017 at 12:03 PM Eugene Kirpichov
>> > <kirpic...@google.com.invalid> wrote:
>> >
>> > > Hi,
>> > >
>> > > Just filed JIRA https://issues.apache.org/jira/browse/BEAM-2644
>> > >
>> > > Many transforms that take ValueProvider's have different codepaths for
>> > when
>> > > the provider is accessible or not. However, as far as I can tell,
>> there
>> > is
>> > > no good way to construct a pipeline with PipelineOptions containing an
>> > > inaccessible ValueProvider, and then test how it would run as a
>> template
>> > > invocation with an actual value supplied.
>> > >
>> > > The only way I could come up with is mimicking
>> > >
>> > > https://github.com/apache/beam/blob/master/sdks/java/
>> >
>> core/src/test/java/org/apache/beam/sdk/options/ValueProviderTest.java#L202
>> > > , which is very ugly.
>> > >
>> > > Am I missing something? Is there already a good way to do this?
>> > >
>> >
>>
>
