Hi,
Just filed JIRA https://issues.apache.org/jira/browse/BEAM-2644
Many transforms that take ValueProvider's have different codepaths for when
the provider is accessible or not. However, as far as I can tell, there is
no good way to construct a pipeline with PipelineOptions containing an
inacces
I also reported something similar to this as
https://issues.apache.org/jira/browse/BEAM-2577. That issue was reported
because we don't have any tests that use a runner and attempt to pass
ValueProviders in. This means that we've found bugs such as
NestedValueProviders used with non-serializable ano
I think (3) sounds good for BEAM-2644. I think keep them both open since
one is to develop the capability and the other is to use it.
On Wed, Jul 19, 2017 at 12:32 PM, Ben Chambers wrote:
> I also reported something similar to this as
> https://issues.apache.org/jira/browse/BEAM-2577. That issue
+1, 3 is a good option.
On Wed, Jul 19, 2017, 12:40 PM Kenneth Knowles
wrote:
> I think (3) sounds good for BEAM-2644. I think keep them both open since
> one is to develop the capability and the other is to use it.
>
> On Wed, Jul 19, 2017 at 12:32 PM, Ben Chambers
> > wrote:
>
> > I also repo
I sent a PR for review with something that I think is a still better option:
https://github.com/apache/beam/pull/3753 +Ben Chambers
Example usage:
p.apply("Read", AvroIO.read(GenericClass.class)
.from(*p.newProvider*(outputFile.getAbsolutePath()
TestPipeline.newProvider() returns a Value
The PR has been submitted.
I think now it'd be nice to add test coverage to all IOs using this
utility. It should be pretty trivial, I wonder how many bugs we'll find.
On Mon, Aug 28, 2017 at 12:06 PM Eugene Kirpichov
wrote:
> I sent a PR for review with something that I think is a still better