I created a ticket to track this:
https://issues.apache.org/jira/browse/BEAM-11914. Thanks everyone for the
comments!
Thanks,
Xinyu
On Mon, Mar 1, 2021 at 4:45 PM Xinyu Liu wrote:
> The reason for not passing it in directly is that we have a large amount
> of configs here at LinkedIn so we use
The reason for not passing it in directly is that we have a large amount of
configs here at LinkedIn so we use an internal config management framework
to hide the actual detailed configs needed to construct these resources.
Internally we put a config map inside the PipelineOptions and then during
@
Why not just pass in the arguments to the DoFn constructor or as a variable
in the containing scope? Do you only know the option after the pipeline is
completely constructed so you need to make the switch at runtime? Makes
sense. I think passing options to @Setup is useful and harmless.
Kenn
On M
Any reason not to simply pass these parameters into the DoFn constructor?
On Mon, Mar 1, 2021 at 3:42 PM Xinyu Liu wrote:
> Hi, all,
>
> Currently the @Setup method signature in DoFn does not support any
> arguments. This is a bit cumbersome to use for use cases such as creating a
> db connectio
