Oki, will try a PR after the classloader one then. Thanks a lot. Le 13 févr. 2018 19:14, "Lukasz Cwik" <lc...@google.com> a écrit :
> The one in Dataflow 1.x was one system property that contained all the > JSON so it wasn't exactly what you were looking for. > > On Tue, Feb 13, 2018 at 9:51 AM, Romain Manni-Bucau <rmannibu...@gmail.com > > wrote: > >> I like your proposal Kenneth. Perfectly fits my use case and deployment >> one as well - when ops configure the env without modifying the code. >> >> How do we move forward on that? Should I send a PR or do you want to >> import what was in dataflow? >> >> >> Romain Manni-Bucau >> @rmannibucau <https://twitter.com/rmannibucau> | Blog >> <https://rmannibucau.metawerx.net/> | Old Blog >> <http://rmannibucau.wordpress.com> | Github >> <https://github.com/rmannibucau> | LinkedIn >> <https://www.linkedin.com/in/rmannibucau> | Book >> <https://www.packtpub.com/application-development/java-ee-8-high-performance> >> >> 2018-02-13 18:49 GMT+01:00 Lukasz Cwik <lc...@google.com>: >> >>> PipelineOptionsFactory.fromSystemProperties did exist in Dataflow 1.x >>> and was dropped for the reason that Ken mentioned. >>> >>> On Tue, Feb 13, 2018 at 9:46 AM, Kenneth Knowles <k...@google.com> wrote: >>> >>>> Pipeline options are not global - they are a property of a single job. >>>> The TestPipeline reads them from a very particular system property because >>>> it is a special testing rule. >>>> >>>> If you want a generic way to build pipeline options from a set of >>>> system properties, it should be from an explicit prefix, not global >>>> defaults. So if a user wants to put a bunch of options into properties, >>>> they choose a namespace like "my.random.namespace" and everything under it >>>> can be interpreted as a pipelineoption: >>>> >>>> my.random.namespace.SomeOption=bizzle >>>> my.random.namespace.SomeOtherOption=whatever >>>> >>>> And you could do PipelineOptionsFactory.fromSys >>>> temProperties("my.random.namespace") >>>> >>>> We should use the full name of the option not split it with dots - dots >>>> represent hierarchy not words separation. >>>> >>>> interface FooOptions extends PipelineOptions { >>>> getSomeOption() >>>> getSomeOtherOption() >>>> } >>>> >>>> I think I can be +0.5 on this. We might also reserve a namespace like >>>> "beam.TestPipeline.options" and use it for specification of test config. >>>> Could be easier than embedding JSON in a property, in some cases. Easier to >>>> override little pieces for sure. >>>> >>>> Kenn >>>> >>>> On Tue, Feb 13, 2018 at 9:29 AM, Romain Manni-Bucau < >>>> rmannibu...@gmail.com> wrote: >>>> >>>>> makes sense, do we want beam.foo.bar -> --foo-bar conversion too? >>>>> >>>>> >>>>> Romain Manni-Bucau >>>>> @rmannibucau <https://twitter.com/rmannibucau> | Blog >>>>> <https://rmannibucau.metawerx.net/> | Old Blog >>>>> <http://rmannibucau.wordpress.com> | Github >>>>> <https://github.com/rmannibucau> | LinkedIn >>>>> <https://www.linkedin.com/in/rmannibucau> | Book >>>>> <https://www.packtpub.com/application-development/java-ee-8-high-performance> >>>>> >>>>> 2018-02-13 18:19 GMT+01:00 Eugene Kirpichov <kirpic...@google.com>: >>>>> >>>>>> Neutral about this one: haven't seen a case where this was needed, >>>>>> but don't see anything wrong with it either. One thing I'd recommend if >>>>>> you >>>>>> go through with it, extract from system properties under "beam." rather >>>>>> than all of them, to avoid clashes. >>>>>> >>>>>> On Tue, Feb 13, 2018, 7:53 AM Jean-Baptiste Onofré <j...@nanthrax.net> >>>>>> wrote: >>>>>> >>>>>>> Hi Romain, >>>>>>> >>>>>>> it sounds interesting to me, and doesn't break anything, so +1 from >>>>>>> my side. >>>>>>> >>>>>>> Regards >>>>>>> JB >>>>>>> >>>>>>> On 02/13/2018 03:42 PM, Romain Manni-Bucau wrote: >>>>>>> > Hi guys, >>>>>>> > >>>>>>> > there are hacks in beam testing code to read the args from a >>>>>>> system property but >>>>>>> > I wonder if we shouldnt add a PipelineOptionsFactory.fromSys >>>>>>> temProperties(). >>>>>>> > >>>>>>> > It would iterate over the system properties and take all --xxx=foo >>>>>>> as potential >>>>>>> > argument it tries to bind. >>>>>>> > >>>>>>> > Rational behind that is to enable users to wrap the pipeline API >>>>>>> but still >>>>>>> > expose the pipeline options to end users for advanced cases. >>>>>>> > >>>>>>> > Any discussion on this kind of usages already? What do you think >>>>>>> of this proposal? >>>>>>> > >>>>>>> > Side note: we can think about a fromEnv() too. >>>>>>> > >>>>>>> > Romain Manni-Bucau >>>>>>> > @rmannibucau <https://twitter.com/rmannibucau> | Blog >>>>>>> > <https://rmannibucau.metawerx.net/> | Old Blog >>>>>>> > <http://rmannibucau.wordpress.com> | Github < >>>>>>> https://github.com/rmannibucau> | >>>>>>> > LinkedIn <https://www.linkedin.com/in/rmannibucau> | Book >>>>>>> > <https://www.packtpub.com/application-development/java-ee-8- >>>>>>> high-performance> >>>>>>> >>>>>>> -- >>>>>>> Jean-Baptiste Onofré >>>>>>> jbono...@apache.org >>>>>>> http://blog.nanthrax.net >>>>>>> Talend - http://www.talend.com >>>>>>> >>>>>> >>>>> >>>> >>> >> >
