Would you be able to put together a doc with a little more motivation and
background on the feature, plus specific proposal? I think there's a lot of
desire to build Beam pipelines in ways that are not programmatic and we can
have a good discussion and make sure that changes meet the goals.
Separately, I've built *many* systems to non-programmatically build things
that were previously done via code. One lesson is that whatever starts as
reflection-based "totally automatic" tooling never suits all your needs
beyond simple prototypes. You'll almost certainly end up building a system
of annotations in the code or a system with configuration schema on the
side. So I would not invest in making things reflection friendly unless we
know this will never need to grow into more.
Kenn
On Fri, Jan 12, 2018 at 10:26 AM, Romain Manni-Bucau <rmannibu...@gmail.com>
wrote:
> Hi guys
>
> I'd like to discuss the IO configuration.
>
> My goal is to be able to instrospect (or equivalent) the IO to instantiate
> them programmatically in a generic manner from a generic config - this is
> not yet linked to the system property topic but can benefit beam on this
> other topic too.
>
> Auto value loosing the final fields ordering is impossible to use until
> you parse sources.
>
> In other words: auto value is nice for programmatic usage but is blocking
> for any automotion on top of it - even using unsafe is not an option ATM :(.
>
> Can we try to get something to solve that need please?
>
> Here are the solutions I see (pick just one ;)):
>
> 1. migrate IO to IOOptions (based on pipeline options kind of design).
> This is a breaking change - but I'm sure we can mitigate it in term of user
> compatibility - but it unifies the beam config and enables to not have IO
> specific configurations which can vary between the IO if not developped by
> the same guy.
> 2. Add an extension to @AutoValue to also generate the field names order
> in the create() (@Fields({"address","username","password"}). Cheap but
> the way to instantiate an IO from a config is still a pain (think
> Factory.createIO(clazz, properties))
> 3. Also generate a plain pojo from the abstract @AutoValue class - this
> requires to modify the source class to make it working but is feasible with
> a processor
> 4. drop autovalue and use plain pojo - writing it cause it is a technical
> option but it leads to break as much as 1 without getting all the benefit
> to have an agnostic config and the tooling we can build on top of it
> 5. probably others
>
>
> Wdyt?
>
> Personally I really like 1 cause it starts to create a clean programming
> model we can then build other features on.
>
>
> Romain Manni-Bucau
> @rmannibucau <https://twitter.com/rmannibucau> | Blog
> <https://rmannibucau.metawerx.net/> | Old Blog
> <http://rmannibucau.wordpress.com> | Github
> <https://github.com/rmannibucau> | LinkedIn
> <https://www.linkedin.com/in/rmannibucau>
>
