I can help review the Java change.
-Rui
On Thu, Aug 6, 2020 at 9:53 AM Brian Hulette wrote:
> The PR for this is up now: https://github.com/apache/beam/pull/12481
> Any volunteers to help review? We may want a separate reviewer for Python
> and Java changes.
>
> Brian
>
> On Wed, Aug 5, 2020 a
The PR for this is up now: https://github.com/apache/beam/pull/12481
Any volunteers to help review? We may want a separate reviewer for Python
and Java changes.
Brian
On Wed, Aug 5, 2020 at 9:00 AM Brian Hulette wrote:
> What I'm working on changes ExternalConfigurationPayload [1] to this:
>
>
What I'm working on changes ExternalConfigurationPayload [1] to this:
message ExternalConfigurationPayload {
// A schema for use in beam:coder:row:v1
Schema schema = 1;
// A payload which can be decoded using beam:coder:row:v1 and the given
schema.
bytes payload = 2;
}
The calling SDK ca
+1
The format to store coders is not set in stone, it was a first version
to make external configuration work. Using the Coder message would be
better.
As for using Schema to store the configuration, could somebody fill me
in how that would work?
-Max
On 04.08.20 02:01, Brian Hulette wrot
I've opened BEAM-10571 [1] for this, and I'm most of the way to an
implementation now. Aiming to have it done before the 2.24.0 cut since it
will be the last release with python 2 support.
[1] https://issues.apache.org/jira/browse/BEAM-10571
On Wed, Jul 15, 2020 at 9:03 AM Chamikara Jayalath
wro
On Fri, Jul 10, 2020 at 4:47 PM Robert Bradshaw wrote:
> On Fri, Jul 10, 2020 at 4:36 PM Brian Hulette wrote:
> >
> > Ah yes I'm +1 for that approach too - it would let us leverage all the
> schema-inference already in the Java SDK for translating configuration
> objects which would be great.
>
On Fri, Jul 10, 2020 at 4:36 PM Brian Hulette wrote:
>
> Ah yes I'm +1 for that approach too - it would let us leverage all the
> schema-inference already in the Java SDK for translating configuration
> objects which would be great.
> Things on the Python side would be trickier as schemas don't
Ah yes I'm +1 for that approach too - it would let us leverage all the
schema-inference already in the Java SDK for translating configuration
objects which would be great.
Things on the Python side would be trickier as schemas don't formally
support all the types you can use in the PayloadBuilder i
I would be in favor of just using a schema to store the entire
configuration. The reason we went with what we have to day is that we
didn't have cross language schemas yet.
On Fri, Jul 10, 2020 at 12:24 PM Brian Hulette wrote:
>
> Hi everyone,
> I noticed that currently the ExternalConfigurationP
Hi everyone,
I noticed that currently the ExternalConfigurationPayload uses a list of
coder URNs to represent the coder that was used to serialize each
configuration field [1]. This seems acceptable at first blush, but there's
one notable issue: it has no place to store a payload for the coder. Mos
10 matches
Mail list logo
