Hi everyone,
I noticed that currently the ExternalConfigurationPayload uses a list of
coder URNs to represent the coder that was used to serialize each
configuration field [1]. This seems acceptable at first blush, but there's
one notable issue: it has no place to store a payload for the coder. Most
standard coders don't use a payload so it's not a problem, but row coder
does use a payload to store it's schema, which means it can't be used in an
ExternalConfigurationPayload today.

Is there a reason not to just use the Coder message [2] in
ExternalConfigurationPayload instead of a list of coder URNs? That would
work with row coder, and it would also make it easier to re-use logic for
translating Pipeline protos.

I'd be happy to make this change, but I wanted to ask on dev@ in case
there's something I'm missing here.

Brian

[1]
https://github.com/apache/beam/blob/c54a0b7f49f2eb4a15df115205e2fa455116ccbe/model/pipeline/src/main/proto/external_transforms.proto#L34-L35
[2]
https://github.com/apache/beam/blob/c54a0b7f49f2eb4a15df115205e2fa455116ccbe/model/pipeline/src/main/proto/beam_runner_api.proto#L542-L555

Reply via email to