[
https://issues.apache.org/jira/browse/BEAM-6753?focusedWorklogId=223384&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-223384
]
ASF GitHub Bot logged work on BEAM-6753:
----------------------------------------
Author: ASF GitHub Bot
Created on: 05/Apr/19 00:22
Start Date: 05/Apr/19 00:22
Worklog Time Spent: 10m
Work Description: reuvenlax commented on issue #7952: [BEAM-6753] Set the
stage to make schema coder update compatible
URL: https://github.com/apache/beam/pull/7952#issuecomment-480108267
FYI this was added right now simply for the DataflowRunner; we used the
portability protos for convenience (and because that's where it will go
eventually anyway). pushing things into portability for real indeed does
require a published design first, which is why the portability runners
currently ignore this.
On Thu, Apr 4, 2019 at 3:42 PM Kenn Knowles <[email protected]>
wrote:
> *@kennknowles* commented on this pull request.
> ------------------------------
>
> In model/pipeline/src/main/proto/beam_runner_api.proto
> <https://github.com/apache/beam/pull/7952#discussion_r272393948>:
>
> > @@ -635,6 +635,61 @@ message StandardCoders {
> }
> }
>
> +// Experimental: A representation of a Beam Schema.
> +message Schema {
> + enum TypeName {
> + BYTE = 0;
> + INT16 = 1;
> + INT32 = 2;
> + INT64 = 3;
> + DECIMAL = 4;
> + FLOAT = 5;
> + DOUBLE = 6;
> + STRING = 7;
> + DATETIME = 8;
>
> Adding schemas to portability needs a design doc and discussion on dev@.
> Did I miss that?
> ------------------------------
>
> In model/pipeline/src/main/proto/beam_runner_api.proto
> <https://github.com/apache/beam/pull/7952#discussion_r272394224>:
>
> > +// Experimental: A representation of a Beam Schema.
> +message Schema {
> + enum TypeName {
> + BYTE = 0;
> + INT16 = 1;
> + INT32 = 2;
> + INT64 = 3;
> + DECIMAL = 4;
> + FLOAT = 5;
> + DOUBLE = 6;
> + STRING = 7;
> + DATETIME = 8;
> + BOOLEAN = 9;
> + BYTES = 10;
> + ARRAY = 11;
> + MAP = 13;
>
> Notably, you don't need this. FieldType has a oneof and one of the
> alternatives can be an atomic type and the others are map, row, array,
> logical. I would recommend breaking out more types in this oneof, for
> example a BytesTypes we may likely want metadata indicating a coder. And we
> discussed - and I think should go ahead with - removing datetime as a
> primitive type.
>
> —
> You are receiving this because you modified the open/close state.
> Reply to this email directly, view it on GitHub
> <https://github.com/apache/beam/pull/7952#pullrequestreview-223035548>,
> or mute the thread
>
<https://github.com/notifications/unsubscribe-auth/AUGE1VPhTpyITXgGc3rJBPfBqSqPCqXIks5vdn_NgaJpZM4bThja>
> .
>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 223384)
Time Spent: 2.5h (was: 2h 20m)
> Create proto representation for schemas
> ---------------------------------------
>
> Key: BEAM-6753
> URL: https://issues.apache.org/jira/browse/BEAM-6753
> Project: Beam
> Issue Type: Sub-task
> Components: beam-model
> Reporter: Reuven Lax
> Assignee: Reuven Lax
> Priority: Major
> Time Spent: 2.5h
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
