[jira] [Updated] (FLINK-21171) Introduce TypedValue to the StateFun request-reply protocol
[
https://issues.apache.org/jira/browse/FLINK-21171?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated FLINK-21171:
---
Labels: pull-request-available (was: )
> Introduce TypedValue to the StateFun request-reply protocol
> ---
>
> Key: FLINK-21171
> URL: https://issues.apache.org/jira/browse/FLINK-21171
> Project: Flink
> Issue Type: New Feature
> Components: Stateful Functions
>Reporter: Tzu-Li (Gordon) Tai
>Assignee: Tzu-Li (Gordon) Tai
>Priority: Major
> Labels: pull-request-available
> Fix For: statefun-3.0.0
>
>
> Currently, all values being passed around via the request-reply protocol, are
> of the Protobuf {{Any}} type. This includes payloads of outgoing messages to
> other functions, and also state values.
> This has a few shortcomings:
> * All user records are strictly required to be modeled and wrapped as a
> Protobuf message - even for simple primitive types. This makes it awkward to
> work with for many common types of messages, for example JSON.
> * For data persisted as state, with each state value being a Protobuf
> {{Any}}, each value would also redundantly store the type urls associated
> with each Protobuf message.
> Instead, we'd like to introduce a {{TypedValue}} construct that replaces
> {{Any}} everywhere in the protocol, for both messages and state values:
> {code}
> message TypedValue {
> string typename = 1;
> bytes value = 2;
> }
> {code}
> The {{typename}} here directly maps to the type concept introduced in
> FLINK-21061.
> For state, we directly write the value bytes of a {{TypedValue}} into state,
> and the {{typename}} is the meta information snapshotted by the state
> serializer (see FLINK-21061).
> For messages, the new {{TypedValue}} opens up the possibility for
> user-defined types. For example, a user can serialize a JSON string as the
> value bytes, and define a custom typename to indentify the type.
> We can also leverage this to define built-in types, for example a
> cross-language unification of primitive types. This would be an extended
> scope of this ticket and will be separately handled.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (FLINK-21171) Introduce TypedValue to the StateFun request-reply protocol
[
https://issues.apache.org/jira/browse/FLINK-21171?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tzu-Li (Gordon) Tai updated FLINK-21171:
Description:
Currently, all values being passed around via the request-reply protocol, are
of the Protobuf {{Any}} type. This includes payloads of outgoing messages to
other functions, and also state values.
This has a few shortcomings:
* All user records are strictly required to be modeled and wrapped as a
Protobuf message - even for simple primitive types. This makes it awkward to
work with for many common types of messages, for example JSON.
* For data persisted as state, with each state value being a Protobuf {{Any}},
each value would also redundantly store the type urls associated with each
Protobuf message.
Instead, we'd like to introduce a {{TypedValue}} construct that replaces
{{Any}} everywhere in the protocol, for both messages and state values:
{code}
message TypedValue {
string typename = 1;
bytes value = 2;
}
{code}
The {{typename}} here directly maps to the type concept introduced in
FLINK-21061.
For state, we directly write the value bytes of a {{TypedValue}} into state,
and the {{typename}} is the meta information snapshotted by the state
serializer (see FLINK-21061).
For messages, the new {{TypedValue}} opens up the possibility for user-defined
types. For example, a user can serialize a JSON string as the value bytes, and
define a custom typename to indentify the type.
We can also leverage this to define built-in types, for example a
cross-language unification of primitive types. This would be an extended scope
of this ticket and will be separately handled.
was:
Currently, all values being passed around via the request-reply protocol, are
of the Protobuf {{Any}} type. This includes payloads of outgoing messages to
other functions, and also state values.
This has a few shortcomings:
* All user records are strictly required to be modeled and wrapped as a
Protobuf message - even for simple primitive types. This makes it awkward to
work with for many common types of messages, for example JSON.
* For data persisted as state, with each state value being a Protobuf {{Any}},
each value would also redundantly store the type urls associated with each
Protobuf message.
Instead, we'd like to introduced a {{TypedValue}} construct that replaces
{{Any}} everywhere in the protocol, for both messages and state values:
{code}
message TypedValue {
string typename = 1;
bytes value = 2;
}
{code}
The {{typename}} here directly maps to the type concept introduced in
FLINK-21061.
For state, we directly write the value bytes of a {{TypedValue}} into state,
and the {{typename}} is the meta information snapshotted by the state
serializer (see FLINK-21061).
> Introduce TypedValue to the StateFun request-reply protocol
> ---
>
> Key: FLINK-21171
> URL: https://issues.apache.org/jira/browse/FLINK-21171
> Project: Flink
> Issue Type: New Feature
> Components: Stateful Functions
>Reporter: Tzu-Li (Gordon) Tai
>Assignee: Tzu-Li (Gordon) Tai
>Priority: Major
> Fix For: statefun-3.0.0
>
>
> Currently, all values being passed around via the request-reply protocol, are
> of the Protobuf {{Any}} type. This includes payloads of outgoing messages to
> other functions, and also state values.
> This has a few shortcomings:
> * All user records are strictly required to be modeled and wrapped as a
> Protobuf message - even for simple primitive types. This makes it awkward to
> work with for many common types of messages, for example JSON.
> * For data persisted as state, with each state value being a Protobuf
> {{Any}}, each value would also redundantly store the type urls associated
> with each Protobuf message.
> Instead, we'd like to introduce a {{TypedValue}} construct that replaces
> {{Any}} everywhere in the protocol, for both messages and state values:
> {code}
> message TypedValue {
> string typename = 1;
> bytes value = 2;
> }
> {code}
> The {{typename}} here directly maps to the type concept introduced in
> FLINK-21061.
> For state, we directly write the value bytes of a {{TypedValue}} into state,
> and the {{typename}} is the meta information snapshotted by the state
> serializer (see FLINK-21061).
> For messages, the new {{TypedValue}} opens up the possibility for
> user-defined types. For example, a user can serialize a JSON string as the
> value bytes, and define a custom typename to indentify the type.
> We can also leverage this to define built-in types, for example a
> cross-language unification of primitive types. This would be an extended
> scope of this ticket and will be separately handled.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (FLINK-21171) Introduce TypedValue to the StateFun request-reply protocol
[
https://issues.apache.org/jira/browse/FLINK-21171?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tzu-Li (Gordon) Tai updated FLINK-21171:
Description:
Currently, all values being passed around via the request-reply protocol, are
of the Protobuf {{Any}} type. This includes payloads of outgoing messages to
other functions, and also state values.
This has a few shortcomings:
* All user records are strictly required to be modeled and wrapped as a
Protobuf message - even for simple primitive types. This makes it awkward to
work with for many common types of messages, for example JSON.
* For data persisted as state, with each state value being a Protobuf {{Any}},
each value would also redundantly store the type urls associated with each
Protobuf message.
Instead, we'd like to introduced a {{TypedValue}} construct that replaces
{{Any}} everywhere in the protocol, for both messages and state values:
{code}
message TypedValue {
string typename = 1;
bytes value = 2;
}
{code}
The {{typename}} here directly maps to the type concept introduced in
FLINK-21061.
For state, we directly write the value bytes of a {{TypedValue}} into state,
and the {{typename}} is the meta information snapshotted by the state
serializer (see FLINK-21061).
was:
Currently, all values being passed around via the request-reply protocol, are
of the Protobuf {{Any}} type. This includes payloads of outgoing messages to
other functions, and also state values.
This has a few shortcomings:
* All user records are strictly required to be modeled and wrapped as a
Protobuf message - even for simple primitive type. This makes it awkward to
work with for many common types of messages, for example JSON.
* For data persisted as state, with each state value being a Protobuf {{Any}},
each value would also redundantly store the type urls associated with each
Protobuf message.
Instead, we'd like to introduced a {{TypedValue}} construct that replaces
{{Any}} everywhere in the protocol, for both messages and state values:
{code}
message TypedValue {
string typename = 1;
bytes value = 2;
}
{code}
The {{typename}} here directly maps to the type concept introduced in
FLINK-21061.
For state, we directly write the value bytes of a {{TypedValue}} into state,
and the {{typename}} is the meta information snapshotted by the state
serializer (see FLINK-21061).
> Introduce TypedValue to the StateFun request-reply protocol
> ---
>
> Key: FLINK-21171
> URL: https://issues.apache.org/jira/browse/FLINK-21171
> Project: Flink
> Issue Type: New Feature
> Components: Stateful Functions
>Reporter: Tzu-Li (Gordon) Tai
>Assignee: Tzu-Li (Gordon) Tai
>Priority: Major
> Fix For: statefun-3.0.0
>
>
> Currently, all values being passed around via the request-reply protocol, are
> of the Protobuf {{Any}} type. This includes payloads of outgoing messages to
> other functions, and also state values.
> This has a few shortcomings:
> * All user records are strictly required to be modeled and wrapped as a
> Protobuf message - even for simple primitive types. This makes it awkward to
> work with for many common types of messages, for example JSON.
> * For data persisted as state, with each state value being a Protobuf
> {{Any}}, each value would also redundantly store the type urls associated
> with each Protobuf message.
> Instead, we'd like to introduced a {{TypedValue}} construct that replaces
> {{Any}} everywhere in the protocol, for both messages and state values:
> {code}
> message TypedValue {
> string typename = 1;
> bytes value = 2;
> }
> {code}
> The {{typename}} here directly maps to the type concept introduced in
> FLINK-21061.
> For state, we directly write the value bytes of a {{TypedValue}} into state,
> and the {{typename}} is the meta information snapshotted by the state
> serializer (see FLINK-21061).
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
