Galen Warren created FLINK-11055:
------------------------------------
Summary: Allow Queryable State to be transformed on the
TaskManager before being returned to the client
Key: FLINK-11055
URL: https://issues.apache.org/jira/browse/FLINK-11055
Project: Flink
Issue Type: Improvement
Components: Queryable State
Reporter: Galen Warren
Fix For: 1.7.0
The proposal here is to enhance the way Queryable State works to allow for the
state object to be transformed on the TaskManager before being returned to the
client. As an example, if some MapState<K,V> were made queryable, such a
transform might look up a specific key in the map and return its corresponding
value, resulting in only that value being returned to the client instead of the
entire map. This could be useful in cases where the client only wants a portion
of the state and the state is large (this is my use case).
At a high level, I think this could be accomplished by adding an (optional)
serializable Function<TState,TTransformed> into KvStateRequest (and related
classes?) and having that transform be applied in the QueryableStateServer (or
QueryableStateClientProxy?). I expect some additional TypeInformation would
also have to be supplied/used in places. It should be doable in a backwards
compatible way such that if the client does not specify a transform it works
exactly as it does now.
Would there be any interested in a PR for this? This would help me for
something I'm currently working on and I'd be willing to take a crack at it. If
there is interest, I'll be happy to do some more research to come up with a
more concrete proposal.
Thanks for Flink - it's great!
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
