[
https://issues.apache.org/jira/browse/FLINK-39988?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gabor Somogyi updated FLINK-39988:
----------------------------------
Description:
FLINK-39637 added predicate push-down when querying state using the _savepoint_
connector. However, it only works with primitive types, such as {_}string{_},
{_}long{_}, etc. It can be improved by adding support for composite types:
* POJOs
* Java records
* Flink Tuple
The general idea is the following:
* user describes key column as _ROW(field1, field2, ..., fieldN)_ in the DDL +
provides {_}value-class{_}, if needed
* _SavepointFilterTranslator_ knows state key type and knows the filtering
which user is applying, e.g. _WHERE k =_ _ROW("Bob", 20)._ It can construct an
object _obj_ of key type, filling its fields with data supplied by user in the
{_}ROW("Bob", 20){_}{_}.{_} Then it can take _obj.hashcode()_ and derive the
exact _InputSplit_ of the key (same approach we do for simple types). We can
then scan only relevant splits.
was:
FLINK-39637 added predicate push-down when querying state using the _savepoint_
connector. However, it only works with primitive types, such as {_}string{_},
{_}long{_}, etc. It can be improved by adding support for composite types:
* POJOs
* Java records
* Flink Tuple
* Flink Row
* Avro records
The general idea is the following:
* user describes key column as _ROW(field1, field2, ..., fieldN)_ in the DDL +
provides {_}value-class{_}, if needed
* _SavepointFilterTranslator_ knows state key type and knows the filtering
which user is applying, e.g. _WHERE k =_ _ROW("Bob", 20)._ It can construct an
object _obj_ of key type, filling its fields with data supplied by user in the
{_}ROW("Bob", 20){_}{_}.{_} Then it can take _obj.hashcode()_ and derive the
exact _InputSplit_ of the key (same approach we do for simple types). We can
then scan only relevant splits.
> Support composite types in filter push-down for savepoint Table API connector
> -----------------------------------------------------------------------------
>
> Key: FLINK-39988
> URL: https://issues.apache.org/jira/browse/FLINK-39988
> Project: Flink
> Issue Type: Improvement
> Components: API / State Processor
> Affects Versions: 2.4.0
> Reporter: Ilya Soin
> Assignee: Ilya Soin
> Priority: Minor
>
> FLINK-39637 added predicate push-down when querying state using the
> _savepoint_ connector. However, it only works with primitive types, such as
> {_}string{_}, {_}long{_}, etc. It can be improved by adding support for
> composite types:
> * POJOs
> * Java records
> * Flink Tuple
> The general idea is the following:
> * user describes key column as _ROW(field1, field2, ..., fieldN)_ in the DDL
> + provides {_}value-class{_}, if needed
> * _SavepointFilterTranslator_ knows state key type and knows the filtering
> which user is applying, e.g. _WHERE k =_ _ROW("Bob", 20)._ It can construct
> an object _obj_ of key type, filling its fields with data supplied by user in
> the {_}ROW("Bob", 20){_}{_}.{_} Then it can take _obj.hashcode()_ and derive
> the exact _InputSplit_ of the key (same approach we do for simple types). We
> can then scan only relevant splits.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
