zoucao created FLINK-23777:
------------------------------
Summary: Support dynamic parameters for sink table in multiple
connectors
Key: FLINK-23777
URL: https://issues.apache.org/jira/browse/FLINK-23777
Project: Flink
Issue Type: Improvement
Components: Connectors / ElasticSearch, Connectors / Kafka
Reporter: zoucao
For now, metadata has been supported in SQL, it is simple, elegant and easy to
use.
But it is mostly used to get meta info from source table for now, I think we
could use it more to support dynamic parameters for sink table.
For example,
* In kafka sink, we can use metadata to achieve dynamic partitions，let record
choose it's partition by calculation logic.
{code:java}
CREATE TABLE KafkaSinkTable (
`user_id` BIGINT,
`item_id` BIGINT,
`behavior` STRING,
`partition` INT METADATA FROM 'partition'
) WITH (
'connector' = 'kafka'
)
{code}
* In Elasticsearch sink, we also can use metadata to support dynamic index,
like
{code:java}
CREATE TABLE ESSinkTable (
index VARCHAR METADATA FROM 'index',
log_ts VARCHAR
) with (
'connector' = 'elasticsearch-7'
)
{code}
which is more elegant than treating a specific field as partial index like
`myusers-\{log_ts|yyyy-MM-dd}`.
If there are some opinions or other dynamic parameters worth to do, it could
be propsed and dicussed here.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
