[
https://issues.apache.org/jira/browse/FLINK-15985?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Danny Chen updated FLINK-15985:
-------------------------------
Fix Version/s: (was: 1.11.0)
> offload runtime params from DDL to table hints in DML/queries
> -------------------------------------------------------------
>
> Key: FLINK-15985
> URL: https://issues.apache.org/jira/browse/FLINK-15985
> Project: Flink
> Issue Type: New Feature
> Components: Table SQL / API
> Reporter: Bowen Li
> Assignee: Danny Chen
> Priority: Major
>
> background:
> Currently Flink DDL mixes three types of params all together:
> * External data’s metadata: defines what the data looks like (schema), where
> it is (location/url), how it should be accessed (username/pwd)
> * Source/sink runtime params: defines how and usually how fast Flink
> source/sink reads/writes data, not affecting the results
> * Kafka “sink-partitioner”
> * Elastic “bulk-flush.interval/max-size/...”
> * Semantics params: defines aspects like how much data Flink reads/writes,
> how the result will look like
> * Kafka “startup-mode”, “offset”
> * Watermark, timestamp column
>
> Problems of the current mix-up: Flink cannot leverage catalogs and external
> system metadata alone to run queries with all the non-metadata params
> involved in DDL. E.g. when we add a catalog for Confluent Schema Registry,
> the expected user experience should be that Flink users just configure the
> catalog with url and usr/pwd, and should be able to run queries immediately;
> however, that’s not the case right now because users still have to use DDL to
> define a bunch params like “startup-mode”, “offset”, timestamp column, etc,
> along with the schema redundantly. We’ve heard many user complaints on this.
>
> cc [~ykt836] [~lirui] [~lzljs3620320] [~jark] [~twalthr] [~dwysakowicz]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
