Hi all,
After an offline talk with Gunnar I review all connectors on branch
release_1.16 (still has virtually all connectors) we can see some of them using
ExecutionConfig..
The basic use is for creating serializers:
TypeInformation.createSerializer(ExecutionConfig config), and since it is a
Flink-core class, I think the easy way is to expose the full ExecutionConfig..
As note on some of them the config is not used on implementations..
For a rapid view (ignoring the use of ExecutionConfig.ClosureCleanerLevel as
its a enum),
- Cassandra connector:
- uses TypeInformation.createSerializer (but builds a new ExecutionConfig
for it)
- Files connector:
- uses TypeInformation.createSerializer
- creates KyroSerializer that use: getDefaultKryoSerializers,
getDefaultKryoSerializerClasses, getRegisteredKryoTypes,
getRegisteredTypesWithKryoSerializerClasses,
getRegisteredTypesWithKryoSerializers
- Hive connector:
- uses TypeInformation.createSerializer (but builds a new ExecutionConfig
for it)
- Jdbc connector:
- uses TypeInformation.createSerializer
- uses isObjectReuseEnable
- Kafka connector:
- uses TypeInformation.createSerializer
- uses getAutoWatermarkInterval
- creates KyroSerializer
- Kinesis connector:
- uses getAutoWatermarkInterval
- Pulsar connector:
- uses TypeInformation.createSerializer
We can create a FLIP to discuss a new ReadOnlyConfig but this will lead us to
discuss lot of new things.
Best regards
On 2023/01/13 14:15:04 Joao Boto wrote:
> Hi flink devs,
>
> I'd like to start a discussion thread for FLIP-287[1].
> This comes from an offline discussion with @Lijie Wang, from FLIP-239[2]
> specially for the sink[3].
>
> Basically to expose the ExecutionConfig and JobId on SinkV2#InitContext.
> This changes are necessary to correct migrate the current sinks to SinkV2
> like JdbcSink, KafkaTableSink and so on, that relies on RuntimeContext
>
> Comments are welcome!
> Thanks,
>
> [1]
> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=240880853
> [2]
> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=217386271
> [3] https://issues.apache.org/jira/browse/FLINK-25421
>
