Re: Relation between Flink Configuration and TableEnv

[Paul Lam]

Hi Caizhi,

Thanks a lot for you clarification! Now I understand the design of TableConfig. 

Best,
Paul Lam

> 2021年9月24日 15:40，Caizhi Weng  写道：
> 
> Hi!
> 
> TableConfig is for configurations related to the Table and SQL API, 
> especially the configurations in OptimizerConfigOptions and 
> ExecutionConfigOptions.
> 
> By Flink Configuration I guess you mean the configuration in Configuration. 
> Sadly, as you say, it can be configured only once when creating the execution 
> environment, and only a few configuration can be modified with the 
> StreamExecutionEnvironment#configure method. This configuration contains 
> datastream and execution related configurations.
> 
> I bet the devs separate the configurations because they belong to different 
> modules, but yes sometimes users will set their configuration in the wrong 
> place.
> 
> Paul Lam mailto:paullin3...@gmail.com>> 于2021年9月24日周五 
> 上午12:26写道：
> Sorry, I mean the relation between Flink Configuration and TableConfig, not 
> TableEnv.
> 
> Best,
> Paul Lam
> 
> Paul Lam mailto:paullin3...@gmail.com>> 于2021年9月24日周五 
> 上午12:24写道：
> Hi all,
> 
> Currently, Flink creates a new Configuration in TableConfig of  
> StreamTableEnvironment, and synchronizes options in it back to the 
> Configuration of the underlying StreamExecutionEnvironment afterward. 
> However, only "relevant" options are set back [1], others are dropped 
> silently. That blocks users from overriding options like 
> DeploymentOptions.ATTACHED from TableConfig. So I'm wondering the reason why 
> not merge all options from TableConfig into the Configuration, and what's the 
> relation between Configuration and TableEnv?
> 
> Thanks a lot!
> 
> [1] 
> https://github.com/apache/flink/blob/36ff71f5ff63a140acc634dd1d98b2bb47a76ba5/flink-streaming-java/src/main/java/org/apache/flink/streaming/api/environment/StreamExecutionEnvironment.java#L904
>  
> 
> 
> Best,
> Paul Lam
> 

Re: Relation between Flink Configuration and TableEnv

[Caizhi Weng]

Hi!

TableConfig is for configurations related to the Table and SQL API,
especially the configurations in OptimizerConfigOptions and
ExecutionConfigOptions.

By Flink Configuration I guess you mean the configuration in Configuration.
Sadly, as you say, it can be configured only once when creating the
execution environment, and only a few configuration can be modified with
the StreamExecutionEnvironment#configure method. This configuration
contains datastream and execution related configurations.

I bet the devs separate the configurations because they belong to different
modules, but yes sometimes users will set their configuration in the wrong
place.

Paul Lam  于2021年9月24日周五 上午12:26写道：

> Sorry, I mean the relation between Flink Configuration and TableConfig,
> not TableEnv.
>
> Best,
> Paul Lam
>
> Paul Lam  于2021年9月24日周五 上午12:24写道：
>
>> Hi all,
>>
>> Currently, Flink creates a new Configuration in TableConfig of
>> StreamTableEnvironment, and synchronizes options in it back to the
>> Configuration of the underlying StreamExecutionEnvironment afterward.
>> However, only "relevant" options are set back [1], others are dropped
>> silently. That blocks users from overriding options like
>> DeploymentOptions.ATTACHED from TableConfig. So I'm wondering the reason
>> why not merge all options from TableConfig into the Configuration, and
>> what's the relation between Configuration and TableEnv?
>>
>> Thanks a lot!
>>
>> [1]
>> https://github.com/apache/flink/blob/36ff71f5ff63a140acc634dd1d98b2bb47a76ba5/flink-streaming-java/src/main/java/org/apache/flink/streaming/api/environment/StreamExecutionEnvironment.java#L904
>>
>> Best,
>> Paul Lam
>>
>>

Re: Relation between Flink Configuration and TableEnv

[Paul Lam]

Sorry, I mean the relation between Flink Configuration and TableConfig, not
TableEnv.

Best,
Paul Lam

Paul Lam  于2021年9月24日周五 上午12:24写道：

> Hi all,
>
> Currently, Flink creates a new Configuration in TableConfig of
> StreamTableEnvironment, and synchronizes options in it back to the
> Configuration of the underlying StreamExecutionEnvironment afterward.
> However, only "relevant" options are set back [1], others are dropped
> silently. That blocks users from overriding options like
> DeploymentOptions.ATTACHED from TableConfig. So I'm wondering the reason
> why not merge all options from TableConfig into the Configuration, and
> what's the relation between Configuration and TableEnv?
>
> Thanks a lot!
>
> [1]
> https://github.com/apache/flink/blob/36ff71f5ff63a140acc634dd1d98b2bb47a76ba5/flink-streaming-java/src/main/java/org/apache/flink/streaming/api/environment/StreamExecutionEnvironment.java#L904
>
> Best,
> Paul Lam
>
>