Re: Is there any way to get the ExecutionConfigurations in Dynamic factory class

2022-04-13 Thread Qingsheng Ren
Hi Anitha, I’m not quite familiar with BigQuery. Is it possible to postpone the operation to SplitEnumerator, in which you could get parallelism by org.apache.flink.api.connector.source.SplitEnumeratorContext#currentParallelism? If you are using SourceFunction you can get access to current

Re: Is there any way to get the ExecutionConfigurations in Dynamic factory class

2022-04-10 Thread Anitha Thankappan
Hi Qingsheng, I am creating BigQuery read sessions in the factory class level. I want to set the number of Streams in the read session using the parallelism set at Execution environment. This is to execute each stream by each executor to achieve parallel execution of streams. Thanks and

Re: Is there any way to get the ExecutionConfigurations in Dynamic factory class

2022-04-08 Thread Qingsheng Ren
Hi Anitha, AFAIK DynamicTableSourceFactory doesn’t expose interface for getting parallelism. Could you elaborate on why you need parallelism in table factory? Maybe we could find other ways to fulfill your requirement. Best regards, Qingsheng > On Apr 7, 2022, at 16:11, Anitha Thankappan

Is there any way to get the ExecutionConfigurations in Dynamic factory class

2022-04-07 Thread Anitha Thankappan
Hi I have developed a BigQuery Flink connector by implementing DynamicTableSourceFactory. I have a requirement to : get the configured parallelism value of StreamExecutionEnvironment in the Factory class. or set the parallelism