Re: Support for ConfigMap for Runtime Arguments in Flink Kubernetes Operator

2024-03-04 Thread Surendra Singh Lilhore
Hi Arjun, I have raised a Jira for this case and attached a patch: https://issues.apache.org/jira/browse/FLINK-34565 -Surendra On Wed, Feb 21, 2024 at 12:48 AM Surendra Singh Lilhore < surendralilh...@apache.org> wrote: > Hi Arjun, > > Yes, direct support for external conf

Re: Support for ConfigMap for Runtime Arguments in Flink Kubernetes Operator

2024-02-20 Thread Surendra Singh Lilhore
Hi Arjun, Yes, direct support for external configuration files within Flink ConfigMaps is somewhat restricted. The current method involves simply copying two local files from the operator. Please check : FlinkConfMountDecorator#getLocalLogConfFiles()

Re: Flink 1.18.0 Checkpoints on cancelled jobs

2023-12-10 Thread Surendra Singh Lilhore
ilesystem > state.checkpoints.dir: {{ .Values.jobManager.checkpointUrl }} > state.savepoints.dir: {{ .Values.jobManager.savepointUrl }} > > The actual checkpoint is there in s3 after cancellation. Can someone point > me to the code where the checkpoint history is maintained? > > Thanks, > Eth

Re: Continuous errors with Azure ABFSS

2023-09-28 Thread Surendra Singh Lilhore
Hi Alexis, Could you please check the TaskManager log for any exceptions? Thanks Surendra On Thu, Sep 28, 2023 at 7:06 AM Alexis Sarda-Espinosa < sarda.espin...@gmail.com> wrote: > Hello, > > We are using ABFSS for RocksDB's backend as well as the storage dir > required for Kubernetes HA. In

Re: MSI Auth to Azure Storage Account with Flink Apache Operator not working

2023-05-17 Thread Surendra Singh Lilhore
> value: flink-azure-fs-hadoop-1.16.1.jar > > > > > > This azure managed service identity configuration should be added to the > flink docs. I couldn’t find anywhere that the > fs.azure.account.oauth.provider.type had to be set as > *org.apache.flink.fs.

Re: MSI Auth to Azure Storage Account with Flink Apache Operator not working

2023-05-16 Thread Surendra Singh Lilhore
Hi DEROCCO, Flink uses shaded jars for the Hadoop Azure Storage plugin, so in order to correct the ClassNotFoundException, you need to adjust the configuration. Please configure the MSITokenProvider as shown below. fs.azure.account.oauth.provider.type: