Unable to locate full stderr logs while using Flink Operator

2023-12-07 Thread Edgar H
Hi all! I've just deployed an Apache Beam job using FlinkRunner in k8s and found that the job failed and has the following field: error: >- {"type":"org.apache.flink.util.SerializedThrowable","message":"org.apache.flink.client.program.ProgramInvocationException: The main method caused an

Mounting files into native k8s Flink pods

2023-05-08 Thread Edgar H
Hi all!Running Flink on k8s (native, not using the operator) I was trying to mount Hadoop configuration files into a certain directory within the pods but I can't manage to do so.My existing deployment consists in a Job spec which launches a .sh file that will trigger the flink run-application

ArgoCD unable to process health from k8s FlinkDeployment - stuck in Processing

2022-12-09 Thread Edgar H
Morning all, Recently been testing the Flink k8s operator (flink.apache.org/v1beta1) and although the jobs do startup and run perfectly fine, their status in ArgoCD is not yet as it should be, some details: When describing the flinkdeployment I'm currently trying to test, the follows appears in

Re: Handling environment variables in Flink and within the Flink Operator

2022-08-24 Thread Edgar H
, 91 Parry Street > > Newcastle NSW 2302 > > *P* (02) 4786 0426 Ext 233 > > E francis.con...@switchdin.com > > www.switchdin.com > > > > > On Tue, 23 Aug 2022 at 22:54, Edgar H wrote: > >> Morning all! >> >> Recently we've been trying to migra

Handling environment variables in Flink and within the Flink Operator

2022-08-23 Thread Edgar H
Morning all! Recently we've been trying to migrate from some custom in-house code we've got to manage our k8s cluster with Flink running into the recently released operator. However, we found an issue when parsing environment variables into the arguments which are sent into the jobs - not being