Trying to read a file from S3 with flink on kubernetes

2024-07-11 Thread gwenael . lebarzic
Hey guys. I'm trying to read a file from an internal S3 with flink on Kubernetes, but get a strange blocking error. Here is the code : MyFlinkJob.scala : ### package com.example.flink import org.apache.flink.api.common.serialization.SimpleStringSchema import

RE: Problem reading a CSV file with pyflink datastream in k8s with Flink operator

2024-06-18 Thread gwenael . lebarzic
Hello Rob. This workaround works indeed ! Cdt. [Logo Orange] Gwenael Le Barzic De : Robert Young Envoyé : mardi 18 juin 2024 03:54 À : LE BARZIC Gwenael DTSI/SI Cc : user@flink.apache.org Objet : Re: Problem reading a CSV file with pyflink datastream in k8s with

RE: Problem reading a CSV file with pyflink datastream in k8s with Flink operator

2024-06-17 Thread gwenael . lebarzic
Hello everyone. Does someone know how to solve this please ? Cdt. [Logo Orange] Gwenael Le Barzic Ingénieur technique techno BigData Orange/OF/DTSI/SI/DATA-IA/SOLID/CXP Mobile : +33 6 48 70 85 75

Problem reading a CSV file with pyflink datastream in k8s with Flink operator

2024-06-14 Thread gwenael . lebarzic
Hello everyone. I get the following error when trying to read a CSV file with pyflink datastream in a k8s environment using the flink operator. ### File "/opt/myworkdir/myscript.py", line 30, in run_flink_job(myfile) File "/opt/myworkdir/myscript.py", line 21, in run_flink_job

RE: Which base image to use for pyflink on k8s with flink operator ?

2024-06-14 Thread gwenael . lebarzic
To be sure about that, I can see this in the doc : # install PyFlink COPY apache-flink*.tar.gz / RUN pip3 install /apache-flink-libraries*.tar.gz && pip3 install /apache-flink*.tar.gz Is the result the same than this command below : RUN pip install --no-cache-dir -r requirements.txt With

RE: Which base image to use for pyflink on k8s with flink operator ?

2024-06-14 Thread gwenael . lebarzic
Thank you for your answer, Mate o/ Cdt. [Logo Orange] Gwenael Le Barzic Ingénieur technique techno BigData Orange/OF/DTSI/SI/DATA-IA/SOLID/CXP Mobile : +33 6 48 70 85 75

Which base image to use for pyflink on k8s with flink operator ?

2024-06-14 Thread gwenael . lebarzic
Hello everyone. I contact you because I'm encountereing some strange difficulties with pyflink on Kubernetes using the flink operator. So, first thing first, I was wondering which base image should I use for my python image that I will then deploy on my Kubernetes cluster ? Can I use flink