Hey, I ran into some issues with PyFlink on Kubernetes myself a while ago. Blogged about it here, perhaps it's useful:
https://www.decodable.co/blog/getting-started-with-pyflink-on-kubernetes Best, --Gunnar Am Fr., 14. Juni 2024 um 20:58 Uhr schrieb Mate Czagany <czmat...@gmail.com >: > Hi, > > You can refer to the example Dockerfile in the Flink docs [1] and you can > also take a look at the example found in the Flink Kubernetes Operator repo > [2]. The second Dockerfile won't work because it is missing all Flink > libraries if I am not mistaken. > > Regards, > Mate > > <gwenael.lebar...@orange.com> ezt írta (időpont: 2024. jún. 14., P, > 17:22): > >> Hello everyone. >> >> >> >> I contact you because I’m encountereing some strange difficulties with >> pyflink on Kubernetes using the flink operator. >> >> So, first thing first, I was wondering which base image should I use for >> my python image that I will then deploy on my Kubernetes cluster ? >> >> >> >> Can I use flink official image 1.18.1 ? >> >> FROM flink:1.18.1 >> >> >> >> RUN mkdir -p /opt/myworkdir >> >> WORKDIR /opt/myworkdir >> >> >> >> RUN apt-get update && \ >> >> apt-get install -y python3 python3-pip python3-dev && \ >> >> apt-get clean && \ >> >> rm -rf /var/lib/apt/lists/* >> >> >> >> RUN alias python=python3 && ln -s /usr/bin/python3 /usr/bin/python >> >> >> >> COPY requirements.txt . >> >> RUN pip install --no-cache-dir -r requirements.txt >> >> >> >> COPY src . >> >> RUN chown -R root:root * >> >> RUN chmod -R 755 * >> >> >> >> Or can I use a base python 3.11 image with installation of java 11 ? >> >> FROM dockerproxy.repos.tech.orange/python:3.11-slim >> >> >> >> RUN mkdir -p /opt/myworkdir >> >> WORKDIR /opt/myworkdir >> >> >> >> RUN apt-get update && apt-get install -y wget tar >> >> >> >> RUN wget -O /tmp/openjdk11.tar.gz >> https://github.com/AdoptOpenJDK/openjdk11-upstream-binaries/releases/download/jdk-11.0.12%2B7/OpenJDK11U-jdk_x64_linux_11.0.12_7.tar.gz >> \ >> >> && mkdir -p /opt/java/openjdk \ >> >> && tar -xvf /tmp/openjdk11.tar.gz -C /opt/java/openjdk >> --strip-components=1 \ >> >> && rm -rf /tmp/openjdk11.tar.gz >> >> >> >> ENV JAVA_HOME=/opt/java/openjdk >> >> ENV PATH="$JAVA_HOME/bin:$PATH" >> >> >> >> COPY requirements.txt . >> >> RUN pip install --no-cache-dir -r requirements.txt >> >> >> >> COPY src . >> >> COPY pyflink*.yaml . >> >> RUN chown -R root:root * >> >> RUN chmod -R 755 * >> >> >> >> Thank you in advance for you answer ! >> >> >> >> Cdt. >> >> *Gwenael Le Barzic * >> >> >> >> Orange Restricted >> >> >> >> Orange Restricted >> >> ____________________________________________________________________________________________________________ >> Ce message et ses pieces jointes peuvent contenir des informations >> confidentielles ou privilegiees et ne doivent donc >> pas etre diffuses, exploites ou copies sans autorisation. Si vous avez recu >> ce message par erreur, veuillez le signaler >> a l'expediteur et le detruire ainsi que les pieces jointes. Les messages >> electroniques etant susceptibles d'alteration, >> Orange decline toute responsabilite si ce message a ete altere, deforme ou >> falsifie. Merci. >> >> This message and its attachments may contain confidential or privileged >> information that may be protected by law; >> they should not be distributed, used or copied without authorisation. >> If you have received this email in error, please notify the sender and >> delete this message and its attachments. >> As emails may be altered, Orange is not liable for messages that have been >> modified, changed or falsified. >> Thank you. >> >>
