Hi George.
We saw the same problem, running Apache Flink 1.19 and 1.20 images. The cause
is that Flink image provides a JRE and you need JDK to build/install PyFlink.
And, oddly enough, I think it was only on ARM64 images. Amd64 was OK, I think.
So, Mac M1, M2, M3…
Our Docker file for building our custom image has some extra steps, to install
JDK, pip-install and then cleanup.
RUN apt-get update -y \
&& apt-get upgrade -y \
&& apt-get install --no-install-recommends -y \
…
openjdk-17-jdk-headless \
ENV JAVA_HOME=/usr/lib/jvm/java-17-openjdk-arm64
After PIP install our requirement.txt, we do cleanup.
# Cleanup JDK
RUN apt-get remove -y openjdk-17-jdk-headless
ENV JAVA_HOME=/opt/java/openjdk
Hope it helps.
Nix.
From: George <[email protected]>
Date: Wednesday, May 14, 2025 at 10:23 AM
To: [email protected] <[email protected]>
Subject: Python based User defined function on Flink 1.19.1
Hi all
My Flink Jobmanager and Taskmanager is based on the community 1.19.1 on the
Java 17 image.
I've extended this by installing Python 3.11 and python3.11-venv.
created my venv environment, activated it,
when trying to pip install apache-flink. I first got an error saying
/opt/java/openjdk/include does not exist... so created it.
Next attempt ended in the below error.
Please advise.
G
```
venv) flink@jobmanager:/sql/prompush$ pip install apache-flink
Collecting apache-flink
Downloading apache-flink-2.0.0.tar.gz (1.5 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.5/1.5 MB 1.9 MB/s eta 0:00:00
Installing build dependencies ... done
Getting requirements to build wheel ... done
Preparing metadata (pyproject.toml) ... done
Collecting python-dateutil<3,>=2.8.0
Using cached python_dateutil-2.9.0.post0-py2.py3-none-any.whl (229 kB)
Collecting requests>=2.26.0
Using cached requests-2.32.3-py3-none-any.whl (64 kB)
Collecting pyarrow>=5.0.0
Downloading pyarrow-20.0.0-cp311-cp311-manylinux_2_28_aarch64.whl (40.7 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 40.7/40.7 MB 415.8 kB/s eta
0:00:00
Collecting pytz>=2018.3
Using cached pytz-2025.2-py2.py3-none-any.whl (509 kB)
Collecting fastavro!=1.8.0,>=1.1.0
Using cached
fastavro-1.10.0-cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
(3.3 MB)
Collecting pemja==0.4.1
Downloading pemja-0.4.1.tar.gz (50 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 50.7/50.7 KB 600.0 kB/s eta
0:00:00
Installing build dependencies ... done
Getting requirements to build wheel ... error
error: subprocess-exited-with-error
× Getting requirements to build wheel did not run successfully.
│ exit code: 255
╰─> [1 lines of output]
jni.h should be in '/opt/java/openjdk/include/jni.h' but doesn't exist.
Please check you've installed the JDK properly.
[end of output]
note: This error originates from a subprocess, and is likely not a problem
with pip.
error: subprocess-exited-with-error
× Getting requirements to build wheel did not run successfully.
│ exit code: 255
╰─> See above for output.
note: This error originates from a subprocess, and is likely not a problem with
pip.
```
--
You have the obligation to inform one honestly of the risk, and as a person
you are committed to educate yourself to the total risk in any activity!
Once informed & totally aware of the risk,
every fool has the right to kill or injure themselves as they see fit!
