GitHub user pykenny edited a comment on the discussion: Kuberentes integration settings/commands for Breeze environment?
Recent atttempt: - Fix crash issue by turning off some of the resource-consuming programs on host system for more RAM. - Modify [`kubernetes_commands.py`](https://github.com/apache/airflow/blob/12cd06d41366fe40ebd8712945d5201c5b0f19b2/dev/breeze/src/airflow_breeze/commands/kubernetes_commands.py#L605) to copy things under `dags` folder into the image (`files` folder is not mounted onto the pods under this mode) during build stage. Once completing `build-k8s-image` and `upload-k8s-image` commands the cluster will automatically run the migration and restart with latest image. ```dockerfile COPY --chown=airflow:0 dags/ /opt/airflow/dags/ ``` Minimum test example: ```python from datetime import timedelta from airflow import DAG from airflow.providers.cncf.kubernetes.operators.job import KubernetesJobOperator from pendulum import datetime with DAG( dag_id="k8s_job_always_fail", schedule="0 * * * *", start_date=datetime(2022, 12, 8), catchup=False, tags=["example", "k8s", "retry"], ) as dag: failing_task = KubernetesJobOperator( task_id="always_fail_job", namespace="default", image="alpine:latest", cmds=["/bin/sh", "-c", "echo failing... && exit 1"], retries=3, retry_delay=timedelta(seconds=30), wait_until_job_complete=True, deferrable=True, backoff_limit=0, do_xcom_push=False, get_logs=False, ) ``` Result: Failed due to lack of permission (Airflow 3 seems to remove log download utils from UI, so using screenshot instead): <img width="1024" alt="image" src="https://github.com/user-attachments/assets/7fbfce3b-4c04-4471-9541-238de8f5dfe1"; /> ``` ApiException: (403) Reason: Forbidden HTTP response headers: HTTPHeaderDict({'Audit-Id': '31359fbd-cdfe-40a8-88f0-93df023e7630', 'Cache-Control': 'no-cache, private', 'Content-Type': 'application/json', 'X-Content-Type-Options': 'nosniff', 'X-Kubernetes-Pf-Flowschema-Uid': 'dfc53d28-fc9b-41f3-8478-7a0282759537', 'X-Kubernetes-Pf-Prioritylevel-Uid': 'e10ca922-38fe-4420-891d-6ca2f1253406', 'Date': 'Tue, 01 Jul 2025 03:26:03 GMT', 'Content-Length': '321'}) HTTP response body: {"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"jobs.batch is forbidden: User \"system:serviceaccount:airflow:airflow-scheduler\" cannot create resource \"jobs\" in API group \"batch\" in the namespace \"default\"","reason":"Forbidden","details":{"group":"batch","kind":"jobs"},"code":403} ``` GitHub link: https://github.com/apache/airflow/discussions/52131#discussioncomment-13624390 ---- This is an automatically sent email for [email protected]. To unsubscribe, please send an email to: [email protected]
