[ https://issues.apache.org/jira/browse/AIRFLOW-6166?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Cabell Maddux updated AIRFLOW-6166: ----------------------------------- Description: Hi, I'm relatively new to airflow and am testing out Airflow running with the Kubernetes executor locally on Minikube based off of the CI configs in the repo ([https://github.com/apache/airflow/tree/master/scripts/ci/kubernetes/app]). Was able to get everything up and running smoothly, however thought default behavior for xcom when no `do_xcom_push` argument is provided to a `KubernetesPodOperator` was a bit confusing. If a user doesn't explicitly create a result.json in the pod default `XCOM_MOUNT_PATH` as part of the task pod, obviously the task will fail ([https://github.com/apache/airflow/blob/a36cfe049a2c5948b24fde7a878fe19cabede5f7/airflow/kubernetes/pod_launcher.py#L214)] ``` cat: can't open '/airflow/xcom/return.json': No such file or directory ``` Rather than the user having to explicitly create result.json as part of their task, maybe more aligned with other operators to me to create `\{XCOM_MOUNT_PATH}/result.json` when creating the xcom sidecar - [https://github.com/apache/airflow/blob/a36cfe049a2c5948b24fde7a878fe19cabede5f7/airflow/kubernetes/pod_generator.py#L36]. Otherwise, maybe would be nice to document that user of KubernetesPodOperator should be careful to explicitly deal with `do_xcom_push` to avoid a bit of confusion. Happy to open a PR if this is something that you think needs to be dealt with either in code or via docs. Also found some older tests associated with this ([https://github.com/apache/airflow/blob/36f3bfb0619cc78698280f6ec3bc985f84e58343/tests/contrib/minikube/test_kubernetes_pod_operator.py#L321)|https://github.com/apache/airflow/blob/36f3bfb0619cc78698280f6ec3bc985f84e58343/tests/contrib/minikube/test_kubernetes_pod_operator.py#L321-] If already in the works, no worries. Thanks! was: Hi, I'm relatively new to airflow and am testing out Airflow running with the Kubernetes executor locally on Minikube based off of the CI configs in the repo ([https://github.com/apache/airflow/tree/master/scripts/ci/kubernetes/app]). Was able to get everything up and running smoothly, however thought default behavior for xcom when no `do_xcom_push` argument is provided to a `KubernetesPodOperator` was a bit confusing. If a user doesn't explicitly create a result.json in the pod default `XCOM_MOUNT_PATH` as part of the task pod, obviously the task will fail ([https://github.com/apache/airflow/blob/a36cfe049a2c5948b24fde7a878fe19cabede5f7/airflow/kubernetes/pod_launcher.py#L214)] ``` cat: can't open '/airflow/xcom/return.json': No such file or directory ``` Rather than the user having to explicitly create result.json as part of their task, maybe more aligned with other operators to me to create `\{XCOM_MOUNT_PATH}/result.json` when creating the xcom sidecar - https://github.com/apache/airflow/blob/a36cfe049a2c5948b24fde7a878fe19cabede5f7/airflow/kubernetes/pod_generator.py#L36. Otherwise, maybe would be nice to document that user of KubernetesPodOperator should be careful to explicitly deal with `do_xcom_push` to avoid a bit of confusion. > KubernetesPodOperator resolve default xcom read behavior or improve > associated documentation > -------------------------------------------------------------------------------------------- > > Key: AIRFLOW-6166 > URL: https://issues.apache.org/jira/browse/AIRFLOW-6166 > Project: Apache Airflow > Issue Type: Improvement > Components: documentation, executor-kubernetes, xcom > Affects Versions: 1.10.7 > Reporter: Cabell Maddux > Assignee: Daniel Imberman > Priority: Minor > > Hi, > I'm relatively new to airflow and am testing out Airflow running with the > Kubernetes executor locally on Minikube based off of the CI configs in the > repo > ([https://github.com/apache/airflow/tree/master/scripts/ci/kubernetes/app]). > Was able to get everything up and running smoothly, however thought default > behavior for xcom when no `do_xcom_push` argument is provided to a > `KubernetesPodOperator` was a bit confusing. > If a user doesn't explicitly create a result.json in the pod default > `XCOM_MOUNT_PATH` as part of the task pod, obviously the task will fail > ([https://github.com/apache/airflow/blob/a36cfe049a2c5948b24fde7a878fe19cabede5f7/airflow/kubernetes/pod_launcher.py#L214)] > ``` > cat: can't open '/airflow/xcom/return.json': No such file or directory > ``` > Rather than the user having to explicitly create result.json as part of their > task, maybe more aligned with other operators to me to create > `\{XCOM_MOUNT_PATH}/result.json` when creating the xcom sidecar - > [https://github.com/apache/airflow/blob/a36cfe049a2c5948b24fde7a878fe19cabede5f7/airflow/kubernetes/pod_generator.py#L36]. > Otherwise, maybe would be nice to document that user of KubernetesPodOperator > should be careful to explicitly deal with `do_xcom_push` to avoid a bit of > confusion. > Happy to open a PR if this is something that you think needs to be dealt with > either in code or via docs. Also found some older tests associated with this > ([https://github.com/apache/airflow/blob/36f3bfb0619cc78698280f6ec3bc985f84e58343/tests/contrib/minikube/test_kubernetes_pod_operator.py#L321)|https://github.com/apache/airflow/blob/36f3bfb0619cc78698280f6ec3bc985f84e58343/tests/contrib/minikube/test_kubernetes_pod_operator.py#L321-] > If already in the works, no worries. > Thanks! -- This message was sent by Atlassian Jira (v8.3.4#803005)