Re: [Kubernetes][1.10.7] XCom extraction fails on KubernetesPodOperator + KubernetesExecutor

Specifying xcom_push along with do_xcom_push seems to work around the issue On Thu, 16 Jan 2020 at 11:04, Daniel Mateus Pires wrote: > Hi there, > > I recently upgraded our set up of Airflow from 1.10.6 to 1.10.7, and I was > surprised to find that the XCom sidecar is not creat

[Kubernetes][1.10.7] XCom extraction fails on KubernetesPodOperator + KubernetesExecutor

Hi there, I recently upgraded our set up of Airflow from 1.10.6 to 1.10.7, and I was surprised to find that the XCom sidecar is not created along the "base" pod using KubernetesPodOperator on top of KubernetesExecutor As a result the KubernetesPodOperators using do_xcom_push=True fail; this was n

KubernetesPodOperator can't use get_logs

Hi there, We use the KubernetesPodOperator and can't turn get_logs to True because of this issue https://issues.apache.org/jira/browse/AIRFLOW-3534, this is a major bug that fails pods Daniel

Re: [Discuss] EMR Step operator/sensor

onable). In our current Dags we resolved this by > >> putting the tasks in a SubDag and changing the on_retry_callback to > >> clear the state of the sub-tasks. But use of SubDags makes navigation in > >> the UI a bit of a pain, so we've planned to merge them into

[Discuss] EMR Step operator/sensor

Hi there! Would it make sense to add an operator that is both the EmrAddStep operator and the step sensor? In a past role we were using Airflow heavily for all things EMR, and I found myself writing an Operator that combined the emr_add_step operator and the sensor, it made the pipelines simpler

Re: [2.0 spring cleaning] Remove the EMR connection type.

In our company we use EMR based operators a lot and it's always been confusing for new users to find the different kinds of EMR clusters as "Connections". Not sure you could just remove the aws_conn_id, because the emr_conn_id doesn't define which AWS account, which region, which profile to use et

Re: Welcoming Xiaodong Deng as a new commiter!

Congrats!! Always happy to see the committers growing Daniel On Thu, 7 Mar 2019 at 13:25, Ash Berlin-Taylor wrote: > Hello Airflowers, > > Please join the Apache Airflow PMC in welcoming our newest committer: > Xiaodong Deng (a.k.a. XD-DENG ). > > XD has been an act

Re: AIRFLOW-3918 Adding SSH authentication to git-sync for Kubernetes

gt; > > On 28 Feb 2019, at 10:24, Daniel Mateus Pires > wrote: > > > > Hi devs! > > > > I'm trying to get some traction on > > https://github.com/apache/airflow/pull/4777, it adds support for SSH > > authentication to private GitHub repositories to

AIRFLOW-3918 Adding SSH authentication to git-sync for Kubernetes

Hi devs! I'm trying to get some traction on https://github.com/apache/airflow/pull/4777, it adds support for SSH authentication to private GitHub repositories to clone Airflow DAGs, the PR passes the tests and it was reviewed once by a core committer. I came back with changes 2 days ago and I'm l