Re: KubernetesPodOperator: Invalid arguments were passed to BaseOperator

I have submitted a patch: https://github.com/apache/incubator-airflow/pull/3442 -- Craig On 2018/05/30 19:45:23, Craig Rodrigues wrote: > Oh, OK, I just saw this in example_kubernetes_operator.py: > > try: > # Kubernetes is optional, so not available in vanilla Airflow > # pip

Re: KubernetesPodOperator: Invalid arguments were passed to BaseOperator

In the example it imports BaseOperator as KubernetesPodOperator, when the kubernetes modules can't be found. On Wed, May 30, 2018 at 3:34 PM, Craig Rodrigues wrote: > For this

Re: KubernetesPodOperator: Invalid arguments were passed to BaseOperator

For this particular DeprecationWarning, this problem is not caused by dependencies on kubernetes stuff. On this line: https://github.com/apache/incubator-airflow/blob/master/airflow/contrib/operators/kubernetes_pod_operator.py#L137 The __init__() method for KubernetesPodOperator is calling the

Re: KubernetesPodOperator: Invalid arguments were passed to BaseOperator

For this particular DeprecationWarning, this problem is not caused by dependencies on kubernetes stuff. On this line: https://github.com/apache/incubator-airflow/blob/master/airflow/contrib/operators/kubernetes_pod_operator.py#L137 The __init__() method for KubernetesPodOperator is calling the

Re: KubernetesPodOperator: Invalid arguments were passed to BaseOperator

Hi Taylor, Thanks, I was thinking about something similar like you're suggesting. But I'm not confident if the sys.exit() won't kill the whole Airflow process. For example, if you do an airflow initdb, also the examples will be initialised, and if you don't have kubernetes installed, it will hit

Re: KubernetesPodOperator: Invalid arguments were passed to BaseOperator

I used requests instead of kube as an example, but what do you think about doing something like this? I'm happy to put this into a PR if it would solve the pain point today. import logging try: import requests except ModuleNotFoundError: import sys logging.warning('kube not

Re: KubernetesPodOperator: Invalid arguments were passed to BaseOperator

Hi Craig, This is something that needs to be fixed. I agree with you this is very dirty. In your installation you're not installing the kubernetes stuff, so the KubernetesPodOperator is ignored. We need to figure out how to have example dags that are not compatible with the vanilla installation,

KubernetesPodOperator: Invalid arguments were passed to BaseOperator

I tested master branch by putting the following in my requirements.txt: git+https://github.com/rodrigc/incubator-airflow@master#egg=apache-airflow[celery,crypto,emr,hive,hdfs,ldap,mysql,postgres,redis,slack,s3] and did a pip install -r requirements.txt When I started the airflow webserver, I