[jira] [Created] (AIRFLOW-6652) Set is_delete_operator_pod=True by default in KubernetesPodOperator()
Mathew Wicks created AIRFLOW-6652: - Summary: Set is_delete_operator_pod=True by default in KubernetesPodOperator() Key: AIRFLOW-6652 URL: https://issues.apache.org/jira/browse/AIRFLOW-6652 Project: Apache Airflow Issue Type: Improvement Components: operators Affects Versions: 1.10.7 Reporter: Mathew Wicks As we are not far from a major release (Airflow 2.0) could we consider changing the default behaviour of the *is_delete_operator_pod* argument in *KubernetesPodOperator*() to *True, r*ather than False? For most usecases, it is not desirable that the pod continues running independent of the airflow task, and having an unsafe default is bad practice. Thoughts? -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (AIRFLOW-6827) PROPOSAL: Implement a run_pool concept for DagRuns
Mathew Wicks created AIRFLOW-6827: - Summary: PROPOSAL: Implement a run_pool concept for DagRuns Key: AIRFLOW-6827 URL: https://issues.apache.org/jira/browse/AIRFLOW-6827 Project: Apache Airflow Issue Type: New Feature Components: DagRun, scheduler Affects Versions: 1.10.9 Reporter: Mathew Wicks Currently we have a concept of resource pools for tasks, where there can be at most N instances of a task running at once. However, no such concept exists for DagRuns. For example, you might have a requirement that two DAGs (with different dag_id) cannot run at the same time, so you might define a run_pool for these DAGs with size 1. Meaning only 1 DagRun may be in the RUNNING state at a time. I am interested in hearing everyone's thoughts. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (AIRFLOW-5033) Cannot install apache-airflow[kerberos] on Python 3
Mathew Wicks created AIRFLOW-5033: - Summary: Cannot install apache-airflow[kerberos] on Python 3 Key: AIRFLOW-5033 URL: https://issues.apache.org/jira/browse/AIRFLOW-5033 Project: Apache Airflow Issue Type: Bug Components: build Affects Versions: 1.10.3 Reporter: Mathew Wicks Currently you cannot install apache-airflow[kerberos] with Python 3. *Steps to reproduce: (For Ubuntu)* # Ensure you have the required build dependencies, _*sudo apt install libsasl2-dev libkrb5-dev*_ # Create a new conda environment. _*conda create -n airflow_36 python=3.6*_ # Activate the environment: _*conda activate airflow_36*_ # Attempt to install _*pip install apache-airflow[kerberos]*_ # **fails** The error is caused by trying to install python-krbV as a depednancy of snakebite[kerberos], as python-krbV does not support Python 3. -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Commented] (AIRFLOW-5033) Cannot install apache-airflow[kerberos] on Python 3
[ https://issues.apache.org/jira/browse/AIRFLOW-5033?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16892213#comment-16892213 ] Mathew Wicks commented on AIRFLOW-5033: --- [~jackjack10] Yes, with _*pip install apache-airflow[kerberos]==1.10.4rc3*_ the same error occurs. > Cannot install apache-airflow[kerberos] on Python 3 > --- > > Key: AIRFLOW-5033 > URL: https://issues.apache.org/jira/browse/AIRFLOW-5033 > Project: Apache Airflow > Issue Type: Bug > Components: build >Affects Versions: 1.10.3 >Reporter: Mathew Wicks >Priority: Major > > Currently you cannot install apache-airflow[kerberos] with Python 3. > *Steps to reproduce: (For Ubuntu)* > # Ensure you have the required build dependencies, _*sudo apt install > libsasl2-dev libkrb5-dev*_ > # Create a new conda environment. _*conda create -n airflow_36 python=3.6*_ > # Activate the environment: _*conda activate airflow_36*_ > # Attempt to install _*pip install apache-airflow[kerberos]*_ > # **fails** > The error is caused by trying to install python-krbV as a depednancy of > snakebite[kerberos], as python-krbV does not support Python 3. -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Commented] (AIRFLOW-5033) Cannot install apache-airflow[kerberos] on Python 3
[ https://issues.apache.org/jira/browse/AIRFLOW-5033?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16892452#comment-16892452 ] Mathew Wicks commented on AIRFLOW-5033: --- [~jackjack10], the issue doesn't happen in CI because we manually exclude it: [https://github.com/apache/airflow/blob/1.10.3/setup.py#L269] > Cannot install apache-airflow[kerberos] on Python 3 > --- > > Key: AIRFLOW-5033 > URL: https://issues.apache.org/jira/browse/AIRFLOW-5033 > Project: Apache Airflow > Issue Type: Bug > Components: build >Affects Versions: 1.10.3 >Reporter: Mathew Wicks >Priority: Major > > Currently you cannot install apache-airflow[kerberos] with Python 3. > *Steps to reproduce: (For Ubuntu)* > # Ensure you have the required build dependencies, _*sudo apt install > libsasl2-dev libkrb5-dev*_ > # Create a new conda environment. _*conda create -n airflow_36 python=3.6*_ > # Activate the environment: _*conda activate airflow_36*_ > # Attempt to install _*pip install apache-airflow[kerberos]*_ > # **fails** > The error is caused by trying to install python-krbV as a depednancy of > snakebite[kerberos], as python-krbV does not support Python 3. -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Created] (AIRFLOW-5261) RBAC can't use LDAP -- 'python-ldap package is missing from setup.py
Mathew Wicks created AIRFLOW-5261:
-
Summary: RBAC can't use LDAP -- 'python-ldap package is missing
from setup.py
Key: AIRFLOW-5261
URL: https://issues.apache.org/jira/browse/AIRFLOW-5261
Project: Apache Airflow
Issue Type: Bug
Components: webserver
Affects Versions: 1.10.4
Reporter: Mathew Wicks
Currently with airflow 1.10.4, if you setup a clean python 3 environment, and
install airflow with:
{code:python}
pip install airflow[ldap]
{code}
You will be missing a required package for LDAP to work with the new RBAC,
giving:
{code:java}
Traceback (most recent call last):
File
"PYTHON_HOME/lib/python3.6/site-packages/flask_appbuilder/security/manager.py",
line 725, in auth_user_ldap
ldap.set_option(ldap.OPT_X_TLS_CACERTFILE, self.auth_ldap_tls_cacertfile)
ModuleNotFoundError: No module named 'ldap'
{code}
This is because the 'ldap' tag in setup.py doesn't include the 'python-ldap'
package, which is needed for 'import ldap' to succeed.
The change would need to be made here:
[https://github.com/apache/airflow/blob/1.10.4/setup.py#L229]
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Updated] (AIRFLOW-5261) RBAC can't use LDAP -- 'python-ldap' package is missing from setup.py
[
https://issues.apache.org/jira/browse/AIRFLOW-5261?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mathew Wicks updated AIRFLOW-5261:
--
Summary: RBAC can't use LDAP -- 'python-ldap' package is missing from
setup.py (was: RBAC can't use LDAP -- 'python-ldap package is missing from
setup.py)
> RBAC can't use LDAP -- 'python-ldap' package is missing from setup.py
> -
>
> Key: AIRFLOW-5261
> URL: https://issues.apache.org/jira/browse/AIRFLOW-5261
> Project: Apache Airflow
> Issue Type: Bug
> Components: webserver
>Affects Versions: 1.10.4
>Reporter: Mathew Wicks
>Priority: Major
>
> Currently with airflow 1.10.4, if you setup a clean python 3 environment, and
> install airflow with:
> {code:python}
> pip install airflow[ldap]
> {code}
> You will be missing a required package for LDAP to work with the new RBAC,
> giving:
> {code:java}
> Traceback (most recent call last):
> File
> "PYTHON_HOME/lib/python3.6/site-packages/flask_appbuilder/security/manager.py",
> line 725, in auth_user_ldap
> ldap.set_option(ldap.OPT_X_TLS_CACERTFILE, self.auth_ldap_tls_cacertfile)
> ModuleNotFoundError: No module named 'ldap'
> {code}
> This is because the 'ldap' tag in setup.py doesn't include the 'python-ldap'
> package, which is needed for 'import ldap' to succeed.
> The change would need to be made here:
> [https://github.com/apache/airflow/blob/1.10.4/setup.py#L229]
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
