[jira] [Resolved] (AIRFLOW-4800) GKE operators call GKEClusterHook with wrong (old) arguments

2019-06-17 Thread Jarek Potiuk (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-4800?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jarek Potiuk resolved AIRFLOW-4800. --- Resolution: Fixed Fix Version/s: 2.0.0 1.10.4 > GKE operators

[jira] [Resolved] (AIRFLOW-4765) Recursion error in DataProcPigOperator

2019-06-17 Thread Jarek Potiuk (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-4765?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jarek Potiuk resolved AIRFLOW-4765. --- Resolution: Fixed Fix Version/s: 2.0.0 > Recursion error in DataProcPigOperator >

[jira] [Resolved] (AIRFLOW-4807) Make GCS operators Pylint compatible

2019-06-19 Thread Jarek Potiuk (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-4807?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jarek Potiuk resolved AIRFLOW-4807. --- Resolution: Fixed Fix Version/s: 2.0.0 > Make GCS operators Pylint compatible >

[jira] [Resolved] (AIRFLOW-4784) Make GCP operators Pylint compatible

2019-06-19 Thread Jarek Potiuk (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-4784?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jarek Potiuk resolved AIRFLOW-4784. --- Resolution: Fixed Fix Version/s: 2.0.0 > Make GCP operators Pylint compatible >

[jira] [Closed] (AIRFLOW-4518) GKEClusterCreateOperator and GkeClusterDeleteOperator do not properly instantiate GKEClusterHook

2019-06-19 Thread Jarek Potiuk (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-4518?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jarek Potiuk closed AIRFLOW-4518. - Resolution: Fixed > GKEClusterCreateOperator and GkeClusterDeleteOperator do not properly >

[jira] [Resolved] (AIRFLOW-4817) Remove deprecated methods from tests

2019-06-19 Thread Jarek Potiuk (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-4817?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jarek Potiuk resolved AIRFLOW-4817. --- Resolution: Fixed Fix Version/s: 2.0.0 > Remove deprecated methods from tests >

[jira] [Resolved] (AIRFLOW-4818) Remove valid file from todo list

2019-06-19 Thread Jarek Potiuk (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-4818?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jarek Potiuk resolved AIRFLOW-4818. --- Resolution: Fixed Fix Version/s: 2.0.0 > Remove valid file from todo list >

[jira] [Resolved] (AIRFLOW-4820) Fix unnecessary-pass errors in pylint

2019-06-19 Thread Jarek Potiuk (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-4820?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jarek Potiuk resolved AIRFLOW-4820. --- Resolution: Fixed Fix Version/s: 2.0.0 > Fix unnecessary-pass errors in pylint >

[jira] [Resolved] (AIRFLOW-4819) Fix singleton-comparison errors in pylint

2019-06-19 Thread Jarek Potiuk (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-4819?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jarek Potiuk resolved AIRFLOW-4819. --- Resolution: Fixed Fix Version/s: 2.0.0 > Fix singleton-comparison errors in pylint

[jira] [Resolved] (AIRFLOW-4812) Add batch images annotation

2019-06-19 Thread Jarek Potiuk (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-4812?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jarek Potiuk resolved AIRFLOW-4812. --- Resolution: Fixed Fix Version/s: 1.10.4 > Add batch images annotation >

[jira] [Commented] (AIRFLOW-3019) Remove tox from the test environment

2019-06-23 Thread Jarek Potiuk (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-3019?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16870694#comment-16870694 ] Jarek Potiuk commented on AIRFLOW-3019: --- Correct. I just linked thoe two. > Remove tox from the

[jira] [Comment Edited] (AIRFLOW-3019) Remove tox from the test environment

2019-06-23 Thread Jarek Potiuk (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-3019?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16870694#comment-16870694 ] Jarek Potiuk edited comment on AIRFLOW-3019 at 6/23/19 11:11 PM: -

[jira] [Comment Edited] (AIRFLOW-3019) Remove tox from the test environment

2019-06-23 Thread Jarek Potiuk (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-3019?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16870694#comment-16870694 ] Jarek Potiuk edited comment on AIRFLOW-3019 at 6/23/19 11:11 PM: -

[jira] [Resolved] (AIRFLOW-4116) Support for Main/CI images in single Dockerfile

2019-06-23 Thread Jarek Potiuk (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-4116?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jarek Potiuk resolved AIRFLOW-4116. --- Resolution: Fixed Fix Version/s: 2.0.0 > Support for Main/CI images in single

[jira] [Resolved] (AIRFLOW-4115) Docker file for Main airflow image is multi-staging and has multiple layers

2019-06-23 Thread Jarek Potiuk (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-4115?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jarek Potiuk resolved AIRFLOW-4115. --- Resolution: Fixed Fix Version/s: 2.0.0 > Docker file for Main airflow image is

[jira] [Created] (AIRFLOW-4841) Doc check in master started to failed with cyclic import problem suddenly

2019-06-23 Thread Jarek Potiuk (JIRA)
Jarek Potiuk created AIRFLOW-4841: - Summary: Doc check in master started to failed with cyclic import problem suddenly Key: AIRFLOW-4841 URL: https://issues.apache.org/jira/browse/AIRFLOW-4841

[jira] [Updated] (AIRFLOW-3718) Make official CI Airflow Image multi-layered

2019-06-23 Thread Jarek Potiuk (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-3718?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jarek Potiuk updated AIRFLOW-3718: -- Summary: Make official CI Airflow Image multi-layered (was: Make official Airflow Image

[jira] [Resolved] (AIRFLOW-4839) Fix pylint errors regarding superfluous-parens

2019-06-23 Thread Jarek Potiuk (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-4839?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jarek Potiuk resolved AIRFLOW-4839. --- Resolution: Fixed Fix Version/s: 2.0.0 > Fix pylint errors regarding

[jira] [Resolved] (AIRFLOW-4837) Fix pylint errors regarding ungrouped imports

2019-06-23 Thread Jarek Potiuk (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-4837?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jarek Potiuk resolved AIRFLOW-4837. --- Resolution: Fixed Fix Version/s: 2.0.0 > Fix pylint errors regarding ungrouped

[jira] [Commented] (AIRFLOW-4841) Doc check in master started to failed with cyclic import problem suddenly

2019-06-24 Thread Jarek Potiuk (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-4841?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16870831#comment-16870831 ] Jarek Potiuk commented on AIRFLOW-4841: --- Unfortunately this patch is from a different commit I

[jira] [Updated] (AIRFLOW-4841) Doc check in master started to failed with cyclic import problem suddenly

2019-06-24 Thread Jarek Potiuk (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-4841?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jarek Potiuk updated AIRFLOW-4841: -- Description: Doc check started to fail suddenly after merging 

[jira] [Resolved] (AIRFLOW-4782) Make GCP hooks Pylint compatible

2019-06-24 Thread Jarek Potiuk (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-4782?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jarek Potiuk resolved AIRFLOW-4782. --- Resolution: Fixed Fix Version/s: 2.0.0 > Make GCP hooks Pylint compatible >

[jira] [Commented] (AIRFLOW-4364) Integrate Pylint

2019-06-10 Thread Jarek Potiuk (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-4364?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16859864#comment-16859864 ] Jarek Potiuk commented on AIRFLOW-4364: --- [~BasPH]-> we discussed with [~kamil.bregula] that maybe

[jira] [Created] (AIRFLOW-4757) Selectively disable pylint missing-docstrings for tests

2019-06-10 Thread Jarek Potiuk (JIRA)
Jarek Potiuk created AIRFLOW-4757: - Summary: Selectively disable pylint missing-docstrings for tests Key: AIRFLOW-4757 URL: https://issues.apache.org/jira/browse/AIRFLOW-4757 Project: Apache Airflow

[jira] [Resolved] (AIRFLOW-4757) Selectively disable some pylint checks for tests

2019-06-10 Thread Jarek Potiuk (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-4757?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jarek Potiuk resolved AIRFLOW-4757. --- Resolution: Fixed Fix Version/s: 2.0.0 > Selectively disable some pylint checks for

[jira] [Resolved] (AIRFLOW-4795) Upgrade alembic

2019-06-17 Thread Jarek Potiuk (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-4795?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jarek Potiuk resolved AIRFLOW-4795. --- Resolution: Fixed Fix Version/s: 2.0.0 1.10.4 > Upgrade alembic

[jira] [Resolved] (AIRFLOW-4799) tests using bash operator fail flakily because jinja2 rendering of environment variables

2019-06-17 Thread Jarek Potiuk (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-4799?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jarek Potiuk resolved AIRFLOW-4799. --- Resolution: Fixed Fix Version/s: 2.0.0 1.10.4 > tests using bash

[jira] [Updated] (AIRFLOW-3341) Example for function that returns a DAG object

2019-05-09 Thread Jarek Potiuk (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-3341?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jarek Potiuk updated AIRFLOW-3341: -- Fix Version/s: 1.10.4 > Example for function that returns a DAG object >

[jira] [Resolved] (AIRFLOW-3341) Example for function that returns a DAG object

2019-05-09 Thread Jarek Potiuk (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-3341?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jarek Potiuk resolved AIRFLOW-3341. --- Resolution: Fixed Fix Version/s: 2.0.0 > Example for function that returns a DAG

[jira] [Work started] (AIRFLOW-4511) Travis CI builds fail randomly - usually when pulling Docker images or pip install

2019-05-13 Thread Jarek Potiuk (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-4511?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on AIRFLOW-4511 started by Jarek Potiuk. - > Travis CI builds fail randomly - usually when pulling Docker images or

[jira] [Created] (AIRFLOW-4511) Travis CI builds fail randomly - usually when pulling Docker images or pip install

2019-05-13 Thread Jarek Potiuk (JIRA)
Jarek Potiuk created AIRFLOW-4511: - Summary: Travis CI builds fail randomly - usually when pulling Docker images or pip install Key: AIRFLOW-4511 URL: https://issues.apache.org/jira/browse/AIRFLOW-4511

[jira] [Commented] (AIRFLOW-4457) Enhance Task logs by providing the task context

2019-05-14 Thread Jarek Potiuk (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-4457?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16839139#comment-16839139 ] Jarek Potiuk commented on AIRFLOW-4457: --- I removed it from v1-10-test as it breaks py27 builds. I

[jira] [Resolved] (AIRFLOW-4503) Support fully pig options, such as -useHCatalog, -x tez,...

2019-05-13 Thread Jarek Potiuk (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-4503?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jarek Potiuk resolved AIRFLOW-4503. --- Resolution: Fixed Fix Version/s: 2.0.0 1.10.4 > Support fully

[jira] [Resolved] (AIRFLOW-4457) Enhance Task logs by providing the task context

2019-05-13 Thread Jarek Potiuk (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-4457?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jarek Potiuk resolved AIRFLOW-4457. --- Resolution: Fixed Fix Version/s: 2.0.0 > Enhance Task logs by providing the task

[jira] [Resolved] (AIRFLOW-4511) Travis CI builds fail randomly - usually when pulling Docker images or pip install

2019-05-14 Thread Jarek Potiuk (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-4511?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jarek Potiuk resolved AIRFLOW-4511. --- Resolution: Fixed Fix Version/s: 2.0.0 1.10.4 > Travis CI builds

[jira] [Updated] (AIRFLOW-4457) Enhance Task logs by providing the task context

2019-05-13 Thread Jarek Potiuk (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-4457?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jarek Potiuk updated AIRFLOW-4457: -- Fix Version/s: 1.10.4 > Enhance Task logs by providing the task context >

[jira] [Commented] (AIRFLOW-4545) Github Enterprise OAuth auto registration fails due to bug in Flask-AppBuilder Lib.

2019-05-21 Thread Jarek Potiuk (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-4545?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16844568#comment-16844568 ] Jarek Potiuk commented on AIRFLOW-4545: --- It was fixed by cherry-picking

[jira] [Resolved] (AIRFLOW-4545) Github Enterprise OAuth auto registration fails due to bug in Flask-AppBuilder Lib.

2019-05-21 Thread Jarek Potiuk (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-4545?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jarek Potiuk resolved AIRFLOW-4545. --- Resolution: Fixed Fix Version/s: 1.10.4 > Github Enterprise OAuth auto registration

[jira] [Resolved] (AIRFLOW-4089) pytz.utc has no attribute name

2019-05-23 Thread Jarek Potiuk (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-4089?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jarek Potiuk resolved AIRFLOW-4089. --- Resolution: Fixed Fix Version/s: 2.0.0 1.10.4 > pytz.utc has no

[jira] [Resolved] (AIRFLOW-4554) run_unit_tests.sh uses sudo unconditionally

2019-05-23 Thread Jarek Potiuk (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-4554?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jarek Potiuk resolved AIRFLOW-4554. --- Resolution: Fixed Fix Version/s: 2.0.0 1.10.4 >

[jira] [Resolved] (AIRFLOW-4557) The get_sqlproxy_runner() method of the CloudSqlDatabaseHook requires you to use the google_cloud_default_connection

2019-05-23 Thread Jarek Potiuk (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-4557?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jarek Potiuk resolved AIRFLOW-4557. --- Resolution: Fixed Fix Version/s: 2.0.0 1.10.4 > The

[jira] [Reopened] (AIRFLOW-1684) Branching based on XCOM variable

2019-05-19 Thread Jarek Potiuk (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-1684?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jarek Potiuk reopened AIRFLOW-1684: --- > Branching based on XCOM variable > > > Key:

[jira] [Resolved] (AIRFLOW-1684) Branching based on XCOM variable

2019-05-19 Thread Jarek Potiuk (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-1684?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jarek Potiuk resolved AIRFLOW-1684. --- Resolution: Fixed > Branching based on XCOM variable > >

[jira] [Created] (AIRFLOW-4542) Replace os.makedirs usage with pathlib

2019-05-19 Thread Jarek Potiuk (JIRA)
Jarek Potiuk created AIRFLOW-4542: - Summary: Replace os.makedirs usage with pathlib Key: AIRFLOW-4542 URL: https://issues.apache.org/jira/browse/AIRFLOW-4542 Project: Apache Airflow Issue

[jira] [Updated] (AIRFLOW-4541) Replace mkdirs usage with pathlib

2019-05-19 Thread Jarek Potiuk (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-4541?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jarek Potiuk updated AIRFLOW-4541: -- Description: _makedirs is used in 'airlfow.utils.file.mkdirs'  - it could be replaced with

[jira] [Resolved] (AIRFLOW-4537) Cleanup the mkdir_p function

2019-05-19 Thread Jarek Potiuk (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-4537?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jarek Potiuk resolved AIRFLOW-4537. --- Resolution: Fixed Fix Version/s: 2.0.0 > Cleanup the mkdir_p function >

[jira] [Created] (AIRFLOW-4541) Replace mkdirs usage with pathlib

2019-05-19 Thread Jarek Potiuk (JIRA)
Jarek Potiuk created AIRFLOW-4541: - Summary: Replace mkdirs usage with pathlib Key: AIRFLOW-4541 URL: https://issues.apache.org/jira/browse/AIRFLOW-4541 Project: Apache Airflow Issue Type:

[jira] [Resolved] (AIRFLOW-4532) Optimise iteration in deepcopy

2019-05-19 Thread Jarek Potiuk (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-4532?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jarek Potiuk resolved AIRFLOW-4532. --- Resolution: Fixed Fix Version/s: 2.0.0 > Optimise iteration in deepcopy >

[jira] [Updated] (AIRFLOW-1684) Branching based on XCOM variable

2019-05-19 Thread Jarek Potiuk (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-1684?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jarek Potiuk updated AIRFLOW-1684: -- Fix Version/s: 1.10.4 > Branching based on XCOM variable >

[jira] [Resolved] (AIRFLOW-4519) Optimise operators collection type conversions in views

2019-05-20 Thread Jarek Potiuk (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-4519?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jarek Potiuk resolved AIRFLOW-4519. --- Resolution: Fixed Fix Version/s: 2.0.0 1.10.4 > Optimise

[jira] [Commented] (AIRFLOW-4528) DataProcOperators do not stop underlying jobs on timeout

2019-05-20 Thread Jarek Potiuk (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-4528?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16844296#comment-16844296 ] Jarek Potiuk commented on AIRFLOW-4528: --- Hey [~AlexisBRENON_ayl] - it's merged to 2.0.0 now but

[jira] [Comment Edited] (AIRFLOW-4528) DataProcOperators do not stop underlying jobs on timeout

2019-05-20 Thread Jarek Potiuk (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-4528?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16844296#comment-16844296 ] Jarek Potiuk edited comment on AIRFLOW-4528 at 5/20/19 9:00 PM: Hey

[jira] [Resolved] (AIRFLOW-4546) Fix version conflict in gcp_api extras

2019-05-20 Thread Jarek Potiuk (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-4546?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jarek Potiuk resolved AIRFLOW-4546. --- Resolution: Fixed Fix Version/s: 2.0.0 1.10.4 > Fix version

[jira] [Updated] (AIRFLOW-4528) DataProcOperators do not stop underlying jobs on timeout

2019-05-20 Thread Jarek Potiuk (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-4528?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jarek Potiuk updated AIRFLOW-4528: -- Fix Version/s: 1.10.4 > DataProcOperators do not stop underlying jobs on timeout >

[jira] [Updated] (AIRFLOW-3359) Add CMEK as a disk encryption option to the Dataproc operator

2019-05-20 Thread Jarek Potiuk (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-3359?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jarek Potiuk updated AIRFLOW-3359: -- Fix Version/s: 1.10.4 > Add CMEK as a disk encryption option to the Dataproc operator >

[jira] [Updated] (AIRFLOW-4528) DataProcOperators do not stop underlying jobs on timeout

2019-05-20 Thread Jarek Potiuk (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-4528?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jarek Potiuk updated AIRFLOW-4528: -- Fix Version/s: 2.0.0 > DataProcOperators do not stop underlying jobs on timeout >

[jira] [Updated] (AIRFLOW-4401) multiprocessing.Queue.empty() is unreliable

2019-04-28 Thread Jarek Potiuk (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-4401?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jarek Potiuk updated AIRFLOW-4401: -- Description: After discussing with [~ash] and [~BasPH] potential reasons for flakiness of

[jira] [Updated] (AIRFLOW-4401) multiprocessing.Queue.empty() is unreliable

2019-04-29 Thread Jarek Potiuk (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-4401?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jarek Potiuk updated AIRFLOW-4401: -- Description: After discussing with [~ash] and [~BasPH] potential reasons for flakiness of

[jira] [Resolved] (AIRFLOW-3143) Support auto-zone in DataprocClusterCreateOperator

2019-04-29 Thread Jarek Potiuk (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-3143?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jarek Potiuk resolved AIRFLOW-3143. --- Resolution: Fixed Fix Version/s: 2.0.0 1.10.4 > Support

[jira] [Updated] (AIRFLOW-4401) multiprocessing.Queue.empty() is unreliable

2019-04-29 Thread Jarek Potiuk (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-4401?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jarek Potiuk updated AIRFLOW-4401: -- Description: After discussing with [~ash] and [~BasPH] potential reasons for flakiness of

[jira] [Commented] (AIRFLOW-4401) multiprocessing.Queue.empty() is unreliable

2019-04-29 Thread Jarek Potiuk (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-4401?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16828944#comment-16828944 ] Jarek Potiuk commented on AIRFLOW-4401: --- [~ash] and [~BasPH] : Both solutions candidates are now

[jira] [Resolved] (AIRFLOW-4397) Add GCSUploadSessionCompleteSensor

2019-04-27 Thread Jarek Potiuk (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-4397?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jarek Potiuk resolved AIRFLOW-4397. --- Resolution: Fixed Fix Version/s: 2.0.0 > Add GCSUploadSessionCompleteSensor >

[jira] [Updated] (AIRFLOW-4168) Create Google Cloud Video Intelligence Operators

2019-04-27 Thread Jarek Potiuk (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-4168?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jarek Potiuk updated AIRFLOW-4168: -- Fix Version/s: 2.0.0 1.10.4 > Create Google Cloud Video Intelligence

[jira] [Resolved] (AIRFLOW-4168) Create Google Cloud Video Intelligence Operators

2019-04-27 Thread Jarek Potiuk (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-4168?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jarek Potiuk resolved AIRFLOW-4168. --- Resolution: Fixed > Create Google Cloud Video Intelligence Operators >

[jira] [Resolved] (AIRFLOW-3720) GoogleCloudStorageToS3Operator - incorrect folder compare

2019-04-27 Thread Jarek Potiuk (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-3720?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jarek Potiuk resolved AIRFLOW-3720. --- Resolution: Fixed > GoogleCloudStorageToS3Operator - incorrect folder compare >

[jira] [Updated] (AIRFLOW-3720) GoogleCloudStorageToS3Operator - incorrect folder compare

2019-04-27 Thread Jarek Potiuk (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-3720?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jarek Potiuk updated AIRFLOW-3720: -- Fix Version/s: 1.10.4 > GoogleCloudStorageToS3Operator - incorrect folder compare >

[jira] [Updated] (AIRFLOW-4397) Add GCSUploadSessionCompleteSensor

2019-04-27 Thread Jarek Potiuk (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-4397?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jarek Potiuk updated AIRFLOW-4397: -- Fix Version/s: 1.10.4 > Add GCSUploadSessionCompleteSensor >

[jira] [Resolved] (AIRFLOW-4401) multiprocessing.Queue.empty() is unreliable

2019-04-30 Thread Jarek Potiuk (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-4401?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jarek Potiuk resolved AIRFLOW-4401. --- Resolution: Fixed Fix Version/s: 2.0.0 > multiprocessing.Queue.empty() is

[jira] [Updated] (AIRFLOW-4358) Decouple running actual tasks from test_jobs.py

2019-04-30 Thread Jarek Potiuk (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-4358?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jarek Potiuk updated AIRFLOW-4358: -- Fix Version/s: 2.0.0 > Decouple running actual tasks from test_jobs.py >

[jira] [Resolved] (AIRFLOW-4382) Limited Parallelism test is flaky

2019-04-30 Thread Jarek Potiuk (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-4382?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jarek Potiuk resolved AIRFLOW-4382. --- Resolution: Fixed Fix Version/s: 2.0.0 Resolved by AIRFLOW-4401 > Limited

[jira] [Commented] (AIRFLOW-4424) Scheduler does not terminate after num_runs when executor is KubernetesExecutor

2019-04-30 Thread Jarek Potiuk (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-4424?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16830162#comment-16830162 ] Jarek Potiuk commented on AIRFLOW-4424: --- It's likely that the issue is related to a problem

[jira] [Commented] (AIRFLOW-4440) Avoid terminating processes using shared resources

2019-04-30 Thread Jarek Potiuk (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-4440?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16830207#comment-16830207 ] Jarek Potiuk commented on AIRFLOW-4440: --- CC: [~ash] > Avoid terminating processes using shared

[jira] [Created] (AIRFLOW-4440) Avoid terminating processes using shared resources

2019-04-30 Thread Jarek Potiuk (JIRA)
Jarek Potiuk created AIRFLOW-4440: - Summary: Avoid terminating processes using shared resources Key: AIRFLOW-4440 URL: https://issues.apache.org/jira/browse/AIRFLOW-4440 Project: Apache Airflow

[jira] [Resolved] (AIRFLOW-4381) Use get_direct_relative_ids to get task relatives

2019-05-06 Thread Jarek Potiuk (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-4381?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jarek Potiuk resolved AIRFLOW-4381. --- Resolution: Fixed Fix Version/s: 2.0.0 1.10.4 > Use

[jira] [Updated] (AIRFLOW-865) Configure FTP connection mode

2019-05-07 Thread Jarek Potiuk (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-865?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jarek Potiuk updated AIRFLOW-865: - Fix Version/s: 1.10.4 > Configure FTP connection mode > - > >

[jira] [Resolved] (AIRFLOW-4467) Add dataproc_jars to templated fields in Dataproc operators

2019-05-07 Thread Jarek Potiuk (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-4467?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jarek Potiuk resolved AIRFLOW-4467. --- Resolution: Fixed Fix Version/s: 2.0.0 > Add dataproc_jars to templated fields in

[jira] [Updated] (AIRFLOW-4136) Fix overwrite of key_file by constructor in SSHHook

2019-05-07 Thread Jarek Potiuk (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-4136?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jarek Potiuk updated AIRFLOW-4136: -- Fix Version/s: 1.10.4 > Fix overwrite of key_file by constructor in SSHHook >

[jira] [Commented] (AIRFLOW-4358) Decouple running actual tasks from test_jobs.py

2019-04-19 Thread Jarek Potiuk (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-4358?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16822049#comment-16822049 ] Jarek Potiuk commented on AIRFLOW-4358: --- I would love to take a look after I finalise the

[jira] [Commented] (AIRFLOW-2910) models.Connection cannot use https

2019-05-04 Thread Jarek Potiuk (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-2910?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16833032#comment-16833032 ] Jarek Potiuk commented on AIRFLOW-2910: --- I think we should improve it and possibly in

[jira] [Resolved] (AIRFLOW-4206) Remove explicit inheritance of object

2019-05-06 Thread Jarek Potiuk (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-4206?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jarek Potiuk resolved AIRFLOW-4206. --- Resolution: Fixed > Remove explicit inheritance of object >

[jira] [Resolved] (AIRFLOW-1501) Google Cloud Storage delete operator

2019-05-03 Thread Jarek Potiuk (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-1501?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jarek Potiuk resolved AIRFLOW-1501. --- Resolution: Fixed Fix Version/s: 2.0.0 1.10.4 > Google Cloud

[jira] [Reopened] (AIRFLOW-1501) Google Cloud Storage delete operator

2019-05-03 Thread Jarek Potiuk (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-1501?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jarek Potiuk reopened AIRFLOW-1501: --- > Google Cloud Storage delete operator > > >

[jira] [Commented] (AIRFLOW-4401) multiprocessing.Queue.empty() is unreliable

2019-04-28 Thread Jarek Potiuk (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-4401?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16828045#comment-16828045 ] Jarek Potiuk commented on AIRFLOW-4401: --- And we have now a competing (simpler) implementation 

[jira] [Updated] (AIRFLOW-4382) Limited Parallelism test is flaky

2019-04-28 Thread Jarek Potiuk (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-4382?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jarek Potiuk updated AIRFLOW-4382: -- Description: -The SFTPOperator tests seems to fail many times.- Limited parallelism test is

[jira] [Commented] (AIRFLOW-4401) multiprocessing.Queue.empty() is unreliable

2019-04-28 Thread Jarek Potiuk (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-4401?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16828027#comment-16828027 ] Jarek Potiuk commented on AIRFLOW-4401: --- [~BasPH] [~ash] -> I think the new PR 

[jira] [Updated] (AIRFLOW-4382) Limited Parallelism test is flaky

2019-04-28 Thread Jarek Potiuk (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-4382?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jarek Potiuk updated AIRFLOW-4382: -- Description: -The SFTPOperator tests seems to fail many times.- Limited parallelism test is

[jira] [Updated] (AIRFLOW-4382) Limited Parallelism test is flaky

2019-04-28 Thread Jarek Potiuk (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-4382?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jarek Potiuk updated AIRFLOW-4382: -- Summary: Limited Parallelism test is flaky (was: SFTPOperator tests are flaky) > Limited

[jira] [Commented] (AIRFLOW-4401) multiprocessing.Queue.empty() is unreliable

2019-04-28 Thread Jarek Potiuk (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-4401?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16828780#comment-16828780 ] Jarek Potiuk commented on AIRFLOW-4401: --- I had to also add some shutdowns to clean-up the extra

[jira] [Updated] (AIRFLOW-2903) Default owner should be "airflow" and not "Airflow"

2019-05-04 Thread Jarek Potiuk (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-2903?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jarek Potiuk updated AIRFLOW-2903: -- Fix Version/s: 1.10.4 > Default owner should be "airflow" and not "Airflow" >

[jira] [Resolved] (AIRFLOW-4198) Remove all try import py2 version, except import py3 version imports

2019-05-05 Thread Jarek Potiuk (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-4198?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jarek Potiuk resolved AIRFLOW-4198. --- Resolution: Fixed > Remove all try import py2 version, except import py3 version imports >

[jira] [Resolved] (AIRFLOW-4207) Replace __metaclass__ attribute by metaclass as a class keyword argument

2019-05-05 Thread Jarek Potiuk (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-4207?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jarek Potiuk resolved AIRFLOW-4207. --- Resolution: Fixed > Replace __metaclass__ attribute by metaclass as a class keyword

[jira] [Resolved] (AIRFLOW-3626) Unable to Trigger Zipped Dags through Api

2019-05-05 Thread Jarek Potiuk (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-3626?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jarek Potiuk resolved AIRFLOW-3626. --- Resolution: Fixed > Unable to Trigger Zipped Dags through Api >

[jira] [Updated] (AIRFLOW-3626) Unable to Trigger Zipped Dags through Api

2019-05-05 Thread Jarek Potiuk (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-3626?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jarek Potiuk updated AIRFLOW-3626: -- Fix Version/s: 2.0.0 1.10.4 > Unable to Trigger Zipped Dags through Api >

[jira] [Updated] (AIRFLOW-4468) Add max_overflow parameter for configuration of SQLAlchemy connection pool

2019-05-08 Thread Jarek Potiuk (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-4468?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jarek Potiuk updated AIRFLOW-4468: -- Fix Version/s: 2.0.0 > Add max_overflow parameter for configuration of SQLAlchemy connection

[jira] [Resolved] (AIRFLOW-4468) Add max_overflow parameter for configuration of SQLAlchemy connection pool

2019-05-08 Thread Jarek Potiuk (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-4468?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jarek Potiuk resolved AIRFLOW-4468. --- Resolution: Fixed > Add max_overflow parameter for configuration of SQLAlchemy connection

[jira] [Resolved] (AIRFLOW-4876) Make CoreTest test cases rerunnable

2019-07-05 Thread Jarek Potiuk (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-4876?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jarek Potiuk resolved AIRFLOW-4876. --- Resolution: Fixed Fix Version/s: 2.0.0 > Make CoreTest test cases rerunnable >

[jira] [Resolved] (AIRFLOW-4900) Incompatible version of Werkzeug after Flask upgrade

2019-07-05 Thread Jarek Potiuk (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-4900?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jarek Potiuk resolved AIRFLOW-4900. --- Resolution: Fixed Fix Version/s: 2.0.0 > Incompatible version of Werkzeug after

[jira] [Resolved] (AIRFLOW-4859) Extend list of pylint good-names

2019-06-27 Thread Jarek Potiuk (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-4859?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jarek Potiuk resolved AIRFLOW-4859. --- Resolution: Fixed Fix Version/s: 2.0.0 > Extend list of pylint good-names >

[jira] [Updated] (AIRFLOW-4859) Extend list of pylint good-names

2019-06-27 Thread Jarek Potiuk (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-4859?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jarek Potiuk updated AIRFLOW-4859: -- Issue Type: Sub-task (was: Improvement) Parent: AIRFLOW-4364 > Extend list of

[jira] [Resolved] (AIRFLOW-4919) Add 'dataproc_properties' parameter to templated fields of DataProcJobBaseOperator child classes

2019-07-10 Thread Jarek Potiuk (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-4919?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jarek Potiuk resolved AIRFLOW-4919. --- Resolution: Fixed Fix Version/s: 1.10.4 > Add 'dataproc_properties' parameter to

<    1   2   3   4   5   6   7   8   9   10   >