[jira] [Commented] (AIRFLOW-6824) EMRAddStepsOperator does not work well with multi-step XCom

2020-02-25 Thread ASF subversion and git services (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6824?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17045157#comment-17045157 ] ASF subversion and git services commented on AIRFLOW-6824: -- Commit

[jira] [Resolved] (AIRFLOW-6824) EMRAddStepsOperator does not work well with multi-step XCom

2020-02-25 Thread Kamil Bregula (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6824?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kamil Bregula resolved AIRFLOW-6824. Fix Version/s: 2.0.0 Resolution: Fixed > EMRAddStepsOperator does not work well

[jira] [Commented] (AIRFLOW-6824) EMRAddStepsOperator does not work well with multi-step XCom

2020-02-25 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6824?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17045156#comment-17045156 ] ASF GitHub Bot commented on AIRFLOW-6824: - mik-laj commented on pull request #7443:

[GitHub] [airflow] mik-laj merged pull request #7443: [AIRFLOW-6824] EMRAddStepsOperator problem with multi-step XCom

2020-02-25 Thread GitBox
mik-laj merged pull request #7443: [AIRFLOW-6824] EMRAddStepsOperator problem with multi-step XCom URL: https://github.com/apache/airflow/pull/7443 This is an automated message from the Apache Git Service. To respond to the

[jira] [Commented] (AIRFLOW-6822) AWS hooks dont always cache the boto3 client

2020-02-25 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17045153#comment-17045153 ] ASF GitHub Bot commented on AIRFLOW-6822: - baolsen commented on pull request #7541:

[GitHub] [airflow] baolsen opened a new pull request #7541: [AIRFLOW-6822] AWS hooks should cache boto3 client

2020-02-25 Thread GitBox
baolsen opened a new pull request #7541: [AIRFLOW-6822] AWS hooks should cache boto3 client URL: https://github.com/apache/airflow/pull/7541 --- Issue link: WILL BE INSERTED BY [boring-cyborg](https://github.com/kaxil/boring-cyborg) Make sure to mark the boxes below before

[GitHub] [airflow] codecov-io edited a comment on issue #7476: [AIRFLOW-6856][depends on AIRFLOW-6862][WIP] Bulk fetch paused_dag_ids

2020-02-25 Thread GitBox
codecov-io edited a comment on issue #7476: [AIRFLOW-6856][depends on AIRFLOW-6862][WIP] Bulk fetch paused_dag_ids URL: https://github.com/apache/airflow/pull/7476#issuecomment-589383726 # [Codecov](https://codecov.io/gh/apache/airflow/pull/7476?src=pr=h1) Report > Merging

[GitHub] [airflow] codecov-io edited a comment on issue #7476: [AIRFLOW-6856][depends on AIRFLOW-6862][WIP] Bulk fetch paused_dag_ids

2020-02-25 Thread GitBox
codecov-io edited a comment on issue #7476: [AIRFLOW-6856][depends on AIRFLOW-6862][WIP] Bulk fetch paused_dag_ids URL: https://github.com/apache/airflow/pull/7476#issuecomment-589383726 # [Codecov](https://codecov.io/gh/apache/airflow/pull/7476?src=pr=h1) Report > Merging

[GitHub] [airflow] mik-laj commented on a change in pull request #7537: [AIRFLOW-6454] And script to benchmark scheduler dag-run time

2020-02-25 Thread GitBox
mik-laj commented on a change in pull request #7537: [AIRFLOW-6454] And script to benchmark scheduler dag-run time URL: https://github.com/apache/airflow/pull/7537#discussion_r384258420 ## File path: scripts/perf/scheduler_dag_execution_timing.py ## @@ -0,0 +1,221 @@

[GitHub] [airflow] codecov-io commented on issue #7539: [AIRFLOW-6919] Make Breeze DAG-test friedly

2020-02-25 Thread GitBox
codecov-io commented on issue #7539: [AIRFLOW-6919] Make Breeze DAG-test friedly URL: https://github.com/apache/airflow/pull/7539#issuecomment-591192171 # [Codecov](https://codecov.io/gh/apache/airflow/pull/7539?src=pr=h1) Report > Merging

[GitHub] [airflow] codecov-io edited a comment on issue #7492: [AIRFLOW-6871] optimize tree view for large DAGs

2020-02-25 Thread GitBox
codecov-io edited a comment on issue #7492: [AIRFLOW-6871] optimize tree view for large DAGs URL: https://github.com/apache/airflow/pull/7492#issuecomment-589921201 # [Codecov](https://codecov.io/gh/apache/airflow/pull/7492?src=pr=h1) Report > Merging

[GitHub] [airflow] kaxil merged pull request #7540: [AIRFLOW-XXXX] Add instructions on using templating in Bash Script

2020-02-25 Thread GitBox
kaxil merged pull request #7540: [AIRFLOW-] Add instructions on using templating in Bash Script URL: https://github.com/apache/airflow/pull/7540 This is an automated message from the Apache Git Service. To respond to

[GitHub] [airflow] codecov-io edited a comment on issue #6342: [AIRFLOW-5662] fix incorrect naming for scheduler used slot metric

2020-02-25 Thread GitBox
codecov-io edited a comment on issue #6342: [AIRFLOW-5662] fix incorrect naming for scheduler used slot metric URL: https://github.com/apache/airflow/pull/6342#issuecomment-547121627 # [Codecov](https://codecov.io/gh/apache/airflow/pull/6342?src=pr=h1) Report > Merging

[GitHub] [airflow] kaxil opened a new pull request #7540: [AIRFLOW-XXXX] Add instructions on using templating in Bash Script

2020-02-25 Thread GitBox
kaxil opened a new pull request #7540: [AIRFLOW-] Add instructions on using templating in Bash Script URL: https://github.com/apache/airflow/pull/7540 Add instructions on using Jinja templating in Bash Script passed to BashOperator --- Issue link: WILL BE INSERTED BY

[GitHub] [airflow] ashb commented on a change in pull request #7492: [AIRFLOW-6871] optimize tree view for large DAGs

2020-02-25 Thread GitBox
ashb commented on a change in pull request #7492: [AIRFLOW-6871] optimize tree view for large DAGs URL: https://github.com/apache/airflow/pull/7492#discussion_r384191672 ## File path: airflow/www/views.py ## @@ -1374,90 +1376,115 @@ def tree(self): .all()

[jira] [Updated] (AIRFLOW-6389) add config for 'allow_multi_scheduler_instances' default True

2020-02-25 Thread t oo (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6389?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] t oo updated AIRFLOW-6389: -- Description: right now common deployment pattern with blue/green build is: 1. on EC2 1, start scheduler 2.

[jira] [Closed] (AIRFLOW-4502) new cli command - task_states_for_dag_run

2020-02-25 Thread t oo (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-4502?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] t oo closed AIRFLOW-4502. - Resolution: Fixed > new cli command - task_states_for_dag_run > - > >

[jira] [Commented] (AIRFLOW-6919) Make Breeeze more Dag-test friendly

2020-02-25 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6919?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17045000#comment-17045000 ] ASF GitHub Bot commented on AIRFLOW-6919: - potiuk commented on pull request #7539:

[jira] [Created] (AIRFLOW-6920) AIRFLOW Feature Parity with LUIGI & CONTROLM

2020-02-25 Thread t oo (Jira)
t oo created AIRFLOW-6920: - Summary: AIRFLOW Feature Parity with LUIGI & CONTROLM Key: AIRFLOW-6920 URL: https://issues.apache.org/jira/browse/AIRFLOW-6920 Project: Apache Airflow Issue Type:

[GitHub] [airflow] potiuk opened a new pull request #7539: [AIRFLOW-6919] Make Breeze DAG-test friedly

2020-02-25 Thread GitBox
potiuk opened a new pull request #7539: [AIRFLOW-6919] Make Breeze DAG-test friedly URL: https://github.com/apache/airflow/pull/7539 Originally Breeze was used to run unit and integration tests, recently system tests and finally we make it a bit more friendly to test your DAGs there.

[jira] [Created] (AIRFLOW-6919) Make Breeeze more Dag-test friendly

2020-02-25 Thread Jarek Potiuk (Jira)
Jarek Potiuk created AIRFLOW-6919: - Summary: Make Breeeze more Dag-test friendly Key: AIRFLOW-6919 URL: https://issues.apache.org/jira/browse/AIRFLOW-6919 Project: Apache Airflow Issue Type:

[jira] [Commented] (AIRFLOW-5307) Move the BaseOperator to the operators package

2020-02-25 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-5307?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17044991#comment-17044991 ] ASF GitHub Bot commented on AIRFLOW-5307: - BasPH commented on pull request #5910:

[GitHub] [airflow] kaxil commented on issue #5910: [WIP][AIRFLOW-5307] Move BaseOperator to airflow.operators.base_operator

2020-02-25 Thread GitBox
kaxil commented on issue #5910: [WIP][AIRFLOW-5307] Move BaseOperator to airflow.operators.base_operator URL: https://github.com/apache/airflow/pull/5910#issuecomment-591124831 Agree, this should be moved to operator. Can you rebase to the latest master?

[GitHub] [airflow] BasPH opened a new pull request #5910: [WIP][AIRFLOW-5307] Move BaseOperator to airflow.operators.base_operator

2020-02-25 Thread GitBox
BasPH opened a new pull request #5910: [WIP][AIRFLOW-5307] Move BaseOperator to airflow.operators.base_operator URL: https://github.com/apache/airflow/pull/5910 **WIP: working out a bug in the docs.** Make sure you have checked _all_ steps below. ### Jira - [x] My PR

[jira] [Updated] (AIRFLOW-6389) add config for 'allow_multi_scheduler_instances' default True

2020-02-25 Thread t oo (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6389?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] t oo updated AIRFLOW-6389: -- Description: right now common deployment pattern with blue/green build is: 1. on EC2 1, start scheduler 2.

[jira] [Updated] (AIRFLOW-6389) add config for 'allow_multi_scheduler_instances' default True

2020-02-25 Thread t oo (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6389?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] t oo updated AIRFLOW-6389: -- Description: right now common deployment pattern with blue/green build is: 1. on EC2 1, start scheduler 2.

[jira] [Updated] (AIRFLOW-6388) SparkSubmitOperator polling should not 'consume' a pool slot

2020-02-25 Thread t oo (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6388?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] t oo updated AIRFLOW-6388: -- Summary: SparkSubmitOperator polling should not 'consume' a pool slot (was: SparkSubmitOperator polling

[jira] [Updated] (AIRFLOW-6443) Pool name - increase length > 50, make cli give error if too large

2020-02-25 Thread t oo (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6443?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] t oo updated AIRFLOW-6443: -- Labels: gsoc gsoc2020 mentor pool (was: pool) > Pool name - increase length > 50, make cli give error if

[GitHub] [airflow] saguziel commented on issue #7269: [AIRFLOW-6651] Add Redis Heartbeat option

2020-02-25 Thread GitBox
saguziel commented on issue #7269: [AIRFLOW-6651] Add Redis Heartbeat option URL: https://github.com/apache/airflow/pull/7269#issuecomment-591116708 Celery does not support this type of thing, and it would not be portable across executors. Celery's consistency model is really simple, which

[GitHub] [airflow] tooptoop4 commented on issue #7537: [AIRFLOW-XXXX] And script to benchmark scheduler dag-run time

2020-02-25 Thread GitBox
tooptoop4 commented on issue #7537: [AIRFLOW-] And script to benchmark scheduler dag-run time URL: https://github.com/apache/airflow/pull/7537#issuecomment-591113663 maybe link to https://jira.apache.org/jira/browse/AIRFLOW-6454 ?

[GitHub] [airflow] ashb commented on a change in pull request #7537: [AIRFLOW-XXXX] And script to benchmark scheduler dag-run time

2020-02-25 Thread GitBox
ashb commented on a change in pull request #7537: [AIRFLOW-] And script to benchmark scheduler dag-run time URL: https://github.com/apache/airflow/pull/7537#discussion_r384173851 ## File path: scripts/perf/scheduler_dag_execution_timing.py ## @@ -0,0 +1,221 @@

[GitHub] [airflow] ashb commented on a change in pull request #7537: [AIRFLOW-XXXX] And script to benchmark scheduler dag-run time

2020-02-25 Thread GitBox
ashb commented on a change in pull request #7537: [AIRFLOW-] And script to benchmark scheduler dag-run time URL: https://github.com/apache/airflow/pull/7537#discussion_r384173467 ## File path: scripts/perf/scheduler_dag_execution_timing.py ## @@ -0,0 +1,221 @@

[GitHub] [airflow] kaxil commented on a change in pull request #7537: [AIRFLOW-XXXX] And script to benchmark scheduler dag-run time

2020-02-25 Thread GitBox
kaxil commented on a change in pull request #7537: [AIRFLOW-] And script to benchmark scheduler dag-run time URL: https://github.com/apache/airflow/pull/7537#discussion_r384173156 ## File path: scripts/perf/scheduler_dag_execution_timing.py ## @@ -0,0 +1,221 @@

[GitHub] [airflow] ashb commented on issue #7510: [AIRFLOW-6887][WIP] Do not check the state of fresh DAGRun

2020-02-25 Thread GitBox
ashb commented on issue #7510: [AIRFLOW-6887][WIP] Do not check the state of fresh DAGRun URL: https://github.com/apache/airflow/pull/7510#issuecomment-591112184 @Fokko `run.refresh_from_db()` just loads the id and state columns. The case you are talking about is still handled correctly

[GitHub] [airflow] houqp commented on a change in pull request #7492: [AIRFLOW-6871] optimize tree view for large DAGs

2020-02-25 Thread GitBox
houqp commented on a change in pull request #7492: [AIRFLOW-6871] optimize tree view for large DAGs URL: https://github.com/apache/airflow/pull/7492#discussion_r384172061 ## File path: airflow/www/views.py ## @@ -1374,90 +1376,115 @@ def tree(self):

[GitHub] [airflow] ashb commented on a change in pull request #7492: [AIRFLOW-6871] optimize tree view for large DAGs

2020-02-25 Thread GitBox
ashb commented on a change in pull request #7492: [AIRFLOW-6871] optimize tree view for large DAGs URL: https://github.com/apache/airflow/pull/7492#discussion_r384171265 ## File path: airflow/www/templates/airflow/tree.html ## @@ -85,8 +85,59 @@

[jira] [Commented] (AIRFLOW-6518) Task did not retry when there was temporary metastore db connectivity loss

2020-02-25 Thread t oo (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6518?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17044969#comment-17044969 ] t oo commented on AIRFLOW-6518: --- [~ash] [~potiuk] thoughts? > Task did not retry when there was

[jira] [Updated] (AIRFLOW-6747) UI - Show count of tasks in each dag on the main dags page

2020-02-25 Thread t oo (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6747?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] t oo updated AIRFLOW-6747: -- Labels: gsoc gsoc2020 mentor (was: ) > UI - Show count of tasks in each dag on the main dags page >

[GitHub] [airflow] ashb commented on issue #7538: [AIRFLOW-6382] Add reason for pre-commit rule

2020-02-25 Thread GitBox
ashb commented on issue #7538: [AIRFLOW-6382] Add reason for pre-commit rule URL: https://github.com/apache/airflow/pull/7538#issuecomment-591108820 (so much for the CI skip) This is an automated message from the Apache Git

[jira] [Updated] (AIRFLOW-6918) don't use 'is' in if conditions comparing STATE

2020-02-25 Thread t oo (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6918?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] t oo updated AIRFLOW-6918: -- Description: Trivial code tidy-up - 'is' usually is used for testing booleans or None presence. This change

[GitHub] [airflow] ashb commented on a change in pull request #7492: [AIRFLOW-6871] optimize tree view for large DAGs

2020-02-25 Thread GitBox
ashb commented on a change in pull request #7492: [AIRFLOW-6871] optimize tree view for large DAGs URL: https://github.com/apache/airflow/pull/7492#discussion_r384168703 ## File path: airflow/www/templates/airflow/tree.html ## @@ -85,8 +85,59 @@

[GitHub] [airflow] tooptoop4 commented on issue #7536: [AIRFLOW-6918] don't use 'is' in if conditions comparing STATE

2020-02-25 Thread GitBox
tooptoop4 commented on issue #7536: [AIRFLOW-6918] don't use 'is' in if conditions comparing STATE URL: https://github.com/apache/airflow/pull/7536#issuecomment-591107895 @kaxil updated This is an automated message from the

[GitHub] [airflow] ashb commented on a change in pull request #7492: [AIRFLOW-6871] optimize tree view for large DAGs

2020-02-25 Thread GitBox
ashb commented on a change in pull request #7492: [AIRFLOW-6871] optimize tree view for large DAGs URL: https://github.com/apache/airflow/pull/7492#discussion_r384164611 ## File path: airflow/www/templates/airflow/tree.html ## @@ -85,8 +85,59 @@

[GitHub] [airflow] ashb commented on a change in pull request #7537: [AIRFLOW-XXXX] And script to benchmark scheduler dag-run time

2020-02-25 Thread GitBox
ashb commented on a change in pull request #7537: [AIRFLOW-] And script to benchmark scheduler dag-run time URL: https://github.com/apache/airflow/pull/7537#discussion_r384161251 ## File path: scripts/perf/scheduler_dag_execution_timing.py ## @@ -0,0 +1,220 @@

[jira] [Commented] (AIRFLOW-6866) export EXTRA_DC_OPTIONS+= fails on mac

2020-02-25 Thread ASF subversion and git services (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6866?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17044952#comment-17044952 ] ASF subversion and git services commented on AIRFLOW-6866: -- Commit

[jira] [Commented] (AIRFLOW-6763) Make system test ready for backport tests

2020-02-25 Thread ASF subversion and git services (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6763?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17044954#comment-17044954 ] ASF subversion and git services commented on AIRFLOW-6763: -- Commit

[jira] [Commented] (AIRFLOW-6892) Fix broken non-wheel releases

2020-02-25 Thread ASF subversion and git services (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6892?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17044953#comment-17044953 ] ASF subversion and git services commented on AIRFLOW-6892: -- Commit

[jira] [Commented] (AIRFLOW-6838) Introduce real subcommands for Breeze

2020-02-25 Thread ASF subversion and git services (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6838?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17044955#comment-17044955 ] ASF subversion and git services commented on AIRFLOW-6838: -- Commit

[GitHub] [airflow] houqp commented on a change in pull request #7492: [AIRFLOW-6871] optimize tree view for large DAGs

2020-02-25 Thread GitBox
houqp commented on a change in pull request #7492: [AIRFLOW-6871] optimize tree view for large DAGs URL: https://github.com/apache/airflow/pull/7492#discussion_r384160648 ## File path: airflow/www/views.py ## @@ -1374,90 +1376,115 @@ def tree(self):

[jira] [Commented] (AIRFLOW-6382) Extract provide/create session to session module

2020-02-25 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6382?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17044949#comment-17044949 ] ASF GitHub Bot commented on AIRFLOW-6382: - ashb commented on pull request #7538: [AIRFLOW-6382]

[GitHub] [airflow] ashb opened a new pull request #7538: [AIRFLOW-6382] Add reason for pre-commit rule

2020-02-25 Thread GitBox
ashb opened a new pull request #7538: [AIRFLOW-6382] Add reason for pre-commit rule URL: https://github.com/apache/airflow/pull/7538 [ci-skip] --- Issue link: WILL BE INSERTED BY [boring-cyborg](https://github.com/kaxil/boring-cyborg) Make sure to mark the boxes below

[GitHub] [airflow] ashb commented on issue #6938: [AIRFLOW-6382] Extract provide/create session to session module

2020-02-25 Thread GitBox
ashb commented on issue #6938: [AIRFLOW-6382] Extract provide/create session to session module URL: https://github.com/apache/airflow/pull/6938#issuecomment-591098934 https://github.com/apache/airflow/pull/7538 will I think solve my issue/brain fart.

[GitHub] [airflow] kaxil commented on issue #7536: [AIRFLOW-6918] don't use 'is' in if conditions comparing STATE

2020-02-25 Thread GitBox
kaxil commented on issue #7536: [AIRFLOW-6918] don't use 'is' in if conditions comparing STATE URL: https://github.com/apache/airflow/pull/7536#issuecomment-591097807 The PR and JIRA has no description, please add it before someone can review

[GitHub] [airflow] houqp commented on a change in pull request #7492: [AIRFLOW-6871] optimize tree view for large DAGs

2020-02-25 Thread GitBox
houqp commented on a change in pull request #7492: [AIRFLOW-6871] optimize tree view for large DAGs URL: https://github.com/apache/airflow/pull/7492#discussion_r384158366 ## File path: airflow/www/templates/airflow/tree.html ## @@ -85,8 +85,59 @@

[GitHub] [airflow] houqp commented on a change in pull request #7492: [AIRFLOW-6871] optimize tree view for large DAGs

2020-02-25 Thread GitBox
houqp commented on a change in pull request #7492: [AIRFLOW-6871] optimize tree view for large DAGs URL: https://github.com/apache/airflow/pull/7492#discussion_r384158262 ## File path: airflow/www/templates/airflow/tree.html ## @@ -85,8 +85,59 @@

[GitHub] [airflow] ashb commented on a change in pull request #7492: [AIRFLOW-6871] optimize tree view for large DAGs

2020-02-25 Thread GitBox
ashb commented on a change in pull request #7492: [AIRFLOW-6871] optimize tree view for large DAGs URL: https://github.com/apache/airflow/pull/7492#discussion_r384158218 ## File path: airflow/www/views.py ## @@ -1374,90 +1376,115 @@ def tree(self): .all()

[GitHub] [airflow] ashb commented on issue #6938: [AIRFLOW-6382] Extract provide/create session to session module

2020-02-25 Thread GitBox
ashb commented on issue #6938: [AIRFLOW-6382] Extract provide/create session to session module URL: https://github.com/apache/airflow/pull/6938#issuecomment-591096260 It's probably fine as it is. Def not worth changing all the files again anyway.

[GitHub] [airflow] houqp edited a comment on issue #7537: [AIRFLOW-XXXX] And script to benchmark scheduler dag-run time

2020-02-25 Thread GitBox
houqp edited a comment on issue #7537: [AIRFLOW-] And script to benchmark scheduler dag-run time URL: https://github.com/apache/airflow/pull/7537#issuecomment-591094143 Looks like a useful script that's worth mentioning in CONTRIBUTING.rst?

[GitHub] [airflow] houqp edited a comment on issue #7537: [AIRFLOW-XXXX] And script to benchmark scheduler dag-run time

2020-02-25 Thread GitBox
houqp edited a comment on issue #7537: [AIRFLOW-] And script to benchmark scheduler dag-run time URL: https://github.com/apache/airflow/pull/7537#issuecomment-591094143 Looks like something that's worth mentioning in CONTRIBUTING.rst?

[GitHub] [airflow] ashb commented on issue #7537: [AIRFLOW-XXXX] And script to benchmark scheduler dag-run time

2020-02-25 Thread GitBox
ashb commented on issue #7537: [AIRFLOW-] And script to benchmark scheduler dag-run time URL: https://github.com/apache/airflow/pull/7537#issuecomment-591094351 Example invocation, and final line of output (there are a lot of logs in the middle) ```

[GitHub] [airflow] houqp commented on issue #7537: [AIRFLOW-XXXX] And script to benchmark scheduler dag-run time

2020-02-25 Thread GitBox
houqp commented on issue #7537: [AIRFLOW-] And script to benchmark scheduler dag-run time URL: https://github.com/apache/airflow/pull/7537#issuecomment-591094143 Looks like something that's worth mentioning in CONTRIBUTING.rst?

[GitHub] [airflow] ashb commented on a change in pull request #7537: [AIRFLOW-XXXX] And script to benchmark scheduler dag-run time

2020-02-25 Thread GitBox
ashb commented on a change in pull request #7537: [AIRFLOW-] And script to benchmark scheduler dag-run time URL: https://github.com/apache/airflow/pull/7537#discussion_r384154553 ## File path: scripts/perf/scheduler_dag_execution_timing.py ## @@ -0,0 +1,220 @@

[GitHub] [airflow] ashb commented on a change in pull request #7492: [AIRFLOW-6871] optimize tree view for large DAGs

2020-02-25 Thread GitBox
ashb commented on a change in pull request #7492: [AIRFLOW-6871] optimize tree view for large DAGs URL: https://github.com/apache/airflow/pull/7492#discussion_r384154242 ## File path: airflow/www/templates/airflow/tree.html ## @@ -85,8 +85,59 @@

[GitHub] [airflow] houqp commented on a change in pull request #7537: [AIRFLOW-XXXX] And script to benchmark scheduler dag-run time

2020-02-25 Thread GitBox
houqp commented on a change in pull request #7537: [AIRFLOW-] And script to benchmark scheduler dag-run time URL: https://github.com/apache/airflow/pull/7537#discussion_r384154018 ## File path: scripts/perf/scheduler_dag_execution_timing.py ## @@ -0,0 +1,220 @@

[GitHub] [airflow] nuclearpinguin commented on issue #6938: [AIRFLOW-6382] Extract provide/create session to session module

2020-02-25 Thread GitBox
nuclearpinguin commented on issue #6938: [AIRFLOW-6382] Extract provide/create session to session module URL: https://github.com/apache/airflow/pull/6938#issuecomment-591092356 That was a big cycle removal. As discussed above, I don't know any other "session" in airflow so the

[GitHub] [airflow] ashb commented on issue #7537: [AIRFLOW-XXXX] And script to benchmark scheduler dag-run time

2020-02-25 Thread GitBox
ashb commented on issue #7537: [AIRFLOW-] And script to benchmark scheduler dag-run time URL: https://github.com/apache/airflow/pull/7537#issuecomment-591092242 I haven't created a Jira for this, figuring it's not really "code" as it doesn't get shipped with the app, but I can do if

[GitHub] [airflow] nuclearpinguin commented on issue #6938: [AIRFLOW-6382] Extract provide/create session to session module

2020-02-25 Thread GitBox
nuclearpinguin commented on issue #6938: [AIRFLOW-6382] Extract provide/create session to session module URL: https://github.com/apache/airflow/pull/6938#issuecomment-591091572 Yeah, so long that I at first edited your comment instead of reply :D

[GitHub] [airflow] ashb commented on a change in pull request #7492: [AIRFLOW-6871] optimize tree view for large DAGs

2020-02-25 Thread GitBox
ashb commented on a change in pull request #7492: [AIRFLOW-6871] optimize tree view for large DAGs URL: https://github.com/apache/airflow/pull/7492#discussion_r384152129 ## File path: airflow/www/views.py ## @@ -1374,90 +1376,113 @@ def tree(self): .all()

[GitHub] [airflow] ashb commented on issue #6938: [AIRFLOW-6382] Extract provide/create session to session module

2020-02-25 Thread GitBox
ashb commented on issue #6938: [AIRFLOW-6382] Extract provide/create session to session module URL: https://github.com/apache/airflow/pull/6938#issuecomment-591091381 :man_facepalming: oh yeah. Sorry, long day. This is an

[GitHub] [airflow] nuclearpinguin commented on issue #6938: [AIRFLOW-6382] Extract provide/create session to session module

2020-02-25 Thread GitBox
nuclearpinguin commented on issue #6938: [AIRFLOW-6382] Extract provide/create session to session module URL: https://github.com/apache/airflow/pull/6938#issuecomment-591091208 >And how did this reduce cyclic imports when utils.db imports the new file still?! That is not a problem.

[GitHub] [airflow] ashb edited a comment on issue #6938: [AIRFLOW-6382] Extract provide/create session to session module

2020-02-25 Thread GitBox
ashb edited a comment on issue #6938: [AIRFLOW-6382] Extract provide/create session to session module URL: https://github.com/apache/airflow/pull/6938#issuecomment-591081255 Why did we make this change? Neither the PR nor the linked Jira give any justification for the change. It

[GitHub] [airflow] ashb edited a comment on issue #6938: [AIRFLOW-6382] Extract provide/create session to session module

2020-02-25 Thread GitBox
ashb edited a comment on issue #6938: [AIRFLOW-6382] Extract provide/create session to session module URL: https://github.com/apache/airflow/pull/6938#issuecomment-591081255 > And how did this reduce cyclic imports when utils.db imports the new file still?! That is not a problem.

[GitHub] [airflow] ashb opened a new pull request #7537: [AIRFLOW-XXXX] And script to benchmark scheduler dag-run time

2020-02-25 Thread GitBox
ashb opened a new pull request #7537: [AIRFLOW-] And script to benchmark scheduler dag-run time URL: https://github.com/apache/airflow/pull/7537 This script will run the SchedulerJob for the specified dags "to completion". That is it creates a fixed number of DAG runs for the

[GitHub] [airflow] ashb edited a comment on issue #6938: [AIRFLOW-6382] Extract provide/create session to session module

2020-02-25 Thread GitBox
ashb edited a comment on issue #6938: [AIRFLOW-6382] Extract provide/create session to session module URL: https://github.com/apache/airflow/pull/6938#issuecomment-591081255 Why did we make this change? Neither the PR nor the linked Jira give _any_ justification for the change. It

[GitHub] [airflow] ashb commented on issue #6938: [AIRFLOW-6382] Extract provide/create session to session module

2020-02-25 Thread GitBox
ashb commented on issue #6938: [AIRFLOW-6382] Extract provide/create session to session module URL: https://github.com/apache/airflow/pull/6938#issuecomment-591084955 (excuse the grump, but this change, and the pre commit rule we put in place meant I had to work around it have code live

[GitHub] [airflow-on-k8s-operator] barney-s opened a new issue #8: Setup CI for the repo

2020-02-25 Thread GitBox
barney-s opened a new issue #8: Setup CI for the repo URL: https://github.com/apache/airflow-on-k8s-operator/issues/8 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [airflow] ashb commented on issue #6938: [AIRFLOW-6382] Extract provide/create session to session module

2020-02-25 Thread GitBox
ashb commented on issue #6938: [AIRFLOW-6382] Extract provide/create session to session module URL: https://github.com/apache/airflow/pull/6938#issuecomment-591081255 Why did we make this change? Neither the PR nor the linked Jira give _any_ justification for the change. It doesn't

[GitHub] [airflow] tooptoop4 commented on issue #7169: [AIRFLOW-6555] mushroom cloud error when clicking 'task instance details' from graph view of dag that has never been run yet

2020-02-25 Thread GitBox
tooptoop4 commented on issue #7169: [AIRFLOW-6555] mushroom cloud error when clicking 'task instance details' from graph view of dag that has never been run yet URL: https://github.com/apache/airflow/pull/7169#issuecomment-591075723 @ashb are there any similar tests I could 'reuse' as a

[jira] [Updated] (AIRFLOW-6918) don't use 'is' in if conditions comparing STATE

2020-02-25 Thread t oo (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6918?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] t oo updated AIRFLOW-6918: -- Priority: Trivial (was: Major) > don't use 'is' in if conditions comparing STATE >

[jira] [Work started] (AIRFLOW-6918) don't use 'is' in if conditions comparing STATE

2020-02-25 Thread t oo (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6918?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on AIRFLOW-6918 started by t oo. - > don't use 'is' in if conditions comparing STATE >

[GitHub] [airflow] tooptoop4 opened a new pull request #7536: [AIRFLOW-6918] don't use 'is' in if conditions comparing STATE

2020-02-25 Thread GitBox
tooptoop4 opened a new pull request #7536: [AIRFLOW-6918] don't use 'is' in if conditions comparing STATE URL: https://github.com/apache/airflow/pull/7536 --- Issue link: WILL BE INSERTED BY [boring-cyborg](https://github.com/kaxil/boring-cyborg) Make sure to mark the boxes

[jira] [Commented] (AIRFLOW-6918) don't use 'is' in if conditions comparing STATE

2020-02-25 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6918?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17044890#comment-17044890 ] ASF GitHub Bot commented on AIRFLOW-6918: - tooptoop4 commented on pull request #7536:

[jira] [Updated] (AIRFLOW-6918) don't use 'is' in if conditions comparing STATE

2020-02-25 Thread t oo (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6918?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] t oo updated AIRFLOW-6918: -- Summary: don't use 'is' in if conditions comparing STATE (was: don't use is in if comparing state) > don't

[jira] [Created] (AIRFLOW-6918) don't use is in if comparing state

2020-02-25 Thread t oo (Jira)
t oo created AIRFLOW-6918: - Summary: don't use is in if comparing state Key: AIRFLOW-6918 URL: https://issues.apache.org/jira/browse/AIRFLOW-6918 Project: Apache Airflow Issue Type: Improvement

[jira] [Commented] (AIRFLOW-6864) Make airfow/jobs pylint compatible

2020-02-25 Thread ASF subversion and git services (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6864?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17044861#comment-17044861 ] ASF subversion and git services commented on AIRFLOW-6864: -- Commit

[jira] [Commented] (AIRFLOW-6864) Make airfow/jobs pylint compatible

2020-02-25 Thread ASF subversion and git services (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6864?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17044868#comment-17044868 ] ASF subversion and git services commented on AIRFLOW-6864: -- Commit

[jira] [Commented] (AIRFLOW-6864) Make airfow/jobs pylint compatible

2020-02-25 Thread ASF subversion and git services (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6864?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17044860#comment-17044860 ] ASF subversion and git services commented on AIRFLOW-6864: -- Commit

[jira] [Commented] (AIRFLOW-6864) Make airfow/jobs pylint compatible

2020-02-25 Thread ASF subversion and git services (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6864?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17044866#comment-17044866 ] ASF subversion and git services commented on AIRFLOW-6864: -- Commit

[jira] [Commented] (AIRFLOW-6864) Make airfow/jobs pylint compatible

2020-02-25 Thread ASF subversion and git services (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6864?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17044863#comment-17044863 ] ASF subversion and git services commented on AIRFLOW-6864: -- Commit

[jira] [Commented] (AIRFLOW-6864) Make airfow/jobs pylint compatible

2020-02-25 Thread ASF subversion and git services (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6864?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17044867#comment-17044867 ] ASF subversion and git services commented on AIRFLOW-6864: -- Commit

[jira] [Commented] (AIRFLOW-6864) Make airfow/jobs pylint compatible

2020-02-25 Thread ASF subversion and git services (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6864?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17044864#comment-17044864 ] ASF subversion and git services commented on AIRFLOW-6864: -- Commit

[jira] [Commented] (AIRFLOW-6864) Make airfow/jobs pylint compatible

2020-02-25 Thread ASF subversion and git services (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6864?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17044862#comment-17044862 ] ASF subversion and git services commented on AIRFLOW-6864: -- Commit

[jira] [Commented] (AIRFLOW-6864) Make airfow/jobs pylint compatible

2020-02-25 Thread ASF subversion and git services (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6864?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17044865#comment-17044865 ] ASF subversion and git services commented on AIRFLOW-6864: -- Commit

[jira] [Resolved] (AIRFLOW-6864) Make airfow/jobs pylint compatible

2020-02-25 Thread Tomasz Urbaszek (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6864?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tomasz Urbaszek resolved AIRFLOW-6864. -- Fix Version/s: 2.0.0 Resolution: Done > Make airfow/jobs pylint compatible >

[jira] [Commented] (AIRFLOW-6864) Make airfow/jobs pylint compatible

2020-02-25 Thread ASF subversion and git services (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6864?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17044859#comment-17044859 ] ASF subversion and git services commented on AIRFLOW-6864: -- Commit

[GitHub] [airflow] nuclearpinguin merged pull request #7484: [AIRFLOW-6864] Make airflow/jobs pylint compatible

2020-02-25 Thread GitBox
nuclearpinguin merged pull request #7484: [AIRFLOW-6864] Make airflow/jobs pylint compatible URL: https://github.com/apache/airflow/pull/7484 This is an automated message from the Apache Git Service. To respond to the

[jira] [Commented] (AIRFLOW-6864) Make airfow/jobs pylint compatible

2020-02-25 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6864?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17044858#comment-17044858 ] ASF GitHub Bot commented on AIRFLOW-6864: - nuclearpinguin commented on pull request #7484:

[jira] [Updated] (AIRFLOW-6917) Remove Pickling

2020-02-25 Thread Kaxil Naik (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6917?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kaxil Naik updated AIRFLOW-6917: Labels: dag-serialization (was: ) > Remove Pickling > > >

[jira] [Created] (AIRFLOW-6917) Remove Pickling

2020-02-25 Thread Kaxil Naik (Jira)
Kaxil Naik created AIRFLOW-6917: --- Summary: Remove Pickling Key: AIRFLOW-6917 URL: https://issues.apache.org/jira/browse/AIRFLOW-6917 Project: Apache Airflow Issue Type: Improvement

[jira] [Resolved] (AIRFLOW-3831) Remove DagBag from /pickle_info

2020-02-25 Thread Kaxil Naik (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-3831?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kaxil Naik resolved AIRFLOW-3831. - Resolution: Fixed > Remove DagBag from /pickle_info > --- > >

  1   2   3   >