[GitHub] [airflow] AlexandrKhabarov commented on issue #7446: [AIRFLOW-6474] list_dag_runs cli command should allow exec_date between start/end range and print start/end times

2020-02-17 Thread GitBox 
AlexandrKhabarov commented on issue #7446: [AIRFLOW-6474] list_dag_runs cli 
command should allow exec_date between start/end range and print start/end times
URL: https://github.com/apache/airflow/pull/7446#issuecomment-587311872
 
 
   Sorry, my fault. Of course, I will do it.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] codecov-io edited a comment on issue #6931: [AIRFLOW-6372] Align WASB remote logging URI scheme

2020-02-17 Thread GitBox 
codecov-io edited a comment on issue #6931: [AIRFLOW-6372] Align WASB remote 
logging URI scheme
URL: https://github.com/apache/airflow/pull/6931#issuecomment-587309404
 
 
   # [Codecov](https://codecov.io/gh/apache/airflow/pull/6931?src=pr&el=h1) 
Report
   > Merging 
[#6931](https://codecov.io/gh/apache/airflow/pull/6931?src=pr&el=desc) into 
[master](https://codecov.io/gh/apache/airflow/commit/dbcd3d8787741fd8203b6d9bdbc5d1da4b10a15b?src=pr&el=desc)
 will **decrease** coverage by `0.31%`.
   > The diff coverage is `13.33%`.
   
   [![Impacted file tree 
graph](https://codecov.io/gh/apache/airflow/pull/6931/graphs/tree.svg?width=650&token=WdLKlKHOAU&height=150&src=pr)](https://codecov.io/gh/apache/airflow/pull/6931?src=pr&el=tree)
   
   ```diff
   @@Coverage Diff @@
   ##   master#6931  +/-   ##
   ==
   - Coverage   86.66%   86.35%   -0.32% 
   ==
 Files 877  877  
 Lines   4124141254  +13 
   ==
   - Hits3574235625 -117 
   - Misses   5499 5629 +130
   ```
   
   
   | [Impacted 
Files](https://codecov.io/gh/apache/airflow/pull/6931?src=pr&el=tree) | 
Coverage Δ | |
   |---|---|---|
   | 
[airflow/config\_templates/airflow\_local\_settings.py](https://codecov.io/gh/apache/airflow/pull/6931/diff?src=pr&el=tree#diff-YWlyZmxvdy9jb25maWdfdGVtcGxhdGVzL2FpcmZsb3dfbG9jYWxfc2V0dGluZ3MucHk=)
 | `65.38% <ø> (ø)` | :arrow_up: |
   | 
[airflow/utils/log/wasb\_task\_handler.py](https://codecov.io/gh/apache/airflow/pull/6931/diff?src=pr&el=tree#diff-YWlyZmxvdy91dGlscy9sb2cvd2FzYl90YXNrX2hhbmRsZXIucHk=)
 | `38.37% <13.33%> (-4.1%)` | :arrow_down: |
   | 
[airflow/kubernetes/volume\_mount.py](https://codecov.io/gh/apache/airflow/pull/6931/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3ZvbHVtZV9tb3VudC5weQ==)
 | `44.44% <0%> (-55.56%)` | :arrow_down: |
   | 
[airflow/kubernetes/volume.py](https://codecov.io/gh/apache/airflow/pull/6931/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3ZvbHVtZS5weQ==)
 | `52.94% <0%> (-47.06%)` | :arrow_down: |
   | 
[airflow/kubernetes/pod\_launcher.py](https://codecov.io/gh/apache/airflow/pull/6931/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3BvZF9sYXVuY2hlci5weQ==)
 | `47.18% <0%> (-45.08%)` | :arrow_down: |
   | 
[...viders/cncf/kubernetes/operators/kubernetes\_pod.py](https://codecov.io/gh/apache/airflow/pull/6931/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvY25jZi9rdWJlcm5ldGVzL29wZXJhdG9ycy9rdWJlcm5ldGVzX3BvZC5weQ==)
 | `69.38% <0%> (-25.52%)` | :arrow_down: |
   | 
[airflow/kubernetes/refresh\_config.py](https://codecov.io/gh/apache/airflow/pull/6931/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3JlZnJlc2hfY29uZmlnLnB5)
 | `50.98% <0%> (-23.53%)` | :arrow_down: |
   
   --
   
   [Continue to review full report at 
Codecov](https://codecov.io/gh/apache/airflow/pull/6931?src=pr&el=continue).
   > **Legend** - [Click here to learn 
more](https://docs.codecov.io/docs/codecov-delta)
   > `Δ = absolute  (impact)`, `ø = not affected`, `? = missing data`
   > Powered by 
[Codecov](https://codecov.io/gh/apache/airflow/pull/6931?src=pr&el=footer). 
Last update 
[dbcd3d8...d1f4e21](https://codecov.io/gh/apache/airflow/pull/6931?src=pr&el=lastupdated).
 Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] codecov-io commented on issue #6931: [AIRFLOW-6372] Align WASB remote logging URI scheme

2020-02-17 Thread GitBox 
codecov-io commented on issue #6931: [AIRFLOW-6372] Align WASB remote logging 
URI scheme
URL: https://github.com/apache/airflow/pull/6931#issuecomment-587309404
 
 
   # [Codecov](https://codecov.io/gh/apache/airflow/pull/6931?src=pr&el=h1) 
Report
   > Merging 
[#6931](https://codecov.io/gh/apache/airflow/pull/6931?src=pr&el=desc) into 
[master](https://codecov.io/gh/apache/airflow/commit/dbcd3d8787741fd8203b6d9bdbc5d1da4b10a15b?src=pr&el=desc)
 will **decrease** coverage by `0.48%`.
   > The diff coverage is `13.33%`.
   
   [![Impacted file tree 
graph](https://codecov.io/gh/apache/airflow/pull/6931/graphs/tree.svg?width=650&token=WdLKlKHOAU&height=150&src=pr)](https://codecov.io/gh/apache/airflow/pull/6931?src=pr&el=tree)
   
   ```diff
   @@Coverage Diff @@
   ##   master#6931  +/-   ##
   ==
   - Coverage   86.66%   86.18%   -0.49% 
   ==
 Files 877  877  
 Lines   4124141254  +13 
   ==
   - Hits3574235553 -189 
   - Misses   5499 5701 +202
   ```
   
   
   | [Impacted 
Files](https://codecov.io/gh/apache/airflow/pull/6931?src=pr&el=tree) | 
Coverage Δ | |
   |---|---|---|
   | 
[airflow/config\_templates/airflow\_local\_settings.py](https://codecov.io/gh/apache/airflow/pull/6931/diff?src=pr&el=tree#diff-YWlyZmxvdy9jb25maWdfdGVtcGxhdGVzL2FpcmZsb3dfbG9jYWxfc2V0dGluZ3MucHk=)
 | `65.38% <ø> (ø)` | :arrow_up: |
   | 
[airflow/utils/log/wasb\_task\_handler.py](https://codecov.io/gh/apache/airflow/pull/6931/diff?src=pr&el=tree#diff-YWlyZmxvdy91dGlscy9sb2cvd2FzYl90YXNrX2hhbmRsZXIucHk=)
 | `38.37% <13.33%> (-4.1%)` | :arrow_down: |
   | 
[...w/providers/apache/hive/operators/mysql\_to\_hive.py](https://codecov.io/gh/apache/airflow/pull/6931/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvYXBhY2hlL2hpdmUvb3BlcmF0b3JzL215c3FsX3RvX2hpdmUucHk=)
 | `35.84% <0%> (-64.16%)` | :arrow_down: |
   | 
[airflow/kubernetes/volume\_mount.py](https://codecov.io/gh/apache/airflow/pull/6931/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3ZvbHVtZV9tb3VudC5weQ==)
 | `44.44% <0%> (-55.56%)` | :arrow_down: |
   | 
[airflow/kubernetes/volume.py](https://codecov.io/gh/apache/airflow/pull/6931/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3ZvbHVtZS5weQ==)
 | `52.94% <0%> (-47.06%)` | :arrow_down: |
   | 
[airflow/security/kerberos.py](https://codecov.io/gh/apache/airflow/pull/6931/diff?src=pr&el=tree#diff-YWlyZmxvdy9zZWN1cml0eS9rZXJiZXJvcy5weQ==)
 | `30.43% <0%> (-45.66%)` | :arrow_down: |
   | 
[airflow/kubernetes/pod\_launcher.py](https://codecov.io/gh/apache/airflow/pull/6931/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3BvZF9sYXVuY2hlci5weQ==)
 | `47.18% <0%> (-45.08%)` | :arrow_down: |
   | 
[airflow/providers/mysql/operators/mysql.py](https://codecov.io/gh/apache/airflow/pull/6931/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvbXlzcWwvb3BlcmF0b3JzL215c3FsLnB5)
 | `55% <0%> (-45%)` | :arrow_down: |
   | 
[...viders/cncf/kubernetes/operators/kubernetes\_pod.py](https://codecov.io/gh/apache/airflow/pull/6931/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvY25jZi9rdWJlcm5ldGVzL29wZXJhdG9ycy9rdWJlcm5ldGVzX3BvZC5weQ==)
 | `69.38% <0%> (-25.52%)` | :arrow_down: |
   | 
[airflow/kubernetes/refresh\_config.py](https://codecov.io/gh/apache/airflow/pull/6931/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3JlZnJlc2hfY29uZmlnLnB5)
 | `50.98% <0%> (-23.53%)` | :arrow_down: |
   | ... and [3 
more](https://codecov.io/gh/apache/airflow/pull/6931/diff?src=pr&el=tree-more) 
| |
   
   --
   
   [Continue to review full report at 
Codecov](https://codecov.io/gh/apache/airflow/pull/6931?src=pr&el=continue).
   > **Legend** - [Click here to learn 
more](https://docs.codecov.io/docs/codecov-delta)
   > `Δ = absolute  (impact)`, `ø = not affected`, `? = missing data`
   > Powered by 
[Codecov](https://codecov.io/gh/apache/airflow/pull/6931?src=pr&el=footer). 
Last update 
[dbcd3d8...d1f4e21](https://codecov.io/gh/apache/airflow/pull/6931?src=pr&el=lastupdated).
 Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] codecov-io commented on issue #7451: [AIRFLOW-6830] Add Subject/MessageAttributes to SNS hook and operator

2020-02-17 Thread GitBox 
codecov-io commented on issue #7451: [AIRFLOW-6830] Add 
Subject/MessageAttributes to SNS hook and operator
URL: https://github.com/apache/airflow/pull/7451#issuecomment-587304168
 
 
   # [Codecov](https://codecov.io/gh/apache/airflow/pull/7451?src=pr&el=h1) 
Report
   > Merging 
[#7451](https://codecov.io/gh/apache/airflow/pull/7451?src=pr&el=desc) into 
[master](https://codecov.io/gh/apache/airflow/commit/dbcd3d8787741fd8203b6d9bdbc5d1da4b10a15b?src=pr&el=desc)
 will **decrease** coverage by `0.46%`.
   > The diff coverage is `95%`.
   
   [![Impacted file tree 
graph](https://codecov.io/gh/apache/airflow/pull/7451/graphs/tree.svg?width=650&token=WdLKlKHOAU&height=150&src=pr)](https://codecov.io/gh/apache/airflow/pull/7451?src=pr&el=tree)
   
   ```diff
   @@Coverage Diff@@
   ##   master   #7451  +/-   ##
   =
   - Coverage   86.66%   86.2%   -0.47% 
   =
 Files 877 877  
 Lines   41241   41255  +14 
   =
   - Hits35742   35563 -179 
   - Misses   54995692 +193
   ```
   
   
   | [Impacted 
Files](https://codecov.io/gh/apache/airflow/pull/7451?src=pr&el=tree) | 
Coverage Δ | |
   |---|---|---|
   | 
[airflow/providers/amazon/aws/operators/sns.py](https://codecov.io/gh/apache/airflow/pull/7451/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvYW1hem9uL2F3cy9vcGVyYXRvcnMvc25zLnB5)
 | `100% <100%> (ø)` | :arrow_up: |
   | 
[airflow/providers/amazon/aws/hooks/sns.py](https://codecov.io/gh/apache/airflow/pull/7451/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvYW1hem9uL2F3cy9ob29rcy9zbnMucHk=)
 | `96.42% <94.11%> (-3.58%)` | :arrow_down: |
   | 
[...w/providers/apache/hive/operators/mysql\_to\_hive.py](https://codecov.io/gh/apache/airflow/pull/7451/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvYXBhY2hlL2hpdmUvb3BlcmF0b3JzL215c3FsX3RvX2hpdmUucHk=)
 | `35.84% <0%> (-64.16%)` | :arrow_down: |
   | 
[airflow/operators/generic\_transfer.py](https://codecov.io/gh/apache/airflow/pull/7451/diff?src=pr&el=tree#diff-YWlyZmxvdy9vcGVyYXRvcnMvZ2VuZXJpY190cmFuc2Zlci5weQ==)
 | `100% <0%> (ø)` | :arrow_up: |
   | 
[airflow/kubernetes/volume\_mount.py](https://codecov.io/gh/apache/airflow/pull/7451/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3ZvbHVtZV9tb3VudC5weQ==)
 | `44.44% <0%> (-55.56%)` | :arrow_down: |
   | 
[airflow/providers/postgres/operators/postgres.py](https://codecov.io/gh/apache/airflow/pull/7451/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvcG9zdGdyZXMvb3BlcmF0b3JzL3Bvc3RncmVzLnB5)
 | `100% <0%> (ø)` | :arrow_up: |
   | 
[airflow/kubernetes/volume.py](https://codecov.io/gh/apache/airflow/pull/7451/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3ZvbHVtZS5weQ==)
 | `52.94% <0%> (-47.06%)` | :arrow_down: |
   | 
[airflow/security/kerberos.py](https://codecov.io/gh/apache/airflow/pull/7451/diff?src=pr&el=tree#diff-YWlyZmxvdy9zZWN1cml0eS9rZXJiZXJvcy5weQ==)
 | `30.43% <0%> (-45.66%)` | :arrow_down: |
   | 
[airflow/kubernetes/pod\_launcher.py](https://codecov.io/gh/apache/airflow/pull/7451/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3BvZF9sYXVuY2hlci5weQ==)
 | `47.18% <0%> (-45.08%)` | :arrow_down: |
   | 
[airflow/providers/mysql/operators/mysql.py](https://codecov.io/gh/apache/airflow/pull/7451/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvbXlzcWwvb3BlcmF0b3JzL215c3FsLnB5)
 | `55% <0%> (-45%)` | :arrow_down: |
   | ... and [10 
more](https://codecov.io/gh/apache/airflow/pull/7451/diff?src=pr&el=tree-more) 
| |
   
   --
   
   [Continue to review full report at 
Codecov](https://codecov.io/gh/apache/airflow/pull/7451?src=pr&el=continue).
   > **Legend** - [Click here to learn 
more](https://docs.codecov.io/docs/codecov-delta)
   > `Δ = absolute  (impact)`, `ø = not affected`, `? = missing data`
   > Powered by 
[Codecov](https://codecov.io/gh/apache/airflow/pull/7451?src=pr&el=footer). 
Last update 
[dbcd3d8...3621fa7](https://codecov.io/gh/apache/airflow/pull/7451?src=pr&el=lastupdated).
 Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] TV4Fun commented on issue #7098: [AIRFLOW-4453] Make behavior of `none_failed` consistent with documentation

2020-02-17 Thread GitBox 
TV4Fun commented on issue #7098: [AIRFLOW-4453] Make behavior of `none_failed` 
consistent with documentation
URL: https://github.com/apache/airflow/pull/7098#issuecomment-587300834
 
 
   Thank you @yuqian90! 👍 


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] yuqian90 commented on issue #7098: [AIRFLOW-4453] Make behavior of `none_failed` consistent with documentation

2020-02-17 Thread GitBox 
yuqian90 commented on issue #7098: [AIRFLOW-4453] Make behavior of 
`none_failed` consistent with documentation
URL: https://github.com/apache/airflow/pull/7098#issuecomment-587298029
 
 
   > Hi @yuqian90! I would like to work more on this, but am going to he very 
busy for the foreseeable future. If someone would like to take this over or 
make their own version, I have no objection.
   > […](#)
   > On Mon, Feb 17, 2020, 5:28 PM yuqian90 ***@***.***> wrote: Hi, @TV4Fun 
 would you like to continue working on this PR? This 
is definitely a good change to make. The issue regarding nested 
BranchPythonOperator can be addressed with something like the create_join() in 
my previous comment. Please consider it. Thanks! — You are receiving this 
because you were mentioned. Reply to this email directly, view it on GitHub 
<#7098?email_source=notifications&email_token=ABD4ECE6QP4BQNOAQCR5SJ3RDM2T5A5CNFSM4KEBP7VKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEMAHBQA#issuecomment-587231424>,
 or unsubscribe 

 .
   
   Thanks! I'll take it from where you left it and put up a new PR soon. 


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] codecov-io commented on issue #7413: [AIRFLOW-6793] Respect env variable in airflow config command

2020-02-17 Thread GitBox 
codecov-io commented on issue #7413: [AIRFLOW-6793] Respect env variable in 
airflow config command
URL: https://github.com/apache/airflow/pull/7413#issuecomment-587288473
 
 
   # [Codecov](https://codecov.io/gh/apache/airflow/pull/7413?src=pr&el=h1) 
Report
   > Merging 
[#7413](https://codecov.io/gh/apache/airflow/pull/7413?src=pr&el=desc) into 
[master](https://codecov.io/gh/apache/airflow/commit/33af616b83738b6dde31ac8e939237515dc0390b?src=pr&el=desc)
 will **decrease** coverage by `0.25%`.
   > The diff coverage is `90%`.
   
   [![Impacted file tree 
graph](https://codecov.io/gh/apache/airflow/pull/7413/graphs/tree.svg?width=650&token=WdLKlKHOAU&height=150&src=pr)](https://codecov.io/gh/apache/airflow/pull/7413?src=pr&el=tree)
   
   ```diff
   @@Coverage Diff @@
   ##   master#7413  +/-   ##
   ==
   - Coverage   86.63%   86.37%   -0.26% 
   ==
 Files 874  877   +3 
 Lines   4086841253 +385 
   ==
   + Hits3540535632 +227 
   - Misses   5463 5621 +158
   ```
   
   
   | [Impacted 
Files](https://codecov.io/gh/apache/airflow/pull/7413?src=pr&el=tree) | 
Coverage Δ | |
   |---|---|---|
   | 
[airflow/config\_templates/default\_celery.py](https://codecov.io/gh/apache/airflow/pull/7413/diff?src=pr&el=tree#diff-YWlyZmxvdy9jb25maWdfdGVtcGxhdGVzL2RlZmF1bHRfY2VsZXJ5LnB5)
 | `52.94% <100%> (ø)` | :arrow_up: |
   | 
[airflow/configuration.py](https://codecov.io/gh/apache/airflow/pull/7413/diff?src=pr&el=tree#diff-YWlyZmxvdy9jb25maWd1cmF0aW9uLnB5)
 | `87.78% <89.47%> (-0.23%)` | :arrow_down: |
   | 
[airflow/kubernetes/volume\_mount.py](https://codecov.io/gh/apache/airflow/pull/7413/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3ZvbHVtZV9tb3VudC5weQ==)
 | `44.44% <0%> (-55.56%)` | :arrow_down: |
   | 
[airflow/kubernetes/volume.py](https://codecov.io/gh/apache/airflow/pull/7413/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3ZvbHVtZS5weQ==)
 | `52.94% <0%> (-47.06%)` | :arrow_down: |
   | 
[airflow/kubernetes/pod\_launcher.py](https://codecov.io/gh/apache/airflow/pull/7413/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3BvZF9sYXVuY2hlci5weQ==)
 | `47.18% <0%> (-45.08%)` | :arrow_down: |
   | 
[...viders/cncf/kubernetes/operators/kubernetes\_pod.py](https://codecov.io/gh/apache/airflow/pull/7413/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvY25jZi9rdWJlcm5ldGVzL29wZXJhdG9ycy9rdWJlcm5ldGVzX3BvZC5weQ==)
 | `69.38% <0%> (-25.52%)` | :arrow_down: |
   | 
[airflow/kubernetes/refresh\_config.py](https://codecov.io/gh/apache/airflow/pull/7413/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3JlZnJlc2hfY29uZmlnLnB5)
 | `50.98% <0%> (-23.53%)` | :arrow_down: |
   | 
[airflow/utils/sqlalchemy.py](https://codecov.io/gh/apache/airflow/pull/7413/diff?src=pr&el=tree#diff-YWlyZmxvdy91dGlscy9zcWxhbGNoZW15LnB5)
 | `84.93% <0%> (-11.74%)` | :arrow_down: |
   | 
[airflow/stats.py](https://codecov.io/gh/apache/airflow/pull/7413/diff?src=pr&el=tree#diff-YWlyZmxvdy9zdGF0cy5weQ==)
 | `85.29% <0%> (-5.19%)` | :arrow_down: |
   | 
[airflow/cli/commands/dag\_command.py](https://codecov.io/gh/apache/airflow/pull/7413/diff?src=pr&el=tree#diff-YWlyZmxvdy9jbGkvY29tbWFuZHMvZGFnX2NvbW1hbmQucHk=)
 | `84.41% <0%> (-1.5%)` | :arrow_down: |
   | ... and [44 
more](https://codecov.io/gh/apache/airflow/pull/7413/diff?src=pr&el=tree-more) 
| |
   
   --
   
   [Continue to review full report at 
Codecov](https://codecov.io/gh/apache/airflow/pull/7413?src=pr&el=continue).
   > **Legend** - [Click here to learn 
more](https://docs.codecov.io/docs/codecov-delta)
   > `Δ = absolute  (impact)`, `ø = not affected`, `? = missing data`
   > Powered by 
[Codecov](https://codecov.io/gh/apache/airflow/pull/7413?src=pr&el=footer). 
Last update 
[33af616...52f5677](https://codecov.io/gh/apache/airflow/pull/7413?src=pr&el=lastupdated).
 Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[jira] [Commented] (AIRFLOW-6830) AWS SNS Hook and Operator should both support Subject and MessageAttributes

2020-02-17 Thread ASF GitHub Bot (Jira) 


[ 
https://issues.apache.org/jira/browse/AIRFLOW-6830?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17038786#comment-17038786
 ] 

ASF GitHub Bot commented on AIRFLOW-6830:
-

snazzyfox commented on pull request #7451: [AIRFLOW-6830] Add 
Subject/MessageAttributes to SNS hook and operator
URL: https://github.com/apache/airflow/pull/7451
 
 
   Add `subject` and `message_attributes` as possible parameters for 
`AwsSnsHook` and `SnsPublishOperator`. Both defaults to none and are backwards 
compatible.
   
   ---
   Issue link: WILL BE INSERTED BY 
[boring-cyborg](https://github.com/kaxil/boring-cyborg)
   
   Make sure to mark the boxes below before creating PR: [x]
   
   - [x] Description above provides context of the change
   - [x] Commit message/PR title starts with `[AIRFLOW-]`. AIRFLOW- = 
JIRA ID*
   - [x] Unit tests coverage for changes (not needed for documentation changes)
   - [x] Commits follow "[How to write a good git commit 
message](http://chris.beams.io/posts/git-commit/)"
   - [x] Relevant documentation is updated including usage instructions.
   - [x] I will engage committers as explained in [Contribution Workflow 
Example](https://github.com/apache/airflow/blob/master/CONTRIBUTING.rst#contribution-workflow-example).
   
   * For document-only changes commit message can start with 
`[AIRFLOW-]`.
   
   ---
   In case of fundamental code change, Airflow Improvement Proposal 
([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvements+Proposals))
 is needed.
   In case of a new dependency, check compliance with the [ASF 3rd Party 
License Policy](https://www.apache.org/legal/resolved.html#category-x).
   In case of backwards incompatible changes please leave a note in 
[UPDATING.md](https://github.com/apache/airflow/blob/master/UPDATING.md).
   Read the [Pull Request 
Guidelines](https://github.com/apache/airflow/blob/master/CONTRIBUTING.rst#pull-request-guidelines)
 for more information.
   
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


> AWS SNS Hook and Operator should both support Subject and MessageAttributes
> ---
>
> Key: AIRFLOW-6830
> URL: https://issues.apache.org/jira/browse/AIRFLOW-6830
> Project: Apache Airflow
>  Issue Type: Improvement
>  Components: aws, hooks, operators
>Affects Versions: 1.10.9
>Reporter: Luyao Yang
>Assignee: Luyao Yang
>Priority: Minor
>  Labels: AWS
>
> Support for the Subject field for AWS SNS was added to AwsSnsHook in 
> [AIRFLOW-5082|https://issues.apache.org/jira/browse/AIRFLOW-5082].  
> MessageAttributes is another field that can be passed with any message type 
> and should be supported.
> Furthermore, both fields should be available through SnsPublishOperator as 
> well so we don't have to write a Python function whenever one of these fields 
> is needed.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[GitHub] [airflow] snazzyfox opened a new pull request #7451: [AIRFLOW-6830] Add Subject/MessageAttributes to SNS hook and operator

2020-02-17 Thread GitBox 
snazzyfox opened a new pull request #7451: [AIRFLOW-6830] Add 
Subject/MessageAttributes to SNS hook and operator
URL: https://github.com/apache/airflow/pull/7451
 
 
   Add `subject` and `message_attributes` as possible parameters for 
`AwsSnsHook` and `SnsPublishOperator`. Both defaults to none and are backwards 
compatible.
   
   ---
   Issue link: WILL BE INSERTED BY 
[boring-cyborg](https://github.com/kaxil/boring-cyborg)
   
   Make sure to mark the boxes below before creating PR: [x]
   
   - [x] Description above provides context of the change
   - [x] Commit message/PR title starts with `[AIRFLOW-]`. AIRFLOW- = 
JIRA ID*
   - [x] Unit tests coverage for changes (not needed for documentation changes)
   - [x] Commits follow "[How to write a good git commit 
message](http://chris.beams.io/posts/git-commit/)"
   - [x] Relevant documentation is updated including usage instructions.
   - [x] I will engage committers as explained in [Contribution Workflow 
Example](https://github.com/apache/airflow/blob/master/CONTRIBUTING.rst#contribution-workflow-example).
   
   * For document-only changes commit message can start with 
`[AIRFLOW-]`.
   
   ---
   In case of fundamental code change, Airflow Improvement Proposal 
([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvements+Proposals))
 is needed.
   In case of a new dependency, check compliance with the [ASF 3rd Party 
License Policy](https://www.apache.org/legal/resolved.html#category-x).
   In case of backwards incompatible changes please leave a note in 
[UPDATING.md](https://github.com/apache/airflow/blob/master/UPDATING.md).
   Read the [Pull Request 
Guidelines](https://github.com/apache/airflow/blob/master/CONTRIBUTING.rst#pull-request-guidelines)
 for more information.
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] boring-cyborg[bot] commented on issue #7451: [AIRFLOW-6830] Add Subject/MessageAttributes to SNS hook and operator

2020-02-17 Thread GitBox 
boring-cyborg[bot] commented on issue #7451: [AIRFLOW-6830] Add 
Subject/MessageAttributes to SNS hook and operator
URL: https://github.com/apache/airflow/pull/7451#issuecomment-587282521
 
 
   Congratulations on your first Pull Request and welcome to the Apache Airflow 
community! If you have any issues or are unsure about any anything please check 
our Contribution Guide 
(https://github.com/apache/airflow/blob/master/CONTRIBUTING.rst)
   Here are some useful points:
   - Pay attention to the quality of your code (flake8, pylint and type 
annotations). Our [pre-commits]( 
https://github.com/apache/airflow/blob/master/STATIC_CODE_CHECKS.rst#prerequisites-for-pre-commit-hooks)
 will help you with that.
   - In case of a new feature add useful documentation (in docstrings or in 
`docs/` directory). Adding a new operator? Check this short 
[guide](https://github.com/apache/airflow/blob/master/docs/howto/custom-operator.rst)
 Consider adding an example DAG that shows how users should use it.
   - Consider using [Breeze 
environment](https://github.com/apache/airflow/blob/master/BREEZE.rst) for 
testing locally, it’s a heavy docker but it ships with a working Airflow and a 
lot of integrations.
   - Be patient and persistent. It might take some time to get a review or get 
the final approval from Committers.
   Apache Airflow is a community-driven project and together we are making it 
better 🚀.
   In case of doubts contact the developers at:
   Mailing List: d...@airflow.apache.org
   Slack: https://apache-airflow-slack.herokuapp.com/
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[jira] [Created] (AIRFLOW-6830) AWS SNS Hook and Operator should both support Subject and MessageAttributes

2020-02-17 Thread Luyao Yang (Jira) 
Luyao Yang created AIRFLOW-6830:
---

 Summary: AWS SNS Hook and Operator should both support Subject and 
MessageAttributes
 Key: AIRFLOW-6830
 URL: https://issues.apache.org/jira/browse/AIRFLOW-6830
 Project: Apache Airflow
  Issue Type: Improvement
  Components: aws, hooks, operators
Affects Versions: 1.10.9
Reporter: Luyao Yang
Assignee: Luyao Yang


Support for the Subject field for AWS SNS was added to AwsSnsHook in 
[AIRFLOW-5082|https://issues.apache.org/jira/browse/AIRFLOW-5082].  
MessageAttributes is another field that can be passed with any message type and 
should be supported.

Furthermore, both fields should be available through SnsPublishOperator as well 
so we don't have to write a Python function whenever one of these fields is 
needed.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[GitHub] [airflow] codecov-io commented on issue #7448: [AIRFLOW-6828] Stop using the zope library

2020-02-17 Thread GitBox 
codecov-io commented on issue #7448: [AIRFLOW-6828] Stop using the zope library
URL: https://github.com/apache/airflow/pull/7448#issuecomment-587265175
 
 
   # [Codecov](https://codecov.io/gh/apache/airflow/pull/7448?src=pr&el=h1) 
Report
   > Merging 
[#7448](https://codecov.io/gh/apache/airflow/pull/7448?src=pr&el=desc) into 
[master](https://codecov.io/gh/apache/airflow/commit/9cbd7de6d115795aba8bfb8addb060bfdfbdf87b?src=pr&el=desc)
 will **decrease** coverage by `0.17%`.
   > The diff coverage is `87.87%`.
   
   [![Impacted file tree 
graph](https://codecov.io/gh/apache/airflow/pull/7448/graphs/tree.svg?width=650&token=WdLKlKHOAU&height=150&src=pr)](https://codecov.io/gh/apache/airflow/pull/7448?src=pr&el=tree)
   
   ```diff
   @@Coverage Diff @@
   ##   master#7448  +/-   ##
   ==
   - Coverage   86.37%   86.19%   -0.18% 
   ==
 Files 877  877  
 Lines   4122141260  +39 
   ==
   - Hits3560335566  -37 
   - Misses   5618 5694  +76
   ```
   
   
   | [Impacted 
Files](https://codecov.io/gh/apache/airflow/pull/7448?src=pr&el=tree) | 
Coverage Δ | |
   |---|---|---|
   | 
[.../microsoft/azure/hooks/azure\_container\_instance.py](https://codecov.io/gh/apache/airflow/pull/7448/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvbWljcm9zb2Z0L2F6dXJlL2hvb2tzL2F6dXJlX2NvbnRhaW5lcl9pbnN0YW5jZS5weQ==)
 | `65.38% <33.33%> (-2.62%)` | :arrow_down: |
   | 
[airflow/configuration.py](https://codecov.io/gh/apache/airflow/pull/7448/diff?src=pr&el=tree#diff-YWlyZmxvdy9jb25maWd1cmF0aW9uLnB5)
 | `88.02% <93.33%> (+0.01%)` | :arrow_up: |
   | 
[...w/providers/apache/hive/operators/mysql\_to\_hive.py](https://codecov.io/gh/apache/airflow/pull/7448/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvYXBhY2hlL2hpdmUvb3BlcmF0b3JzL215c3FsX3RvX2hpdmUucHk=)
 | `35.84% <0%> (-64.16%)` | :arrow_down: |
   | 
[airflow/security/kerberos.py](https://codecov.io/gh/apache/airflow/pull/7448/diff?src=pr&el=tree#diff-YWlyZmxvdy9zZWN1cml0eS9rZXJiZXJvcy5weQ==)
 | `30.43% <0%> (-45.66%)` | :arrow_down: |
   | 
[airflow/providers/mysql/operators/mysql.py](https://codecov.io/gh/apache/airflow/pull/7448/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvbXlzcWwvb3BlcmF0b3JzL215c3FsLnB5)
 | `55% <0%> (-45%)` | :arrow_down: |
   | 
[airflow/providers/apache/hive/hooks/hive.py](https://codecov.io/gh/apache/airflow/pull/7448/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvYXBhY2hlL2hpdmUvaG9va3MvaGl2ZS5weQ==)
 | `76.02% <0%> (-1.54%)` | :arrow_down: |
   | 
[airflow/hooks/dbapi\_hook.py](https://codecov.io/gh/apache/airflow/pull/7448/diff?src=pr&el=tree#diff-YWlyZmxvdy9ob29rcy9kYmFwaV9ob29rLnB5)
 | `90.9% <0%> (-0.83%)` | :arrow_down: |
   | 
[airflow/jobs/scheduler\_job.py](https://codecov.io/gh/apache/airflow/pull/7448/diff?src=pr&el=tree#diff-YWlyZmxvdy9qb2JzL3NjaGVkdWxlcl9qb2IucHk=)
 | `89.51% <0%> (-0.15%)` | :arrow_down: |
   | 
[...viders/docker/example\_dags/example\_docker\_swarm.py](https://codecov.io/gh/apache/airflow/pull/7448/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvZG9ja2VyL2V4YW1wbGVfZGFncy9leGFtcGxlX2RvY2tlcl9zd2FybS5weQ==)
 | `100% <0%> (ø)` | :arrow_up: |
   | 
[...ow/providers/docker/example\_dags/example\_docker.py](https://codecov.io/gh/apache/airflow/pull/7448/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvZG9ja2VyL2V4YW1wbGVfZGFncy9leGFtcGxlX2RvY2tlci5weQ==)
 | `100% <0%> (ø)` | :arrow_up: |
   
   --
   
   [Continue to review full report at 
Codecov](https://codecov.io/gh/apache/airflow/pull/7448?src=pr&el=continue).
   > **Legend** - [Click here to learn 
more](https://docs.codecov.io/docs/codecov-delta)
   > `Δ = absolute  (impact)`, `ø = not affected`, `? = missing data`
   > Powered by 
[Codecov](https://codecov.io/gh/apache/airflow/pull/7448?src=pr&el=footer). 
Last update 
[9cbd7de...a390889](https://codecov.io/gh/apache/airflow/pull/7448?src=pr&el=lastupdated).
 Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] codecov-io commented on issue #7449: [AIRFLOW-6823] Fix root owned files automatically in Breeze

2020-02-17 Thread GitBox 
codecov-io commented on issue #7449: [AIRFLOW-6823] Fix root owned files 
automatically in Breeze
URL: https://github.com/apache/airflow/pull/7449#issuecomment-587264083
 
 
   # [Codecov](https://codecov.io/gh/apache/airflow/pull/7449?src=pr&el=h1) 
Report
   > Merging 
[#7449](https://codecov.io/gh/apache/airflow/pull/7449?src=pr&el=desc) into 
[master](https://codecov.io/gh/apache/airflow/commit/dbcd3d8787741fd8203b6d9bdbc5d1da4b10a15b?src=pr&el=desc)
 will **not change** coverage.
   > The diff coverage is `n/a`.
   
   [![Impacted file tree 
graph](https://codecov.io/gh/apache/airflow/pull/7449/graphs/tree.svg?width=650&token=WdLKlKHOAU&height=150&src=pr)](https://codecov.io/gh/apache/airflow/pull/7449?src=pr&el=tree)
   
   ```diff
   @@   Coverage Diff   @@
   ##   master#7449   +/-   ##
   ===
 Coverage   86.66%   86.66%   
   ===
 Files 877  877   
 Lines   4124141241   
   ===
 Hits3574235742   
 Misses   5499 5499
   ```
   
   
   
   --
   
   [Continue to review full report at 
Codecov](https://codecov.io/gh/apache/airflow/pull/7449?src=pr&el=continue).
   > **Legend** - [Click here to learn 
more](https://docs.codecov.io/docs/codecov-delta)
   > `Δ = absolute  (impact)`, `ø = not affected`, `? = missing data`
   > Powered by 
[Codecov](https://codecov.io/gh/apache/airflow/pull/7449?src=pr&el=footer). 
Last update 
[dbcd3d8...23f55ad](https://codecov.io/gh/apache/airflow/pull/7449?src=pr&el=lastupdated).
 Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[jira] [Commented] (AIRFLOW-6829) Auto apply apply_default decorator to all sub-classess

2020-02-17 Thread ASF GitHub Bot (Jira) 


[ 
https://issues.apache.org/jira/browse/AIRFLOW-6829?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17038748#comment-17038748
 ] 

ASF GitHub Bot commented on AIRFLOW-6829:
-

mik-laj commented on pull request #7450: [AIRFLOW-6829] Introduce 
BaseOperatorMetaClass - auto-apply apply_default
URL: https://github.com/apache/airflow/pull/7450
 
 
   This decorator is confusing to new users. However, it can be applied 
automatically without the need to involve users through the use of metaclass.
   
   ---
   Issue link: WILL BE INSERTED BY 
[boring-cyborg](https://github.com/kaxil/boring-cyborg)
   
   Make sure to mark the boxes below before creating PR: [x]
   
   - [ ] Description above provides context of the change
   - [ ] Commit message/PR title starts with `[AIRFLOW-]`. AIRFLOW- = 
JIRA ID*
   - [ ] Unit tests coverage for changes (not needed for documentation changes)
   - [ ] Commits follow "[How to write a good git commit 
message](http://chris.beams.io/posts/git-commit/)"
   - [ ] Relevant documentation is updated including usage instructions.
   - [ ] I will engage committers as explained in [Contribution Workflow 
Example](https://github.com/apache/airflow/blob/master/CONTRIBUTING.rst#contribution-workflow-example).
   
   * For document-only changes commit message can start with 
`[AIRFLOW-]`.
   
   ---
   In case of fundamental code change, Airflow Improvement Proposal 
([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvements+Proposals))
 is needed.
   In case of a new dependency, check compliance with the [ASF 3rd Party 
License Policy](https://www.apache.org/legal/resolved.html#category-x).
   In case of backwards incompatible changes please leave a note in 
[UPDATING.md](https://github.com/apache/airflow/blob/master/UPDATING.md).
   Read the [Pull Request 
Guidelines](https://github.com/apache/airflow/blob/master/CONTRIBUTING.rst#pull-request-guidelines)
 for more information.
   
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


> Auto apply apply_default decorator to all sub-classess
> --
>
> Key: AIRFLOW-6829
> URL: https://issues.apache.org/jira/browse/AIRFLOW-6829
> Project: Apache Airflow
>  Issue Type: Bug
>  Components: core
>Affects Versions: 1.10.9
>Reporter: Kamil Bregula
>Priority: Major
>




--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[GitHub] [airflow] mik-laj opened a new pull request #7450: [AIRFLOW-6829] Introduce BaseOperatorMetaClass - auto-apply apply_default

2020-02-17 Thread GitBox 
mik-laj opened a new pull request #7450: [AIRFLOW-6829] Introduce 
BaseOperatorMetaClass - auto-apply apply_default
URL: https://github.com/apache/airflow/pull/7450
 
 
   This decorator is confusing to new users. However, it can be applied 
automatically without the need to involve users through the use of metaclass.
   
   ---
   Issue link: WILL BE INSERTED BY 
[boring-cyborg](https://github.com/kaxil/boring-cyborg)
   
   Make sure to mark the boxes below before creating PR: [x]
   
   - [ ] Description above provides context of the change
   - [ ] Commit message/PR title starts with `[AIRFLOW-]`. AIRFLOW- = 
JIRA ID*
   - [ ] Unit tests coverage for changes (not needed for documentation changes)
   - [ ] Commits follow "[How to write a good git commit 
message](http://chris.beams.io/posts/git-commit/)"
   - [ ] Relevant documentation is updated including usage instructions.
   - [ ] I will engage committers as explained in [Contribution Workflow 
Example](https://github.com/apache/airflow/blob/master/CONTRIBUTING.rst#contribution-workflow-example).
   
   * For document-only changes commit message can start with 
`[AIRFLOW-]`.
   
   ---
   In case of fundamental code change, Airflow Improvement Proposal 
([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvements+Proposals))
 is needed.
   In case of a new dependency, check compliance with the [ASF 3rd Party 
License Policy](https://www.apache.org/legal/resolved.html#category-x).
   In case of backwards incompatible changes please leave a note in 
[UPDATING.md](https://github.com/apache/airflow/blob/master/UPDATING.md).
   Read the [Pull Request 
Guidelines](https://github.com/apache/airflow/blob/master/CONTRIBUTING.rst#pull-request-guidelines)
 for more information.
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[jira] [Created] (AIRFLOW-6829) Auto apply apply_default decorator to all sub-classess

2020-02-17 Thread Kamil Bregula (Jira) 
Kamil Bregula created AIRFLOW-6829:
--

 Summary: Auto apply apply_default decorator to all sub-classess
 Key: AIRFLOW-6829
 URL: https://issues.apache.org/jira/browse/AIRFLOW-6829
 Project: Apache Airflow
  Issue Type: Bug
  Components: core
Affects Versions: 1.10.9
Reporter: Kamil Bregula






--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (AIRFLOW-6823) Breeze broken on current master on Linux

2020-02-17 Thread Jarek Potiuk (Jira) 


[ 
https://issues.apache.org/jira/browse/AIRFLOW-6823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17038743#comment-17038743
 ] 

Jarek Potiuk commented on AIRFLOW-6823:
---

Detecting before doing the build is definitely doable. Fix ready to review[: 
|https://github.com/apache/airflow/pull/7449]

[https://github.com/apache/airflow/pull/7449]

> Breeze broken on current master on Linux
> 
>
> Key: AIRFLOW-6823
> URL: https://issues.apache.org/jira/browse/AIRFLOW-6823
> Project: Apache Airflow
>  Issue Type: Bug
>  Components: breeze
>Affects Versions: 2.0.0
>Reporter: Aleksander Nitecki
>Assignee: Ash Berlin-Taylor
>Priority: Minor
> Attachments: Breeze on current master log.txt
>
>
> See the attachment for log.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (AIRFLOW-6823) Breeze broken on current master on Linux

2020-02-17 Thread ASF GitHub Bot (Jira) 


[ 
https://issues.apache.org/jira/browse/AIRFLOW-6823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17038741#comment-17038741
 ] 

ASF GitHub Bot commented on AIRFLOW-6823:
-

potiuk commented on pull request #7449: [AIRFLOW-6823] Fix root owned files 
automatically in Breeze
URL: https://github.com/apache/airflow/pull/7449
 
 
   ---
   Issue link: WILL BE INSERTED BY 
[boring-cyborg](https://github.com/kaxil/boring-cyborg)
   
   Make sure to mark the boxes below before creating PR: [x]
   
   - [x] Description above provides context of the change
   - [x] Commit message/PR title starts with `[AIRFLOW-]`. AIRFLOW- = 
JIRA ID*
   - [x] Unit tests coverage for changes (not needed for documentation changes)
   - [x] Commits follow "[How to write a good git commit 
message](http://chris.beams.io/posts/git-commit/)"
   - [x] Relevant documentation is updated including usage instructions.
   - [x] I will engage committers as explained in [Contribution Workflow 
Example](https://github.com/apache/airflow/blob/master/CONTRIBUTING.rst#contribution-workflow-example).
   
   * For document-only changes commit message can start with 
`[AIRFLOW-]`.
   
   ---
   In case of fundamental code change, Airflow Improvement Proposal 
([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvements+Proposals))
 is needed.
   In case of a new dependency, check compliance with the [ASF 3rd Party 
License Policy](https://www.apache.org/legal/resolved.html#category-x).
   In case of backwards incompatible changes please leave a note in 
[UPDATING.md](https://github.com/apache/airflow/blob/master/UPDATING.md).
   Read the [Pull Request 
Guidelines](https://github.com/apache/airflow/blob/master/CONTRIBUTING.rst#pull-request-guidelines)
 for more information.
   
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


> Breeze broken on current master on Linux
> 
>
> Key: AIRFLOW-6823
> URL: https://issues.apache.org/jira/browse/AIRFLOW-6823
> Project: Apache Airflow
>  Issue Type: Bug
>  Components: breeze
>Affects Versions: 2.0.0
>Reporter: Aleksander Nitecki
>Assignee: Ash Berlin-Taylor
>Priority: Minor
> Attachments: Breeze on current master log.txt
>
>
> See the attachment for log.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[GitHub] [airflow] potiuk opened a new pull request #7449: [AIRFLOW-6823] Fix root owned files automatically in Breeze

2020-02-17 Thread GitBox 
potiuk opened a new pull request #7449: [AIRFLOW-6823] Fix root owned files 
automatically in Breeze
URL: https://github.com/apache/airflow/pull/7449
 
 
   ---
   Issue link: WILL BE INSERTED BY 
[boring-cyborg](https://github.com/kaxil/boring-cyborg)
   
   Make sure to mark the boxes below before creating PR: [x]
   
   - [x] Description above provides context of the change
   - [x] Commit message/PR title starts with `[AIRFLOW-]`. AIRFLOW- = 
JIRA ID*
   - [x] Unit tests coverage for changes (not needed for documentation changes)
   - [x] Commits follow "[How to write a good git commit 
message](http://chris.beams.io/posts/git-commit/)"
   - [x] Relevant documentation is updated including usage instructions.
   - [x] I will engage committers as explained in [Contribution Workflow 
Example](https://github.com/apache/airflow/blob/master/CONTRIBUTING.rst#contribution-workflow-example).
   
   * For document-only changes commit message can start with 
`[AIRFLOW-]`.
   
   ---
   In case of fundamental code change, Airflow Improvement Proposal 
([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvements+Proposals))
 is needed.
   In case of a new dependency, check compliance with the [ASF 3rd Party 
License Policy](https://www.apache.org/legal/resolved.html#category-x).
   In case of backwards incompatible changes please leave a note in 
[UPDATING.md](https://github.com/apache/airflow/blob/master/UPDATING.md).
   Read the [Pull Request 
Guidelines](https://github.com/apache/airflow/blob/master/CONTRIBUTING.rst#pull-request-guidelines)
 for more information.
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] codecov-io commented on issue #6837: [AIRFLOW-6266] Better asserts in test_views.py

2020-02-17 Thread GitBox 
codecov-io commented on issue #6837: [AIRFLOW-6266] Better asserts in 
test_views.py
URL: https://github.com/apache/airflow/pull/6837#issuecomment-587241797
 
 
   # [Codecov](https://codecov.io/gh/apache/airflow/pull/6837?src=pr&el=h1) 
Report
   > Merging 
[#6837](https://codecov.io/gh/apache/airflow/pull/6837?src=pr&el=desc) into 
[master](https://codecov.io/gh/apache/airflow/commit/9cbd7de6d115795aba8bfb8addb060bfdfbdf87b?src=pr&el=desc)
 will **increase** coverage by `0.11%`.
   > The diff coverage is `n/a`.
   
   [![Impacted file tree 
graph](https://codecov.io/gh/apache/airflow/pull/6837/graphs/tree.svg?width=650&token=WdLKlKHOAU&height=150&src=pr)](https://codecov.io/gh/apache/airflow/pull/6837?src=pr&el=tree)
   
   ```diff
   @@Coverage Diff @@
   ##   master#6837  +/-   ##
   ==
   + Coverage   86.26%   86.37%   +0.11% 
   ==
 Files 877  877  
 Lines   4122141241  +20 
   ==
   + Hits3555935623  +64 
   + Misses   5662 5618  -44
   ```
   
   
   | [Impacted 
Files](https://codecov.io/gh/apache/airflow/pull/6837?src=pr&el=tree) | 
Coverage Δ | |
   |---|---|---|
   | 
[...viders/docker/example\_dags/example\_docker\_swarm.py](https://codecov.io/gh/apache/airflow/pull/6837/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvZG9ja2VyL2V4YW1wbGVfZGFncy9leGFtcGxlX2RvY2tlcl9zd2FybS5weQ==)
 | `100% <0%> (ø)` | :arrow_up: |
   | 
[...ow/providers/docker/example\_dags/example\_docker.py](https://codecov.io/gh/apache/airflow/pull/6837/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvZG9ja2VyL2V4YW1wbGVfZGFncy9leGFtcGxlX2RvY2tlci5weQ==)
 | `100% <0%> (ø)` | :arrow_up: |
   | 
[airflow/utils/dag\_processing.py](https://codecov.io/gh/apache/airflow/pull/6837/diff?src=pr&el=tree#diff-YWlyZmxvdy91dGlscy9kYWdfcHJvY2Vzc2luZy5weQ==)
 | `88.12% <0%> (+0.19%)` | :arrow_up: |
   | 
[airflow/jobs/scheduler\_job.py](https://codecov.io/gh/apache/airflow/pull/6837/diff?src=pr&el=tree#diff-YWlyZmxvdy9qb2JzL3NjaGVkdWxlcl9qb2IucHk=)
 | `89.66% <0%> (+0.43%)` | :arrow_up: |
   | 
[airflow/jobs/backfill\_job.py](https://codecov.io/gh/apache/airflow/pull/6837/diff?src=pr&el=tree#diff-YWlyZmxvdy9qb2JzL2JhY2tmaWxsX2pvYi5weQ==)
 | `92.09% <0%> (+1.12%)` | :arrow_up: |
   | 
[airflow/hooks/dbapi\_hook.py](https://codecov.io/gh/apache/airflow/pull/6837/diff?src=pr&el=tree#diff-YWlyZmxvdy9ob29rcy9kYmFwaV9ob29rLnB5)
 | `91.73% <0%> (+1.65%)` | :arrow_up: |
   | 
[airflow/providers/postgres/hooks/postgres.py](https://codecov.io/gh/apache/airflow/pull/6837/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvcG9zdGdyZXMvaG9va3MvcG9zdGdyZXMucHk=)
 | `94.36% <0%> (+16.9%)` | :arrow_up: |
   | 
[...roviders/google/cloud/operators/postgres\_to\_gcs.py](https://codecov.io/gh/apache/airflow/pull/6837/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvZ29vZ2xlL2Nsb3VkL29wZXJhdG9ycy9wb3N0Z3Jlc190b19nY3MucHk=)
 | `85.29% <0%> (+32.35%)` | :arrow_up: |
   | 
[airflow/providers/postgres/operators/postgres.py](https://codecov.io/gh/apache/airflow/pull/6837/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvcG9zdGdyZXMvb3BlcmF0b3JzL3Bvc3RncmVzLnB5)
 | `100% <0%> (+50%)` | :arrow_up: |
   
   --
   
   [Continue to review full report at 
Codecov](https://codecov.io/gh/apache/airflow/pull/6837?src=pr&el=continue).
   > **Legend** - [Click here to learn 
more](https://docs.codecov.io/docs/codecov-delta)
   > `Δ = absolute  (impact)`, `ø = not affected`, `? = missing data`
   > Powered by 
[Codecov](https://codecov.io/gh/apache/airflow/pull/6837?src=pr&el=footer). 
Last update 
[9cbd7de...f7fca6f](https://codecov.io/gh/apache/airflow/pull/6837?src=pr&el=lastupdated).
 Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] TV4Fun commented on issue #7098: [AIRFLOW-4453] Make behavior of `none_failed` consistent with documentation

2020-02-17 Thread GitBox 
TV4Fun commented on issue #7098: [AIRFLOW-4453] Make behavior of `none_failed` 
consistent with documentation
URL: https://github.com/apache/airflow/pull/7098#issuecomment-587240265
 
 
   Hi @yuqian90! I would like to work more on this, but am going to he very
   busy for the foreseeable future. If someone would like to take this over or
   make their own version, I have no objection.
   
   On Mon, Feb 17, 2020, 5:28 PM yuqian90  wrote:
   
   > Hi, @TV4Fun  would you like to continue
   > working on this PR? This is definitely a good change to make. The issue
   > regarding nested BranchPythonOperator can be addressed with something
   > like the create_join() in my previous comment. Please consider it. Thanks!
   >
   > —
   > You are receiving this because you were mentioned.
   > Reply to this email directly, view it on GitHub
   > 
,
   > or unsubscribe
   > 

   > .
   >
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] codecov-io edited a comment on issue #7389: [AIRFLOW-6763] Make systems tests ready for backport tests

2020-02-17 Thread GitBox 
codecov-io edited a comment on issue #7389: [AIRFLOW-6763] Make systems tests 
ready for backport tests
URL: https://github.com/apache/airflow/pull/7389#issuecomment-583898179
 
 
   # [Codecov](https://codecov.io/gh/apache/airflow/pull/7389?src=pr&el=h1) 
Report
   > Merging 
[#7389](https://codecov.io/gh/apache/airflow/pull/7389?src=pr&el=desc) into 
[master](https://codecov.io/gh/apache/airflow/commit/9cbd7de6d115795aba8bfb8addb060bfdfbdf87b?src=pr&el=desc)
 will **increase** coverage by `0.43%`.
   > The diff coverage is `n/a`.
   
   [![Impacted file tree 
graph](https://codecov.io/gh/apache/airflow/pull/7389/graphs/tree.svg?width=650&token=WdLKlKHOAU&height=150&src=pr)](https://codecov.io/gh/apache/airflow/pull/7389?src=pr&el=tree)
   
   ```diff
   @@Coverage Diff @@
   ##   master#7389  +/-   ##
   ==
   + Coverage   86.26%   86.69%   +0.43% 
   ==
 Files 877  877  
 Lines   4122141242  +21 
   ==
   + Hits3555935755 +196 
   + Misses   5662 5487 -175
   ```
   
   
   | [Impacted 
Files](https://codecov.io/gh/apache/airflow/pull/7389?src=pr&el=tree) | 
Coverage Δ | |
   |---|---|---|
   | 
[...viders/docker/example\_dags/example\_docker\_swarm.py](https://codecov.io/gh/apache/airflow/pull/7389/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvZG9ja2VyL2V4YW1wbGVfZGFncy9leGFtcGxlX2RvY2tlcl9zd2FybS5weQ==)
 | `100% <0%> (ø)` | :arrow_up: |
   | 
[...ow/providers/docker/example\_dags/example\_docker.py](https://codecov.io/gh/apache/airflow/pull/7389/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvZG9ja2VyL2V4YW1wbGVfZGFncy9leGFtcGxlX2RvY2tlci5weQ==)
 | `100% <0%> (ø)` | :arrow_up: |
   | 
[airflow/utils/dag\_processing.py](https://codecov.io/gh/apache/airflow/pull/7389/diff?src=pr&el=tree#diff-YWlyZmxvdy91dGlscy9kYWdfcHJvY2Vzc2luZy5weQ==)
 | `88.12% <0%> (+0.19%)` | :arrow_up: |
   | 
[airflow/models/dag.py](https://codecov.io/gh/apache/airflow/pull/7389/diff?src=pr&el=tree#diff-YWlyZmxvdy9tb2RlbHMvZGFnLnB5)
 | `91.16% <0%> (+0.25%)` | :arrow_up: |
   | 
[airflow/jobs/scheduler\_job.py](https://codecov.io/gh/apache/airflow/pull/7389/diff?src=pr&el=tree#diff-YWlyZmxvdy9qb2JzL3NjaGVkdWxlcl9qb2IucHk=)
 | `89.66% <0%> (+0.43%)` | :arrow_up: |
   | 
[airflow/jobs/backfill\_job.py](https://codecov.io/gh/apache/airflow/pull/7389/diff?src=pr&el=tree#diff-YWlyZmxvdy9qb2JzL2JhY2tmaWxsX2pvYi5weQ==)
 | `92.09% <0%> (+1.12%)` | :arrow_up: |
   | 
[airflow/hooks/dbapi\_hook.py](https://codecov.io/gh/apache/airflow/pull/7389/diff?src=pr&el=tree#diff-YWlyZmxvdy9ob29rcy9kYmFwaV9ob29rLnB5)
 | `91.73% <0%> (+1.65%)` | :arrow_up: |
   | 
[airflow/configuration.py](https://codecov.io/gh/apache/airflow/pull/7389/diff?src=pr&el=tree#diff-YWlyZmxvdy9jb25maWd1cmF0aW9uLnB5)
 | `91.43% <0%> (+3.42%)` | :arrow_up: |
   | 
[airflow/providers/postgres/hooks/postgres.py](https://codecov.io/gh/apache/airflow/pull/7389/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvcG9zdGdyZXMvaG9va3MvcG9zdGdyZXMucHk=)
 | `94.36% <0%> (+16.9%)` | :arrow_up: |
   | 
[airflow/kubernetes/refresh\_config.py](https://codecov.io/gh/apache/airflow/pull/7389/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3JlZnJlc2hfY29uZmlnLnB5)
 | `74.5% <0%> (+23.52%)` | :arrow_up: |
   | ... and [6 
more](https://codecov.io/gh/apache/airflow/pull/7389/diff?src=pr&el=tree-more) 
| |
   
   --
   
   [Continue to review full report at 
Codecov](https://codecov.io/gh/apache/airflow/pull/7389?src=pr&el=continue).
   > **Legend** - [Click here to learn 
more](https://docs.codecov.io/docs/codecov-delta)
   > `Δ = absolute  (impact)`, `ø = not affected`, `? = missing data`
   > Powered by 
[Codecov](https://codecov.io/gh/apache/airflow/pull/7389?src=pr&el=footer). 
Last update 
[9cbd7de...9751b69](https://codecov.io/gh/apache/airflow/pull/7389?src=pr&el=lastupdated).
 Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[jira] [Comment Edited] (AIRFLOW-6823) Breeze broken on current master on Linux

2020-02-17 Thread Jarek Potiuk (Jira) 


[ 
https://issues.apache.org/jira/browse/AIRFLOW-6823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17038727#comment-17038727
 ] 

Jarek Potiuk edited comment on AIRFLOW-6823 at 2/18/20 1:53 AM:


It wont work [~ash] - > the problem is that the airflow files need to be 
accessible (runnable scripts for example) by the logged-in user. There are a 
number of scripts and binaries that are only runnable for the owner, not for 
groups or others - similarly the directories are not group executable nor other 
executable. That will prevent the new user from accessing them. 

We would have to know that user id while we are building Docker (but we can't 
as the user might be different on each installation). Another workaround is 
changing ownership of all files at entry (but this makes no sense for us). What 
could work is user remap feature 
[https://www.jujens.eu/posts/en/2017/Jul/02/docker-userns-remap/] but it needs 
to be configured at docker engine level.


was (Author: higrys):
It wont work [~ash] - > the problem is that the airflow files need to be 
accessible (runnable scripts for example) by the logged user. There are a 
number of scripts and binaries that are only runnable for the owner, not for 
groups or others - similarly the directories are not group executable nor other 
executable. That will prevent the new user from accessing them. 

We would have to know that user id while we are building Docker (but we can't 
as the user might be different on each installation). Another workaround is 
changing ownership of all files at entry (but this makes no sense for us). What 
could work is user remap feature 
[https://www.jujens.eu/posts/en/2017/Jul/02/docker-userns-remap/] but it needs 
to be configured at docker engine level.

> Breeze broken on current master on Linux
> 
>
> Key: AIRFLOW-6823
> URL: https://issues.apache.org/jira/browse/AIRFLOW-6823
> Project: Apache Airflow
>  Issue Type: Bug
>  Components: breeze
>Affects Versions: 2.0.0
>Reporter: Aleksander Nitecki
>Assignee: Ash Berlin-Taylor
>Priority: Minor
> Attachments: Breeze on current master log.txt
>
>
> See the attachment for log.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (AIRFLOW-6823) Breeze broken on current master on Linux

2020-02-17 Thread Jarek Potiuk (Jira) 


[ 
https://issues.apache.org/jira/browse/AIRFLOW-6823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17038727#comment-17038727
 ] 

Jarek Potiuk commented on AIRFLOW-6823:
---

It wont work [~ash] - > the problem is that the airflow files need to be 
accessible (runnable scripts for example) by the logged user. There are a 
number of scripts and binaries that are only runnable for the owner, not for 
groups or others - similarly the directories are not group executable nor other 
executable. That will prevent the new user from accessing them. 

We would have to know that user id while we are building Docker (but we can't 
as the user might be different on each installation). Another workaround is 
changing ownership of all files at entry (but this makes no sense for us). What 
could work is user remap feature 
[https://www.jujens.eu/posts/en/2017/Jul/02/docker-userns-remap/] but it needs 
to be configured at docker engine level.

> Breeze broken on current master on Linux
> 
>
> Key: AIRFLOW-6823
> URL: https://issues.apache.org/jira/browse/AIRFLOW-6823
> Project: Apache Airflow
>  Issue Type: Bug
>  Components: breeze
>Affects Versions: 2.0.0
>Reporter: Aleksander Nitecki
>Assignee: Ash Berlin-Taylor
>Priority: Minor
> Attachments: Breeze on current master log.txt
>
>
> See the attachment for log.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[GitHub] [airflow] yuqian90 commented on issue #7098: [AIRFLOW-4453] Make behavior of `none_failed` consistent with documentation

2020-02-17 Thread GitBox 
yuqian90 commented on issue #7098: [AIRFLOW-4453] Make behavior of 
`none_failed` consistent with documentation
URL: https://github.com/apache/airflow/pull/7098#issuecomment-587231424
 
 
   Hi, @TV4Fun would you like to continue working on this PR? This is 
definitely a good change to make. The issue regarding nested 
`BranchPythonOperator` can be addressed with something like the `create_join()` 
in my previous comment. Please consider it. Thanks!


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] yuqian90 edited a comment on issue #7276: [AIRFLOW-5391] Do not run skipped tasks when they are cleared

2020-02-17 Thread GitBox 
yuqian90 edited a comment on issue #7276: [AIRFLOW-5391] Do not run skipped 
tasks when they are cleared
URL: https://github.com/apache/airflow/pull/7276#issuecomment-587230400
 
 
   Hi, @ashb  thanks for pointing out. I've fixed those. 
   
   Travis is still unhappy, but I'm pretty sure this is unrelated to my change. 
Not sure if anyone already brought this up in a JIRA somewhere:
   
   ```
   tests/cli/commands/test_task_command.py .F
   _ TestCliTasks.test_local_run 
__
   self = 
   def test_local_run(self):
   args = self.parser.parse_args([
   ...
   def _recv(self, size, read=_read):
   buf = io.BytesIO()
   handle = self._handle
   remaining = size
   while remaining > 0:
   chunk = read(handle, remaining)
   n = len(chunk)
   if n == 0:
   if remaining == size:
   >   raise EOFError
   E   EOFError
   /usr/local/lib/python3.7/multiprocessing/connection.py:383: EOFError
   ```


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] yuqian90 commented on issue #7276: [AIRFLOW-5391] Do not run skipped tasks when they are cleared

2020-02-17 Thread GitBox 
yuqian90 commented on issue #7276: [AIRFLOW-5391] Do not run skipped tasks when 
they are cleared
URL: https://github.com/apache/airflow/pull/7276#issuecomment-587230400
 
 
   Hi, @ashb  thanks for pointing out. I've fixed those. 
   
   Travis is still unhappy, but I'm pretty sure this is unrelated to my change:


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] potiuk commented on issue #7433: [AIRFLOW-6820] split breeze into functions

2020-02-17 Thread GitBox 
potiuk commented on issue #7433: [AIRFLOW-6820] split breeze into functions
URL: https://github.com/apache/airflow/pull/7433#issuecomment-587228520
 
 
   > We should make sure this works for pyenv, but at a code level this LGTM
   
   Confirmed it works fr both pyenv and virtualenv.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[jira] [Commented] (AIRFLOW-6804) Add the basic test for example DAGs

2020-02-17 Thread ASF subversion and git services (Jira) 


[ 
https://issues.apache.org/jira/browse/AIRFLOW-6804?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17038712#comment-17038712
 ] 

ASF subversion and git services commented on AIRFLOW-6804:
--

Commit dbcd3d8787741fd8203b6d9bdbc5d1da4b10a15b in airflow's branch 
refs/heads/master from Kamil Breguła
[ https://gitbox.apache.org/repos/asf?p=airflow.git;h=dbcd3d8 ]

[AIRFLOW-6804] Add the basic test for all example DAGs (#7419)

Co-Authored-By: Tomek Urbaszek 


> Add the basic test for example DAGs
> ---
>
> Key: AIRFLOW-6804
> URL: https://issues.apache.org/jira/browse/AIRFLOW-6804
> Project: Apache Airflow
>  Issue Type: Bug
>  Components: documentation
>Affects Versions: 1.10.9
>Reporter: Kamil Bregula
>Priority: Major
> Fix For: 2.0.0
>
>




--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Resolved] (AIRFLOW-6804) Add the basic test for example DAGs

2020-02-17 Thread Kamil Bregula (Jira) 


 [ 
https://issues.apache.org/jira/browse/AIRFLOW-6804?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kamil Bregula resolved AIRFLOW-6804.

Fix Version/s: 2.0.0
   Resolution: Fixed

> Add the basic test for example DAGs
> ---
>
> Key: AIRFLOW-6804
> URL: https://issues.apache.org/jira/browse/AIRFLOW-6804
> Project: Apache Airflow
>  Issue Type: Bug
>  Components: documentation
>Affects Versions: 1.10.9
>Reporter: Kamil Bregula
>Priority: Major
> Fix For: 2.0.0
>
>




--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (AIRFLOW-6804) Add the basic test for example DAGs

2020-02-17 Thread ASF GitHub Bot (Jira) 


[ 
https://issues.apache.org/jira/browse/AIRFLOW-6804?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17038711#comment-17038711
 ] 

ASF GitHub Bot commented on AIRFLOW-6804:
-

mik-laj commented on pull request #7419: [AIRFLOW-6804] Add the basic test for 
all example DAGs
URL: https://github.com/apache/airflow/pull/7419
 
 
   
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


> Add the basic test for example DAGs
> ---
>
> Key: AIRFLOW-6804
> URL: https://issues.apache.org/jira/browse/AIRFLOW-6804
> Project: Apache Airflow
>  Issue Type: Bug
>  Components: documentation
>Affects Versions: 1.10.9
>Reporter: Kamil Bregula
>Priority: Major
>




--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[GitHub] [airflow] mik-laj merged pull request #7419: [AIRFLOW-6804] Add the basic test for all example DAGs

2020-02-17 Thread GitBox 
mik-laj merged pull request #7419: [AIRFLOW-6804] Add the basic test for all 
example DAGs
URL: https://github.com/apache/airflow/pull/7419
 
 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[jira] [Commented] (AIRFLOW-6828) Drop zope package

2020-02-17 Thread ASF GitHub Bot (Jira) 


[ 
https://issues.apache.org/jira/browse/AIRFLOW-6828?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17038705#comment-17038705
 ] 

ASF GitHub Bot commented on AIRFLOW-6828:
-

mik-laj commented on pull request #7448: [AIRFLOW-6828] Stop using the zope 
library
URL: https://github.com/apache/airflow/pull/7448
 
 
   In very many places, we use a native method to report a deprecation warning.
   ```
   01:53 $ git ls-files |grep -v "tests/" | xargs grep -l DeprecationWarning  | 
wc -l
318
   ```
   On other hands, zope deprecation is used in two files only, so I think we 
should abandon the use of this library to unify the code.
   
   The native method is also better supported by the IDE. 
   
   ---
   Issue link: WILL BE INSERTED BY 
[boring-cyborg](https://github.com/kaxil/boring-cyborg)
   
   Make sure to mark the boxes below before creating PR: [x]
   
   - [X] Description above provides context of the change
   - [X] Commit message/PR title starts with `[AIRFLOW-]`. AIRFLOW- = 
JIRA ID*
   - [X] Unit tests coverage for changes (not needed for documentation changes)
   - [X] Commits follow "[How to write a good git commit 
message](http://chris.beams.io/posts/git-commit/)"
   - [X] Relevant documentation is updated including usage instructions.
   - [X] I will engage committers as explained in [Contribution Workflow 
Example](https://github.com/apache/airflow/blob/master/CONTRIBUTING.rst#contribution-workflow-example).
   
   * For document-only changes commit message can start with 
`[AIRFLOW-]`.
   
   ---
   In case of fundamental code change, Airflow Improvement Proposal 
([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvements+Proposals))
 is needed.
   In case of a new dependency, check compliance with the [ASF 3rd Party 
License Policy](https://www.apache.org/legal/resolved.html#category-x).
   In case of backwards incompatible changes please leave a note in 
[UPDATING.md](https://github.com/apache/airflow/blob/master/UPDATING.md).
   Read the [Pull Request 
Guidelines](https://github.com/apache/airflow/blob/master/CONTRIBUTING.rst#pull-request-guidelines)
 for more information.
   
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


> Drop zope package
> -
>
> Key: AIRFLOW-6828
> URL: https://issues.apache.org/jira/browse/AIRFLOW-6828
> Project: Apache Airflow
>  Issue Type: Bug
>  Components: core
>Affects Versions: 1.10.9
>Reporter: Kamil Bregula
>Priority: Major
>




--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[GitHub] [airflow] mik-laj opened a new pull request #7448: [AIRFLOW-6828] Stop using the zope library

2020-02-17 Thread GitBox 
mik-laj opened a new pull request #7448: [AIRFLOW-6828] Stop using the zope 
library
URL: https://github.com/apache/airflow/pull/7448
 
 
   In very many places, we use a native method to report a deprecation warning.
   ```
   01:53 $ git ls-files |grep -v "tests/" | xargs grep -l DeprecationWarning  | 
wc -l
318
   ```
   On other hands, zope deprecation is used in two files only, so I think we 
should abandon the use of this library to unify the code.
   
   The native method is also better supported by the IDE. 
   
   ---
   Issue link: WILL BE INSERTED BY 
[boring-cyborg](https://github.com/kaxil/boring-cyborg)
   
   Make sure to mark the boxes below before creating PR: [x]
   
   - [X] Description above provides context of the change
   - [X] Commit message/PR title starts with `[AIRFLOW-]`. AIRFLOW- = 
JIRA ID*
   - [X] Unit tests coverage for changes (not needed for documentation changes)
   - [X] Commits follow "[How to write a good git commit 
message](http://chris.beams.io/posts/git-commit/)"
   - [X] Relevant documentation is updated including usage instructions.
   - [X] I will engage committers as explained in [Contribution Workflow 
Example](https://github.com/apache/airflow/blob/master/CONTRIBUTING.rst#contribution-workflow-example).
   
   * For document-only changes commit message can start with 
`[AIRFLOW-]`.
   
   ---
   In case of fundamental code change, Airflow Improvement Proposal 
([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvements+Proposals))
 is needed.
   In case of a new dependency, check compliance with the [ASF 3rd Party 
License Policy](https://www.apache.org/legal/resolved.html#category-x).
   In case of backwards incompatible changes please leave a note in 
[UPDATING.md](https://github.com/apache/airflow/blob/master/UPDATING.md).
   Read the [Pull Request 
Guidelines](https://github.com/apache/airflow/blob/master/CONTRIBUTING.rst#pull-request-guidelines)
 for more information.
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[jira] [Created] (AIRFLOW-6828) Drop zope package

2020-02-17 Thread Kamil Bregula (Jira) 
Kamil Bregula created AIRFLOW-6828:
--

 Summary: Drop zope package
 Key: AIRFLOW-6828
 URL: https://issues.apache.org/jira/browse/AIRFLOW-6828
 Project: Apache Airflow
  Issue Type: Bug
  Components: core
Affects Versions: 1.10.9
Reporter: Kamil Bregula






--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[GitHub] [airflow] potiuk commented on a change in pull request #7433: [AIRFLOW-6820] split breeze into functions

2020-02-17 Thread GitBox 
potiuk commented on a change in pull request #7433: [AIRFLOW-6820] split breeze 
into functions
URL: https://github.com/apache/airflow/pull/7433#discussion_r380409876
 
 

 ##
 File path: breeze
 ##
 @@ -16,146 +16,245 @@
 # KIND, either express or implied.  See the License for the
 # specific language governing permissions and limitations
 # under the License.
-
-# Bash sanity settings (error on exit, complain for undefined vars, error when 
pipe fails)
 set -euo pipefail
 
-export BREEZE=true
-
 MY_DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )"
 
-export AIRFLOW_SOURCES="${MY_DIR}"
-
-# Directory where all CI scripts are located
-export SCRIPTS_CI_DIR="${MY_DIR}/scripts/ci"
-
-BUILD_CACHE_DIR="${MY_DIR}/.build"
-FILES_DIR="${MY_DIR}/files"
-TMP_DIR="${MY_DIR}/tmp"
-
-mkdir -pv "${BUILD_CACHE_DIR}"
-mkdir -pv "${TMP_DIR}"
-mkdir -pv "${FILES_DIR}"
-
-# Note - we do not use __script_init.sh here because it can only be used from 
within
-# the CI directory and we need to overrride PYTHON_VERSION based on what we 
store
-# in the .build directory
+# Bash arrays need to be defined outside of functions unfortunately :(
+# Array with extra options for Docker compose
+declare -a EXTRA_DC_OPTIONS
+# Array with selected integrations
+declare -a INTEGRATIONS
+# This is where remaining args are passed
+declare -a REMAINING_ARGS
 
-# Beginning of the initialisation here
+function setup_default_breeze_variables() {
+# Whether to actually run docker compose with the command set given
+export COMMAND_TO_RUN="enter_breeze"
 
 Review comment:
   Reviewed


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] potiuk commented on issue #7204: [AIRFLOW-XXXX] - add communication chapter to contributing

2020-02-17 Thread GitBox 
potiuk commented on issue #7204: [AIRFLOW-] - add communication chapter to 
contributing
URL: https://github.com/apache/airflow/pull/7204#issuecomment-587213272
 
 
   thanks @mschickensoup  for comments
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] mik-laj commented on a change in pull request #7419: [AIRFLOW-6804][depends on AIRFLOW-6792] Add the basic test for all example DAGs

2020-02-17 Thread GitBox 
mik-laj commented on a change in pull request #7419:  [AIRFLOW-6804][depends on 
AIRFLOW-6792] Add the basic test for all example DAGs
URL: https://github.com/apache/airflow/pull/7419#discussion_r380403509
 
 

 ##
 File path: airflow/providers/qubole/example_dags/example_qubole.py
 ##
 @@ -51,6 +42,18 @@
 schedule_interval=None,
 tags=['example'],
 ) as dag:
+dag.doc_md = textwrap.dedent(
 
 Review comment:
   Because now this file contains two DAGs, so they contain two different 
documentation. 


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[jira] [Commented] (AIRFLOW-6792) Remove _operator/_hook/_sensor in providers package and add tests

2020-02-17 Thread ASF subversion and git services (Jira) 


[ 
https://issues.apache.org/jira/browse/AIRFLOW-6792?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17038690#comment-17038690
 ] 

ASF subversion and git services commented on AIRFLOW-6792:
--

Commit 9cbd7de6d115795aba8bfb8addb060bfdfbdf87b in airflow's branch 
refs/heads/master from Kamil Breguła
[ https://gitbox.apache.org/repos/asf?p=airflow.git;h=9cbd7de ]

[AIRFLOW-6792] Remove _operator/_hook/_sensor in providers package and add 
tests (#7412)



> Remove _operator/_hook/_sensor in providers package and add tests
> -
>
> Key: AIRFLOW-6792
> URL: https://issues.apache.org/jira/browse/AIRFLOW-6792
> Project: Apache Airflow
>  Issue Type: Improvement
>  Components: hooks, operators
>Affects Versions: 1.10.9
>Reporter: Kamil Bregula
>Priority: Major
> Fix For: 2.0.0
>
>




--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Resolved] (AIRFLOW-6792) Remove _operator/_hook/_sensor in providers package and add tests

2020-02-17 Thread Kamil Bregula (Jira) 


 [ 
https://issues.apache.org/jira/browse/AIRFLOW-6792?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kamil Bregula resolved AIRFLOW-6792.

Fix Version/s: 2.0.0
   Resolution: Fixed

> Remove _operator/_hook/_sensor in providers package and add tests
> -
>
> Key: AIRFLOW-6792
> URL: https://issues.apache.org/jira/browse/AIRFLOW-6792
> Project: Apache Airflow
>  Issue Type: Improvement
>  Components: hooks, operators
>Affects Versions: 1.10.9
>Reporter: Kamil Bregula
>Priority: Major
> Fix For: 2.0.0
>
>




--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (AIRFLOW-6792) Remove _operator/_hook/_sensor in providers package and add tests

2020-02-17 Thread ASF GitHub Bot (Jira) 


[ 
https://issues.apache.org/jira/browse/AIRFLOW-6792?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17038689#comment-17038689
 ] 

ASF GitHub Bot commented on AIRFLOW-6792:
-

mik-laj commented on pull request #7412: [AIRFLOW-6792] Remove 
_operator/_hook/_sensor in providers package and add tests
URL: https://github.com/apache/airflow/pull/7412
 
 
   
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


> Remove _operator/_hook/_sensor in providers package and add tests
> -
>
> Key: AIRFLOW-6792
> URL: https://issues.apache.org/jira/browse/AIRFLOW-6792
> Project: Apache Airflow
>  Issue Type: Improvement
>  Components: hooks, operators
>Affects Versions: 1.10.9
>Reporter: Kamil Bregula
>Priority: Major
>




--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[GitHub] [airflow] mik-laj merged pull request #7412: [AIRFLOW-6792] Remove _operator/_hook/_sensor in providers package and add tests

2020-02-17 Thread GitBox 
mik-laj merged pull request #7412: [AIRFLOW-6792] Remove 
_operator/_hook/_sensor in providers package and add tests
URL: https://github.com/apache/airflow/pull/7412
 
 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] codecov-io commented on issue #7412: [AIRFLOW-6792] Remove _operator/_hook/_sensor in providers package and add tests

2020-02-17 Thread GitBox 
codecov-io commented on issue #7412: [AIRFLOW-6792] Remove 
_operator/_hook/_sensor in providers package and add tests
URL: https://github.com/apache/airflow/pull/7412#issuecomment-587207652
 
 
   # [Codecov](https://codecov.io/gh/apache/airflow/pull/7412?src=pr&el=h1) 
Report
   > Merging 
[#7412](https://codecov.io/gh/apache/airflow/pull/7412?src=pr&el=desc) into 
[master](https://codecov.io/gh/apache/airflow/commit/5b199cb86be5b1aefbd8620185033d6f635713c1?src=pr&el=desc)
 will **decrease** coverage by `0.38%`.
   > The diff coverage is `100%`.
   
   [![Impacted file tree 
graph](https://codecov.io/gh/apache/airflow/pull/7412/graphs/tree.svg?width=650&token=WdLKlKHOAU&height=150&src=pr)](https://codecov.io/gh/apache/airflow/pull/7412?src=pr&el=tree)
   
   ```diff
   @@Coverage Diff @@
   ##   master#7412  +/-   ##
   ==
   - Coverage   86.66%   86.27%   -0.39% 
   ==
 Files 878  877   -1 
 Lines   4121941221   +2 
   ==
   - Hits3572235564 -158 
   - Misses   5497 5657 +160
   ```
   
   
   | [Impacted 
Files](https://codecov.io/gh/apache/airflow/pull/7412?src=pr&el=tree) | 
Coverage Δ | |
   |---|---|---|
   | 
[...ow/providers/docker/example\_dags/example\_docker.py](https://codecov.io/gh/apache/airflow/pull/7412/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvZG9ja2VyL2V4YW1wbGVfZGFncy9leGFtcGxlX2RvY2tlci5weQ==)
 | `100% <ø> (ø)` | |
   | 
[...w/providers/apache/pig/example\_dags/example\_pig.py](https://codecov.io/gh/apache/airflow/pull/7412/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvYXBhY2hlL3BpZy9leGFtcGxlX2RhZ3MvZXhhbXBsZV9waWcucHk=)
 | `100% <ø> (ø)` | |
   | 
[...viders/docker/example\_dags/example\_docker\_swarm.py](https://codecov.io/gh/apache/airflow/pull/7412/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvZG9ja2VyL2V4YW1wbGVfZGFncy9leGFtcGxlX2RvY2tlcl9zd2FybS5weQ==)
 | `100% <ø> (ø)` | |
   | 
[.../providers/google/suite/operators/gcs\_to\_gdrive.py](https://codecov.io/gh/apache/airflow/pull/7412/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvZ29vZ2xlL3N1aXRlL29wZXJhdG9ycy9nY3NfdG9fZ2RyaXZlLnB5)
 | `100% <ø> (ø)` | |
   | 
[...irflow/providers/http/example\_dags/example\_http.py](https://codecov.io/gh/apache/airflow/pull/7412/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvaHR0cC9leGFtcGxlX2RhZ3MvZXhhbXBsZV9odHRwLnB5)
 | `100% <ø> (ø)` | |
   | 
[...cncf/kubernetes/example\_dags/example\_kubernetes.py](https://codecov.io/gh/apache/airflow/pull/7412/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvY25jZi9rdWJlcm5ldGVzL2V4YW1wbGVfZGFncy9leGFtcGxlX2t1YmVybmV0ZXMucHk=)
 | `78.57% <ø> (ø)` | |
   | 
[...providers/google/cloud/example\_dags/example\_dlp.py](https://codecov.io/gh/apache/airflow/pull/7412/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvZ29vZ2xlL2Nsb3VkL2V4YW1wbGVfZGFncy9leGFtcGxlX2RscC5weQ==)
 | `100% <ø> (ø)` | |
   | 
[...low/providers/yandex/hooks/yandexcloud\_dataproc.py](https://codecov.io/gh/apache/airflow/pull/7412/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMveWFuZGV4L2hvb2tzL3lhbmRleGNsb3VkX2RhdGFwcm9jLnB5)
 | `100% <ø> (ø)` | |
   | 
[.../cloud/operators/cloud\_storage\_transfer\_service.py](https://codecov.io/gh/apache/airflow/pull/7412/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvZ29vZ2xlL2Nsb3VkL29wZXJhdG9ycy9jbG91ZF9zdG9yYWdlX3RyYW5zZmVyX3NlcnZpY2UucHk=)
 | `95.63% <100%> (ø)` | :arrow_up: |
   | 
[...flow/providers/amazon/aws/hooks/lambda\_function.py](https://codecov.io/gh/apache/airflow/pull/7412/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvYW1hem9uL2F3cy9ob29rcy9sYW1iZGFfZnVuY3Rpb24ucHk=)
 | `100% <100%> (ø)` | :arrow_up: |
   | ... and [43 
more](https://codecov.io/gh/apache/airflow/pull/7412/diff?src=pr&el=tree-more) 
| |
   
   --
   
   [Continue to review full report at 
Codecov](https://codecov.io/gh/apache/airflow/pull/7412?src=pr&el=continue).
   > **Legend** - [Click here to learn 
more](https://docs.codecov.io/docs/codecov-delta)
   > `Δ = absolute  (impact)`, `ø = not affected`, `? = missing data`
   > Powered by 
[Codecov](https://codecov.io/gh/apache/airflow/pull/7412?src=pr&el=footer). 
Last update 
[5b199cb...8344222](https://codecov.io/gh/apache/airflow/pull/7412?src=pr&el=lastupdated).
 Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] codecov-io edited a comment on issue #7412: [AIRFLOW-6792] Remove _operator/_hook/_sensor in providers package and add tests

2020-02-17 Thread GitBox 
codecov-io edited a comment on issue #7412: [AIRFLOW-6792] Remove 
_operator/_hook/_sensor in providers package and add tests
URL: https://github.com/apache/airflow/pull/7412#issuecomment-587207652
 
 
   # [Codecov](https://codecov.io/gh/apache/airflow/pull/7412?src=pr&el=h1) 
Report
   > Merging 
[#7412](https://codecov.io/gh/apache/airflow/pull/7412?src=pr&el=desc) into 
[master](https://codecov.io/gh/apache/airflow/commit/5b199cb86be5b1aefbd8620185033d6f635713c1?src=pr&el=desc)
 will **decrease** coverage by `0.29%`.
   > The diff coverage is `100%`.
   
   [![Impacted file tree 
graph](https://codecov.io/gh/apache/airflow/pull/7412/graphs/tree.svg?width=650&token=WdLKlKHOAU&height=150&src=pr)](https://codecov.io/gh/apache/airflow/pull/7412?src=pr&el=tree)
   
   ```diff
   @@Coverage Diff@@
   ##   master#7412 +/-   ##
   =
   - Coverage   86.66%   86.37%   -0.3% 
   =
 Files 878  877  -1 
 Lines   4121941221  +2 
   =
   - Hits3572235603-119 
   - Misses   5497 5618+121
   ```
   
   
   | [Impacted 
Files](https://codecov.io/gh/apache/airflow/pull/7412?src=pr&el=tree) | 
Coverage Δ | |
   |---|---|---|
   | 
[...ow/providers/docker/example\_dags/example\_docker.py](https://codecov.io/gh/apache/airflow/pull/7412/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvZG9ja2VyL2V4YW1wbGVfZGFncy9leGFtcGxlX2RvY2tlci5weQ==)
 | `100% <ø> (ø)` | |
   | 
[...w/providers/apache/pig/example\_dags/example\_pig.py](https://codecov.io/gh/apache/airflow/pull/7412/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvYXBhY2hlL3BpZy9leGFtcGxlX2RhZ3MvZXhhbXBsZV9waWcucHk=)
 | `100% <ø> (ø)` | |
   | 
[...viders/docker/example\_dags/example\_docker\_swarm.py](https://codecov.io/gh/apache/airflow/pull/7412/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvZG9ja2VyL2V4YW1wbGVfZGFncy9leGFtcGxlX2RvY2tlcl9zd2FybS5weQ==)
 | `100% <ø> (ø)` | |
   | 
[.../providers/google/suite/operators/gcs\_to\_gdrive.py](https://codecov.io/gh/apache/airflow/pull/7412/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvZ29vZ2xlL3N1aXRlL29wZXJhdG9ycy9nY3NfdG9fZ2RyaXZlLnB5)
 | `100% <ø> (ø)` | |
   | 
[...irflow/providers/http/example\_dags/example\_http.py](https://codecov.io/gh/apache/airflow/pull/7412/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvaHR0cC9leGFtcGxlX2RhZ3MvZXhhbXBsZV9odHRwLnB5)
 | `100% <ø> (ø)` | |
   | 
[...cncf/kubernetes/example\_dags/example\_kubernetes.py](https://codecov.io/gh/apache/airflow/pull/7412/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvY25jZi9rdWJlcm5ldGVzL2V4YW1wbGVfZGFncy9leGFtcGxlX2t1YmVybmV0ZXMucHk=)
 | `78.57% <ø> (ø)` | |
   | 
[...providers/google/cloud/example\_dags/example\_dlp.py](https://codecov.io/gh/apache/airflow/pull/7412/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvZ29vZ2xlL2Nsb3VkL2V4YW1wbGVfZGFncy9leGFtcGxlX2RscC5weQ==)
 | `100% <ø> (ø)` | |
   | 
[...low/providers/yandex/hooks/yandexcloud\_dataproc.py](https://codecov.io/gh/apache/airflow/pull/7412/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMveWFuZGV4L2hvb2tzL3lhbmRleGNsb3VkX2RhdGFwcm9jLnB5)
 | `100% <ø> (ø)` | |
   | 
[.../cloud/operators/cloud\_storage\_transfer\_service.py](https://codecov.io/gh/apache/airflow/pull/7412/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvZ29vZ2xlL2Nsb3VkL29wZXJhdG9ycy9jbG91ZF9zdG9yYWdlX3RyYW5zZmVyX3NlcnZpY2UucHk=)
 | `95.63% <100%> (ø)` | :arrow_up: |
   | 
[...flow/providers/amazon/aws/hooks/lambda\_function.py](https://codecov.io/gh/apache/airflow/pull/7412/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvYW1hem9uL2F3cy9ob29rcy9sYW1iZGFfZnVuY3Rpb24ucHk=)
 | `100% <100%> (ø)` | :arrow_up: |
   | ... and [43 
more](https://codecov.io/gh/apache/airflow/pull/7412/diff?src=pr&el=tree-more) 
| |
   
   --
   
   [Continue to review full report at 
Codecov](https://codecov.io/gh/apache/airflow/pull/7412?src=pr&el=continue).
   > **Legend** - [Click here to learn 
more](https://docs.codecov.io/docs/codecov-delta)
   > `Δ = absolute  (impact)`, `ø = not affected`, `? = missing data`
   > Powered by 
[Codecov](https://codecov.io/gh/apache/airflow/pull/7412?src=pr&el=footer). 
Last update 
[5b199cb...8344222](https://codecov.io/gh/apache/airflow/pull/7412?src=pr&el=lastupdated).
 Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[jira] [Created] (AIRFLOW-6827) PROPOSAL: Implement a run_pool concept for DagRuns

2020-02-17 Thread Mathew Wicks (Jira) 
Mathew Wicks created AIRFLOW-6827:
-

 Summary: PROPOSAL: Implement a run_pool concept for DagRuns
 Key: AIRFLOW-6827
 URL: https://issues.apache.org/jira/browse/AIRFLOW-6827
 Project: Apache Airflow
  Issue Type: New Feature
  Components: DagRun, scheduler
Affects Versions: 1.10.9
Reporter: Mathew Wicks


Currently we have a concept of resource pools for tasks, where there can be at 
most N instances of a task running at once. However, no such concept exists for 
DagRuns.

For example, you might have a requirement that two DAGs (with different dag_id) 
cannot run at the same time, so you might define a run_pool for these DAGs with 
size 1. Meaning only 1 DagRun may be in the RUNNING state at a time.

I am interested in hearing everyone's thoughts.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[GitHub] [airflow] codecov-io edited a comment on issue #7267: [AIRFLOW-6645] Pass the TaskHandlers configuration using the constructor

2020-02-17 Thread GitBox 
codecov-io edited a comment on issue #7267: [AIRFLOW-6645] Pass the 
TaskHandlers configuration using the constructor
URL: https://github.com/apache/airflow/pull/7267#issuecomment-579544056
 
 
   # [Codecov](https://codecov.io/gh/apache/airflow/pull/7267?src=pr&el=h1) 
Report
   > Merging 
[#7267](https://codecov.io/gh/apache/airflow/pull/7267?src=pr&el=desc) into 
[master](https://codecov.io/gh/apache/airflow/commit/5b199cb86be5b1aefbd8620185033d6f635713c1?src=pr&el=desc)
 will **decrease** coverage by `0.18%`.
   > The diff coverage is `95.23%`.
   
   [![Impacted file tree 
graph](https://codecov.io/gh/apache/airflow/pull/7267/graphs/tree.svg?width=650&token=WdLKlKHOAU&height=150&src=pr)](https://codecov.io/gh/apache/airflow/pull/7267?src=pr&el=tree)
   
   ```diff
   @@Coverage Diff @@
   ##   master#7267  +/-   ##
   ==
   - Coverage   86.66%   86.48%   -0.19% 
   ==
 Files 878  878  
 Lines   4121941216   -3 
   ==
   - Hits3572235644  -78 
   - Misses   5497 5572  +75
   ```
   
   
   | [Impacted 
Files](https://codecov.io/gh/apache/airflow/pull/7267?src=pr&el=tree) | 
Coverage Δ | |
   |---|---|---|
   | 
[airflow/utils/log/gcs\_task\_handler.py](https://codecov.io/gh/apache/airflow/pull/7267/diff?src=pr&el=tree#diff-YWlyZmxvdy91dGlscy9sb2cvZ2NzX3Rhc2tfaGFuZGxlci5weQ==)
 | `37.17% <100%> (+37.17%)` | :arrow_up: |
   | 
[airflow/config\_templates/airflow\_local\_settings.py](https://codecov.io/gh/apache/airflow/pull/7267/diff?src=pr&el=tree#diff-YWlyZmxvdy9jb25maWdfdGVtcGxhdGVzL2FpcmZsb3dfbG9jYWxfc2V0dGluZ3MucHk=)
 | `91.07% <100%> (+25.68%)` | :arrow_up: |
   | 
[airflow/utils/log/s3\_task\_handler.py](https://codecov.io/gh/apache/airflow/pull/7267/diff?src=pr&el=tree#diff-YWlyZmxvdy91dGlscy9sb2cvczNfdGFza19oYW5kbGVyLnB5)
 | `98.57% <100%> (ø)` | :arrow_up: |
   | 
[airflow/utils/log/es\_task\_handler.py](https://codecov.io/gh/apache/airflow/pull/7267/diff?src=pr&el=tree#diff-YWlyZmxvdy91dGlscy9sb2cvZXNfdGFza19oYW5kbGVyLnB5)
 | `93.33% <100%> (-0.07%)` | :arrow_down: |
   | 
[airflow/utils/log/file\_task\_handler.py](https://codecov.io/gh/apache/airflow/pull/7267/diff?src=pr&el=tree#diff-YWlyZmxvdy91dGlscy9sb2cvZmlsZV90YXNrX2hhbmRsZXIucHk=)
 | `90.58% <100%> (+1.82%)` | :arrow_up: |
   | 
[airflow/utils/log/wasb\_task\_handler.py](https://codecov.io/gh/apache/airflow/pull/7267/diff?src=pr&el=tree#diff-YWlyZmxvdy91dGlscy9sb2cvd2FzYl90YXNrX2hhbmRsZXIucHk=)
 | `43.05% <66.66%> (+0.58%)` | :arrow_up: |
   | 
[airflow/kubernetes/volume\_mount.py](https://codecov.io/gh/apache/airflow/pull/7267/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3ZvbHVtZV9tb3VudC5weQ==)
 | `44.44% <0%> (-55.56%)` | :arrow_down: |
   | 
[airflow/kubernetes/volume.py](https://codecov.io/gh/apache/airflow/pull/7267/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3ZvbHVtZS5weQ==)
 | `52.94% <0%> (-47.06%)` | :arrow_down: |
   | 
[airflow/kubernetes/pod\_launcher.py](https://codecov.io/gh/apache/airflow/pull/7267/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3BvZF9sYXVuY2hlci5weQ==)
 | `47.18% <0%> (-45.08%)` | :arrow_down: |
   | 
[...viders/cncf/kubernetes/operators/kubernetes\_pod.py](https://codecov.io/gh/apache/airflow/pull/7267/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvY25jZi9rdWJlcm5ldGVzL29wZXJhdG9ycy9rdWJlcm5ldGVzX3BvZC5weQ==)
 | `69.38% <0%> (-25.52%)` | :arrow_down: |
   | ... and [4 
more](https://codecov.io/gh/apache/airflow/pull/7267/diff?src=pr&el=tree-more) 
| |
   
   --
   
   [Continue to review full report at 
Codecov](https://codecov.io/gh/apache/airflow/pull/7267?src=pr&el=continue).
   > **Legend** - [Click here to learn 
more](https://docs.codecov.io/docs/codecov-delta)
   > `Δ = absolute  (impact)`, `ø = not affected`, `? = missing data`
   > Powered by 
[Codecov](https://codecov.io/gh/apache/airflow/pull/7267?src=pr&el=footer). 
Last update 
[5b199cb...d1fe777](https://codecov.io/gh/apache/airflow/pull/7267?src=pr&el=lastupdated).
 Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] potiuk commented on a change in pull request #7204: [AIRFLOW-XXXX] - add communication chapter to contributing

2020-02-17 Thread GitBox 
potiuk commented on a change in pull request #7204: [AIRFLOW-] - add 
communication chapter to contributing
URL: https://github.com/apache/airflow/pull/7204#discussion_r380398180
 
 

 ##
 File path: CONTRIBUTING.rst
 ##
 @@ -664,6 +664,73 @@ Later on
above depending if you have more commits that cause conflicts in your PR 
(rebasing applies each
commit from your PR one-by-one).
 
+How to communicate
+==
+
+Apache Airflow is a Community within Apache Software Foundation. As the motto 
of
+the Apache Software Foundation states "Community over Code" - people in the
+community are far more important than their contribution.
+
+This means that communication is very important, and this chapter is all about 
it.
+
+We have various channel of communication - starting from the official devlist, 
comments
+in the Pull Requests, Slack, wiki.
+
+All those channels can be used for different purposes:
+
+* The devlist for official communication, discussing proposals, voting, 
general issues, asking community for
+  opinion etc.
+* Wiki for detailed discussions on big proposals (AIPs) and some helpful, 
shared resources.
+* PRs for discussing implementation details of this PR (not for architectural 
discussions)
+* Slack for ad-hoc questions, troubleshooting, occassional discussions, group 
talks (including SIG - special
+interest groups), notifications, random queries, newbie questions etc.
+
+
+It is important to remember that all contributors, committers and even PMC 
members are individuals
+that contribute a lot of their free time on top of their work time to develop 
Apache Airflow.
+They have families, personal life, sometimes they are swamped with work, 
sometimes they are on vacations
+sometimes they are simply tired and need a time off. They are in various 
timezones and their responsiveness
+will vary from time to time. As a community project we have no formal 
structure, we have no managers nor
+departments and most of the people are autonomous in their opinions and 
decisions, so sometimes even
+committers will have different opinions on the same subjects.
+
+Disagreements are expected, discussions might include strong opinions and 
contradicting statements.
+Sometimes you might get two committers asking you to do things differently. 
This all happened in the past
+and will continue to happen. As a community we have some mechanisms to 
facilitate discussion and come to
+a consensus, conclusions and sometime we end up voting certain decisions. It 
is important that those
+decisions are not treated as personal wins or looses. At the end it's the 
community that we all care about
+and what's good for community, should be accepted even if you have different 
opinion. There is the nice
+motto that you should follow in case you disagree with community decision 
"Disagree but engage". Even
+if you do not agree with a community decision, you should follow it and 
embrace (but you are free to
+express your opinion that you don't agree with it).
+
+As a community - we have high requirements for code quality. This is mainly 
because we are a distributed
+and loosely organised team - sometime we have contributors that commit one 
commit only, sometimes people add
+more commits, sometimes people assume informal "stewardship" over parts of 
code for some time - but at any
+time we should be sure that the code can be taken over by others - without 
excessive communication. Setting
 
 Review comment:
   Done


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] potiuk commented on a change in pull request #7204: [AIRFLOW-XXXX] - add communication chapter to contributing

2020-02-17 Thread GitBox 
potiuk commented on a change in pull request #7204: [AIRFLOW-] - add 
communication chapter to contributing
URL: https://github.com/apache/airflow/pull/7204#discussion_r380397983
 
 

 ##
 File path: CONTRIBUTING.rst
 ##
 @@ -664,6 +664,73 @@ Later on
above depending if you have more commits that cause conflicts in your PR 
(rebasing applies each
commit from your PR one-by-one).
 
+How to communicate
+==
+
+Apache Airflow is a Community within Apache Software Foundation. As the motto 
of
+the Apache Software Foundation states "Community over Code" - people in the
+community are far more important than their contribution.
+
+This means that communication is very important, and this chapter is all about 
it.
+
+We have various channel of communication - starting from the official devlist, 
comments
+in the Pull Requests, Slack, wiki.
+
+All those channels can be used for different purposes:
+
+* The devlist for official communication, discussing proposals, voting, 
general issues, asking community for
+  opinion etc.
+* Wiki for detailed discussions on big proposals (AIPs) and some helpful, 
shared resources.
+* PRs for discussing implementation details of this PR (not for architectural 
discussions)
+* Slack for ad-hoc questions, troubleshooting, occassional discussions, group 
talks (including SIG - special
+interest groups), notifications, random queries, newbie questions etc.
+
+
+It is important to remember that all contributors, committers and even PMC 
members are individuals
+that contribute a lot of their free time on top of their work time to develop 
Apache Airflow.
+They have families, personal life, sometimes they are swamped with work, 
sometimes they are on vacations
+sometimes they are simply tired and need a time off. They are in various 
timezones and their responsiveness
+will vary from time to time. As a community project we have no formal 
structure, we have no managers nor
+departments and most of the people are autonomous in their opinions and 
decisions, so sometimes even
+committers will have different opinions on the same subjects.
+
+Disagreements are expected, discussions might include strong opinions and 
contradicting statements.
+Sometimes you might get two committers asking you to do things differently. 
This all happened in the past
+and will continue to happen. As a community we have some mechanisms to 
facilitate discussion and come to
+a consensus, conclusions and sometime we end up voting certain decisions. It 
is important that those
+decisions are not treated as personal wins or looses. At the end it's the 
community that we all care about
+and what's good for community, should be accepted even if you have different 
opinion. There is the nice
+motto that you should follow in case you disagree with community decision 
"Disagree but engage". Even
+if you do not agree with a community decision, you should follow it and 
embrace (but you are free to
+express your opinion that you don't agree with it).
+
+As a community - we have high requirements for code quality. This is mainly 
because we are a distributed
+and loosely organised team - sometime we have contributors that commit one 
commit only, sometimes people add
 
 Review comment:
   Done


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] potiuk commented on a change in pull request #7204: [AIRFLOW-XXXX] - add communication chapter to contributing

2020-02-17 Thread GitBox 
potiuk commented on a change in pull request #7204: [AIRFLOW-] - add 
communication chapter to contributing
URL: https://github.com/apache/airflow/pull/7204#discussion_r380397958
 
 

 ##
 File path: CONTRIBUTING.rst
 ##
 @@ -664,6 +664,73 @@ Later on
above depending if you have more commits that cause conflicts in your PR 
(rebasing applies each
commit from your PR one-by-one).
 
+How to communicate
+==
+
+Apache Airflow is a Community within Apache Software Foundation. As the motto 
of
+the Apache Software Foundation states "Community over Code" - people in the
+community are far more important than their contribution.
+
+This means that communication is very important, and this chapter is all about 
it.
+
+We have various channel of communication - starting from the official devlist, 
comments
+in the Pull Requests, Slack, wiki.
+
+All those channels can be used for different purposes:
+
+* The devlist for official communication, discussing proposals, voting, 
general issues, asking community for
+  opinion etc.
+* Wiki for detailed discussions on big proposals (AIPs) and some helpful, 
shared resources.
+* PRs for discussing implementation details of this PR (not for architectural 
discussions)
+* Slack for ad-hoc questions, troubleshooting, occassional discussions, group 
talks (including SIG - special
+interest groups), notifications, random queries, newbie questions etc.
+
+
+It is important to remember that all contributors, committers and even PMC 
members are individuals
+that contribute a lot of their free time on top of their work time to develop 
Apache Airflow.
+They have families, personal life, sometimes they are swamped with work, 
sometimes they are on vacations
+sometimes they are simply tired and need a time off. They are in various 
timezones and their responsiveness
+will vary from time to time. As a community project we have no formal 
structure, we have no managers nor
+departments and most of the people are autonomous in their opinions and 
decisions, so sometimes even
+committers will have different opinions on the same subjects.
+
+Disagreements are expected, discussions might include strong opinions and 
contradicting statements.
+Sometimes you might get two committers asking you to do things differently. 
This all happened in the past
+and will continue to happen. As a community we have some mechanisms to 
facilitate discussion and come to
+a consensus, conclusions and sometime we end up voting certain decisions. It 
is important that those
+decisions are not treated as personal wins or looses. At the end it's the 
community that we all care about
+and what's good for community, should be accepted even if you have different 
opinion. There is the nice
 
 Review comment:
   Done


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] potiuk commented on a change in pull request #7204: [AIRFLOW-XXXX] - add communication chapter to contributing

2020-02-17 Thread GitBox 
potiuk commented on a change in pull request #7204: [AIRFLOW-] - add 
communication chapter to contributing
URL: https://github.com/apache/airflow/pull/7204#discussion_r380397903
 
 

 ##
 File path: CONTRIBUTING.rst
 ##
 @@ -664,6 +664,73 @@ Later on
above depending if you have more commits that cause conflicts in your PR 
(rebasing applies each
commit from your PR one-by-one).
 
+How to communicate
+==
+
+Apache Airflow is a Community within Apache Software Foundation. As the motto 
of
+the Apache Software Foundation states "Community over Code" - people in the
+community are far more important than their contribution.
+
+This means that communication is very important, and this chapter is all about 
it.
+
+We have various channel of communication - starting from the official devlist, 
comments
+in the Pull Requests, Slack, wiki.
+
+All those channels can be used for different purposes:
+
+* The devlist for official communication, discussing proposals, voting, 
general issues, asking community for
+  opinion etc.
+* Wiki for detailed discussions on big proposals (AIPs) and some helpful, 
shared resources.
+* PRs for discussing implementation details of this PR (not for architectural 
discussions)
+* Slack for ad-hoc questions, troubleshooting, occassional discussions, group 
talks (including SIG - special
+interest groups), notifications, random queries, newbie questions etc.
+
+
+It is important to remember that all contributors, committers and even PMC 
members are individuals
+that contribute a lot of their free time on top of their work time to develop 
Apache Airflow.
+They have families, personal life, sometimes they are swamped with work, 
sometimes they are on vacations
+sometimes they are simply tired and need a time off. They are in various 
timezones and their responsiveness
+will vary from time to time. As a community project we have no formal 
structure, we have no managers nor
+departments and most of the people are autonomous in their opinions and 
decisions, so sometimes even
+committers will have different opinions on the same subjects.
+
+Disagreements are expected, discussions might include strong opinions and 
contradicting statements.
+Sometimes you might get two committers asking you to do things differently. 
This all happened in the past
+and will continue to happen. As a community we have some mechanisms to 
facilitate discussion and come to
+a consensus, conclusions and sometime we end up voting certain decisions. It 
is important that those
 
 Review comment:
   Done


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] potiuk commented on a change in pull request #7204: [AIRFLOW-XXXX] - add communication chapter to contributing

2020-02-17 Thread GitBox 
potiuk commented on a change in pull request #7204: [AIRFLOW-] - add 
communication chapter to contributing
URL: https://github.com/apache/airflow/pull/7204#discussion_r380397786
 
 

 ##
 File path: CONTRIBUTING.rst
 ##
 @@ -664,6 +664,73 @@ Later on
above depending if you have more commits that cause conflicts in your PR 
(rebasing applies each
commit from your PR one-by-one).
 
+How to communicate
+==
+
+Apache Airflow is a Community within Apache Software Foundation. As the motto 
of
+the Apache Software Foundation states "Community over Code" - people in the
+community are far more important than their contribution.
+
+This means that communication is very important, and this chapter is all about 
it.
+
+We have various channel of communication - starting from the official devlist, 
comments
+in the Pull Requests, Slack, wiki.
+
+All those channels can be used for different purposes:
+
+* The devlist for official communication, discussing proposals, voting, 
general issues, asking community for
+  opinion etc.
+* Wiki for detailed discussions on big proposals (AIPs) and some helpful, 
shared resources.
+* PRs for discussing implementation details of this PR (not for architectural 
discussions)
+* Slack for ad-hoc questions, troubleshooting, occassional discussions, group 
talks (including SIG - special
+interest groups), notifications, random queries, newbie questions etc.
+
+
+It is important to remember that all contributors, committers and even PMC 
members are individuals
+that contribute a lot of their free time on top of their work time to develop 
Apache Airflow.
+They have families, personal life, sometimes they are swamped with work, 
sometimes they are on vacations
+sometimes they are simply tired and need a time off. They are in various 
timezones and their responsiveness
+will vary from time to time. As a community project we have no formal 
structure, we have no managers nor
+departments and most of the people are autonomous in their opinions and 
decisions, so sometimes even
+committers will have different opinions on the same subjects.
+
+Disagreements are expected, discussions might include strong opinions and 
contradicting statements.
+Sometimes you might get two committers asking you to do things differently. 
This all happened in the past
+and will continue to happen. As a community we have some mechanisms to 
facilitate discussion and come to
+a consensus, conclusions and sometime we end up voting certain decisions. It 
is important that those
+decisions are not treated as personal wins or looses. At the end it's the 
community that we all care about
+and what's good for community, should be accepted even if you have different 
opinion. There is the nice
+motto that you should follow in case you disagree with community decision 
"Disagree but engage". Even
+if you do not agree with a community decision, you should follow it and 
embrace (but you are free to
+express your opinion that you don't agree with it).
+
+As a community - we have high requirements for code quality. This is mainly 
because we are a distributed
+and loosely organised team - sometime we have contributors that commit one 
commit only, sometimes people add
+more commits, sometimes people assume informal "stewardship" over parts of 
code for some time - but at any
+time we should be sure that the code can be taken over by others - without 
excessive communication. Setting
+high requirements for the code (fairly strict code review, static code checks, 
requirements of automated
+tests, pre-commit checks ) is the best way that we can achieve that - by only 
accepting good quality
+code with full test coverage we make sure that we can work with the code in 
the future.
+So do not be surprised if you are asked to add more tests or make the code 
cleaner -
+this is for the sake of maintainability.
+
+Here are a few rules that are important to keep in mind when you enter our 
community:
+
+ * Do not be afraid to ask questions
+ * The communication is asynchronous - do not expect immediate answers, ping 
others if you are blocked
+ * There is a #newbie-questions channel in slack as a safe place to ask 
questions
 
 Review comment:
   Done


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] potiuk commented on a change in pull request #7204: [AIRFLOW-XXXX] - add communication chapter to contributing

2020-02-17 Thread GitBox 
potiuk commented on a change in pull request #7204: [AIRFLOW-] - add 
communication chapter to contributing
URL: https://github.com/apache/airflow/pull/7204#discussion_r380397085
 
 

 ##
 File path: CONTRIBUTING.rst
 ##
 @@ -664,6 +664,73 @@ Later on
above depending if you have more commits that cause conflicts in your PR 
(rebasing applies each
commit from your PR one-by-one).
 
+How to communicate
+==
+
+Apache Airflow is a Community within Apache Software Foundation. As the motto 
of
+the Apache Software Foundation states "Community over Code" - people in the
+community are far more important than their contribution.
+
+This means that communication is very important, and this chapter is all about 
it.
+
+We have various channel of communication - starting from the official devlist, 
comments
+in the Pull Requests, Slack, wiki.
+
+All those channels can be used for different purposes:
+
+* The devlist for official communication, discussing proposals, voting, 
general issues, asking community for
+  opinion etc.
+* Wiki for detailed discussions on big proposals (AIPs) and some helpful, 
shared resources.
+* PRs for discussing implementation details of this PR (not for architectural 
discussions)
+* Slack for ad-hoc questions, troubleshooting, occassional discussions, group 
talks (including SIG - special
+interest groups), notifications, random queries, newbie questions etc.
+
+
+It is important to remember that all contributors, committers and even PMC 
members are individuals
+that contribute a lot of their free time on top of their work time to develop 
Apache Airflow.
+They have families, personal life, sometimes they are swamped with work, 
sometimes they are on vacations
+sometimes they are simply tired and need a time off. They are in various 
timezones and their responsiveness
+will vary from time to time. As a community project we have no formal 
structure, we have no managers nor
+departments and most of the people are autonomous in their opinions and 
decisions, so sometimes even
+committers will have different opinions on the same subjects.
+
+Disagreements are expected, discussions might include strong opinions and 
contradicting statements.
+Sometimes you might get two committers asking you to do things differently. 
This all happened in the past
+and will continue to happen. As a community we have some mechanisms to 
facilitate discussion and come to
+a consensus, conclusions and sometime we end up voting certain decisions. It 
is important that those
+decisions are not treated as personal wins or looses. At the end it's the 
community that we all care about
+and what's good for community, should be accepted even if you have different 
opinion. There is the nice
+motto that you should follow in case you disagree with community decision 
"Disagree but engage". Even
+if you do not agree with a community decision, you should follow it and 
embrace (but you are free to
+express your opinion that you don't agree with it).
+
+As a community - we have high requirements for code quality. This is mainly 
because we are a distributed
+and loosely organised team - sometime we have contributors that commit one 
commit only, sometimes people add
+more commits, sometimes people assume informal "stewardship" over parts of 
code for some time - but at any
+time we should be sure that the code can be taken over by others - without 
excessive communication. Setting
+high requirements for the code (fairly strict code review, static code checks, 
requirements of automated
+tests, pre-commit checks ) is the best way that we can achieve that - by only 
accepting good quality
+code with full test coverage we make sure that we can work with the code in 
the future.
+So do not be surprised if you are asked to add more tests or make the code 
cleaner -
+this is for the sake of maintainability.
+
+Here are a few rules that are important to keep in mind when you enter our 
community:
+
+ * Do not be afraid to ask questions
+ * The communication is asynchronous - do not expect immediate answers, ping 
others if you are blocked
 
 Review comment:
   added


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] potiuk commented on a change in pull request #7204: [AIRFLOW-XXXX] - add communication chapter to contributing

2020-02-17 Thread GitBox 
potiuk commented on a change in pull request #7204: [AIRFLOW-] - add 
communication chapter to contributing
URL: https://github.com/apache/airflow/pull/7204#discussion_r380396906
 
 

 ##
 File path: CONTRIBUTING.rst
 ##
 @@ -664,6 +664,73 @@ Later on
above depending if you have more commits that cause conflicts in your PR 
(rebasing applies each
commit from your PR one-by-one).
 
+How to communicate
+==
+
+Apache Airflow is a Community within Apache Software Foundation. As the motto 
of
+the Apache Software Foundation states "Community over Code" - people in the
+community are far more important than their contribution.
+
+This means that communication is very important, and this chapter is all about 
it.
+
+We have various channel of communication - starting from the official devlist, 
comments
+in the Pull Requests, Slack, wiki.
+
+All those channels can be used for different purposes:
+
+* The devlist for official communication, discussing proposals, voting, 
general issues, asking community for
+  opinion etc.
+* Wiki for detailed discussions on big proposals (AIPs) and some helpful, 
shared resources.
+* PRs for discussing implementation details of this PR (not for architectural 
discussions)
+* Slack for ad-hoc questions, troubleshooting, occassional discussions, group 
talks (including SIG - special
+interest groups), notifications, random queries, newbie questions etc.
+
+
+It is important to remember that all contributors, committers and even PMC 
members are individuals
+that contribute a lot of their free time on top of their work time to develop 
Apache Airflow.
+They have families, personal life, sometimes they are swamped with work, 
sometimes they are on vacations
+sometimes they are simply tired and need a time off. They are in various 
timezones and their responsiveness
+will vary from time to time. As a community project we have no formal 
structure, we have no managers nor
+departments and most of the people are autonomous in their opinions and 
decisions, so sometimes even
+committers will have different opinions on the same subjects.
 
 Review comment:
   Agreed.!


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] potiuk commented on a change in pull request #7204: [AIRFLOW-XXXX] - add communication chapter to contributing

2020-02-17 Thread GitBox 
potiuk commented on a change in pull request #7204: [AIRFLOW-] - add 
communication chapter to contributing
URL: https://github.com/apache/airflow/pull/7204#discussion_r380396248
 
 

 ##
 File path: CONTRIBUTING.rst
 ##
 @@ -664,6 +664,73 @@ Later on
above depending if you have more commits that cause conflicts in your PR 
(rebasing applies each
commit from your PR one-by-one).
 
+How to communicate
+==
+
+Apache Airflow is a Community within Apache Software Foundation. As the motto 
of
+the Apache Software Foundation states "Community over Code" - people in the
+community are far more important than their contribution.
+
+This means that communication is very important, and this chapter is all about 
it.
+
+We have various channel of communication - starting from the official devlist, 
comments
+in the Pull Requests, Slack, wiki.
+
+All those channels can be used for different purposes:
+
+* The devlist for official communication, discussing proposals, voting, 
general issues, asking community for
+  opinion etc.
+* Wiki for detailed discussions on big proposals (AIPs) and some helpful, 
shared resources.
+* PRs for discussing implementation details of this PR (not for architectural 
discussions)
+* Slack for ad-hoc questions, troubleshooting, occassional discussions, group 
talks (including SIG - special
+interest groups), notifications, random queries, newbie questions etc.
 
 Review comment:
   * Those will be bullet points on second level
   * I added links to all the channels
   * I added information about the prefixes


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] codecov-io commented on issue #7447: [AIRFLOW-6821] - Success callback not called when task marked as succ…

2020-02-17 Thread GitBox 
codecov-io commented on issue #7447: [AIRFLOW-6821] - Success callback not 
called when task marked as succ…
URL: https://github.com/apache/airflow/pull/7447#issuecomment-587178126
 
 
   # [Codecov](https://codecov.io/gh/apache/airflow/pull/7447?src=pr&el=h1) 
Report
   > Merging 
[#7447](https://codecov.io/gh/apache/airflow/pull/7447?src=pr&el=desc) into 
[master](https://codecov.io/gh/apache/airflow/commit/d83ce260c074d3f887948921d3ebbdb1100cc800?src=pr&el=desc)
 will **increase** coverage by `53.41%`.
   > The diff coverage is `100%`.
   
   [![Impacted file tree 
graph](https://codecov.io/gh/apache/airflow/pull/7447/graphs/tree.svg?width=650&token=WdLKlKHOAU&height=150&src=pr)](https://codecov.io/gh/apache/airflow/pull/7447?src=pr&el=tree)
   
   ```diff
   @@ Coverage Diff @@
   ##   master#7447   +/-   ##
   ===
   + Coverage   32.95%   86.36%   +53.41% 
   ===
 Files 878  878   
 Lines   4121941222+3 
   ===
   + Hits1358435603+22019 
   + Misses  27635 5619-22016
   ```
   
   
   | [Impacted 
Files](https://codecov.io/gh/apache/airflow/pull/7447?src=pr&el=tree) | 
Coverage Δ | |
   |---|---|---|
   | 
[airflow/jobs/local\_task\_job.py](https://codecov.io/gh/apache/airflow/pull/7447/diff?src=pr&el=tree#diff-YWlyZmxvdy9qb2JzL2xvY2FsX3Rhc2tfam9iLnB5)
 | `90.12% <100%> (+67.04%)` | :arrow_up: |
   | 
[airflow/kubernetes/volume\_mount.py](https://codecov.io/gh/apache/airflow/pull/7447/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3ZvbHVtZV9tb3VudC5weQ==)
 | `44.44% <0%> (-55.56%)` | :arrow_down: |
   | 
[airflow/kubernetes/volume.py](https://codecov.io/gh/apache/airflow/pull/7447/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3ZvbHVtZS5weQ==)
 | `52.94% <0%> (-47.06%)` | :arrow_down: |
   | 
[airflow/kubernetes/pod\_launcher.py](https://codecov.io/gh/apache/airflow/pull/7447/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3BvZF9sYXVuY2hlci5weQ==)
 | `47.18% <0%> (-39.44%)` | :arrow_down: |
   | 
[...viders/cncf/kubernetes/operators/kubernetes\_pod.py](https://codecov.io/gh/apache/airflow/pull/7447/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvY25jZi9rdWJlcm5ldGVzL29wZXJhdG9ycy9rdWJlcm5ldGVzX3BvZC5weQ==)
 | `69.38% <0%> (-25.52%)` | :arrow_down: |
   | 
[airflow/kubernetes/refresh\_config.py](https://codecov.io/gh/apache/airflow/pull/7447/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3JlZnJlc2hfY29uZmlnLnB5)
 | `50.98% <0%> (-23.53%)` | :arrow_down: |
   | 
[airflow/bin/airflow](https://codecov.io/gh/apache/airflow/pull/7447/diff?src=pr&el=tree#diff-YWlyZmxvdy9iaW4vYWlyZmxvdw==)
 | `84.61% <0%> (ø)` | :arrow_up: |
   | 
[airflow/plugins\_manager.py](https://codecov.io/gh/apache/airflow/pull/7447/diff?src=pr&el=tree#diff-YWlyZmxvdy9wbHVnaW5zX21hbmFnZXIucHk=)
 | `86.95% <0%> (+0.72%)` | :arrow_up: |
   | 
[airflow/bin/cli.py](https://codecov.io/gh/apache/airflow/pull/7447/diff?src=pr&el=tree#diff-YWlyZmxvdy9iaW4vY2xpLnB5)
 | `94.73% <0%> (+2.1%)` | :arrow_up: |
   | 
[airflow/executors/celery\_executor.py](https://codecov.io/gh/apache/airflow/pull/7447/diff?src=pr&el=tree#diff-YWlyZmxvdy9leGVjdXRvcnMvY2VsZXJ5X2V4ZWN1dG9yLnB5)
 | `88.51% <0%> (+3.37%)` | :arrow_up: |
   | ... and [749 
more](https://codecov.io/gh/apache/airflow/pull/7447/diff?src=pr&el=tree-more) 
| |
   
   --
   
   [Continue to review full report at 
Codecov](https://codecov.io/gh/apache/airflow/pull/7447?src=pr&el=continue).
   > **Legend** - [Click here to learn 
more](https://docs.codecov.io/docs/codecov-delta)
   > `Δ = absolute  (impact)`, `ø = not affected`, `? = missing data`
   > Powered by 
[Codecov](https://codecov.io/gh/apache/airflow/pull/7447?src=pr&el=footer). 
Last update 
[d83ce26...f0dca49](https://codecov.io/gh/apache/airflow/pull/7447?src=pr&el=lastupdated).
 Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] mik-laj commented on issue #7422: [AIRFLOW-6809] Test for presto operators

2020-02-17 Thread GitBox 
mik-laj commented on issue #7422: [AIRFLOW-6809] Test for presto operators
URL: https://github.com/apache/airflow/pull/7422#issuecomment-587169423
 
 
   If it's a duplicate test, yes.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] saurabhdhupar opened a new pull request #7447: [AIRFLOW-6821] - Success callback not called when task marked as succ…

2020-02-17 Thread GitBox 
saurabhdhupar opened a new pull request #7447: [AIRFLOW-6821] - Success 
callback not called when task marked as succ…
URL: https://github.com/apache/airflow/pull/7447
 
 
   …ess from UI
   
   ---
   Issue link: WILL BE INSERTED BY 
[boring-cyborg](https://github.com/kaxil/boring-cyborg)
   
   Make sure to mark the boxes below before creating PR: [x]
   
   - [ x] Description above provides context of the change
   - [ x] Commit message/PR title starts with `[AIRFLOW-]`. AIRFLOW- = 
JIRA ID*
   - [x ] Unit tests coverage for changes (not needed for documentation changes)
   - [x ] Commits follow "[How to write a good git commit 
message](http://chris.beams.io/posts/git-commit/)"
   - [ x] Relevant documentation is updated including usage instructions.
   - [x ] I will engage committers as explained in [Contribution Workflow 
Example](https://github.com/apache/airflow/blob/master/CONTRIBUTING.rst#contribution-workflow-example).
   
   * For document-only changes commit message can start with 
`[AIRFLOW-]`.
   
   ---
   In case of fundamental code change, Airflow Improvement Proposal 
([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvements+Proposals))
 is needed.
   In case of a new dependency, check compliance with the [ASF 3rd Party 
License Policy](https://www.apache.org/legal/resolved.html#category-x).
   In case of backwards incompatible changes please leave a note in 
[UPDATING.md](https://github.com/apache/airflow/blob/master/UPDATING.md).
   Read the [Pull Request 
Guidelines](https://github.com/apache/airflow/blob/master/CONTRIBUTING.rst#pull-request-guidelines)
 for more information.
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] mik-laj commented on issue #7252: [AIRFLOW-6531] Initial Yandex.Cloud Dataproc support

2020-02-17 Thread GitBox 
mik-laj commented on issue #7252: [AIRFLOW-6531] Initial Yandex.Cloud Dataproc 
support
URL: https://github.com/apache/airflow/pull/7252#issuecomment-587169199
 
 
   @dhegberg @nuclearpinguin 
   Does this integration have a good directory structure?
   Current structure:
   ```
   airflow/providers/yandex/
   ├── __init__.py
   ├── example_dags
   │   ├── __init__.py
   │   └── example_yandexcloud_dataproc.py
   ├── hooks
   │   ├── __init__.py
   │   └── yandexcloud_dataproc_hook.py
   └── operators
   ├── __init__.py
   └── yandexcloud_dataproc_operator.py
   ```
   In my opinion, it not valid according to AIP-21. It should looks as follow:
   ```
   airflow/providers/yandex/
   ├── __init__.py
   └── cloud
   ├── __init__.py
   ├── example_dags
   │   ├── __init__.py
   │   └── example_dataproc.py
   ├── hooks
   │   ├── __init__.py
   │   └── dataproc.py
   └── operators
   ├── __init__.py
   └── dataproc.py
   
   ```


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[jira] [Commented] (AIRFLOW-6821) When task is marked as success through UI, success callback is not triggered

2020-02-17 Thread ASF GitHub Bot (Jira) 


[ 
https://issues.apache.org/jira/browse/AIRFLOW-6821?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17038649#comment-17038649
 ] 

ASF GitHub Bot commented on AIRFLOW-6821:
-

saurabhdhupar commented on pull request #7447: [AIRFLOW-6821] - Success 
callback not called when task marked as succ…
URL: https://github.com/apache/airflow/pull/7447
 
 
   …ess from UI
   
   ---
   Issue link: WILL BE INSERTED BY 
[boring-cyborg](https://github.com/kaxil/boring-cyborg)
   
   Make sure to mark the boxes below before creating PR: [x]
   
   - [ x] Description above provides context of the change
   - [ x] Commit message/PR title starts with `[AIRFLOW-]`. AIRFLOW- = 
JIRA ID*
   - [x ] Unit tests coverage for changes (not needed for documentation changes)
   - [x ] Commits follow "[How to write a good git commit 
message](http://chris.beams.io/posts/git-commit/)"
   - [ x] Relevant documentation is updated including usage instructions.
   - [x ] I will engage committers as explained in [Contribution Workflow 
Example](https://github.com/apache/airflow/blob/master/CONTRIBUTING.rst#contribution-workflow-example).
   
   * For document-only changes commit message can start with 
`[AIRFLOW-]`.
   
   ---
   In case of fundamental code change, Airflow Improvement Proposal 
([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvements+Proposals))
 is needed.
   In case of a new dependency, check compliance with the [ASF 3rd Party 
License Policy](https://www.apache.org/legal/resolved.html#category-x).
   In case of backwards incompatible changes please leave a note in 
[UPDATING.md](https://github.com/apache/airflow/blob/master/UPDATING.md).
   Read the [Pull Request 
Guidelines](https://github.com/apache/airflow/blob/master/CONTRIBUTING.rst#pull-request-guidelines)
 for more information.
   
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


> When task is marked as success through UI, success callback is not triggered
> 
>
> Key: AIRFLOW-6821
> URL: https://issues.apache.org/jira/browse/AIRFLOW-6821
> Project: Apache Airflow
>  Issue Type: Bug
>  Components: core
>Affects Versions: 1.10.9
>Reporter: Saurabh Dhupar
>Assignee: Saurabh Dhupar
>Priority: Minor
>
> When you mark a task as success - Once Worker detects and stops the process 
> but - on_success_callback is not called.
> In this case, I would expect on_success_callbackto be called to complete it's 
> intended functionality. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[GitHub] [airflow] codecov-io edited a comment on issue #7438: [AIRFLOW-6821] - Success callback not called when task marked as success from UI

2020-02-17 Thread GitBox 
codecov-io edited a comment on issue #7438: [AIRFLOW-6821] - Success callback 
not called when task marked as success from UI
URL: https://github.com/apache/airflow/pull/7438#issuecomment-586958485
 
 
   # [Codecov](https://codecov.io/gh/apache/airflow/pull/7438?src=pr&el=h1) 
Report
   > Merging 
[#7438](https://codecov.io/gh/apache/airflow/pull/7438?src=pr&el=desc) into 
[master](https://codecov.io/gh/apache/airflow/commit/d83ce260c074d3f887948921d3ebbdb1100cc800?src=pr&el=desc)
 will **increase** coverage by `53.41%`.
   > The diff coverage is `100%`.
   
   [![Impacted file tree 
graph](https://codecov.io/gh/apache/airflow/pull/7438/graphs/tree.svg?width=650&token=WdLKlKHOAU&height=150&src=pr)](https://codecov.io/gh/apache/airflow/pull/7438?src=pr&el=tree)
   
   ```diff
   @@ Coverage Diff @@
   ##   master#7438   +/-   ##
   ===
   + Coverage   32.95%   86.37%   +53.41% 
   ===
 Files 878  878   
 Lines   4121941222+3 
   ===
   + Hits1358435604+22020 
   + Misses  27635 5618-22017
   ```
   
   
   | [Impacted 
Files](https://codecov.io/gh/apache/airflow/pull/7438?src=pr&el=tree) | 
Coverage Δ | |
   |---|---|---|
   | 
[airflow/jobs/local\_task\_job.py](https://codecov.io/gh/apache/airflow/pull/7438/diff?src=pr&el=tree#diff-YWlyZmxvdy9qb2JzL2xvY2FsX3Rhc2tfam9iLnB5)
 | `90.12% <100%> (+67.04%)` | :arrow_up: |
   | 
[airflow/kubernetes/volume\_mount.py](https://codecov.io/gh/apache/airflow/pull/7438/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3ZvbHVtZV9tb3VudC5weQ==)
 | `44.44% <0%> (-55.56%)` | :arrow_down: |
   | 
[airflow/kubernetes/volume.py](https://codecov.io/gh/apache/airflow/pull/7438/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3ZvbHVtZS5weQ==)
 | `52.94% <0%> (-47.06%)` | :arrow_down: |
   | 
[airflow/kubernetes/pod\_launcher.py](https://codecov.io/gh/apache/airflow/pull/7438/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3BvZF9sYXVuY2hlci5weQ==)
 | `47.18% <0%> (-39.44%)` | :arrow_down: |
   | 
[...viders/cncf/kubernetes/operators/kubernetes\_pod.py](https://codecov.io/gh/apache/airflow/pull/7438/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvY25jZi9rdWJlcm5ldGVzL29wZXJhdG9ycy9rdWJlcm5ldGVzX3BvZC5weQ==)
 | `69.38% <0%> (-25.52%)` | :arrow_down: |
   | 
[airflow/kubernetes/refresh\_config.py](https://codecov.io/gh/apache/airflow/pull/7438/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3JlZnJlc2hfY29uZmlnLnB5)
 | `50.98% <0%> (-23.53%)` | :arrow_down: |
   | 
[airflow/bin/airflow](https://codecov.io/gh/apache/airflow/pull/7438/diff?src=pr&el=tree#diff-YWlyZmxvdy9iaW4vYWlyZmxvdw==)
 | `84.61% <0%> (ø)` | :arrow_up: |
   | 
[airflow/plugins\_manager.py](https://codecov.io/gh/apache/airflow/pull/7438/diff?src=pr&el=tree#diff-YWlyZmxvdy9wbHVnaW5zX21hbmFnZXIucHk=)
 | `86.95% <0%> (+0.72%)` | :arrow_up: |
   | 
[airflow/bin/cli.py](https://codecov.io/gh/apache/airflow/pull/7438/diff?src=pr&el=tree#diff-YWlyZmxvdy9iaW4vY2xpLnB5)
 | `94.73% <0%> (+2.1%)` | :arrow_up: |
   | 
[airflow/executors/celery\_executor.py](https://codecov.io/gh/apache/airflow/pull/7438/diff?src=pr&el=tree#diff-YWlyZmxvdy9leGVjdXRvcnMvY2VsZXJ5X2V4ZWN1dG9yLnB5)
 | `88.51% <0%> (+3.37%)` | :arrow_up: |
   | ... and [749 
more](https://codecov.io/gh/apache/airflow/pull/7438/diff?src=pr&el=tree-more) 
| |
   
   --
   
   [Continue to review full report at 
Codecov](https://codecov.io/gh/apache/airflow/pull/7438?src=pr&el=continue).
   > **Legend** - [Click here to learn 
more](https://docs.codecov.io/docs/codecov-delta)
   > `Δ = absolute  (impact)`, `ø = not affected`, `? = missing data`
   > Powered by 
[Codecov](https://codecov.io/gh/apache/airflow/pull/7438?src=pr&el=footer). 
Last update 
[d83ce26...7a780ba](https://codecov.io/gh/apache/airflow/pull/7438?src=pr&el=lastupdated).
 Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[jira] [Commented] (AIRFLOW-6821) When task is marked as success through UI, success callback is not triggered

2020-02-17 Thread ASF GitHub Bot (Jira) 


[ 
https://issues.apache.org/jira/browse/AIRFLOW-6821?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17038624#comment-17038624
 ] 

ASF GitHub Bot commented on AIRFLOW-6821:
-

saurabhdhupar commented on pull request #7438: [AIRFLOW-6821] - Success 
callback not called when task marked as success from UI
URL: https://github.com/apache/airflow/pull/7438
 
 
   
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


> When task is marked as success through UI, success callback is not triggered
> 
>
> Key: AIRFLOW-6821
> URL: https://issues.apache.org/jira/browse/AIRFLOW-6821
> Project: Apache Airflow
>  Issue Type: Bug
>  Components: core
>Affects Versions: 1.10.9
>Reporter: Saurabh Dhupar
>Assignee: Saurabh Dhupar
>Priority: Minor
>
> When you mark a task as success - Once Worker detects and stops the process 
> but - on_success_callback is not called.
> In this case, I would expect on_success_callbackto be called to complete it's 
> intended functionality. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[GitHub] [airflow] saurabhdhupar closed pull request #7438: [AIRFLOW-6821] - Success callback not called when task marked as success from UI

2020-02-17 Thread GitBox 
saurabhdhupar closed pull request #7438: [AIRFLOW-6821] - Success callback not 
called when task marked as success from UI
URL: https://github.com/apache/airflow/pull/7438
 
 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] tooptoop4 commented on issue #7446: [AIRFLOW-6474] list_dag_runs cli command should allow exec_date between start/end range and print start/end times

2020-02-17 Thread GitBox 
tooptoop4 commented on issue #7446: [AIRFLOW-6474] list_dag_runs cli command 
should allow exec_date between start/end range and print start/end times
URL: https://github.com/apache/airflow/pull/7446#issuecomment-587140855
 
 
can u do point 2? Print start/end times of the dagruns?


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[jira] [Commented] (AIRFLOW-6474) list_dag_runs cli command should allow exec_date between start/end range and print start/end times

2020-02-17 Thread alexander (Jira) 


[ 
https://issues.apache.org/jira/browse/AIRFLOW-6474?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17038593#comment-17038593
 ] 

alexander commented on AIRFLOW-6474:


I missed end date in printed table, right ? Yes, i have tested multiple dag 
runs get returned.

> list_dag_runs cli command should allow exec_date between start/end range and 
> print start/end times
> --
>
> Key: AIRFLOW-6474
> URL: https://issues.apache.org/jira/browse/AIRFLOW-6474
> Project: Apache Airflow
>  Issue Type: Improvement
>  Components: cli
>Affects Versions: 1.10.7
>Reporter: t oo
>Assignee: alexander
>Priority: Major
>  Labels: gsoc, gsoc2020, mentor
>
> 1. accept argument exec_date_from, exec_date_to to filter execution_dates 
> returned, ie show dag runs with exec_date between 20190901 and 20190930
> 2. separate to that in the output print the start_date and end_date of each 
> dagrun (ie execdate for 20190907 had start_date 2019090804:23 and end_date  
> 2019090804:38
> 3. dag_id arg should be optional



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[GitHub] [airflow] mik-laj commented on a change in pull request #7419: [AIRFLOW-6804][depends on AIRFLOW-6792] Add the basic test for all example DAGs

2020-02-17 Thread GitBox 
mik-laj commented on a change in pull request #7419:  [AIRFLOW-6804][depends on 
AIRFLOW-6792] Add the basic test for all example DAGs
URL: https://github.com/apache/airflow/pull/7419#discussion_r380344608
 
 

 ##
 File path: tests/test_example_dags.py
 ##
 @@ -0,0 +1,41 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+import os
+import unittest
+from glob import glob
+
+from airflow.models import DagBag
+
 
 Review comment:
   We had a long discussion about pytest and parameterized. 
https://lists.apache.org/thread.html/1e4df7d4b0cd9b2d2bb76a3336471aa85e852545dd41ada6d4e461b8%40%3Cdev.airflow.apache.org%3E
   
   The use of subtest is more nicely displayed in the IDE.
   ![Screenshot 2020-02-17 at 20 38 
16](https://user-images.githubusercontent.com/12058428/74682183-a06b1180-51c5-11ea-98e1-7582353450de.png)
   Using parameterized will fake us the number of tests, because suddenly the 
number of tests will increase by 80. If it's really not a few tests, but only 
one. I also don't like parameterized when generating a list is expensive. Here 
it is necessary to walk through a large number of directories to create these 
tests. 
   
   I don't see the benefit of changing the place where this message is to be 
displayed. On the other hand, the lack of mock makes this test easier to 
understand.
   
   
   
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[jira] [Commented] (AIRFLOW-6474) list_dag_runs cli command should allow exec_date between start/end range and print start/end times

2020-02-17 Thread t oo (Jira) 


[ 
https://issues.apache.org/jira/browse/AIRFLOW-6474?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17038589#comment-17038589
 ] 

t oo commented on AIRFLOW-6474:
---

[~khabarov]  can u do point 2? Print start/end times of the dagruns?  Have you 
tested multiple dag runs get returned?

> list_dag_runs cli command should allow exec_date between start/end range and 
> print start/end times
> --
>
> Key: AIRFLOW-6474
> URL: https://issues.apache.org/jira/browse/AIRFLOW-6474
> Project: Apache Airflow
>  Issue Type: Improvement
>  Components: cli
>Affects Versions: 1.10.7
>Reporter: t oo
>Assignee: alexander
>Priority: Major
>  Labels: gsoc, gsoc2020, mentor
>
> 1. accept argument exec_date_from, exec_date_to to filter execution_dates 
> returned, ie show dag runs with exec_date between 20190901 and 20190930
> 2. separate to that in the output print the start_date and end_date of each 
> dagrun (ie execdate for 20190907 had start_date 2019090804:23 and end_date  
> 2019090804:38
> 3. dag_id arg should be optional



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[GitHub] [airflow] mik-laj merged pull request #7444: [AIRFLOW-XXXX] Add known issue - example_dags/__init__.py

2020-02-17 Thread GitBox 
mik-laj merged pull request #7444: [AIRFLOW-] Add known issue  - 
example_dags/__init__.py
URL: https://github.com/apache/airflow/pull/7444
 
 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] codecov-io edited a comment on issue #7446: [AIRFLOW-6474] list_dag_runs cli command should allow exec_date between start/end range and print start/end times

2020-02-17 Thread GitBox 
codecov-io edited a comment on issue #7446: [AIRFLOW-6474] list_dag_runs cli 
command should allow exec_date between start/end range and print start/end times
URL: https://github.com/apache/airflow/pull/7446#issuecomment-587128695
 
 
   # [Codecov](https://codecov.io/gh/apache/airflow/pull/7446?src=pr&el=h1) 
Report
   > Merging 
[#7446](https://codecov.io/gh/apache/airflow/pull/7446?src=pr&el=desc) into 
[master](https://codecov.io/gh/apache/airflow/commit/d1ab3705d3d03f68abc2b900ebb35e8497dee2e7?src=pr&el=desc)
 will **decrease** coverage by `0.98%`.
   > The diff coverage is `71.42%`.
   
   [![Impacted file tree 
graph](https://codecov.io/gh/apache/airflow/pull/7446/graphs/tree.svg?width=650&token=WdLKlKHOAU&height=150&src=pr)](https://codecov.io/gh/apache/airflow/pull/7446?src=pr&el=tree)
   
   ```diff
   @@Coverage Diff @@
   ##   master#7446  +/-   ##
   ==
   - Coverage   86.55%   85.56%   -0.99% 
   ==
 Files 878  878  
 Lines   4121941225   +6 
   ==
   - Hits3567635274 -402 
   - Misses   5543 5951 +408
   ```
   
   
   | [Impacted 
Files](https://codecov.io/gh/apache/airflow/pull/7446?src=pr&el=tree) | 
Coverage Δ | |
   |---|---|---|
   | 
[airflow/bin/cli.py](https://codecov.io/gh/apache/airflow/pull/7446/diff?src=pr&el=tree#diff-YWlyZmxvdy9iaW4vY2xpLnB5)
 | `94.73% <ø> (ø)` | :arrow_up: |
   | 
[airflow/cli/commands/dag\_command.py](https://codecov.io/gh/apache/airflow/pull/7446/diff?src=pr&el=tree#diff-YWlyZmxvdy9jbGkvY29tbWFuZHMvZGFnX2NvbW1hbmQucHk=)
 | `84.41% <100%> (ø)` | :arrow_up: |
   | 
[airflow/models/dagrun.py](https://codecov.io/gh/apache/airflow/pull/7446/diff?src=pr&el=tree#diff-YWlyZmxvdy9tb2RlbHMvZGFncnVuLnB5)
 | `95.79% <66.66%> (-0.76%)` | :arrow_down: |
   | 
[...flow/providers/apache/cassandra/hooks/cassandra.py](https://codecov.io/gh/apache/airflow/pull/7446/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvYXBhY2hlL2Nhc3NhbmRyYS9ob29rcy9jYXNzYW5kcmEucHk=)
 | `21.51% <0%> (-72.16%)` | :arrow_down: |
   | 
[...w/providers/apache/hive/operators/mysql\_to\_hive.py](https://codecov.io/gh/apache/airflow/pull/7446/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvYXBhY2hlL2hpdmUvb3BlcmF0b3JzL215c3FsX3RvX2hpdmUucHk=)
 | `35.84% <0%> (-64.16%)` | :arrow_down: |
   | 
[airflow/operators/generic\_transfer.py](https://codecov.io/gh/apache/airflow/pull/7446/diff?src=pr&el=tree#diff-YWlyZmxvdy9vcGVyYXRvcnMvZ2VuZXJpY190cmFuc2Zlci5weQ==)
 | `39.28% <0%> (-60.72%)` | :arrow_down: |
   | 
[airflow/kubernetes/volume\_mount.py](https://codecov.io/gh/apache/airflow/pull/7446/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3ZvbHVtZV9tb3VudC5weQ==)
 | `44.44% <0%> (-55.56%)` | :arrow_down: |
   | 
[airflow/api/auth/backend/kerberos\_auth.py](https://codecov.io/gh/apache/airflow/pull/7446/diff?src=pr&el=tree#diff-YWlyZmxvdy9hcGkvYXV0aC9iYWNrZW5kL2tlcmJlcm9zX2F1dGgucHk=)
 | `28.16% <0%> (-54.93%)` | :arrow_down: |
   | 
[airflow/providers/redis/operators/redis\_publish.py](https://codecov.io/gh/apache/airflow/pull/7446/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvcmVkaXMvb3BlcmF0b3JzL3JlZGlzX3B1Ymxpc2gucHk=)
 | `50% <0%> (-50%)` | :arrow_down: |
   | 
[airflow/kubernetes/volume.py](https://codecov.io/gh/apache/airflow/pull/7446/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3ZvbHVtZS5weQ==)
 | `52.94% <0%> (-47.06%)` | :arrow_down: |
   | ... and [15 
more](https://codecov.io/gh/apache/airflow/pull/7446/diff?src=pr&el=tree-more) 
| |
   
   --
   
   [Continue to review full report at 
Codecov](https://codecov.io/gh/apache/airflow/pull/7446?src=pr&el=continue).
   > **Legend** - [Click here to learn 
more](https://docs.codecov.io/docs/codecov-delta)
   > `Δ = absolute  (impact)`, `ø = not affected`, `? = missing data`
   > Powered by 
[Codecov](https://codecov.io/gh/apache/airflow/pull/7446?src=pr&el=footer). 
Last update 
[d1ab370...174016d](https://codecov.io/gh/apache/airflow/pull/7446?src=pr&el=lastupdated).
 Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] codecov-io commented on issue #7446: [AIRFLOW-6474] list_dag_runs cli command should allow exec_date between start/end range and print start/end times

2020-02-17 Thread GitBox 
codecov-io commented on issue #7446: [AIRFLOW-6474] list_dag_runs cli command 
should allow exec_date between start/end range and print start/end times
URL: https://github.com/apache/airflow/pull/7446#issuecomment-587128695
 
 
   # [Codecov](https://codecov.io/gh/apache/airflow/pull/7446?src=pr&el=h1) 
Report
   > Merging 
[#7446](https://codecov.io/gh/apache/airflow/pull/7446?src=pr&el=desc) into 
[master](https://codecov.io/gh/apache/airflow/commit/d1ab3705d3d03f68abc2b900ebb35e8497dee2e7?src=pr&el=desc)
 will **decrease** coverage by `0.98%`.
   > The diff coverage is `71.42%`.
   
   [![Impacted file tree 
graph](https://codecov.io/gh/apache/airflow/pull/7446/graphs/tree.svg?width=650&token=WdLKlKHOAU&height=150&src=pr)](https://codecov.io/gh/apache/airflow/pull/7446?src=pr&el=tree)
   
   ```diff
   @@Coverage Diff @@
   ##   master#7446  +/-   ##
   ==
   - Coverage   86.55%   85.56%   -0.99% 
   ==
 Files 878  878  
 Lines   4121941225   +6 
   ==
   - Hits3567635274 -402 
   - Misses   5543 5951 +408
   ```
   
   
   | [Impacted 
Files](https://codecov.io/gh/apache/airflow/pull/7446?src=pr&el=tree) | 
Coverage Δ | |
   |---|---|---|
   | 
[airflow/bin/cli.py](https://codecov.io/gh/apache/airflow/pull/7446/diff?src=pr&el=tree#diff-YWlyZmxvdy9iaW4vY2xpLnB5)
 | `94.73% <ø> (ø)` | :arrow_up: |
   | 
[airflow/cli/commands/dag\_command.py](https://codecov.io/gh/apache/airflow/pull/7446/diff?src=pr&el=tree#diff-YWlyZmxvdy9jbGkvY29tbWFuZHMvZGFnX2NvbW1hbmQucHk=)
 | `84.41% <100%> (ø)` | :arrow_up: |
   | 
[airflow/models/dagrun.py](https://codecov.io/gh/apache/airflow/pull/7446/diff?src=pr&el=tree#diff-YWlyZmxvdy9tb2RlbHMvZGFncnVuLnB5)
 | `95.79% <66.66%> (-0.76%)` | :arrow_down: |
   | 
[...flow/providers/apache/cassandra/hooks/cassandra.py](https://codecov.io/gh/apache/airflow/pull/7446/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvYXBhY2hlL2Nhc3NhbmRyYS9ob29rcy9jYXNzYW5kcmEucHk=)
 | `21.51% <0%> (-72.16%)` | :arrow_down: |
   | 
[...w/providers/apache/hive/operators/mysql\_to\_hive.py](https://codecov.io/gh/apache/airflow/pull/7446/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvYXBhY2hlL2hpdmUvb3BlcmF0b3JzL215c3FsX3RvX2hpdmUucHk=)
 | `35.84% <0%> (-64.16%)` | :arrow_down: |
   | 
[airflow/operators/generic\_transfer.py](https://codecov.io/gh/apache/airflow/pull/7446/diff?src=pr&el=tree#diff-YWlyZmxvdy9vcGVyYXRvcnMvZ2VuZXJpY190cmFuc2Zlci5weQ==)
 | `39.28% <0%> (-60.72%)` | :arrow_down: |
   | 
[airflow/kubernetes/volume\_mount.py](https://codecov.io/gh/apache/airflow/pull/7446/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3ZvbHVtZV9tb3VudC5weQ==)
 | `44.44% <0%> (-55.56%)` | :arrow_down: |
   | 
[airflow/api/auth/backend/kerberos\_auth.py](https://codecov.io/gh/apache/airflow/pull/7446/diff?src=pr&el=tree#diff-YWlyZmxvdy9hcGkvYXV0aC9iYWNrZW5kL2tlcmJlcm9zX2F1dGgucHk=)
 | `28.16% <0%> (-54.93%)` | :arrow_down: |
   | 
[airflow/providers/redis/operators/redis\_publish.py](https://codecov.io/gh/apache/airflow/pull/7446/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvcmVkaXMvb3BlcmF0b3JzL3JlZGlzX3B1Ymxpc2gucHk=)
 | `50% <0%> (-50%)` | :arrow_down: |
   | 
[airflow/kubernetes/volume.py](https://codecov.io/gh/apache/airflow/pull/7446/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3ZvbHVtZS5weQ==)
 | `52.94% <0%> (-47.06%)` | :arrow_down: |
   | ... and [15 
more](https://codecov.io/gh/apache/airflow/pull/7446/diff?src=pr&el=tree-more) 
| |
   
   --
   
   [Continue to review full report at 
Codecov](https://codecov.io/gh/apache/airflow/pull/7446?src=pr&el=continue).
   > **Legend** - [Click here to learn 
more](https://docs.codecov.io/docs/codecov-delta)
   > `Δ = absolute  (impact)`, `ø = not affected`, `? = missing data`
   > Powered by 
[Codecov](https://codecov.io/gh/apache/airflow/pull/7446?src=pr&el=footer). 
Last update 
[d1ab370...174016d](https://codecov.io/gh/apache/airflow/pull/7446?src=pr&el=lastupdated).
 Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] konpap94 commented on issue #6337: [AIRFLOW-5659] - Add support for ephemeral storage on KubernetesPodOp…

2020-02-17 Thread GitBox 
konpap94 commented on issue #6337: [AIRFLOW-5659] - Add support for ephemeral 
storage on KubernetesPodOp…
URL: https://github.com/apache/airflow/pull/6337#issuecomment-587121417
 
 
   Is there any update on this being merged in?


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] codecov-io edited a comment on issue #7437: [AIRFLOW-2325] Add CloudwatchTaskHandler option for remote task loggi…

2020-02-17 Thread GitBox 
codecov-io edited a comment on issue #7437: [AIRFLOW-2325] Add 
CloudwatchTaskHandler option for remote task loggi…
URL: https://github.com/apache/airflow/pull/7437#issuecomment-586842394
 
 
   # [Codecov](https://codecov.io/gh/apache/airflow/pull/7437?src=pr&el=h1) 
Report
   > Merging 
[#7437](https://codecov.io/gh/apache/airflow/pull/7437?src=pr&el=desc) into 
[master](https://codecov.io/gh/apache/airflow/commit/946bdc23c039637b0383e1269f99bdd1b2426565?src=pr&el=desc)
 will **increase** coverage by `53.42%`.
   > The diff coverage is `93.18%`.
   
   [![Impacted file tree 
graph](https://codecov.io/gh/apache/airflow/pull/7437/graphs/tree.svg?width=650&token=WdLKlKHOAU&height=150&src=pr)](https://codecov.io/gh/apache/airflow/pull/7437?src=pr&el=tree)
   
   ```diff
   @@ Coverage Diff @@
   ##   master#7437   +/-   ##
   ===
   + Coverage   32.95%   86.37%   +53.42% 
   ===
 Files 878  879+1 
 Lines   4118841263   +75 
   ===
   + Hits1357335641+22068 
   + Misses  27615 5622-21993
   ```
   
   
   | [Impacted 
Files](https://codecov.io/gh/apache/airflow/pull/7437?src=pr&el=tree) | 
Coverage Δ | |
   |---|---|---|
   | 
[airflow/utils/log/cloudwatch\_task\_handler.py](https://codecov.io/gh/apache/airflow/pull/7437/diff?src=pr&el=tree#diff-YWlyZmxvdy91dGlscy9sb2cvY2xvdWR3YXRjaF90YXNrX2hhbmRsZXIucHk=)
 | `100% <100%> (ø)` | |
   | 
[airflow/config\_templates/airflow\_local\_settings.py](https://codecov.io/gh/apache/airflow/pull/7437/diff?src=pr&el=tree#diff-YWlyZmxvdy9jb25maWdfdGVtcGxhdGVzL2FpcmZsb3dfbG9jYWxfc2V0dGluZ3MucHk=)
 | `62.5% <25%> (+20.19%)` | :arrow_up: |
   | 
[airflow/kubernetes/volume\_mount.py](https://codecov.io/gh/apache/airflow/pull/7437/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3ZvbHVtZV9tb3VudC5weQ==)
 | `44.44% <0%> (-55.56%)` | :arrow_down: |
   | 
[airflow/kubernetes/volume.py](https://codecov.io/gh/apache/airflow/pull/7437/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3ZvbHVtZS5weQ==)
 | `52.94% <0%> (-47.06%)` | :arrow_down: |
   | 
[airflow/kubernetes/pod\_launcher.py](https://codecov.io/gh/apache/airflow/pull/7437/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3BvZF9sYXVuY2hlci5weQ==)
 | `47.18% <0%> (-39.44%)` | :arrow_down: |
   | 
[...viders/cncf/kubernetes/operators/kubernetes\_pod.py](https://codecov.io/gh/apache/airflow/pull/7437/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvY25jZi9rdWJlcm5ldGVzL29wZXJhdG9ycy9rdWJlcm5ldGVzX3BvZC5weQ==)
 | `69.38% <0%> (-25.52%)` | :arrow_down: |
   | 
[airflow/kubernetes/refresh\_config.py](https://codecov.io/gh/apache/airflow/pull/7437/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3JlZnJlc2hfY29uZmlnLnB5)
 | `50.98% <0%> (-23.53%)` | :arrow_down: |
   | 
[airflow/bin/airflow](https://codecov.io/gh/apache/airflow/pull/7437/diff?src=pr&el=tree#diff-YWlyZmxvdy9iaW4vYWlyZmxvdw==)
 | `84.61% <0%> (ø)` | :arrow_up: |
   | 
[airflow/plugins\_manager.py](https://codecov.io/gh/apache/airflow/pull/7437/diff?src=pr&el=tree#diff-YWlyZmxvdy9wbHVnaW5zX21hbmFnZXIucHk=)
 | `86.95% <0%> (+0.72%)` | :arrow_up: |
   | ... and [748 
more](https://codecov.io/gh/apache/airflow/pull/7437/diff?src=pr&el=tree-more) 
| |
   
   --
   
   [Continue to review full report at 
Codecov](https://codecov.io/gh/apache/airflow/pull/7437?src=pr&el=continue).
   > **Legend** - [Click here to learn 
more](https://docs.codecov.io/docs/codecov-delta)
   > `Δ = absolute  (impact)`, `ø = not affected`, `? = missing data`
   > Powered by 
[Codecov](https://codecov.io/gh/apache/airflow/pull/7437?src=pr&el=footer). 
Last update 
[946bdc2...0823ca9](https://codecov.io/gh/apache/airflow/pull/7437?src=pr&el=lastupdated).
 Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] codecov-io edited a comment on issue #7437: [AIRFLOW-2325] Add CloudwatchTaskHandler option for remote task loggi…

2020-02-17 Thread GitBox 
codecov-io edited a comment on issue #7437: [AIRFLOW-2325] Add 
CloudwatchTaskHandler option for remote task loggi…
URL: https://github.com/apache/airflow/pull/7437#issuecomment-586842394
 
 
   # [Codecov](https://codecov.io/gh/apache/airflow/pull/7437?src=pr&el=h1) 
Report
   > Merging 
[#7437](https://codecov.io/gh/apache/airflow/pull/7437?src=pr&el=desc) into 
[master](https://codecov.io/gh/apache/airflow/commit/946bdc23c039637b0383e1269f99bdd1b2426565?src=pr&el=desc)
 will **increase** coverage by `53.24%`.
   > The diff coverage is `93.18%`.
   
   [![Impacted file tree 
graph](https://codecov.io/gh/apache/airflow/pull/7437/graphs/tree.svg?width=650&token=WdLKlKHOAU&height=150&src=pr)](https://codecov.io/gh/apache/airflow/pull/7437?src=pr&el=tree)
   
   ```diff
   @@Coverage Diff @@
   ##   master   #7437   +/-   ##
   ==
   + Coverage   32.95%   86.2%   +53.24% 
   ==
 Files 878 879+1 
 Lines   41188   41263   +75 
   ==
   + Hits13573   35569+21996 
   + Misses  276155694-21921
   ```
   
   
   | [Impacted 
Files](https://codecov.io/gh/apache/airflow/pull/7437?src=pr&el=tree) | 
Coverage Δ | |
   |---|---|---|
   | 
[airflow/utils/log/cloudwatch\_task\_handler.py](https://codecov.io/gh/apache/airflow/pull/7437/diff?src=pr&el=tree#diff-YWlyZmxvdy91dGlscy9sb2cvY2xvdWR3YXRjaF90YXNrX2hhbmRsZXIucHk=)
 | `100% <100%> (ø)` | |
   | 
[airflow/config\_templates/airflow\_local\_settings.py](https://codecov.io/gh/apache/airflow/pull/7437/diff?src=pr&el=tree#diff-YWlyZmxvdy9jb25maWdfdGVtcGxhdGVzL2FpcmZsb3dfbG9jYWxfc2V0dGluZ3MucHk=)
 | `62.5% <25%> (+20.19%)` | :arrow_up: |
   | 
[airflow/kubernetes/volume\_mount.py](https://codecov.io/gh/apache/airflow/pull/7437/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3ZvbHVtZV9tb3VudC5weQ==)
 | `44.44% <0%> (-55.56%)` | :arrow_down: |
   | 
[airflow/kubernetes/volume.py](https://codecov.io/gh/apache/airflow/pull/7437/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3ZvbHVtZS5weQ==)
 | `52.94% <0%> (-47.06%)` | :arrow_down: |
   | 
[airflow/kubernetes/pod\_launcher.py](https://codecov.io/gh/apache/airflow/pull/7437/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3BvZF9sYXVuY2hlci5weQ==)
 | `47.18% <0%> (-39.44%)` | :arrow_down: |
   | 
[...viders/cncf/kubernetes/operators/kubernetes\_pod.py](https://codecov.io/gh/apache/airflow/pull/7437/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvY25jZi9rdWJlcm5ldGVzL29wZXJhdG9ycy9rdWJlcm5ldGVzX3BvZC5weQ==)
 | `69.38% <0%> (-25.52%)` | :arrow_down: |
   | 
[airflow/kubernetes/refresh\_config.py](https://codecov.io/gh/apache/airflow/pull/7437/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3JlZnJlc2hfY29uZmlnLnB5)
 | `50.98% <0%> (-23.53%)` | :arrow_down: |
   | 
[airflow/bin/airflow](https://codecov.io/gh/apache/airflow/pull/7437/diff?src=pr&el=tree#diff-YWlyZmxvdy9iaW4vYWlyZmxvdw==)
 | `84.61% <0%> (ø)` | :arrow_up: |
   | 
[airflow/plugins\_manager.py](https://codecov.io/gh/apache/airflow/pull/7437/diff?src=pr&el=tree#diff-YWlyZmxvdy9wbHVnaW5zX21hbmFnZXIucHk=)
 | `86.95% <0%> (+0.72%)` | :arrow_up: |
   | ... and [748 
more](https://codecov.io/gh/apache/airflow/pull/7437/diff?src=pr&el=tree-more) 
| |
   
   --
   
   [Continue to review full report at 
Codecov](https://codecov.io/gh/apache/airflow/pull/7437?src=pr&el=continue).
   > **Legend** - [Click here to learn 
more](https://docs.codecov.io/docs/codecov-delta)
   > `Δ = absolute  (impact)`, `ø = not affected`, `? = missing data`
   > Powered by 
[Codecov](https://codecov.io/gh/apache/airflow/pull/7437?src=pr&el=footer). 
Last update 
[946bdc2...0823ca9](https://codecov.io/gh/apache/airflow/pull/7437?src=pr&el=lastupdated).
 Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] codecov-io edited a comment on issue #7437: [AIRFLOW-2325] Add CloudwatchTaskHandler option for remote task loggi…

2020-02-17 Thread GitBox 
codecov-io edited a comment on issue #7437: [AIRFLOW-2325] Add 
CloudwatchTaskHandler option for remote task loggi…
URL: https://github.com/apache/airflow/pull/7437#issuecomment-586842394
 
 
   # [Codecov](https://codecov.io/gh/apache/airflow/pull/7437?src=pr&el=h1) 
Report
   > Merging 
[#7437](https://codecov.io/gh/apache/airflow/pull/7437?src=pr&el=desc) into 
[master](https://codecov.io/gh/apache/airflow/commit/946bdc23c039637b0383e1269f99bdd1b2426565?src=pr&el=desc)
 will **decrease** coverage by `0.79%`.
   > The diff coverage is `29.54%`.
   
   [![Impacted file tree 
graph](https://codecov.io/gh/apache/airflow/pull/7437/graphs/tree.svg?width=650&token=WdLKlKHOAU&height=150&src=pr)](https://codecov.io/gh/apache/airflow/pull/7437?src=pr&el=tree)
   
   ```diff
   @@Coverage Diff@@
   ##   master#7437 +/-   ##
   =
   - Coverage   32.95%   32.15%   -0.8% 
   =
 Files 878  878 
 Lines   4118841250 +62 
   =
   - Hits1357313265-308 
   - Misses  2761527985+370
   ```
   
   
   | [Impacted 
Files](https://codecov.io/gh/apache/airflow/pull/7437?src=pr&el=tree) | 
Coverage Δ | |
   |---|---|---|
   | 
[airflow/config\_templates/airflow\_local\_settings.py](https://codecov.io/gh/apache/airflow/pull/7437/diff?src=pr&el=tree#diff-YWlyZmxvdy9jb25maWdfdGVtcGxhdGVzL2FpcmZsb3dfbG9jYWxfc2V0dGluZ3MucHk=)
 | `39.28% <0%> (-3.03%)` | :arrow_down: |
   | 
[airflow/utils/log/cloudwatch\_task\_handler.py](https://codecov.io/gh/apache/airflow/pull/7437/diff?src=pr&el=tree#diff-YWlyZmxvdy91dGlscy9sb2cvY2xvdWR3YXRjaF90YXNrX2hhbmRsZXIucHk=)
 | `32.5% <32.5%> (ø)` | |
   | 
[...viders/cncf/kubernetes/operators/kubernetes\_pod.py](https://codecov.io/gh/apache/airflow/pull/7437/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvY25jZi9rdWJlcm5ldGVzL29wZXJhdG9ycy9rdWJlcm5ldGVzX3BvZC5weQ==)
 | `24.48% <0%> (-70.41%)` | :arrow_down: |
   | 
[airflow/kubernetes/pod\_launcher.py](https://codecov.io/gh/apache/airflow/pull/7437/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3BvZF9sYXVuY2hlci5weQ==)
 | `28.16% <0%> (-58.46%)` | :arrow_down: |
   | 
[airflow/kubernetes/volume\_mount.py](https://codecov.io/gh/apache/airflow/pull/7437/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3ZvbHVtZV9tb3VudC5weQ==)
 | `44.44% <0%> (-55.56%)` | :arrow_down: |
   | 
[airflow/kubernetes/pod.py](https://codecov.io/gh/apache/airflow/pull/7437/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3BvZC5weQ==)
 | `44.73% <0%> (-47.37%)` | :arrow_down: |
   | 
[airflow/kubernetes/volume.py](https://codecov.io/gh/apache/airflow/pull/7437/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3ZvbHVtZS5weQ==)
 | `52.94% <0%> (-47.06%)` | :arrow_down: |
   | 
[airflow/kubernetes/kube\_client.py](https://codecov.io/gh/apache/airflow/pull/7437/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL2t1YmVfY2xpZW50LnB5)
 | `37.14% <0%> (-40%)` | :arrow_down: |
   | 
[airflow/kubernetes/pod\_generator.py](https://codecov.io/gh/apache/airflow/pull/7437/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3BvZF9nZW5lcmF0b3IucHk=)
 | `20.5% <0%> (-38%)` | :arrow_down: |
   | 
[airflow/kubernetes/refresh\_config.py](https://codecov.io/gh/apache/airflow/pull/7437/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3JlZnJlc2hfY29uZmlnLnB5)
 | `39.21% <0%> (-35.3%)` | :arrow_down: |
   | ... and [9 
more](https://codecov.io/gh/apache/airflow/pull/7437/diff?src=pr&el=tree-more) 
| |
   
   --
   
   [Continue to review full report at 
Codecov](https://codecov.io/gh/apache/airflow/pull/7437?src=pr&el=continue).
   > **Legend** - [Click here to learn 
more](https://docs.codecov.io/docs/codecov-delta)
   > `Δ = absolute  (impact)`, `ø = not affected`, `? = missing data`
   > Powered by 
[Codecov](https://codecov.io/gh/apache/airflow/pull/7437?src=pr&el=footer). 
Last update 
[946bdc2...0823ca9](https://codecov.io/gh/apache/airflow/pull/7437?src=pr&el=lastupdated).
 Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] codecov-io edited a comment on issue #7437: [AIRFLOW-2325] Add CloudwatchTaskHandler option for remote task loggi…

2020-02-17 Thread GitBox 
codecov-io edited a comment on issue #7437: [AIRFLOW-2325] Add 
CloudwatchTaskHandler option for remote task loggi…
URL: https://github.com/apache/airflow/pull/7437#issuecomment-586842394
 
 
   # [Codecov](https://codecov.io/gh/apache/airflow/pull/7437?src=pr&el=h1) 
Report
   > Merging 
[#7437](https://codecov.io/gh/apache/airflow/pull/7437?src=pr&el=desc) into 
[master](https://codecov.io/gh/apache/airflow/commit/946bdc23c039637b0383e1269f99bdd1b2426565?src=pr&el=desc)
 will **decrease** coverage by `0.79%`.
   > The diff coverage is `29.54%`.
   
   [![Impacted file tree 
graph](https://codecov.io/gh/apache/airflow/pull/7437/graphs/tree.svg?width=650&token=WdLKlKHOAU&height=150&src=pr)](https://codecov.io/gh/apache/airflow/pull/7437?src=pr&el=tree)
   
   ```diff
   @@Coverage Diff@@
   ##   master#7437 +/-   ##
   =
   - Coverage   32.95%   32.15%   -0.8% 
   =
 Files 878  878 
 Lines   4118841250 +62 
   =
   - Hits1357313265-308 
   - Misses  2761527985+370
   ```
   
   
   | [Impacted 
Files](https://codecov.io/gh/apache/airflow/pull/7437?src=pr&el=tree) | 
Coverage Δ | |
   |---|---|---|
   | 
[airflow/config\_templates/airflow\_local\_settings.py](https://codecov.io/gh/apache/airflow/pull/7437/diff?src=pr&el=tree#diff-YWlyZmxvdy9jb25maWdfdGVtcGxhdGVzL2FpcmZsb3dfbG9jYWxfc2V0dGluZ3MucHk=)
 | `39.28% <0%> (-3.03%)` | :arrow_down: |
   | 
[airflow/utils/log/cloudwatch\_task\_handler.py](https://codecov.io/gh/apache/airflow/pull/7437/diff?src=pr&el=tree#diff-YWlyZmxvdy91dGlscy9sb2cvY2xvdWR3YXRjaF90YXNrX2hhbmRsZXIucHk=)
 | `32.5% <32.5%> (ø)` | |
   | 
[...viders/cncf/kubernetes/operators/kubernetes\_pod.py](https://codecov.io/gh/apache/airflow/pull/7437/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvY25jZi9rdWJlcm5ldGVzL29wZXJhdG9ycy9rdWJlcm5ldGVzX3BvZC5weQ==)
 | `24.48% <0%> (-70.41%)` | :arrow_down: |
   | 
[airflow/kubernetes/pod\_launcher.py](https://codecov.io/gh/apache/airflow/pull/7437/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3BvZF9sYXVuY2hlci5weQ==)
 | `28.16% <0%> (-58.46%)` | :arrow_down: |
   | 
[airflow/kubernetes/volume\_mount.py](https://codecov.io/gh/apache/airflow/pull/7437/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3ZvbHVtZV9tb3VudC5weQ==)
 | `44.44% <0%> (-55.56%)` | :arrow_down: |
   | 
[airflow/kubernetes/pod.py](https://codecov.io/gh/apache/airflow/pull/7437/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3BvZC5weQ==)
 | `44.73% <0%> (-47.37%)` | :arrow_down: |
   | 
[airflow/kubernetes/volume.py](https://codecov.io/gh/apache/airflow/pull/7437/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3ZvbHVtZS5weQ==)
 | `52.94% <0%> (-47.06%)` | :arrow_down: |
   | 
[airflow/kubernetes/kube\_client.py](https://codecov.io/gh/apache/airflow/pull/7437/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL2t1YmVfY2xpZW50LnB5)
 | `37.14% <0%> (-40%)` | :arrow_down: |
   | 
[airflow/kubernetes/pod\_generator.py](https://codecov.io/gh/apache/airflow/pull/7437/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3BvZF9nZW5lcmF0b3IucHk=)
 | `20.5% <0%> (-38%)` | :arrow_down: |
   | 
[airflow/kubernetes/refresh\_config.py](https://codecov.io/gh/apache/airflow/pull/7437/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3JlZnJlc2hfY29uZmlnLnB5)
 | `39.21% <0%> (-35.3%)` | :arrow_down: |
   | ... and [9 
more](https://codecov.io/gh/apache/airflow/pull/7437/diff?src=pr&el=tree-more) 
| |
   
   --
   
   [Continue to review full report at 
Codecov](https://codecov.io/gh/apache/airflow/pull/7437?src=pr&el=continue).
   > **Legend** - [Click here to learn 
more](https://docs.codecov.io/docs/codecov-delta)
   > `Δ = absolute  (impact)`, `ø = not affected`, `? = missing data`
   > Powered by 
[Codecov](https://codecov.io/gh/apache/airflow/pull/7437?src=pr&el=footer). 
Last update 
[946bdc2...0823ca9](https://codecov.io/gh/apache/airflow/pull/7437?src=pr&el=lastupdated).
 Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[jira] [Assigned] (AIRFLOW-2325) Task logging with AWS Cloud watch

2020-02-17 Thread Daniel Hegberg (Jira) 


 [ 
https://issues.apache.org/jira/browse/AIRFLOW-2325?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Daniel Hegberg reassigned AIRFLOW-2325:
---

Assignee: Daniel Hegberg

> Task logging with AWS Cloud watch
> -
>
> Key: AIRFLOW-2325
> URL: https://issues.apache.org/jira/browse/AIRFLOW-2325
> Project: Apache Airflow
>  Issue Type: New Feature
>  Components: logging
>Reporter: Fang-Pen Lin
>Assignee: Daniel Hegberg
>Priority: Minor
>
> In many cases, it's ideal to use remote logging while running Airflow in 
> production, as the worker could be easily scale down or scale up. Or the 
> worker is running in containers, where the local storage is not meant to be 
> there forever. In that case, the S3 task logging handler could be used
> [https://github.com/apache/incubator-airflow/blob/master/airflow/utils/log/s3_task_handler.py]
> However, it comes with drawback. S3 logging handler only uploads the log when 
> the task completed or failed. For long running tasks, it's hard to know 
> what's going on with the process until it finishes.
> To make more real-time logging, I built a logging handler based on AWS 
> CloudWatch. It uses a third party python package `watchtower`
>  
> [https://github.com/kislyuk/watchtower/tree/master/watchtower]
>  
> I created a PR here [https://github.com/apache/incubator-airflow/pull/3229], 
> basically I just copy-pasted the code I wrote for my own project, it works 
> fine with 1.9 release, but never tested with master branch. Also, there is a 
> bug in watchtower causing task runner to hang forever when it completes. I 
> created an issue in their repo
> [https://github.com/kislyuk/watchtower/issues/57]
> And a PR for addressing that issue 
> [https://github.com/kislyuk/watchtower/pull/58]
>  
> The PR is still far from ready to be reviewed, but I just want to get some 
> feedback before I spend more time on it. I would like to see if youguys want 
> this cloudwatch handler goes into the main repo, or do youguys prefer it to 
> be a standalone third-party module. If it's that case, I can close this 
> ticket and create a standalone repo on my own. If the PR is welcome, then I 
> can spend more time on polishing it based on your feedback, add tests / 
> documents and other stuff.
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (AIRFLOW-6474) list_dag_runs cli command should allow exec_date between start/end range and print start/end times

2020-02-17 Thread ASF GitHub Bot (Jira) 


[ 
https://issues.apache.org/jira/browse/AIRFLOW-6474?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17038556#comment-17038556
 ] 

ASF GitHub Bot commented on AIRFLOW-6474:
-

AlexandrKhabarov commented on pull request #7446: [AIRFLOW-6474] list_dag_runs 
cli command should allow exec_date between start/end range and print start/end 
times
URL: https://github.com/apache/airflow/pull/7446
 
 
   ---
   Issue link: 
[AIRFLOW-6474](https://issues.apache.org/jira/projects/AIRFLOW/issues/AIRFLOW-6474?filter=myopenissues)
   
   Make sure to mark the boxes below before creating PR: [x]
   
   - [x] Description above provides context of the change
   - [x] Commit message/PR title starts with `[AIRFLOW-]`. AIRFLOW- = 
JIRA ID*
   - [x] Unit tests coverage for changes (not needed for documentation changes)
   - [x] Commits follow "[How to write a good git commit 
message](http://chris.beams.io/posts/git-commit/)"
   - [x] Relevant documentation is updated including usage instructions.
   - [x] I will engage committers as explained in [Contribution Workflow 
Example](https://github.com/apache/airflow/blob/master/CONTRIBUTING.rst#contribution-workflow-example).
   
   * For document-only changes commit message can start with 
`[AIRFLOW-]`.
   
   ---
   In case of fundamental code change, Airflow Improvement Proposal 
([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvements+Proposals))
 is needed.
   In case of a new dependency, check compliance with the [ASF 3rd Party 
License Policy](https://www.apache.org/legal/resolved.html#category-x).
   In case of backwards incompatible changes please leave a note in 
[UPDATING.md](https://github.com/apache/airflow/blob/master/UPDATING.md).
   Read the [Pull Request 
Guidelines](https://github.com/apache/airflow/blob/master/CONTRIBUTING.rst#pull-request-guidelines)
 for more information.
   
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


> list_dag_runs cli command should allow exec_date between start/end range and 
> print start/end times
> --
>
> Key: AIRFLOW-6474
> URL: https://issues.apache.org/jira/browse/AIRFLOW-6474
> Project: Apache Airflow
>  Issue Type: Improvement
>  Components: cli
>Affects Versions: 1.10.7
>Reporter: t oo
>Assignee: alexander
>Priority: Major
>  Labels: gsoc, gsoc2020, mentor
>
> 1. accept argument exec_date_from, exec_date_to to filter execution_dates 
> returned, ie show dag runs with exec_date between 20190901 and 20190930
> 2. separate to that in the output print the start_date and end_date of each 
> dagrun (ie execdate for 20190907 had start_date 2019090804:23 and end_date  
> 2019090804:38
> 3. dag_id arg should be optional



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[GitHub] [airflow] dhegberg commented on issue #7437: [AIRFLOW-2325] Add CloudwatchTaskHandler option for remote task loggi…

2020-02-17 Thread GitBox 
dhegberg commented on issue #7437: [AIRFLOW-2325] Add CloudwatchTaskHandler 
option for remote task loggi…
URL: https://github.com/apache/airflow/pull/7437#issuecomment-587114047
 
 
   > Can you also add a note in the documentation?
   > https://airflow.readthedocs.io/en/latest/howto/write-logs.html
   
   Done


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] boring-cyborg[bot] commented on issue #7446: [AIRFLOW-6474] list_dag_runs cli command should allow exec_date between start/end range and print start/end times

2020-02-17 Thread GitBox 
boring-cyborg[bot] commented on issue #7446: [AIRFLOW-6474] list_dag_runs cli 
command should allow exec_date between start/end range and print start/end times
URL: https://github.com/apache/airflow/pull/7446#issuecomment-587113938
 
 
   Congratulations on your first Pull Request and welcome to the Apache Airflow 
community! If you have any issues or are unsure about any anything please check 
our Contribution Guide 
(https://github.com/apache/airflow/blob/master/CONTRIBUTING.rst)
   Here are some useful points:
   - Pay attention to the quality of your code (flake8, pylint and type 
annotations). Our [pre-commits]( 
https://github.com/apache/airflow/blob/master/STATIC_CODE_CHECKS.rst#prerequisites-for-pre-commit-hooks)
 will help you with that.
   - In case of a new feature add useful documentation (in docstrings or in 
`docs/` directory). Adding a new operator? Check this short 
[guide](https://github.com/apache/airflow/blob/master/docs/howto/custom-operator.rst)
 Consider adding an example DAG that shows how users should use it.
   - Consider using [Breeze 
environment](https://github.com/apache/airflow/blob/master/BREEZE.rst) for 
testing locally, it’s a heavy docker but it ships with a working Airflow and a 
lot of integrations.
   - Be patient and persistent. It might take some time to get a review or get 
the final approval from Committers.
   Apache Airflow is a community-driven project and together we are making it 
better 🚀.
   In case of doubts contact the developers at:
   Mailing List: d...@airflow.apache.org
   Slack: https://apache-airflow-slack.herokuapp.com/
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] AlexandrKhabarov opened a new pull request #7446: [AIRFLOW-6474] list_dag_runs cli command should allow exec_date between start/end range and print start/end times

2020-02-17 Thread GitBox 
AlexandrKhabarov opened a new pull request #7446: [AIRFLOW-6474] list_dag_runs 
cli command should allow exec_date between start/end range and print start/end 
times
URL: https://github.com/apache/airflow/pull/7446
 
 
   ---
   Issue link: 
[AIRFLOW-6474](https://issues.apache.org/jira/projects/AIRFLOW/issues/AIRFLOW-6474?filter=myopenissues)
   
   Make sure to mark the boxes below before creating PR: [x]
   
   - [x] Description above provides context of the change
   - [x] Commit message/PR title starts with `[AIRFLOW-]`. AIRFLOW- = 
JIRA ID*
   - [x] Unit tests coverage for changes (not needed for documentation changes)
   - [x] Commits follow "[How to write a good git commit 
message](http://chris.beams.io/posts/git-commit/)"
   - [x] Relevant documentation is updated including usage instructions.
   - [x] I will engage committers as explained in [Contribution Workflow 
Example](https://github.com/apache/airflow/blob/master/CONTRIBUTING.rst#contribution-workflow-example).
   
   * For document-only changes commit message can start with 
`[AIRFLOW-]`.
   
   ---
   In case of fundamental code change, Airflow Improvement Proposal 
([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvements+Proposals))
 is needed.
   In case of a new dependency, check compliance with the [ASF 3rd Party 
License Policy](https://www.apache.org/legal/resolved.html#category-x).
   In case of backwards incompatible changes please leave a note in 
[UPDATING.md](https://github.com/apache/airflow/blob/master/UPDATING.md).
   Read the [Pull Request 
Guidelines](https://github.com/apache/airflow/blob/master/CONTRIBUTING.rst#pull-request-guidelines)
 for more information.
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] mik-laj commented on a change in pull request #7217: [WIP] [AIRFLOW-NNNN] Store DAG's source code in the serialized_dag table

2020-02-17 Thread GitBox 
mik-laj commented on a change in pull request #7217: [WIP] [AIRFLOW-] Store 
DAG's source code in the serialized_dag table
URL: https://github.com/apache/airflow/pull/7217#discussion_r380316629
 
 

 ##
 File path: airflow/www/views.py
 ##
 @@ -520,16 +521,60 @@ def last_dagruns(self, session=None):
 @has_access
 @provide_session
 def code(self, session=None):
-dm = models.DagModel
-dag_id = request.args.get('dag_id')
-dag = session.query(dm).filter(dm.dag_id == dag_id).first()
-try:
+
+def get_serialized_dag(dag_id):
 
 Review comment:
   These functions look very complex. Can you move them to another module? This 
will allow their use also by other classes, e.g. API


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] feluelle commented on issue #7412: [AIRFLOW-6792] Remove _operator/_hook/_sensor in providers package and add tests

2020-02-17 Thread GitBox 
feluelle commented on issue #7412: [AIRFLOW-6792] Remove 
_operator/_hook/_sensor in providers package and add tests
URL: https://github.com/apache/airflow/pull/7412#issuecomment-587083854
 
 
   > We currently have CloudBaseHook class, so maybe let's call AwsBaseHook and 
GoogleBaseHook. WDYT?
   
   I am fine with this 👍 


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] vsoch commented on issue #7191: [AIRFLOW-4030] second attempt to add singularity to airflow

2020-02-17 Thread GitBox 
vsoch commented on issue #7191: [AIRFLOW-4030] second attempt to add 
singularity to airflow
URL: https://github.com/apache/airflow/pull/7191#issuecomment-587082403
 
 
   Ah thank you! Just added.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] mik-laj edited a comment on issue #7412: [AIRFLOW-6792] Remove _operator/_hook/_sensor in providers package and add tests

2020-02-17 Thread GitBox 
mik-laj edited a comment on issue #7412: [AIRFLOW-6792] Remove 
_operator/_hook/_sensor in providers package and add tests
URL: https://github.com/apache/airflow/pull/7412#issuecomment-587078917
 
 
   > ..but why not just `aws / google ? 
   
   This is too confusing because we also have packages that have the same name.
   
   >  If we want to prefix it with base_ I would suggest to rename the class 
name also to BaseAwsHook. WDYT?
   
   We currently have CloudBaseHook class, so maybe let's call AwsBaseHook and 
GoogleBaseHook. WDYT?


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] mik-laj commented on issue #7412: [AIRFLOW-6792] Remove _operator/_hook/_sensor in providers package and add tests

2020-02-17 Thread GitBox 
mik-laj commented on issue #7412: [AIRFLOW-6792] Remove _operator/_hook/_sensor 
in providers package and add tests
URL: https://github.com/apache/airflow/pull/7412#issuecomment-587078917
 
 
   > ..but why not just `aws / google ? 
   This is too confusing because we also have packages that have the same name.
   
   >  If we want to prefix it with base_ I would suggest to rename the class 
name also to BaseAwsHook. WDYT?
   We currently have CloudBaseHook class, so maybe let's call AwsBaseHook and 
GoogleBaseHook. WDYT?


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] potiuk commented on issue #6756: [AIRFLOW-6198] Add types to core classes.

2020-02-17 Thread GitBox 
potiuk commented on issue #6756: [AIRFLOW-6198] Add types to core classes.
URL: https://github.com/apache/airflow/pull/6756#issuecomment-587079105
 
 
   N .. Please don't do it stalebot !


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] mik-laj commented on a change in pull request #7413: [AIRFLOW-6793] Respect env variable in airflow config command

2020-02-17 Thread GitBox 
mik-laj commented on a change in pull request #7413: [AIRFLOW-6793] Respect env 
variable in airflow config command
URL: https://github.com/apache/airflow/pull/7413#discussion_r380282030
 
 

 ##
 File path: airflow/configuration.py
 ##
 @@ -372,6 +371,26 @@ def getsection(self, section):
 _section[key] = val
 return _section
 
+def write(self, fp, space_around_delimiters=True):
+"""Write an .ini-format representation of the configuration state.
+
+If `space_around_delimiters' is True (the default), delimiters
+between keys and values are surrounded by spaces.
+
+This is based on the configparsser.RawConfigParser.write method code 
to add support for
 
 Review comment:
   We should also support the cmd variant. I added support. 


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] mik-laj commented on a change in pull request #7413: [AIRFLOW-6793] Respect env variable in airflow config command

2020-02-17 Thread GitBox 
mik-laj commented on a change in pull request #7413: [AIRFLOW-6793] Respect env 
variable in airflow config command
URL: https://github.com/apache/airflow/pull/7413#discussion_r380280948
 
 

 ##
 File path: tests/cli/commands/test_config_command.py
 ##
 @@ -43,3 +43,10 @@ def test_cli_show_config_should_display_key(self):
 config_command.show_config(self.parser.parse_args(['config']))
 self.assertIn('[core]', temp_stdout.getvalue())
 self.assertIn('testkey = test_value', temp_stdout.getvalue())
+
+@mock.patch.dict("os.environ", {"AIRFLOW__CORE__DAGS_FOLDER": 
"/tmp/test_folder"})
+def test_cli_show_config_should_respect_env_variable(self):
+with contextlib.redirect_stdout(io.StringIO()) as temp_stdout:
+config_command.show_config(self.parser.parse_args(['config']))
+self.assertIn('[core]', temp_stdout.getvalue())
+self.assertIn('dags_folder = /tmp/test_folder', temp_stdout.getvalue())
 
 Review comment:
   Removed.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] mik-laj commented on a change in pull request #7413: [AIRFLOW-6793] Respect env variable in airflow config command

2020-02-17 Thread GitBox 
mik-laj commented on a change in pull request #7413: [AIRFLOW-6793] Respect env 
variable in airflow config command
URL: https://github.com/apache/airflow/pull/7413#discussion_r380264708
 
 

 ##
 File path: airflow/configuration.py
 ##
 @@ -372,6 +371,26 @@ def getsection(self, section):
 _section[key] = val
 return _section
 
+def write(self, fp, space_around_delimiters=True):
+"""Write an .ini-format representation of the configuration state.
+
+If `space_around_delimiters' is True (the default), delimiters
+between keys and values are surrounded by spaces.
+
+This is based on the configparsser.RawConfigParser.write method code 
to add support for
 
 Review comment:
   ```diff
   def write(self, fp, space_around_delimiters=True):
   """Write an .ini-format representation of the configuration state.
   
   If `space_around_delimiters' is True (the default), delimiters
   between keys and values are surrounded by spaces.
   """
   if space_around_delimiters:
   d = " {} ".format(self._delimiters[0])
   else:
   d = self._delimiters[0]
   if self._defaults:
   self._write_section(fp, self.default_section, 
self._defaults.items(), d)
   for section in self._sections:
   -self._write_section(fp, section, 
self._sections[section].items(), d)
   +self._write_section(fp, section, 
self.getsection(section).items(), d)
   ```


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] potiuk merged pull request #7445: [AIRFLOW-XXXX] Fix breeze build-docs

2020-02-17 Thread GitBox 
potiuk merged pull request #7445: [AIRFLOW-] Fix breeze build-docs
URL: https://github.com/apache/airflow/pull/7445
 
 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] potiuk commented on issue #7433: [AIRFLOW-6820] split breeze into functions

2020-02-17 Thread GitBox 
potiuk commented on issue #7433: [AIRFLOW-6820] split breeze into functions
URL: https://github.com/apache/airflow/pull/7433#issuecomment-587074147
 
 
   > I'm a little bit lost in two-in-one refactor in this PR (moving things to 
functions, and also renaming some env vars) but looks okay from what I can tell
   
   I could not resist the temptation to rename some of the variables and make 
them shorter. Since they are all internal and done automatically I think doing 
it while separating out functions might be a good idea.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] potiuk commented on a change in pull request #7433: [AIRFLOW-6820] split breeze into functions

2020-02-17 Thread GitBox 
potiuk commented on a change in pull request #7433: [AIRFLOW-6820] split breeze 
into functions
URL: https://github.com/apache/airflow/pull/7433#discussion_r380278477
 
 

 ##
 File path: setup.py
 ##
 @@ -493,7 +493,7 @@ def do_setup():
 'tabulate>=0.7.5, <0.9',
 'tenacity==4.12.0',
 'termcolor==1.1.0',
-'text-unidecode==1.3',
+'text-unidecode==1.2',
 
 Review comment:
   yeah. Thanks .. will fix


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] potiuk commented on a change in pull request #7433: [AIRFLOW-6820] split breeze into functions

2020-02-17 Thread GitBox 
potiuk commented on a change in pull request #7433: [AIRFLOW-6820] split breeze 
into functions
URL: https://github.com/apache/airflow/pull/7433#discussion_r380278387
 
 

 ##
 File path: scripts/ci/in_container/_in_container_utils.sh
 ##
 @@ -58,9 +58,11 @@ function print_in_container_info() {
 # Cleans up PYC files (in case they come in mounted folders)
 #
 function in_container_cleanup_pyc() {
-print_in_container_info
-print_in_container_info "Cleaning up .pyc files"
-print_in_container_info
+if [[ ${AIRFLOW_CI_VERBOSE:="false"} == "true" ]]; then
 
 Review comment:
   Ah yeah. IT was a copy&paste victim


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] potiuk commented on a change in pull request #7433: [AIRFLOW-6820] split breeze into functions

2020-02-17 Thread GitBox 
potiuk commented on a change in pull request #7433: [AIRFLOW-6820] split breeze 
into functions
URL: https://github.com/apache/airflow/pull/7433#discussion_r380278085
 
 

 ##
 File path: breeze
 ##
 @@ -16,146 +16,245 @@
 # KIND, either express or implied.  See the License for the
 # specific language governing permissions and limitations
 # under the License.
-
-# Bash sanity settings (error on exit, complain for undefined vars, error when 
pipe fails)
 set -euo pipefail
 
-export BREEZE=true
-
 MY_DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )"
 
-export AIRFLOW_SOURCES="${MY_DIR}"
-
-# Directory where all CI scripts are located
-export SCRIPTS_CI_DIR="${MY_DIR}/scripts/ci"
-
-BUILD_CACHE_DIR="${MY_DIR}/.build"
-FILES_DIR="${MY_DIR}/files"
-TMP_DIR="${MY_DIR}/tmp"
-
-mkdir -pv "${BUILD_CACHE_DIR}"
-mkdir -pv "${TMP_DIR}"
-mkdir -pv "${FILES_DIR}"
-
-# Note - we do not use __script_init.sh here because it can only be used from 
within
-# the CI directory and we need to overrride PYTHON_VERSION based on what we 
store
-# in the .build directory
+# Bash arrays need to be defined outside of functions unfortunately :(
+# Array with extra options for Docker compose
+declare -a EXTRA_DC_OPTIONS
+# Array with selected integrations
+declare -a INTEGRATIONS
+# This is where remaining args are passed
+declare -a REMAINING_ARGS
 
-# Beginning of the initialisation here
+function setup_default_breeze_variables() {
+# Whether to actually run docker compose with the command set given
+export COMMAND_TO_RUN="enter_breeze"
 
 Review comment:
   Yeah. I will review it and cleanup all the exports :). That's a good point


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[jira] [Created] (AIRFLOW-6826) Make sure initialize-virtualenv works with most *env managers

2020-02-17 Thread Jarek Potiuk (Jira) 
Jarek Potiuk created AIRFLOW-6826:
-

 Summary: Make sure initialize-virtualenv works with most *env 
managers
 Key: AIRFLOW-6826
 URL: https://issues.apache.org/jira/browse/AIRFLOW-6826
 Project: Apache Airflow
  Issue Type: Improvement
  Components: breeze
Affects Versions: 2.0.0
Reporter: Jarek Potiuk






--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[GitHub] [airflow] potiuk commented on a change in pull request #7433: [AIRFLOW-6820] split breeze into functions

2020-02-17 Thread GitBox 
potiuk commented on a change in pull request #7433: [AIRFLOW-6820] split breeze 
into functions
URL: https://github.com/apache/airflow/pull/7433#discussion_r380276920
 
 

 ##
 File path: breeze
 ##
 @@ -16,146 +16,245 @@
 # KIND, either express or implied.  See the License for the
 # specific language governing permissions and limitations
 # under the License.
-
-# Bash sanity settings (error on exit, complain for undefined vars, error when 
pipe fails)
 set -euo pipefail
 
-export BREEZE=true
-
 MY_DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )"
 
-export AIRFLOW_SOURCES="${MY_DIR}"
-
-# Directory where all CI scripts are located
-export SCRIPTS_CI_DIR="${MY_DIR}/scripts/ci"
-
-BUILD_CACHE_DIR="${MY_DIR}/.build"
-FILES_DIR="${MY_DIR}/files"
-TMP_DIR="${MY_DIR}/tmp"
-
-mkdir -pv "${BUILD_CACHE_DIR}"
-mkdir -pv "${TMP_DIR}"
-mkdir -pv "${FILES_DIR}"
-
-# Note - we do not use __script_init.sh here because it can only be used from 
within
-# the CI directory and we need to overrride PYTHON_VERSION based on what we 
store
-# in the .build directory
+# Bash arrays need to be defined outside of functions unfortunately :(
+# Array with extra options for Docker compose
+declare -a EXTRA_DC_OPTIONS
+# Array with selected integrations
+declare -a INTEGRATIONS
+# This is where remaining args are passed
+declare -a REMAINING_ARGS
 
-# Beginning of the initialisation here
+function setup_default_breeze_variables() {
+# Whether to actually run docker compose with the command set given
+export COMMAND_TO_RUN="enter_breeze"
+export BREEZE=true
 
-# shellcheck source=scripts/ci/_utils.sh
-. "${SCRIPTS_CI_DIR}/_utils.sh"
+export AIRFLOW_SOURCES="${MY_DIR}"
 
-initialize_breeze_environment
-
-export PYTHON_VERSION="${PYTHON_VERSION:=$(read_from_file PYTHON_VERSION)}"
-
-basic_sanity_checks
+# Directory where all CI scripts are located
+export SCRIPTS_CI_DIR="${MY_DIR}/scripts/ci"
 
-script_start
+export BUILD_CACHE_DIR="${MY_DIR}/.build"
+export FILES_DIR="${MY_DIR}/files"
+export TMP_DIR="${MY_DIR}/tmp"
 
-trap script_end EXIT
+mkdir -pv "${BUILD_CACHE_DIR}"
+mkdir -pv "${TMP_DIR}"
+mkdir -pv "${FILES_DIR}"
 
-# End of initialisation here
+# Note - we do not use __script_init.sh here because it can only be used 
from within
+# the CI directory and we need to overrride PYTHON_VERSION based on what 
we store
+# in the .build directory
 
+# Beginning of the initialisation here
 
-# Sets width of the screen
-SEPARATOR_WIDTH="$(tput cols)"
+# shellcheck source=scripts/ci/_utils.sh
+. "${SCRIPTS_CI_DIR}/_utils.sh"
 
-# Name of the script
-CMDNAME="$(basename -- "$0")"
+export PYTHON_VERSION="${PYTHON_VERSION:=$(read_from_file PYTHON_VERSION)}"
 
-# Update short and long options in the breeze-complete script
-# This way autocomplete will work automatically with all options
-# shellcheck source=breeze-complete
-. "${MY_DIR}/breeze-complete"
+# Sets width of the screen
+SEPARATOR_WIDTH="$(tput cols)"
 
-# Whether to actually run docker compose with the command set given
-ENTER_ENVIRONMENT="true"
+# Name of the script
+CMDNAME="$(basename -- "$0")"
 
+# Update short and long options in the breeze-complete script
+# This way autocomplete will work automatically with all options
+# shellcheck source=breeze-complete
+. "${MY_DIR}/breeze-complete"
 
-# Whether to cleanup local image
-CLEANUP_IMAGES="false"
+# Skips mounting local Airflow sources
+export SKIP_MOUNTING_LOCAL_SOURCES="false"
 
-# Skips mounting local Airflow sources
-SKIP_MOUNTING_LOCAL_SOURCES="false"
+# Holds chosen command if the -x flag is used.
+export RUN_COMMAND=""
 
-# If set, we initialize local virtualenv and install all dependencies
-INITIALIZE_LOCAL_VIRTUALENV=false
+# Holds the test target if the -t flag is used.
+export TEST_TARGET=""
 
-# If set, we setup autocomplete for breeze
-SETUP_AUTOCOMPLETE=false
+# Holds docker compose command if the -d flag is used.
+export DOCKER_COMPOSE_COMMAND=""
 
-# Holds chosen command if the -x flag is used.
-RUN_COMMAND=""
+# If true, the docker images are rebuilt locally.
+export NEEDS_DOCKER_BUILD="false"
 
-# Holds the test target if the -t flag is used.
-TEST_TARGET=""
+# By default we only pull images if we do not have them locally.
+# This can be overridden by -p flag
+export FORCE_PULL_IMAGES="false"
 
-# Holds docker compose command if the -d flag is used.
-DOCKER_COMPOSE_COMMAND=""
+# Runtime is empty initially (might be set to kubernetes in case 
kubernetes is chosen)
+export RUNTIME=""
 
-#extra options for Docker compose
-EXTRA_DC_OPTIONS=()
+# Do not enable Kind Kubernetes cluster by default
+export ENABLE_KIND_CLUSTER="false"
 
-# If true, the docker images are rebuilt locally.
-export AIRFLOW_CONTAINER_DOCKER_BUILD_NEEDED="false"
+# Do not recreate Kubernetes cluster by default
+export RECREATE_KIND_CLUST

[GitHub] [airflow] mik-laj commented on a change in pull request #7413: [AIRFLOW-6793] Respect env variable in airflow config command

2020-02-17 Thread GitBox 
mik-laj commented on a change in pull request #7413: [AIRFLOW-6793] Respect env 
variable in airflow config command
URL: https://github.com/apache/airflow/pull/7413#discussion_r380264708
 
 

 ##
 File path: airflow/configuration.py
 ##
 @@ -372,6 +371,26 @@ def getsection(self, section):
 _section[key] = val
 return _section
 
+def write(self, fp, space_around_delimiters=True):
+"""Write an .ini-format representation of the configuration state.
+
+If `space_around_delimiters' is True (the default), delimiters
+between keys and values are surrounded by spaces.
+
+This is based on the configparsser.RawConfigParser.write method code 
to add support for
 
 Review comment:
   ```diff
   def write(self, fp, space_around_delimiters=True):
   """Write an .ini-format representation of the configuration state.
   
   If `space_around_delimiters' is True (the default), delimiters
   between keys and values are surrounded by spaces.
   """
   if space_around_delimiters:
   d = " {} ".format(self._delimiters[0])
   else:
   d = self._delimiters[0]
   if self._defaults:
   self._write_section(fp, self.default_section, 
self._defaults.items(), d)
   for section in self._sections:
   -self._write_section(fp, section, 
self._sections[section].items(), d)
   +   self._write_section(fp, section, 
self.getsection(section).items(), d)
   ```


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[jira] [Commented] (AIRFLOW-6823) Breeze broken on current master on Linux

2020-02-17 Thread Ash Berlin-Taylor (Jira) 


[ 
https://issues.apache.org/jira/browse/AIRFLOW-6823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17038458#comment-17038458
 ] 

Ash Berlin-Taylor commented on AIRFLOW-6823:


Detecting def seems doable.

Another option which _may_ work is to do {{docker run --user $(id -ur):$(id 
-gr) ...}} when running. The downside to that is that without putting an entry 
in the container with the right (unknowable-in-advance) user id it shows "I 
have no name!".

> Breeze broken on current master on Linux
> 
>
> Key: AIRFLOW-6823
> URL: https://issues.apache.org/jira/browse/AIRFLOW-6823
> Project: Apache Airflow
>  Issue Type: Bug
>  Components: breeze
>Affects Versions: 2.0.0
>Reporter: Aleksander Nitecki
>Assignee: Ash Berlin-Taylor
>Priority: Minor
> Attachments: Breeze on current master log.txt
>
>
> See the attachment for log.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[GitHub] [airflow] mik-laj commented on issue #6875: [AIRFLOW-6076] Fix dag.cli() Key error

2020-02-17 Thread GitBox 
mik-laj commented on issue #6875: [AIRFLOW-6076] Fix dag.cli() Key error
URL: https://github.com/apache/airflow/pull/6875#issuecomment-587051727
 
 
   `dag.cli()` still doesn't work. 
   ```
   16:39 $ python airflow/example_dags/example_complex.py
   Traceback (most recent call last):
 File "airflow/example_dags/example_complex.py", line 224, in 
   dag.cli()
 File 
"/Users/kamilbregula/devel/google-airflow/airflow/airflow/models/dag.py", line 
1398, in cli
   parser = cli.CLIFactory.get_parser(dag_parser=True)
 File 
"/Users/kamilbregula/devel/google-airflow/airflow/airflow/bin/cli.py", line 
1057, in get_parser
   sub = cls.subparsers_dict[sub]
   KeyError: 'backfill'
   ```
   What do you mean by "These PR changes are already there in the master"? CLI 
is developing very intensively, but all the issues have not yet been resolved.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] dimberman commented on a change in pull request #7433: [AIRFLOW-6820] split breeze into functions

2020-02-17 Thread GitBox 
dimberman commented on a change in pull request #7433: [AIRFLOW-6820] split 
breeze into functions
URL: https://github.com/apache/airflow/pull/7433#discussion_r380248947
 
 

 ##
 File path: breeze
 ##
 @@ -16,146 +16,245 @@
 # KIND, either express or implied.  See the License for the
 # specific language governing permissions and limitations
 # under the License.
-
-# Bash sanity settings (error on exit, complain for undefined vars, error when 
pipe fails)
 set -euo pipefail
 
-export BREEZE=true
-
 MY_DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )"
 
-export AIRFLOW_SOURCES="${MY_DIR}"
-
-# Directory where all CI scripts are located
-export SCRIPTS_CI_DIR="${MY_DIR}/scripts/ci"
-
-BUILD_CACHE_DIR="${MY_DIR}/.build"
-FILES_DIR="${MY_DIR}/files"
-TMP_DIR="${MY_DIR}/tmp"
-
-mkdir -pv "${BUILD_CACHE_DIR}"
-mkdir -pv "${TMP_DIR}"
-mkdir -pv "${FILES_DIR}"
-
-# Note - we do not use __script_init.sh here because it can only be used from 
within
-# the CI directory and we need to overrride PYTHON_VERSION based on what we 
store
-# in the .build directory
+# Bash arrays need to be defined outside of functions unfortunately :(
+# Array with extra options for Docker compose
+declare -a EXTRA_DC_OPTIONS
+# Array with selected integrations
+declare -a INTEGRATIONS
+# This is where remaining args are passed
+declare -a REMAINING_ARGS
 
-# Beginning of the initialisation here
+function setup_default_breeze_variables() {
+# Whether to actually run docker compose with the command set given
+export COMMAND_TO_RUN="enter_breeze"
+export BREEZE=true
 
-# shellcheck source=scripts/ci/_utils.sh
-. "${SCRIPTS_CI_DIR}/_utils.sh"
+export AIRFLOW_SOURCES="${MY_DIR}"
 
-initialize_breeze_environment
-
-export PYTHON_VERSION="${PYTHON_VERSION:=$(read_from_file PYTHON_VERSION)}"
-
-basic_sanity_checks
+# Directory where all CI scripts are located
+export SCRIPTS_CI_DIR="${MY_DIR}/scripts/ci"
 
-script_start
+export BUILD_CACHE_DIR="${MY_DIR}/.build"
+export FILES_DIR="${MY_DIR}/files"
+export TMP_DIR="${MY_DIR}/tmp"
 
-trap script_end EXIT
+mkdir -pv "${BUILD_CACHE_DIR}"
+mkdir -pv "${TMP_DIR}"
+mkdir -pv "${FILES_DIR}"
 
-# End of initialisation here
+# Note - we do not use __script_init.sh here because it can only be used 
from within
+# the CI directory and we need to overrride PYTHON_VERSION based on what 
we store
+# in the .build directory
 
+# Beginning of the initialisation here
 
-# Sets width of the screen
-SEPARATOR_WIDTH="$(tput cols)"
+# shellcheck source=scripts/ci/_utils.sh
+. "${SCRIPTS_CI_DIR}/_utils.sh"
 
-# Name of the script
-CMDNAME="$(basename -- "$0")"
+export PYTHON_VERSION="${PYTHON_VERSION:=$(read_from_file PYTHON_VERSION)}"
 
-# Update short and long options in the breeze-complete script
-# This way autocomplete will work automatically with all options
-# shellcheck source=breeze-complete
-. "${MY_DIR}/breeze-complete"
+# Sets width of the screen
+SEPARATOR_WIDTH="$(tput cols)"
 
-# Whether to actually run docker compose with the command set given
-ENTER_ENVIRONMENT="true"
+# Name of the script
+CMDNAME="$(basename -- "$0")"
 
+# Update short and long options in the breeze-complete script
+# This way autocomplete will work automatically with all options
+# shellcheck source=breeze-complete
+. "${MY_DIR}/breeze-complete"
 
-# Whether to cleanup local image
-CLEANUP_IMAGES="false"
+# Skips mounting local Airflow sources
+export SKIP_MOUNTING_LOCAL_SOURCES="false"
 
-# Skips mounting local Airflow sources
-SKIP_MOUNTING_LOCAL_SOURCES="false"
+# Holds chosen command if the -x flag is used.
+export RUN_COMMAND=""
 
-# If set, we initialize local virtualenv and install all dependencies
-INITIALIZE_LOCAL_VIRTUALENV=false
+# Holds the test target if the -t flag is used.
+export TEST_TARGET=""
 
-# If set, we setup autocomplete for breeze
-SETUP_AUTOCOMPLETE=false
+# Holds docker compose command if the -d flag is used.
+export DOCKER_COMPOSE_COMMAND=""
 
-# Holds chosen command if the -x flag is used.
-RUN_COMMAND=""
+# If true, the docker images are rebuilt locally.
+export NEEDS_DOCKER_BUILD="false"
 
-# Holds the test target if the -t flag is used.
-TEST_TARGET=""
+# By default we only pull images if we do not have them locally.
+# This can be overridden by -p flag
+export FORCE_PULL_IMAGES="false"
 
-# Holds docker compose command if the -d flag is used.
-DOCKER_COMPOSE_COMMAND=""
+# Runtime is empty initially (might be set to kubernetes in case 
kubernetes is chosen)
+export RUNTIME=""
 
-#extra options for Docker compose
-EXTRA_DC_OPTIONS=()
+# Do not enable Kind Kubernetes cluster by default
+export ENABLE_KIND_CLUSTER="false"
 
-# If true, the docker images are rebuilt locally.
-export AIRFLOW_CONTAINER_DOCKER_BUILD_NEEDED="false"
+# Do not recreate Kubernetes cluster by default
+export RECREATE_KIND_CL

[GitHub] [airflow] mik-laj commented on issue #7442: [AIRFLOW-6721] [WIP] Organize hive tests

2020-02-17 Thread GitBox 
mik-laj commented on issue #7442: [AIRFLOW-6721] [WIP] Organize hive tests
URL: https://github.com/apache/airflow/pull/7442#issuecomment-587048503
 
 
   Good direction. I look forward to further work.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

  1   2   3   >