date:20190329



 [ 
https://issues.apache.org/jira/browse/AIRFLOW-4138?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Chao-Han Tsai updated AIRFLOW-4138:
---
Summary: [AIP] Introduce DAG manifest  (was: Introduce DAG manifest)

> [AIP] Introduce DAG manifest
> 
>
> Key: AIRFLOW-4138
> URL: https://issues.apache.org/jira/browse/AIRFLOW-4138
> Project: Apache Airflow
>  Issue Type: New Feature
>Reporter: Chao-Han Tsai
>Assignee: Chao-Han Tsai
>Priority: Major
>
> Currently Airflow traverses all the files under $AIRFLOW_HOME/dags and find 
> out the DAGs by inspecting the code. We should explicitly specify which files 
> are DAG.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Closed] (AIRFLOW-4119) Instrument dagrun start time delay



 [ 
https://issues.apache.org/jira/browse/AIRFLOW-4119?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Chao-Han Tsai closed AIRFLOW-4119.
--
Resolution: Won't Fix

> Instrument dagrun start time delay
> --
>
> Key: AIRFLOW-4119
> URL: https://issues.apache.org/jira/browse/AIRFLOW-4119
> Project: Apache Airflow
>  Issue Type: New Feature
>Reporter: Chao-Han Tsai
>Priority: Major
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (AIRFLOW-4139) [AIP] DAG versioning



 [ 
https://issues.apache.org/jira/browse/AIRFLOW-4139?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Chao-Han Tsai updated AIRFLOW-4139:
---
Summary: [AIP] DAG versioning  (was: DAG versioning)

> [AIP] DAG versioning
> 
>
> Key: AIRFLOW-4139
> URL: https://issues.apache.org/jira/browse/AIRFLOW-4139
> Project: Apache Airflow
>  Issue Type: New Feature
>Reporter: Chao-Han Tsai
>Assignee: Chao-Han Tsai
>Priority: Major
>
> Currently, running DagRun will be impacted if we change the DAG file in the 
> middle of the run. After we have 
> https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-5+Remote+DAG+Fetcher 
> and https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-20+DAG+manifest 
> ready, we can start saving each version of the DAG file on the remote system 
> and the running tasks should refer to a specific version of DAG instead of 
> the latest DAG.
> How is it different from 
> https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-12+Persist+DAG+into+DB?
> Please see 
> https://issues.apache.org/jira/browse/AIRFLOW-4139?focusedCommentId=16799264=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-16799264



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (AIRFLOW-4191) Add a pool utilization instrument DAG

Chao-Han Tsai created AIRFLOW-4191:
--

 Summary: Add a pool utilization instrument DAG
 Key: AIRFLOW-4191
 URL: https://issues.apache.org/jira/browse/AIRFLOW-4191
 Project: Apache Airflow
  Issue Type: New Feature
Reporter: Chao-Han Tsai
Assignee: Chao-Han Tsai


This DAG measures the utilization of each pool (open_slots, used_slots, 
queued_slots).



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (AIRFLOW-4190) Add a schedule delay monitoring DAG

Chao-Han Tsai created AIRFLOW-4190:
--

 Summary: Add a schedule delay monitoring DAG
 Key: AIRFLOW-4190
 URL: https://issues.apache.org/jira/browse/AIRFLOW-4190
 Project: Apache Airflow
  Issue Type: New Feature
Reporter: Chao-Han Tsai
Assignee: Chao-Han Tsai


The DAG measures the delay between the scheduled DAG start time (e.g. 
next_execution_date) and the wall clock time when first task executes.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (AIRFLOW-4189) Add an airflowdb retention DAG

Chao-Han Tsai created AIRFLOW-4189:
--

 Summary: Add an airflowdb retention DAG
 Key: AIRFLOW-4189
 URL: https://issues.apache.org/jira/browse/AIRFLOW-4189
 Project: Apache Airflow
  Issue Type: New Feature
Reporter: Chao-Han Tsai
Assignee: Chao-Han Tsai


Create an Airflow DAG that cleans up the records that are out of retention 
period in the metastore. 

- We probably need to first modify each table to record the `last_modfiy_date` 
to support retention period.
- User will specify the retention_period in the airflow.cfg



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] [airflow] codecov-io edited a comment on issue #4708: [AIRFLOW-3888] HA for metastore connection

codecov-io edited a comment on issue #4708: [AIRFLOW-3888] HA for metastore 
connection
URL: https://github.com/apache/airflow/pull/4708#issuecomment-477986178
 
 
   # [Codecov](https://codecov.io/gh/apache/airflow/pull/4708?src=pr=h1) 
Report
   > Merging 
[#4708](https://codecov.io/gh/apache/airflow/pull/4708?src=pr=desc) into 
[master](https://codecov.io/gh/apache/airflow/commit/157f7bff837dda691012da8638dbc570bf5f1844?src=pr=desc)
 will **increase** coverage by `0.06%`.
   > The diff coverage is `83.33%`.
   
   [![Impacted file tree 
graph](https://codecov.io/gh/apache/airflow/pull/4708/graphs/tree.svg?width=650=WdLKlKHOAU=150=pr)](https://codecov.io/gh/apache/airflow/pull/4708?src=pr=tree)
   
   ```diff
   @@Coverage Diff @@
   ##   master#4708  +/-   ##
   ==
   + Coverage   75.76%   75.82%   +0.06% 
   ==
 Files 461  458   -3 
 Lines   2995529895  -60 
   ==
   - Hits2269422667  -27 
   + Misses   7261 7228  -33
   ```
   
   
   | [Impacted 
Files](https://codecov.io/gh/apache/airflow/pull/4708?src=pr=tree) | 
Coverage Δ | |
   |---|---|---|
   | 
[airflow/hooks/hive\_hooks.py](https://codecov.io/gh/apache/airflow/pull/4708/diff?src=pr=tree#diff-YWlyZmxvdy9ob29rcy9oaXZlX2hvb2tzLnB5)
 | `75.63% <83.33%> (+0.3%)` | :arrow_up: |
   | 
[airflow/utils/db.py](https://codecov.io/gh/apache/airflow/pull/4708/diff?src=pr=tree#diff-YWlyZmxvdy91dGlscy9kYi5weQ==)
 | `90.19% <0%> (-0.1%)` | :arrow_down: |
   | 
[airflow/jobs.py](https://codecov.io/gh/apache/airflow/pull/4708/diff?src=pr=tree#diff-YWlyZmxvdy9qb2JzLnB5)
 | `78.83% <0%> (ø)` | :arrow_up: |
   | 
[.../contrib/example\_dags/example\_dingding\_operator.py](https://codecov.io/gh/apache/airflow/pull/4708/diff?src=pr=tree#diff-YWlyZmxvdy9jb250cmliL2V4YW1wbGVfZGFncy9leGFtcGxlX2RpbmdkaW5nX29wZXJhdG9yLnB5)
 | | |
   | 
[airflow/contrib/hooks/dingding\_hook.py](https://codecov.io/gh/apache/airflow/pull/4708/diff?src=pr=tree#diff-YWlyZmxvdy9jb250cmliL2hvb2tzL2RpbmdkaW5nX2hvb2sucHk=)
 | | |
   | 
[airflow/contrib/operators/dingding\_operator.py](https://codecov.io/gh/apache/airflow/pull/4708/diff?src=pr=tree#diff-YWlyZmxvdy9jb250cmliL29wZXJhdG9ycy9kaW5nZGluZ19vcGVyYXRvci5weQ==)
 | | |
   | 
[airflow/models/\_\_init\_\_.py](https://codecov.io/gh/apache/airflow/pull/4708/diff?src=pr=tree#diff-YWlyZmxvdy9tb2RlbHMvX19pbml0X18ucHk=)
 | `93.01% <0%> (+0.04%)` | :arrow_up: |
   | 
[airflow/www/views.py](https://codecov.io/gh/apache/airflow/pull/4708/diff?src=pr=tree#diff-YWlyZmxvdy93d3cvdmlld3MucHk=)
 | `76.29% <0%> (+0.1%)` | :arrow_up: |
   
   --
   
   [Continue to review full report at 
Codecov](https://codecov.io/gh/apache/airflow/pull/4708?src=pr=continue).
   > **Legend** - [Click here to learn 
more](https://docs.codecov.io/docs/codecov-delta)
   > `Δ = absolute  (impact)`, `ø = not affected`, `? = missing data`
   > Powered by 
[Codecov](https://codecov.io/gh/apache/airflow/pull/4708?src=pr=footer). 
Last update 
[157f7bf...52bb24e](https://codecov.io/gh/apache/airflow/pull/4708?src=pr=lastupdated).
 Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] codecov-io edited a comment on issue #4829: [AIRFLOW-3993] Change SalesforceHook and add tests

codecov-io edited a comment on issue #4829: [AIRFLOW-3993] Change 
SalesforceHook and add tests
URL: https://github.com/apache/airflow/pull/4829#issuecomment-470242979
 
 
   # [Codecov](https://codecov.io/gh/apache/airflow/pull/4829?src=pr=h1) 
Report
   > Merging 
[#4829](https://codecov.io/gh/apache/airflow/pull/4829?src=pr=desc) into 
[master](https://codecov.io/gh/apache/airflow/commit/157f7bff837dda691012da8638dbc570bf5f1844?src=pr=desc)
 will **increase** coverage by `0.25%`.
   > The diff coverage is `91.17%`.
   
   [![Impacted file tree 
graph](https://codecov.io/gh/apache/airflow/pull/4829/graphs/tree.svg?width=650=WdLKlKHOAU=150=pr)](https://codecov.io/gh/apache/airflow/pull/4829?src=pr=tree)
   
   ```diff
   @@Coverage Diff @@
   ##   master#4829  +/-   ##
   ==
   + Coverage   75.76%   76.01%   +0.25% 
   ==
 Files 461  461  
 Lines   2995529945  -10 
   ==
   + Hits2269422762  +68 
   + Misses   7261 7183  -78
   ```
   
   
   | [Impacted 
Files](https://codecov.io/gh/apache/airflow/pull/4829?src=pr=tree) | 
Coverage Δ | |
   |---|---|---|
   | 
[airflow/contrib/hooks/salesforce\_hook.py](https://codecov.io/gh/apache/airflow/pull/4829/diff?src=pr=tree#diff-YWlyZmxvdy9jb250cmliL2hvb2tzL3NhbGVzZm9yY2VfaG9vay5weQ==)
 | `91.78% <91.17%> (+91.78%)` | :arrow_up: |
   | 
[airflow/models/\_\_init\_\_.py](https://codecov.io/gh/apache/airflow/pull/4829/diff?src=pr=tree#diff-YWlyZmxvdy9tb2RlbHMvX19pbml0X18ucHk=)
 | `93.01% <0%> (+0.04%)` | :arrow_up: |
   
   --
   
   [Continue to review full report at 
Codecov](https://codecov.io/gh/apache/airflow/pull/4829?src=pr=continue).
   > **Legend** - [Click here to learn 
more](https://docs.codecov.io/docs/codecov-delta)
   > `Δ = absolute  (impact)`, `ø = not affected`, `? = missing data`
   > Powered by 
[Codecov](https://codecov.io/gh/apache/airflow/pull/4829?src=pr=footer). 
Last update 
[157f7bf...6f85b01](https://codecov.io/gh/apache/airflow/pull/4829?src=pr=lastupdated).
 Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] codecov-io edited a comment on issue #4829: [AIRFLOW-3993] Change SalesforceHook and add tests

codecov-io edited a comment on issue #4829: [AIRFLOW-3993] Change 
SalesforceHook and add tests
URL: https://github.com/apache/airflow/pull/4829#issuecomment-470242979
 
 
   # [Codecov](https://codecov.io/gh/apache/airflow/pull/4829?src=pr=h1) 
Report
   > Merging 
[#4829](https://codecov.io/gh/apache/airflow/pull/4829?src=pr=desc) into 
[master](https://codecov.io/gh/apache/airflow/commit/157f7bff837dda691012da8638dbc570bf5f1844?src=pr=desc)
 will **increase** coverage by `0.25%`.
   > The diff coverage is `91.17%`.
   
   [![Impacted file tree 
graph](https://codecov.io/gh/apache/airflow/pull/4829/graphs/tree.svg?width=650=WdLKlKHOAU=150=pr)](https://codecov.io/gh/apache/airflow/pull/4829?src=pr=tree)
   
   ```diff
   @@Coverage Diff @@
   ##   master#4829  +/-   ##
   ==
   + Coverage   75.76%   76.01%   +0.25% 
   ==
 Files 461  461  
 Lines   2995529945  -10 
   ==
   + Hits2269422762  +68 
   + Misses   7261 7183  -78
   ```
   
   
   | [Impacted 
Files](https://codecov.io/gh/apache/airflow/pull/4829?src=pr=tree) | 
Coverage Δ | |
   |---|---|---|
   | 
[airflow/contrib/hooks/salesforce\_hook.py](https://codecov.io/gh/apache/airflow/pull/4829/diff?src=pr=tree#diff-YWlyZmxvdy9jb250cmliL2hvb2tzL3NhbGVzZm9yY2VfaG9vay5weQ==)
 | `91.78% <91.17%> (+91.78%)` | :arrow_up: |
   | 
[airflow/models/\_\_init\_\_.py](https://codecov.io/gh/apache/airflow/pull/4829/diff?src=pr=tree#diff-YWlyZmxvdy9tb2RlbHMvX19pbml0X18ucHk=)
 | `93.01% <0%> (+0.04%)` | :arrow_up: |
   
   --
   
   [Continue to review full report at 
Codecov](https://codecov.io/gh/apache/airflow/pull/4829?src=pr=continue).
   > **Legend** - [Click here to learn 
more](https://docs.codecov.io/docs/codecov-delta)
   > `Δ = absolute  (impact)`, `ø = not affected`, `? = missing data`
   > Powered by 
[Codecov](https://codecov.io/gh/apache/airflow/pull/4829?src=pr=footer). 
Last update 
[157f7bf...6f85b01](https://codecov.io/gh/apache/airflow/pull/4829?src=pr=lastupdated).
 Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] feluelle commented on a change in pull request #4842: [AIRFLOW-4014] Change DatastoreHook and add tests

feluelle commented on a change in pull request #4842: [AIRFLOW-4014] Change 
DatastoreHook and add tests
URL: https://github.com/apache/airflow/pull/4842#discussion_r270606155
 
 

 ##
 File path: airflow/contrib/hooks/datastore_hook.py
 ##
 @@ -19,58 +19,73 @@
 #
 
 import time
+
 from googleapiclient.discovery import build
+
 from airflow.contrib.hooks.gcp_api_base_hook import GoogleCloudBaseHook
 
 
 class DatastoreHook(GoogleCloudBaseHook):
 """
-Interact with Google Cloud Datastore. This hook uses the Google Cloud 
Platform
-connection.
+Interact with Google Cloud Datastore. This hook uses the Google Cloud 
Platform connection.
 
 This object is not threads safe. If you want to make multiple requests
 simultaneously, you will need to create a hook per thread.
+
+:param version: The version of the API it is going to connect to.
+:type version: str
 """
 
-def __init__(self,
- datastore_conn_id='google_cloud_datastore_default',
- delegate_to=None):
+def __init__(self, datastore_conn_id='google_cloud_datastore_default', 
delegate_to=None, version='v1'):
 
 Review comment:
   Agree. I like `api_version` more. I've changed it.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] mik-laj commented on issue #4989: [AIRFLOW-819] - remove unused dateutil lib from macros init file

mik-laj commented on issue #4989: [AIRFLOW-819] - remove unused dateutil lib 
from macros init file
URL: https://github.com/apache/airflow/pull/4989#issuecomment-478185903
 
 
   This import is needed.It is not used in this file, but is passed to the 
Jinja context and used in DAGs. 
   
   
https://github.com/apache/airflow/blob/ab58eb67f63fa53b68bf50382f5eaf9303e3bdeb/airflow/models/__init__.py#L1580
   
https://github.com/apache/airflow/blob/ab58eb67f63fa53b68bf50382f5eaf9303e3bdeb/airflow/models/__init__.py#L1698


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] mik-laj commented on a change in pull request #5005: [AIRFLOW-3623] Fix bugs in Download task logs

mik-laj commented on a change in pull request #5005: [AIRFLOW-3623] Fix bugs in 
Download task logs
URL: https://github.com/apache/airflow/pull/5005#discussion_r270603410
 
 

 ##
 File path: airflow/www/templates/airflow/graph.html
 ##
 @@ -169,9 +169,9 @@
 d3.selectAll("g.node").on("click", function(d){
 task = tasks[d];
 if (task.task_type == "SubDagOperator")
-call_modal(d, execution_date, true);
+call_modal(d, execution_date, task_instances[d].try_number, true);
 else
-call_modal(d, execution_date);
+call_modal(d, execution_date,task_instances[d].try_number);
 
 Review comment:
   Javascript is not analyzed by the linter.  I am working on a PR that will 
delete the jinja directives from Javascript code so that the linter can read 
the files correctly. 
   Reference: 
   https://github.com/apache/airflow/pull/4787
   
   Other peoples also working on linting: 
   https://github.com/apache/airflow/pull/3656
   
   I have a ready patch that integrates the linter with Travis, but is waiting 
for the right moment.
   https://github.com/PolideaInternal/airflow/pull/75/files
   
   I hope my explanations will be helpful to you.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] mik-laj commented on issue #5008: [AIRFLOW-XXX] Fix typo in README

mik-laj commented on issue #5008: [AIRFLOW-XXX] Fix typo in README
URL: https://github.com/apache/airflow/pull/5008#issuecomment-478183636
 
 
   @ad-m You're right. My comment referred to the non-last version of your PR.  
 I would like to thank you for responding quickly to my comments. I would like 
to cooperate with people like you. You are a good example for other members of 
the community.  Your reaction time is perfect. I am sure that many other 
co-workers wanted to cooperate only with you.  You are an amazing man!


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] mik-laj edited a comment on issue #5008: [AIRFLOW-XXX] Fix typo in README

mik-laj edited a comment on issue #5008: [AIRFLOW-XXX] Fix typo in README
URL: https://github.com/apache/airflow/pull/5008#issuecomment-478183636
 
 
   @ad-m You're right. My comment referred to the non-last version of your PR.  
 I would like to thank you for responding quickly to my comments. I would like 
to cooperate with people like you. You are a good example for other members of 
the community.  Your reaction time is perfect. I am sure that many other 
co-workers wanted to cooperate only with you.  You are an amazing man and 
developer!


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] ad-m commented on issue #5008: [AIRFLOW-XXX] Fix typo in README

ad-m commented on issue #5008: [AIRFLOW-XXX] Fix typo in README
URL: https://github.com/apache/airflow/pull/5008#issuecomment-478182701
 
 
   I do not agree with your assessment. It seems that your last comment does 
not refer to the up-to-date version of the changes. If it is different, please 
point specific, up-to-date remarks instead of throwing words into the wind.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] ad-m commented on a change in pull request #5008: [AIRFLOW-XXX] Fix typo in README

ad-m commented on a change in pull request #5008: [AIRFLOW-XXX] Fix typo in 
README
URL: https://github.com/apache/airflow/pull/5008#discussion_r270601753
 
 

 ##
 File path: README.md
 ##
 @@ -160,14 +160,14 @@ Currently **officially** using Airflow:
 1. [Change.org](https://www.change.org) [[@change](https://github.com/change), 
[@vijaykramesh](https://github.com/vijaykramesh)]
 1. [Chartboost](https://www.chartboost.com) 
[[@cgelman](https://github.com/cgelman) & [@dclubb](https://github.com/dclubb)]
 1. [Checkr](https://checkr.com) [[@tongboh](https://github.com/tongboh)]
-1. [Children's Hospital of Philadelphia Division of Genomic 
Diagnostics](http://www.chop.edu/centers-programs/division-genomic-diagnostics) 
[[@genomics-geek]](https://github.com/genomics-geek/)
+1. [Children's Hospital of Philadelphia Division of Genomic 
Diagnostics](http://www.chop.edu/centers-programs/division-genomic-diagnostics) 
[[@genomics-geek]](https://github.com/genomics-geek/)]
 
 Review comment:
   Thank you for showing how to achieve the perfection of the Air-Flow project. 
I think that you judge it wrong and you look at it in a bad way, because the 
total number of errors in the project, thanks to my pull requests, has dropped. 
Of course, I have not solved all problems, but it is not possible to bring the 
world to perfection in a short time.
   
   I think that achieving the perfection of the Air-Flow project can be 
achieved in a larger number of pull requests of more people.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] mik-laj edited a comment on issue #5008: [AIRFLOW-XXX] Fix typo in README

mik-laj edited a comment on issue #5008: [AIRFLOW-XXX] Fix typo in README
URL: https://github.com/apache/airflow/pull/5008#issuecomment-478181785
 
 
   I looked at the patch and saw one small mistake. Could you look at her?  If 
you do not correct this issues, your patch will not be accepted. The Airflow 
community is very committed to quality. We do not accept changes that are 
flawed. I would like every PR to be accepted, but it was not good for the 
quality of our project.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] wwlian commented on a change in pull request #4980: [AIRFLOW-3971] Add Google Cloud Natural Language operators

wwlian commented on a change in pull request #4980: [AIRFLOW-3971] Add Google 
Cloud Natural Language operators
URL: https://github.com/apache/airflow/pull/4980#discussion_r270601531
 
 

 ##
 File path: docs/howto/operator/gcp/_partials/prerequisite_tasks.rst
 ##
 @@ -0,0 +1,31 @@
+..  Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements.  See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership.  The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License.  You may obtain a copy of the License at
+
+..http://www.apache.org/licenses/LICENSE-2.0
+
+..  Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied.  See the License for the
+specific language governing permissions and limitations
+under the License.
+
+To use these operators, you must do a few things:
+
+  * Select or create a Cloud Platform project using `Cloud Console 
`__.
+  * Enable billing for your project, as described in `Google Cloud 
documentation 
`__.
+  * Enable the Google Cloud Language API using `Cloud Console 
`__.
 
 Review comment:
   Good point, yes, I think that would make sense.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] mik-laj commented on issue #5008: [AIRFLOW-XXX] Fix typo in README

mik-laj commented on issue #5008: [AIRFLOW-XXX] Fix typo in README
URL: https://github.com/apache/airflow/pull/5008#issuecomment-478181785
 
 
   I looked at the patch and saw one small mistake. Could you look at her?  If 
you do not correct this issues, your patch will not be accepted. The Airflow 
community is very committed to quality. We do not accept changes that are 
flawed. I would like every amendment to be accepted, but it was not good for 
the quality of our project.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] mik-laj commented on a change in pull request #5008: [AIRFLOW-XXX] Fix typo in README

mik-laj commented on a change in pull request #5008: [AIRFLOW-XXX] Fix typo in 
README
URL: https://github.com/apache/airflow/pull/5008#discussion_r270600631
 
 

 ##
 File path: README.md
 ##
 @@ -160,14 +160,14 @@ Currently **officially** using Airflow:
 1. [Change.org](https://www.change.org) [[@change](https://github.com/change), 
[@vijaykramesh](https://github.com/vijaykramesh)]
 1. [Chartboost](https://www.chartboost.com) 
[[@cgelman](https://github.com/cgelman) & [@dclubb](https://github.com/dclubb)]
 1. [Checkr](https://checkr.com) [[@tongboh](https://github.com/tongboh)]
-1. [Children's Hospital of Philadelphia Division of Genomic 
Diagnostics](http://www.chop.edu/centers-programs/division-genomic-diagnostics) 
[[@genomics-geek]](https://github.com/genomics-geek/)
+1. [Children's Hospital of Philadelphia Division of Genomic 
Diagnostics](http://www.chop.edu/centers-programs/division-genomic-diagnostics) 
[[@genomics-geek]](https://github.com/genomics-geek/)]
 
 Review comment:
   I have doubts whether this change is certainly correct. Could you justify 
the need for this change?
   
   I add a screenshot to help you find the error
   https://user-images.githubusercontent.com/12058428/55267738-67ac0c00-5284-11e9-8585-eae32f2cf3fe.png;>
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] mik-laj edited a comment on issue #5008: [AIRFLOW-XXX] Fix typo in README

mik-laj edited a comment on issue #5008: [AIRFLOW-XXX] Fix typo in README
URL: https://github.com/apache/airflow/pull/5008#issuecomment-478179448
 
 
   Thanks. Your work for our community is very valuable.  If you are interested 
in further development of the project, I invite you to read the document 
[CONTRIBUTING.md](https://github.com/apache/airflow/blob/master/CONTRIBUTING.md).
   
   Travis is not green, but your PR touches areas of the project that are not 
covered by the tests, so this is probably a temporary environmental problem. 
Please, ignore it. Time saved on ignoring the state of Travis, you can use for 
further development of the project. I hope your further work will be equally or 
even more useful.  
   
   I would like to praise you that you have a very nice avatar. I love a cats  
My life motto is "Cats is a serious matter".


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] mik-laj commented on issue #5008: [AIRFLOW-XXX] Fix typo in README

mik-laj commented on issue #5008: [AIRFLOW-XXX] Fix typo in README
URL: https://github.com/apache/airflow/pull/5008#issuecomment-478179448
 
 
   Thanks. Your work for our community is very valuable.  If you are interested 
in further development of the project, I invite you to read the document 
[CONTRIBUTING.md](https://github.com/apache/airflow/blob/master/CONTRIBUTING.md).
   
   Travis is not green, but your PR touches areas of the project that are not 
covered by the tests, so this is probably a temporary environmental problem. 
Please, ignore it. Time saved on ignoring the state of Travis, you can use for 
further development of the project. I hope your further work will be equally or 
even more useful.  
   
   I would like to praise you that you have a very nice avatar. I love a cats  
.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] ad-m commented on issue #5008: [AIRFLOW-XXX] Fix typo in README

ad-m commented on issue #5008: [AIRFLOW-XXX] Fix typo in README
URL: https://github.com/apache/airflow/pull/5008#issuecomment-478178497
 
 
   > Could be a flaky k8s test, restarted the failing test +1
   > – Fokko


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] ad-m removed a comment on issue #5008: [AIRFLOW-XXX] Fix typo in README

ad-m removed a comment on issue #5008: [AIRFLOW-XXX] Fix typo in README
URL: https://github.com/apache/airflow/pull/5008#issuecomment-478178497
 
 
   > Could be a flaky k8s test, restarted the failing test +1
   > – Fokko


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] codecov-io commented on issue #4992: [AIRFLOW-4172] Fix changes for driver class path option in Spark Subm…

codecov-io commented on issue #4992: [AIRFLOW-4172] Fix changes for driver 
class path option in Spark Subm…
URL: https://github.com/apache/airflow/pull/4992#issuecomment-478175436
 
 
   # [Codecov](https://codecov.io/gh/apache/airflow/pull/4992?src=pr=h1) 
Report
   > Merging 
[#4992](https://codecov.io/gh/apache/airflow/pull/4992?src=pr=desc) into 
[master](https://codecov.io/gh/apache/airflow/commit/157f7bff837dda691012da8638dbc570bf5f1844?src=pr=desc)
 will **increase** coverage by `65.36%`.
   > The diff coverage is `80%`.
   
   [![Impacted file tree 
graph](https://codecov.io/gh/apache/airflow/pull/4992/graphs/tree.svg?width=650=WdLKlKHOAU=150=pr)](https://codecov.io/gh/apache/airflow/pull/4992?src=pr=tree)
   
   ```diff
   @@ Coverage Diff @@
   ##   master#4992   +/-   ##
   ===
   + Coverage   10.61%   75.98%   +65.36% 
   ===
 Files 461  461   
 Lines   2995530444  +489 
   ===
   + Hits 317923132+19953 
   + Misses  26776 7312-19464
   ```
   
   
   | [Impacted 
Files](https://codecov.io/gh/apache/airflow/pull/4992?src=pr=tree) | 
Coverage Δ | |
   |---|---|---|
   | 
[airflow/contrib/operators/spark\_submit\_operator.py](https://codecov.io/gh/apache/airflow/pull/4992/diff?src=pr=tree#diff-YWlyZmxvdy9jb250cmliL29wZXJhdG9ycy9zcGFya19zdWJtaXRfb3BlcmF0b3IucHk=)
 | `92.5% <100%> (+92.5%)` | :arrow_up: |
   | 
[airflow/contrib/hooks/spark\_submit\_hook.py](https://codecov.io/gh/apache/airflow/pull/4992/diff?src=pr=tree#diff-YWlyZmxvdy9jb250cmliL2hvb2tzL3NwYXJrX3N1Ym1pdF9ob29rLnB5)
 | `82.27% <66.66%> (+82.27%)` | :arrow_up: |
   | 
[airflow/plugins\_manager.py](https://codecov.io/gh/apache/airflow/pull/4992/diff?src=pr=tree#diff-YWlyZmxvdy9wbHVnaW5zX21hbmFnZXIucHk=)
 | `86.91% <0%> (+0.93%)` | :arrow_up: |
   | 
[airflow/executors/dask\_executor.py](https://codecov.io/gh/apache/airflow/pull/4992/diff?src=pr=tree#diff-YWlyZmxvdy9leGVjdXRvcnMvZGFza19leGVjdXRvci5weQ==)
 | `2% <0%> (+2%)` | :arrow_up: |
   | 
[airflow/exceptions.py](https://codecov.io/gh/apache/airflow/pull/4992/diff?src=pr=tree#diff-YWlyZmxvdy9leGNlcHRpb25zLnB5)
 | `100% <0%> (+2.56%)` | :arrow_up: |
   | 
[airflow/config\_templates/airflow\_local\_settings.py](https://codecov.io/gh/apache/airflow/pull/4992/diff?src=pr=tree#diff-YWlyZmxvdy9jb25maWdfdGVtcGxhdGVzL2FpcmZsb3dfbG9jYWxfc2V0dGluZ3MucHk=)
 | `76.47% <0%> (+2.94%)` | :arrow_up: |
   | 
[airflow/utils/operator\_resources.py](https://codecov.io/gh/apache/airflow/pull/4992/diff?src=pr=tree#diff-YWlyZmxvdy91dGlscy9vcGVyYXRvcl9yZXNvdXJjZXMucHk=)
 | `86.95% <0%> (+4.34%)` | :arrow_up: |
   | 
[...etes\_request\_factory/kubernetes\_request\_factory.py](https://codecov.io/gh/apache/airflow/pull/4992/diff?src=pr=tree#diff-YWlyZmxvdy9jb250cmliL2t1YmVybmV0ZXMva3ViZXJuZXRlc19yZXF1ZXN0X2ZhY3Rvcnkva3ViZXJuZXRlc19yZXF1ZXN0X2ZhY3RvcnkucHk=)
 | `72.11% <0%> (+4.8%)` | :arrow_up: |
   | 
[airflow/executors/\_\_init\_\_.py](https://codecov.io/gh/apache/airflow/pull/4992/diff?src=pr=tree#diff-YWlyZmxvdy9leGVjdXRvcnMvX19pbml0X18ucHk=)
 | `63.46% <0%> (+5.76%)` | :arrow_up: |
   | 
[airflow/settings.py](https://codecov.io/gh/apache/airflow/pull/4992/diff?src=pr=tree#diff-YWlyZmxvdy9zZXR0aW5ncy5weQ==)
 | `84.25% <0%> (+11.81%)` | :arrow_up: |
   | ... and [361 
more](https://codecov.io/gh/apache/airflow/pull/4992/diff?src=pr=tree-more) 
| |
   
   --
   
   [Continue to review full report at 
Codecov](https://codecov.io/gh/apache/airflow/pull/4992?src=pr=continue).
   > **Legend** - [Click here to learn 
more](https://docs.codecov.io/docs/codecov-delta)
   > `Δ = absolute  (impact)`, `ø = not affected`, `? = missing data`
   > Powered by 
[Codecov](https://codecov.io/gh/apache/airflow/pull/4992?src=pr=footer). 
Last update 
[157f7bf...c55b858](https://codecov.io/gh/apache/airflow/pull/4992?src=pr=lastupdated).
 Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] codecov-io edited a comment on issue #4992: [AIRFLOW-4172] Fix changes for driver class path option in Spark Subm…

codecov-io edited a comment on issue #4992: [AIRFLOW-4172] Fix changes for 
driver class path option in Spark Subm…
URL: https://github.com/apache/airflow/pull/4992#issuecomment-478175436
 
 
   # [Codecov](https://codecov.io/gh/apache/airflow/pull/4992?src=pr=h1) 
Report
   > Merging 
[#4992](https://codecov.io/gh/apache/airflow/pull/4992?src=pr=desc) into 
[master](https://codecov.io/gh/apache/airflow/commit/157f7bff837dda691012da8638dbc570bf5f1844?src=pr=desc)
 will **increase** coverage by `65.36%`.
   > The diff coverage is `80%`.
   
   [![Impacted file tree 
graph](https://codecov.io/gh/apache/airflow/pull/4992/graphs/tree.svg?width=650=WdLKlKHOAU=150=pr)](https://codecov.io/gh/apache/airflow/pull/4992?src=pr=tree)
   
   ```diff
   @@ Coverage Diff @@
   ##   master#4992   +/-   ##
   ===
   + Coverage   10.61%   75.98%   +65.36% 
   ===
 Files 461  461   
 Lines   2995530444  +489 
   ===
   + Hits 317923132+19953 
   + Misses  26776 7312-19464
   ```
   
   
   | [Impacted 
Files](https://codecov.io/gh/apache/airflow/pull/4992?src=pr=tree) | 
Coverage Δ | |
   |---|---|---|
   | 
[airflow/contrib/operators/spark\_submit\_operator.py](https://codecov.io/gh/apache/airflow/pull/4992/diff?src=pr=tree#diff-YWlyZmxvdy9jb250cmliL29wZXJhdG9ycy9zcGFya19zdWJtaXRfb3BlcmF0b3IucHk=)
 | `92.5% <100%> (+92.5%)` | :arrow_up: |
   | 
[airflow/contrib/hooks/spark\_submit\_hook.py](https://codecov.io/gh/apache/airflow/pull/4992/diff?src=pr=tree#diff-YWlyZmxvdy9jb250cmliL2hvb2tzL3NwYXJrX3N1Ym1pdF9ob29rLnB5)
 | `82.27% <66.66%> (+82.27%)` | :arrow_up: |
   | 
[airflow/plugins\_manager.py](https://codecov.io/gh/apache/airflow/pull/4992/diff?src=pr=tree#diff-YWlyZmxvdy9wbHVnaW5zX21hbmFnZXIucHk=)
 | `86.91% <0%> (+0.93%)` | :arrow_up: |
   | 
[airflow/executors/dask\_executor.py](https://codecov.io/gh/apache/airflow/pull/4992/diff?src=pr=tree#diff-YWlyZmxvdy9leGVjdXRvcnMvZGFza19leGVjdXRvci5weQ==)
 | `2% <0%> (+2%)` | :arrow_up: |
   | 
[airflow/exceptions.py](https://codecov.io/gh/apache/airflow/pull/4992/diff?src=pr=tree#diff-YWlyZmxvdy9leGNlcHRpb25zLnB5)
 | `100% <0%> (+2.56%)` | :arrow_up: |
   | 
[airflow/config\_templates/airflow\_local\_settings.py](https://codecov.io/gh/apache/airflow/pull/4992/diff?src=pr=tree#diff-YWlyZmxvdy9jb25maWdfdGVtcGxhdGVzL2FpcmZsb3dfbG9jYWxfc2V0dGluZ3MucHk=)
 | `76.47% <0%> (+2.94%)` | :arrow_up: |
   | 
[airflow/utils/operator\_resources.py](https://codecov.io/gh/apache/airflow/pull/4992/diff?src=pr=tree#diff-YWlyZmxvdy91dGlscy9vcGVyYXRvcl9yZXNvdXJjZXMucHk=)
 | `86.95% <0%> (+4.34%)` | :arrow_up: |
   | 
[...etes\_request\_factory/kubernetes\_request\_factory.py](https://codecov.io/gh/apache/airflow/pull/4992/diff?src=pr=tree#diff-YWlyZmxvdy9jb250cmliL2t1YmVybmV0ZXMva3ViZXJuZXRlc19yZXF1ZXN0X2ZhY3Rvcnkva3ViZXJuZXRlc19yZXF1ZXN0X2ZhY3RvcnkucHk=)
 | `72.11% <0%> (+4.8%)` | :arrow_up: |
   | 
[airflow/executors/\_\_init\_\_.py](https://codecov.io/gh/apache/airflow/pull/4992/diff?src=pr=tree#diff-YWlyZmxvdy9leGVjdXRvcnMvX19pbml0X18ucHk=)
 | `63.46% <0%> (+5.76%)` | :arrow_up: |
   | 
[airflow/settings.py](https://codecov.io/gh/apache/airflow/pull/4992/diff?src=pr=tree#diff-YWlyZmxvdy9zZXR0aW5ncy5weQ==)
 | `84.25% <0%> (+11.81%)` | :arrow_up: |
   | ... and [361 
more](https://codecov.io/gh/apache/airflow/pull/4992/diff?src=pr=tree-more) 
| |
   
   --
   
   [Continue to review full report at 
Codecov](https://codecov.io/gh/apache/airflow/pull/4992?src=pr=continue).
   > **Legend** - [Click here to learn 
more](https://docs.codecov.io/docs/codecov-delta)
   > `Δ = absolute  (impact)`, `ø = not affected`, `? = missing data`
   > Powered by 
[Codecov](https://codecov.io/gh/apache/airflow/pull/4992?src=pr=footer). 
Last update 
[157f7bf...c55b858](https://codecov.io/gh/apache/airflow/pull/4992?src=pr=lastupdated).
 Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] codecov-io edited a comment on issue #4983: [AIRFLOW-4163] IntervalCheckOperator supports relative diff.

codecov-io edited a comment on issue #4983: [AIRFLOW-4163] 
IntervalCheckOperator supports relative diff.
URL: https://github.com/apache/airflow/pull/4983#issuecomment-478175160
 
 
   # [Codecov](https://codecov.io/gh/apache/airflow/pull/4983?src=pr=h1) 
Report
   > Merging 
[#4983](https://codecov.io/gh/apache/airflow/pull/4983?src=pr=desc) into 
[master](https://codecov.io/gh/apache/airflow/commit/157f7bff837dda691012da8638dbc570bf5f1844?src=pr=desc)
 will **increase** coverage by `65.28%`.
   > The diff coverage is `100%`.
   
   [![Impacted file tree 
graph](https://codecov.io/gh/apache/airflow/pull/4983/graphs/tree.svg?width=650=WdLKlKHOAU=150=pr)](https://codecov.io/gh/apache/airflow/pull/4983?src=pr=tree)
   
   ```diff
   @@Coverage Diff @@
   ##   master   #4983   +/-   ##
   ==
   + Coverage   10.61%   75.9%   +65.28% 
   ==
 Files 461 461   
 Lines   29955   29968   +13 
   ==
   + Hits 3179   22746+19567 
   + Misses  267767222-19554
   ```
   
   
   | [Impacted 
Files](https://codecov.io/gh/apache/airflow/pull/4983?src=pr=tree) | 
Coverage Δ | |
   |---|---|---|
   | 
[airflow/operators/check\_operator.py](https://codecov.io/gh/apache/airflow/pull/4983/diff?src=pr=tree#diff-YWlyZmxvdy9vcGVyYXRvcnMvY2hlY2tfb3BlcmF0b3IucHk=)
 | `91.97% <100%> (+64.55%)` | :arrow_up: |
   | 
[airflow/plugins\_manager.py](https://codecov.io/gh/apache/airflow/pull/4983/diff?src=pr=tree#diff-YWlyZmxvdy9wbHVnaW5zX21hbmFnZXIucHk=)
 | `86.91% <0%> (+0.93%)` | :arrow_up: |
   | 
[airflow/executors/dask\_executor.py](https://codecov.io/gh/apache/airflow/pull/4983/diff?src=pr=tree#diff-YWlyZmxvdy9leGVjdXRvcnMvZGFza19leGVjdXRvci5weQ==)
 | `2% <0%> (+2%)` | :arrow_up: |
   | 
[airflow/exceptions.py](https://codecov.io/gh/apache/airflow/pull/4983/diff?src=pr=tree#diff-YWlyZmxvdy9leGNlcHRpb25zLnB5)
 | `100% <0%> (+2.56%)` | :arrow_up: |
   | 
[airflow/config\_templates/airflow\_local\_settings.py](https://codecov.io/gh/apache/airflow/pull/4983/diff?src=pr=tree#diff-YWlyZmxvdy9jb25maWdfdGVtcGxhdGVzL2FpcmZsb3dfbG9jYWxfc2V0dGluZ3MucHk=)
 | `76.47% <0%> (+2.94%)` | :arrow_up: |
   | 
[airflow/utils/operator\_resources.py](https://codecov.io/gh/apache/airflow/pull/4983/diff?src=pr=tree#diff-YWlyZmxvdy91dGlscy9vcGVyYXRvcl9yZXNvdXJjZXMucHk=)
 | `86.95% <0%> (+4.34%)` | :arrow_up: |
   | 
[...etes\_request\_factory/kubernetes\_request\_factory.py](https://codecov.io/gh/apache/airflow/pull/4983/diff?src=pr=tree#diff-YWlyZmxvdy9jb250cmliL2t1YmVybmV0ZXMva3ViZXJuZXRlc19yZXF1ZXN0X2ZhY3Rvcnkva3ViZXJuZXRlc19yZXF1ZXN0X2ZhY3RvcnkucHk=)
 | `72.11% <0%> (+4.8%)` | :arrow_up: |
   | 
[airflow/executors/\_\_init\_\_.py](https://codecov.io/gh/apache/airflow/pull/4983/diff?src=pr=tree#diff-YWlyZmxvdy9leGVjdXRvcnMvX19pbml0X18ucHk=)
 | `63.46% <0%> (+5.76%)` | :arrow_up: |
   | 
[airflow/settings.py](https://codecov.io/gh/apache/airflow/pull/4983/diff?src=pr=tree#diff-YWlyZmxvdy9zZXR0aW5ncy5weQ==)
 | `84.25% <0%> (+11.81%)` | :arrow_up: |
   | 
[airflow/\_vendor/nvd3/ipynb.py](https://codecov.io/gh/apache/airflow/pull/4983/diff?src=pr=tree#diff-YWlyZmxvdy9fdmVuZG9yL252ZDMvaXB5bmIucHk=)
 | `14.28% <0%> (+14.28%)` | :arrow_up: |
   | ... and [360 
more](https://codecov.io/gh/apache/airflow/pull/4983/diff?src=pr=tree-more) 
| |
   
   --
   
   [Continue to review full report at 
Codecov](https://codecov.io/gh/apache/airflow/pull/4983?src=pr=continue).
   > **Legend** - [Click here to learn 
more](https://docs.codecov.io/docs/codecov-delta)
   > `Δ = absolute  (impact)`, `ø = not affected`, `? = missing data`
   > Powered by 
[Codecov](https://codecov.io/gh/apache/airflow/pull/4983?src=pr=footer). 
Last update 
[157f7bf...7fffd65](https://codecov.io/gh/apache/airflow/pull/4983?src=pr=lastupdated).
 Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] codecov-io commented on issue #4983: [AIRFLOW-4163] IntervalCheckOperator supports relative diff.

codecov-io commented on issue #4983: [AIRFLOW-4163] IntervalCheckOperator 
supports relative diff.
URL: https://github.com/apache/airflow/pull/4983#issuecomment-478175160
 
 
   # [Codecov](https://codecov.io/gh/apache/airflow/pull/4983?src=pr=h1) 
Report
   > Merging 
[#4983](https://codecov.io/gh/apache/airflow/pull/4983?src=pr=desc) into 
[master](https://codecov.io/gh/apache/airflow/commit/157f7bff837dda691012da8638dbc570bf5f1844?src=pr=desc)
 will **increase** coverage by `65.28%`.
   > The diff coverage is `100%`.
   
   [![Impacted file tree 
graph](https://codecov.io/gh/apache/airflow/pull/4983/graphs/tree.svg?width=650=WdLKlKHOAU=150=pr)](https://codecov.io/gh/apache/airflow/pull/4983?src=pr=tree)
   
   ```diff
   @@Coverage Diff @@
   ##   master   #4983   +/-   ##
   ==
   + Coverage   10.61%   75.9%   +65.28% 
   ==
 Files 461 461   
 Lines   29955   29968   +13 
   ==
   + Hits 3179   22746+19567 
   + Misses  267767222-19554
   ```
   
   
   | [Impacted 
Files](https://codecov.io/gh/apache/airflow/pull/4983?src=pr=tree) | 
Coverage Δ | |
   |---|---|---|
   | 
[airflow/operators/check\_operator.py](https://codecov.io/gh/apache/airflow/pull/4983/diff?src=pr=tree#diff-YWlyZmxvdy9vcGVyYXRvcnMvY2hlY2tfb3BlcmF0b3IucHk=)
 | `91.97% <100%> (+64.55%)` | :arrow_up: |
   | 
[airflow/plugins\_manager.py](https://codecov.io/gh/apache/airflow/pull/4983/diff?src=pr=tree#diff-YWlyZmxvdy9wbHVnaW5zX21hbmFnZXIucHk=)
 | `86.91% <0%> (+0.93%)` | :arrow_up: |
   | 
[airflow/executors/dask\_executor.py](https://codecov.io/gh/apache/airflow/pull/4983/diff?src=pr=tree#diff-YWlyZmxvdy9leGVjdXRvcnMvZGFza19leGVjdXRvci5weQ==)
 | `2% <0%> (+2%)` | :arrow_up: |
   | 
[airflow/exceptions.py](https://codecov.io/gh/apache/airflow/pull/4983/diff?src=pr=tree#diff-YWlyZmxvdy9leGNlcHRpb25zLnB5)
 | `100% <0%> (+2.56%)` | :arrow_up: |
   | 
[airflow/config\_templates/airflow\_local\_settings.py](https://codecov.io/gh/apache/airflow/pull/4983/diff?src=pr=tree#diff-YWlyZmxvdy9jb25maWdfdGVtcGxhdGVzL2FpcmZsb3dfbG9jYWxfc2V0dGluZ3MucHk=)
 | `76.47% <0%> (+2.94%)` | :arrow_up: |
   | 
[airflow/utils/operator\_resources.py](https://codecov.io/gh/apache/airflow/pull/4983/diff?src=pr=tree#diff-YWlyZmxvdy91dGlscy9vcGVyYXRvcl9yZXNvdXJjZXMucHk=)
 | `86.95% <0%> (+4.34%)` | :arrow_up: |
   | 
[...etes\_request\_factory/kubernetes\_request\_factory.py](https://codecov.io/gh/apache/airflow/pull/4983/diff?src=pr=tree#diff-YWlyZmxvdy9jb250cmliL2t1YmVybmV0ZXMva3ViZXJuZXRlc19yZXF1ZXN0X2ZhY3Rvcnkva3ViZXJuZXRlc19yZXF1ZXN0X2ZhY3RvcnkucHk=)
 | `72.11% <0%> (+4.8%)` | :arrow_up: |
   | 
[airflow/executors/\_\_init\_\_.py](https://codecov.io/gh/apache/airflow/pull/4983/diff?src=pr=tree#diff-YWlyZmxvdy9leGVjdXRvcnMvX19pbml0X18ucHk=)
 | `63.46% <0%> (+5.76%)` | :arrow_up: |
   | 
[airflow/settings.py](https://codecov.io/gh/apache/airflow/pull/4983/diff?src=pr=tree#diff-YWlyZmxvdy9zZXR0aW5ncy5weQ==)
 | `84.25% <0%> (+11.81%)` | :arrow_up: |
   | 
[airflow/\_vendor/nvd3/ipynb.py](https://codecov.io/gh/apache/airflow/pull/4983/diff?src=pr=tree#diff-YWlyZmxvdy9fdmVuZG9yL252ZDMvaXB5bmIucHk=)
 | `14.28% <0%> (+14.28%)` | :arrow_up: |
   | ... and [360 
more](https://codecov.io/gh/apache/airflow/pull/4983/diff?src=pr=tree-more) 
| |
   
   --
   
   [Continue to review full report at 
Codecov](https://codecov.io/gh/apache/airflow/pull/4983?src=pr=continue).
   > **Legend** - [Click here to learn 
more](https://docs.codecov.io/docs/codecov-delta)
   > `Δ = absolute  (impact)`, `ø = not affected`, `? = missing data`
   > Powered by 
[Codecov](https://codecov.io/gh/apache/airflow/pull/4983?src=pr=footer). 
Last update 
[157f7bf...7fffd65](https://codecov.io/gh/apache/airflow/pull/4983?src=pr=lastupdated).
 Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] Fokko commented on issue #4896: [AIRFLOW-4061] Remove incubator in CI process

2019-03-29 Thread ASF subversion and git services (JIRA)

Fokko commented on issue #4896: [AIRFLOW-4061] Remove incubator in CI process
URL: https://github.com/apache/airflow/pull/4896#issuecomment-478169558
 
 
   No, I'm confused. I thought the old incubator repo was pointing to your 
repository. 


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[jira] [Commented] (AIRFLOW-3996) Include full source code examples in the documentation



[ 
https://issues.apache.org/jira/browse/AIRFLOW-3996?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16805517#comment-16805517
 ] 

ASF subversion and git services commented on AIRFLOW-3996:
--

Commit 157f7bff837dda691012da8638dbc570bf5f1844 in airflow's branch 
refs/heads/master from Kamil Breguła
[ https://gitbox.apache.org/repos/asf?p=airflow.git;h=157f7bf ]

[AIRFLOW-3996] Add view source link to included fragments


> Include full source code examples in the documentation
> --
>
> Key: AIRFLOW-3996
> URL: https://issues.apache.org/jira/browse/AIRFLOW-3996
> Project: Apache Airflow
>  Issue Type: Improvement
>Reporter: Kamil Bregula
>Priority: Major
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] [airflow] Fokko commented on issue #4894: [AIRFLOW-3996] Add view source link to included fragments

Fokko commented on issue #4894: [AIRFLOW-3996] Add view source link to included 
fragments
URL: https://github.com/apache/airflow/pull/4894#issuecomment-478169167
 
 
   The commit wasn't correct:
   
![image](https://user-images.githubusercontent.com/1134248/55265976-eef48200-527a-11e9-89d6-3ff610873412.png)
   
   I've corrected this.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[jira] [Commented] (AIRFLOW-3996) Include full source code examples in the documentation



[ 
https://issues.apache.org/jira/browse/AIRFLOW-3996?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16805514#comment-16805514
 ] 

ASF GitHub Bot commented on AIRFLOW-3996:
-

Fokko commented on pull request #4894: [AIRFLOW-3996] Add view source link to 
included fragments
URL: https://github.com/apache/airflow/pull/4894
 
 
   
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


> Include full source code examples in the documentation
> --
>
> Key: AIRFLOW-3996
> URL: https://issues.apache.org/jira/browse/AIRFLOW-3996
> Project: Apache Airflow
>  Issue Type: Improvement
>Reporter: Kamil Bregula
>Priority: Major
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] [airflow] Fokko merged pull request #4894: [AIRFLOW-3996] Add view source link to included fragments

2019-03-29 Thread ASF subversion and git services (JIRA)

Fokko merged pull request #4894: [AIRFLOW-3996] Add view source link to 
included fragments
URL: https://github.com/apache/airflow/pull/4894
 
 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[jira] [Commented] (AIRFLOW-4120) SLA of zero timedelta is not respected



[ 
https://issues.apache.org/jira/browse/AIRFLOW-4120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16805515#comment-16805515
 ] 

ASF subversion and git services commented on AIRFLOW-4120:
--

Commit 2614c02905a94fa36cddecb059768e41b5516f10 in airflow's branch 
refs/heads/master from Kamil Breguła
[ https://gitbox.apache.org/repos/asf?p=airflow.git;h=2614c02 ]

[AIRFLOW-4120] Modify SchedulerJob.manage_slas to respect zero timedelta SLAs 
(#4939) (#4894)

Modify SchedulerJob.manage_slas to respect zero timedelta SLAs

> SLA of zero timedelta is not respected
> --
>
> Key: AIRFLOW-4120
> URL: https://issues.apache.org/jira/browse/AIRFLOW-4120
> Project: Apache Airflow
>  Issue Type: Bug
>  Components: models
>Reporter: Markan Patel
>Assignee: Simon Poortman
>Priority: Minor
> Fix For: 2.0.0
>
>
> When passing a zero timedelta SLA to any operator, it is not properly 
> respected if the task misses it.
>  
> {code:python}
> BaseOperator(
>   sla=timedelta(), # or timedelta(0)
>   **kwargs)
> {code}
> The cause can be narrowed down to the fact that
> {code:python}
> bool(timedelta()) == False
> {code}
> So any code that checks if the SLA parameter is falsy should be updated 
> accordingly.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (AIRFLOW-3807) Fix graph view not highlighting correct tasks when hovering over or clicking on task state

2019-03-29 Thread ASF subversion and git services (JIRA)



[ 
https://issues.apache.org/jira/browse/AIRFLOW-3807?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16805500#comment-16805500
 ] 

ASF subversion and git services commented on AIRFLOW-3807:
--

Commit 0901f337a00356ac348246d21e1c9f4e9c8fbe58 in airflow's branch 
refs/heads/v1-10-test from Felix
[ https://gitbox.apache.org/repos/asf?p=airflow.git;h=0901f33 ]

[AIRFLOW-3807] Fix Graph View Highlighting of Tasks (#4653)

- fixes up_for_retry and up_for_reschedule tasks when hovering over 
retry/rescheduled task state
- adds missing task states to stateFocusMap that will be used for highlighting 
tasks when clicking on task state
- removed invalid attributes for some tags
- reformatted accordingly to rules from .editorconfig

[AIRFLOW-3807] Fix no_status tasks highlighting in graph view

[AIRFLOW-3807] Change "no status" string to "no_status"

[AIRFLOW-3807] Fix syntax issue in js statement

[AIRFLOW-3807] Correct tree view tasks' status labels

- reformat tree.html file
- remove invalid attributes from tree.html tags


> Fix graph view not highlighting correct tasks when hovering over or clicking 
> on task state
> --
>
> Key: AIRFLOW-3807
> URL: https://issues.apache.org/jira/browse/AIRFLOW-3807
> Project: Apache Airflow
>  Issue Type: Bug
>  Components: ui
>Affects Versions: 1.10.2
>Reporter: Felix Uellendall
>Assignee: Felix Uellendall
>Priority: Major
> Fix For: 1.10.3
>
>
> There are actually three issues that this ticket is going to address:
> * tasks with state *up_for_retry* are not highlighted when hovering over task 
> state *retry*
> * tasks with state *up_for_reschedule* are not highlighted when hovering over 
> task state *rescheduled*
> * tasks with state *up_for_reschedule*, *up_for_retry* and *skipped* will not 
> be highlighted when clicking on task state
> *EDIT:*
> * tasks with state *null* are not highlighted when hovering over task state 
> *no status* - See: https://issues.apache.org/jira/browse/AIRFLOW-3748
> * tasks with state *null* will not be highlighted when clicking on task state
> *EDIT:*
> Updates Tree View task status labels to be the same as in the Graph View



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] [airflow] ad-m commented on issue #5008: [AIRFLOW-XXX] Fix typo in README

2019-03-29 Thread ASF subversion and git services (JIRA)

ad-m commented on issue #5008: [AIRFLOW-XXX] Fix typo in README
URL: https://github.com/apache/airflow/pull/5008#issuecomment-478167721
 
 
   @mik-laj, I have heard that you are friendly and helpful. Could you give me 
a hint what I can still do to achieve perfection of this open-source project?


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[jira] [Commented] (AIRFLOW-3807) Fix graph view not highlighting correct tasks when hovering over or clicking on task state



[ 
https://issues.apache.org/jira/browse/AIRFLOW-3807?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16805501#comment-16805501
 ] 

ASF subversion and git services commented on AIRFLOW-3807:
--

Commit 0901f337a00356ac348246d21e1c9f4e9c8fbe58 in airflow's branch 
refs/heads/v1-10-test from Felix
[ https://gitbox.apache.org/repos/asf?p=airflow.git;h=0901f33 ]

[AIRFLOW-3807] Fix Graph View Highlighting of Tasks (#4653)

- fixes up_for_retry and up_for_reschedule tasks when hovering over 
retry/rescheduled task state
- adds missing task states to stateFocusMap that will be used for highlighting 
tasks when clicking on task state
- removed invalid attributes for some tags
- reformatted accordingly to rules from .editorconfig

[AIRFLOW-3807] Fix no_status tasks highlighting in graph view

[AIRFLOW-3807] Change "no status" string to "no_status"

[AIRFLOW-3807] Fix syntax issue in js statement

[AIRFLOW-3807] Correct tree view tasks' status labels

- reformat tree.html file
- remove invalid attributes from tree.html tags


> Fix graph view not highlighting correct tasks when hovering over or clicking 
> on task state
> --
>
> Key: AIRFLOW-3807
> URL: https://issues.apache.org/jira/browse/AIRFLOW-3807
> Project: Apache Airflow
>  Issue Type: Bug
>  Components: ui
>Affects Versions: 1.10.2
>Reporter: Felix Uellendall
>Assignee: Felix Uellendall
>Priority: Major
> Fix For: 1.10.3
>
>
> There are actually three issues that this ticket is going to address:
> * tasks with state *up_for_retry* are not highlighted when hovering over task 
> state *retry*
> * tasks with state *up_for_reschedule* are not highlighted when hovering over 
> task state *rescheduled*
> * tasks with state *up_for_reschedule*, *up_for_retry* and *skipped* will not 
> be highlighted when clicking on task state
> *EDIT:*
> * tasks with state *null* are not highlighted when hovering over task state 
> *no status* - See: https://issues.apache.org/jira/browse/AIRFLOW-3748
> * tasks with state *null* will not be highlighted when clicking on task state
> *EDIT:*
> Updates Tree View task status labels to be the same as in the Graph View



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] [airflow] ad-m opened a new pull request #5008: [AIRFLOW-XXX] Fix typo in README

2019-03-29 Thread ASF subversion and git services (JIRA)

ad-m opened a new pull request #5008: [AIRFLOW-XXX] Fix typo in README
URL: https://github.com/apache/airflow/pull/5008
 
 
   Make sure you have checked _all_ steps below.
   
   ### Jira
   
   - [x] My PR addresses the following [Airflow 
Jira](https://issues.apache.org/jira/browse/AIRFLOW/) issues and references 
them in the PR title. For example, "\[AIRFLOW-XXX\] My Airflow PR"
 - https://issues.apache.org/jira/browse/AIRFLOW-XXX
 - In case you are fixing a typo in the documentation you can prepend your 
commit with \[AIRFLOW-XXX\], code changes always need a Jira issue.
 - In case you are proposing a fundamental code change, you need to create 
an Airflow Improvement Proposal 
([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvements+Proposals)).
 - In case you are adding a dependency, check if the license complies with 
the [ASF 3rd Party License 
Policy](https://www.apache.org/legal/resolved.html#category-x).
   
   ### Description
   
   - [x] Here are some details about my PR, including screenshots of any UI 
changes:
   
   ### Tests
   
   - [x] My PR adds the following unit tests __OR__ does not need testing for 
this extremely good reason:
   
 - This is a minor change in documentation.
   
   ### Commits
   
   - [ ] My commits all reference Jira issues in their subject lines, and I 
have squashed multiple commits if they address the same issue. In addition, my 
commits follow the guidelines from "[How to write a good git commit 
message](http://chris.beams.io/posts/git-commit/)":
 1. Subject is separated from body by a blank line
 1. Subject is limited to 50 characters (not including Jira issue reference)
 1. Subject does not end with a period
 1. Subject uses the imperative mood ("add", not "adding")
 1. Body wraps at 72 characters
 1. Body explains "what" and "why", not "how"
   
   ### Documentation
   
   - [X] In case of new functionality, my PR adds documentation that describes 
how to use it.
 - All the public functions and the classes in the PR contain docstrings 
that explain what it does
 - If you implement backwards incompatible changes, please leave a note in 
the [Updating.md](https://github.com/apache/airflow/blob/master/UPDATING.md) so 
we can assign it to a appropriate release
   
   ### Code Quality
   
   - [X] Passes `flake8`
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[jira] [Commented] (AIRFLOW-3807) Fix graph view not highlighting correct tasks when hovering over or clicking on task state



[ 
https://issues.apache.org/jira/browse/AIRFLOW-3807?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16805498#comment-16805498
 ] 

ASF subversion and git services commented on AIRFLOW-3807:
--

Commit 0901f337a00356ac348246d21e1c9f4e9c8fbe58 in airflow's branch 
refs/heads/v1-10-test from Felix
[ https://gitbox.apache.org/repos/asf?p=airflow.git;h=0901f33 ]

[AIRFLOW-3807] Fix Graph View Highlighting of Tasks (#4653)

- fixes up_for_retry and up_for_reschedule tasks when hovering over 
retry/rescheduled task state
- adds missing task states to stateFocusMap that will be used for highlighting 
tasks when clicking on task state
- removed invalid attributes for some tags
- reformatted accordingly to rules from .editorconfig

[AIRFLOW-3807] Fix no_status tasks highlighting in graph view

[AIRFLOW-3807] Change "no status" string to "no_status"

[AIRFLOW-3807] Fix syntax issue in js statement

[AIRFLOW-3807] Correct tree view tasks' status labels

- reformat tree.html file
- remove invalid attributes from tree.html tags


> Fix graph view not highlighting correct tasks when hovering over or clicking 
> on task state
> --
>
> Key: AIRFLOW-3807
> URL: https://issues.apache.org/jira/browse/AIRFLOW-3807
> Project: Apache Airflow
>  Issue Type: Bug
>  Components: ui
>Affects Versions: 1.10.2
>Reporter: Felix Uellendall
>Assignee: Felix Uellendall
>Priority: Major
> Fix For: 1.10.3
>
>
> There are actually three issues that this ticket is going to address:
> * tasks with state *up_for_retry* are not highlighted when hovering over task 
> state *retry*
> * tasks with state *up_for_reschedule* are not highlighted when hovering over 
> task state *rescheduled*
> * tasks with state *up_for_reschedule*, *up_for_retry* and *skipped* will not 
> be highlighted when clicking on task state
> *EDIT:*
> * tasks with state *null* are not highlighted when hovering over task state 
> *no status* - See: https://issues.apache.org/jira/browse/AIRFLOW-3748
> * tasks with state *null* will not be highlighted when clicking on task state
> *EDIT:*
> Updates Tree View task status labels to be the same as in the Graph View



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (AIRFLOW-3623) Support download log file from UI

2019-03-29 Thread ASF subversion and git services (JIRA)



[ 
https://issues.apache.org/jira/browse/AIRFLOW-3623?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16805495#comment-16805495
 ] 

ASF subversion and git services commented on AIRFLOW-3623:
--

Commit 6f360ab797bfebca8a0b4f7de17b15bcbe8bb515 in airflow's branch 
refs/heads/v1-10-test from Ping Zhang
[ https://gitbox.apache.org/repos/asf?p=airflow.git;h=6f360ab ]

[AIRFLOW-3623] Support download logs by attempts from UI (#4425)


> Support download log file from UI
> -
>
> Key: AIRFLOW-3623
> URL: https://issues.apache.org/jira/browse/AIRFLOW-3623
> Project: Apache Airflow
>  Issue Type: Improvement
>  Components: ui
>Reporter: Ping Zhang
>Assignee: Ash Berlin-Taylor
>Priority: Major
>  Labels: newbie
> Fix For: 2.0.0, 1.10.3
>
>
> for some large log files, it is not a good idea to fetch and render in the 
> UI. Adding the ability to let users to download by try_number in the dag 
> modal.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (AIRFLOW-3807) Fix graph view not highlighting correct tasks when hovering over or clicking on task state

2019-03-29 Thread ASF subversion and git services (JIRA)



[ 
https://issues.apache.org/jira/browse/AIRFLOW-3807?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16805497#comment-16805497
 ] 

ASF subversion and git services commented on AIRFLOW-3807:
--

Commit 0901f337a00356ac348246d21e1c9f4e9c8fbe58 in airflow's branch 
refs/heads/v1-10-test from Felix
[ https://gitbox.apache.org/repos/asf?p=airflow.git;h=0901f33 ]

[AIRFLOW-3807] Fix Graph View Highlighting of Tasks (#4653)

- fixes up_for_retry and up_for_reschedule tasks when hovering over 
retry/rescheduled task state
- adds missing task states to stateFocusMap that will be used for highlighting 
tasks when clicking on task state
- removed invalid attributes for some tags
- reformatted accordingly to rules from .editorconfig

[AIRFLOW-3807] Fix no_status tasks highlighting in graph view

[AIRFLOW-3807] Change "no status" string to "no_status"

[AIRFLOW-3807] Fix syntax issue in js statement

[AIRFLOW-3807] Correct tree view tasks' status labels

- reformat tree.html file
- remove invalid attributes from tree.html tags


> Fix graph view not highlighting correct tasks when hovering over or clicking 
> on task state
> --
>
> Key: AIRFLOW-3807
> URL: https://issues.apache.org/jira/browse/AIRFLOW-3807
> Project: Apache Airflow
>  Issue Type: Bug
>  Components: ui
>Affects Versions: 1.10.2
>Reporter: Felix Uellendall
>Assignee: Felix Uellendall
>Priority: Major
> Fix For: 1.10.3
>
>
> There are actually three issues that this ticket is going to address:
> * tasks with state *up_for_retry* are not highlighted when hovering over task 
> state *retry*
> * tasks with state *up_for_reschedule* are not highlighted when hovering over 
> task state *rescheduled*
> * tasks with state *up_for_reschedule*, *up_for_retry* and *skipped* will not 
> be highlighted when clicking on task state
> *EDIT:*
> * tasks with state *null* are not highlighted when hovering over task state 
> *no status* - See: https://issues.apache.org/jira/browse/AIRFLOW-3748
> * tasks with state *null* will not be highlighted when clicking on task state
> *EDIT:*
> Updates Tree View task status labels to be the same as in the Graph View



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (AIRFLOW-3541) Include --useAvroLogicalTypes setting for the Airflow operator GoogleCloudStorageToBigQueryOperator

2019-03-29 Thread ASF subversion and git services (JIRA)



[ 
https://issues.apache.org/jira/browse/AIRFLOW-3541?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16805502#comment-16805502
 ] 

ASF subversion and git services commented on AIRFLOW-3541:
--

Commit 2f6bcc9f350174e295426fd41f14a8e72bc1b75b in airflow's branch 
refs/heads/v1-10-test from BrechtDeVlieger
[ https://gitbox.apache.org/repos/asf?p=airflow.git;h=2f6bcc9 ]

[AIRFLOW-3541] Add Avro logical type conversion to bigquery hook (#4553)



> Include --useAvroLogicalTypes setting for the Airflow operator 
> GoogleCloudStorageToBigQueryOperator
> ---
>
> Key: AIRFLOW-3541
> URL: https://issues.apache.org/jira/browse/AIRFLOW-3541
> Project: Apache Airflow
>  Issue Type: New Feature
>  Components: operators
> Environment: Composer
>Reporter: Monica Parpi
>Priority: Major
>  Labels: pull-request-available
> Fix For: 1.10.3
>
>
> Getting an error [1] when loading data from GCS into BigQuery using the 
> GoogleCloudStorageToBigQueryOperator for a Avro schema containing this column:
> {
> "name": "date_requested",
> "type": "int",
> "logicalType": "date",
> }
> This column contains the number of days since the epoch as indicated here [2].
> It is possible to load this data into a DATE column by using the 
> "--useAvroLogicalTypes" parameter when initiating a load using the CLI [3]
> It would be very useful if this setting could be enabled via the Airflow 
> operator that performs the equivalent data load [4].
> Thank you!
> [ [1] 
> https://issuetracker.google.com/35905894|https://issuetracker.google.com/35905894]
>  
> [[2] 
> http://avro.apache.org/docs/current/spec.html#Date|http://avro.apache.org/docs/current/spec.html#Date]
> [3] 
> [https://cloud.google.com/bigquery/docs/reference/rest/v2/jobs#configuration.load.useAvroLogicalTypes]
> [4] 
> [https://airflow.apache.org/_modules/airflow/contrib/operators/gcs_to_bq.html#GoogleCloudStorageToBigQueryOperator]
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (AIRFLOW-3636) Fix a test introduced in PR #4425

2019-03-29 Thread ASF subversion and git services (JIRA)



[ 
https://issues.apache.org/jira/browse/AIRFLOW-3636?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16805496#comment-16805496
 ] 

ASF subversion and git services commented on AIRFLOW-3636:
--

Commit 6adad35f9a2662a1da6c606b2313a87a5aa24acc in airflow's branch 
refs/heads/v1-10-test from Ping Zhang
[ https://gitbox.apache.org/repos/asf?p=airflow.git;h=6adad35 ]

[AIRFLOW-3636] Fix a test introduced in #4425 (#4446)



> Fix a test introduced in PR #4425
> -
>
> Key: AIRFLOW-3636
> URL: https://issues.apache.org/jira/browse/AIRFLOW-3636
> Project: Apache Airflow
>  Issue Type: Bug
>Reporter: Ping Zhang
>Assignee: Ping Zhang
>Priority: Major
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (AIRFLOW-3807) Fix graph view not highlighting correct tasks when hovering over or clicking on task state

2019-03-29 Thread ASF subversion and git services (JIRA)



[ 
https://issues.apache.org/jira/browse/AIRFLOW-3807?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16805499#comment-16805499
 ] 

ASF subversion and git services commented on AIRFLOW-3807:
--

Commit 0901f337a00356ac348246d21e1c9f4e9c8fbe58 in airflow's branch 
refs/heads/v1-10-test from Felix
[ https://gitbox.apache.org/repos/asf?p=airflow.git;h=0901f33 ]

[AIRFLOW-3807] Fix Graph View Highlighting of Tasks (#4653)

- fixes up_for_retry and up_for_reschedule tasks when hovering over 
retry/rescheduled task state
- adds missing task states to stateFocusMap that will be used for highlighting 
tasks when clicking on task state
- removed invalid attributes for some tags
- reformatted accordingly to rules from .editorconfig

[AIRFLOW-3807] Fix no_status tasks highlighting in graph view

[AIRFLOW-3807] Change "no status" string to "no_status"

[AIRFLOW-3807] Fix syntax issue in js statement

[AIRFLOW-3807] Correct tree view tasks' status labels

- reformat tree.html file
- remove invalid attributes from tree.html tags


> Fix graph view not highlighting correct tasks when hovering over or clicking 
> on task state
> --
>
> Key: AIRFLOW-3807
> URL: https://issues.apache.org/jira/browse/AIRFLOW-3807
> Project: Apache Airflow
>  Issue Type: Bug
>  Components: ui
>Affects Versions: 1.10.2
>Reporter: Felix Uellendall
>Assignee: Felix Uellendall
>Priority: Major
> Fix For: 1.10.3
>
>
> There are actually three issues that this ticket is going to address:
> * tasks with state *up_for_retry* are not highlighted when hovering over task 
> state *retry*
> * tasks with state *up_for_reschedule* are not highlighted when hovering over 
> task state *rescheduled*
> * tasks with state *up_for_reschedule*, *up_for_retry* and *skipped* will not 
> be highlighted when clicking on task state
> *EDIT:*
> * tasks with state *null* are not highlighted when hovering over task state 
> *no status* - See: https://issues.apache.org/jira/browse/AIRFLOW-3748
> * tasks with state *null* will not be highlighted when clicking on task state
> *EDIT:*
> Updates Tree View task status labels to be the same as in the Graph View



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (AIRFLOW-3623) Support download log file from UI

2019-03-29 Thread ASF subversion and git services (JIRA)



[ 
https://issues.apache.org/jira/browse/AIRFLOW-3623?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16805503#comment-16805503
 ] 

ASF subversion and git services commented on AIRFLOW-3623:
--

Commit 1ccd0a4bb423adfdb1b48edb429bb918264bf04e in airflow's branch 
refs/heads/v1-10-test from Fokko Driesprong
[ https://gitbox.apache.org/repos/asf?p=airflow.git;h=1ccd0a4 ]

[AIRFLOW-3623] Fix bugs in Download task logs


> Support download log file from UI
> -
>
> Key: AIRFLOW-3623
> URL: https://issues.apache.org/jira/browse/AIRFLOW-3623
> Project: Apache Airflow
>  Issue Type: Improvement
>  Components: ui
>Reporter: Ping Zhang
>Assignee: Ash Berlin-Taylor
>Priority: Major
>  Labels: newbie
> Fix For: 2.0.0, 1.10.3
>
>
> for some large log files, it is not a good idea to fetch and render in the 
> UI. Adding the ability to let users to download by try_number in the dag 
> modal.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (AIRFLOW-4177) Check types in tests



[ 
https://issues.apache.org/jira/browse/AIRFLOW-4177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16805483#comment-16805483
 ] 

ASF GitHub Bot commented on AIRFLOW-4177:
-

Fokko commented on pull request #4994: [AIRFLOW-4177] Check types in tests.
URL: https://github.com/apache/airflow/pull/4994
 
 
   
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


> Check types in tests
> 
>
> Key: AIRFLOW-4177
> URL: https://issues.apache.org/jira/browse/AIRFLOW-4177
> Project: Apache Airflow
>  Issue Type: Improvement
>Reporter: Josh Carp
>Priority: Trivial
>
> We're checking types with mypy in the airflow module; let's check types in 
> the tests module as well.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] [airflow] Fokko commented on a change in pull request #5005: [AIRFLOW-3623] Fix bugs in Download task logs

Fokko commented on a change in pull request #5005: [AIRFLOW-3623] Fix bugs in 
Download task logs
URL: https://github.com/apache/airflow/pull/5005#discussion_r270588292
 
 

 ##
 File path: airflow/www/templates/airflow/graph.html
 ##
 @@ -169,9 +169,9 @@
 d3.selectAll("g.node").on("click", function(d){
 task = tasks[d];
 if (task.task_type == "SubDagOperator")
-call_modal(d, execution_date, true);
+call_modal(d, execution_date, task_instances[d].try_number, true);
 else
-call_modal(d, execution_date);
+call_modal(d, execution_date,task_instances[d].try_number);
 
 Review comment:
   Why did the CI not complain 樂 


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] Fokko commented on issue #5005: [AIRFLOW-3623] Fix bugs in Download task logs

2019-03-29 Thread ASF subversion and git services (JIRA)

Fokko commented on issue #5005: [AIRFLOW-3623] Fix bugs in Download task logs
URL: https://github.com/apache/airflow/pull/5005#issuecomment-478165613
 
 
   @ashb pushed it to `v1-10-test` as well :-)


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[jira] [Commented] (AIRFLOW-3623) Support download log file from UI



[ 
https://issues.apache.org/jira/browse/AIRFLOW-3623?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16805489#comment-16805489
 ] 

ASF subversion and git services commented on AIRFLOW-3623:
--

Commit cb6711482e3c92bcae0c3d3fbf08da5a9e6e9466 in airflow's branch 
refs/heads/master from Ash Berlin-Taylor
[ https://gitbox.apache.org/repos/asf?p=airflow.git;h=cb67114 ]

[AIRFLOW-3623] Fix bugs in Download task logs (#5005)

They didn't work in Python 3 (Bytes vs Strings), the graph_view wasn't
specifying the try_number correctly, and the "All" logs had "None" as
the suffix instead of "all".

> Support download log file from UI
> -
>
> Key: AIRFLOW-3623
> URL: https://issues.apache.org/jira/browse/AIRFLOW-3623
> Project: Apache Airflow
>  Issue Type: Improvement
>  Components: ui
>Reporter: Ping Zhang
>Assignee: Ping Zhang
>Priority: Major
>  Labels: newbie
> Fix For: 1.10.3
>
>
> for some large log files, it is not a good idea to fetch and render in the 
> UI. Adding the ability to let users to download by try_number in the dag 
> modal.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] [airflow] Fokko commented on a change in pull request #5005: [AIRFLOW-3623] Fix bugs in Download task logs

Fokko commented on a change in pull request #5005: [AIRFLOW-3623] Fix bugs in 
Download task logs
URL: https://github.com/apache/airflow/pull/5005#discussion_r270587394
 
 

 ##
 File path: airflow/www/templates/airflow/graph.html
 ##
 @@ -169,9 +169,9 @@
 d3.selectAll("g.node").on("click", function(d){
 task = tasks[d];
 if (task.task_type == "SubDagOperator")
-call_modal(d, execution_date, true);
+call_modal(d, execution_date, task_instances[d].try_number, true);
 else
-call_modal(d, execution_date);
+call_modal(d, execution_date,task_instances[d].try_number);
 
 Review comment:
   Missing space here, auch!


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[jira] [Closed] (AIRFLOW-3623) Support download log file from UI



 [ 
https://issues.apache.org/jira/browse/AIRFLOW-3623?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Fokko Driesprong closed AIRFLOW-3623.
-
Assignee: Ash Berlin-Taylor  (was: Ping Zhang)

> Support download log file from UI
> -
>
> Key: AIRFLOW-3623
> URL: https://issues.apache.org/jira/browse/AIRFLOW-3623
> Project: Apache Airflow
>  Issue Type: Improvement
>  Components: ui
>Reporter: Ping Zhang
>Assignee: Ash Berlin-Taylor
>Priority: Major
>  Labels: newbie
> Fix For: 2.0.0, 1.10.3
>
>
> for some large log files, it is not a good idea to fetch and render in the 
> UI. Adding the ability to let users to download by try_number in the dag 
> modal.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (AIRFLOW-3623) Support download log file from UI



[ 
https://issues.apache.org/jira/browse/AIRFLOW-3623?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16805488#comment-16805488
 ] 

ASF GitHub Bot commented on AIRFLOW-3623:
-

Fokko commented on pull request #5005: [AIRFLOW-3623] Fix bugs in Download task 
logs
URL: https://github.com/apache/airflow/pull/5005
 
 
   
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


> Support download log file from UI
> -
>
> Key: AIRFLOW-3623
> URL: https://issues.apache.org/jira/browse/AIRFLOW-3623
> Project: Apache Airflow
>  Issue Type: Improvement
>  Components: ui
>Reporter: Ping Zhang
>Assignee: Ping Zhang
>Priority: Major
>  Labels: newbie
> Fix For: 1.10.3
>
>
> for some large log files, it is not a good idea to fetch and render in the 
> UI. Adding the ability to let users to download by try_number in the dag 
> modal.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] [airflow] Fokko merged pull request #5005: [AIRFLOW-3623] Fix bugs in Download task logs

Fokko merged pull request #5005: [AIRFLOW-3623] Fix bugs in Download task logs
URL: https://github.com/apache/airflow/pull/5005
 
 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[jira] [Updated] (AIRFLOW-3623) Support download log file from UI



 [ 
https://issues.apache.org/jira/browse/AIRFLOW-3623?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Fokko Driesprong updated AIRFLOW-3623:
--
Fix Version/s: 2.0.0

> Support download log file from UI
> -
>
> Key: AIRFLOW-3623
> URL: https://issues.apache.org/jira/browse/AIRFLOW-3623
> Project: Apache Airflow
>  Issue Type: Improvement
>  Components: ui
>Reporter: Ping Zhang
>Assignee: Ping Zhang
>Priority: Major
>  Labels: newbie
> Fix For: 2.0.0, 1.10.3
>
>
> for some large log files, it is not a good idea to fetch and render in the 
> UI. Adding the ability to let users to download by try_number in the dag 
> modal.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (AIRFLOW-4177) Check types in tests



 [ 
https://issues.apache.org/jira/browse/AIRFLOW-4177?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Fokko Driesprong updated AIRFLOW-4177:
--
Fix Version/s: 2.0.0

> Check types in tests
> 
>
> Key: AIRFLOW-4177
> URL: https://issues.apache.org/jira/browse/AIRFLOW-4177
> Project: Apache Airflow
>  Issue Type: Improvement
>Reporter: Josh Carp
>Assignee: Josh Carp
>Priority: Trivial
> Fix For: 2.0.0
>
>
> We're checking types with mypy in the airflow module; let's check types in 
> the tests module as well.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] [airflow] Fokko commented on issue #4994: [AIRFLOW-4177] Check types in tests.

Fokko commented on issue #4994: [AIRFLOW-4177] Check types in tests.
URL: https://github.com/apache/airflow/pull/4994#issuecomment-478163607
 
 
   Thanks @jmcarp Appreciate it.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[jira] [Assigned] (AIRFLOW-4177) Check types in tests

2019-03-29 Thread ASF subversion and git services (JIRA)



 [ 
https://issues.apache.org/jira/browse/AIRFLOW-4177?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Fokko Driesprong reassigned AIRFLOW-4177:
-

Assignee: Josh Carp

> Check types in tests
> 
>
> Key: AIRFLOW-4177
> URL: https://issues.apache.org/jira/browse/AIRFLOW-4177
> Project: Apache Airflow
>  Issue Type: Improvement
>Reporter: Josh Carp
>Assignee: Josh Carp
>Priority: Trivial
>
> We're checking types with mypy in the airflow module; let's check types in 
> the tests module as well.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (AIRFLOW-4177) Check types in tests



[ 
https://issues.apache.org/jira/browse/AIRFLOW-4177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16805484#comment-16805484
 ] 

ASF subversion and git services commented on AIRFLOW-4177:
--

Commit 0e30793849c7af288f491f8db4cded73c6fe265f in airflow's branch 
refs/heads/master from Joshua Carp
[ https://gitbox.apache.org/repos/asf?p=airflow.git;h=0e30793 ]

[AIRFLOW-4177] Check types in tests (#4994)



> Check types in tests
> 
>
> Key: AIRFLOW-4177
> URL: https://issues.apache.org/jira/browse/AIRFLOW-4177
> Project: Apache Airflow
>  Issue Type: Improvement
>Reporter: Josh Carp
>Priority: Trivial
>
> We're checking types with mypy in the airflow module; let's check types in 
> the tests module as well.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] [airflow] Fokko merged pull request #4994: [AIRFLOW-4177] Check types in tests.

Fokko merged pull request #4994: [AIRFLOW-4177] Check types in tests.
URL: https://github.com/apache/airflow/pull/4994
 
 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] codecov-io commented on issue #5007: [AIRFLOW-XXX] Remove duplicate information

codecov-io commented on issue #5007: [AIRFLOW-XXX] Remove duplicate information
URL: https://github.com/apache/airflow/pull/5007#issuecomment-478162798
 
 
   # [Codecov](https://codecov.io/gh/apache/airflow/pull/5007?src=pr=h1) 
Report
   > Merging 
[#5007](https://codecov.io/gh/apache/airflow/pull/5007?src=pr=desc) into 
[master](https://codecov.io/gh/apache/airflow/commit/9d8719e17d9b506b01ac2de59e5da26089a93555?src=pr=desc)
 will **not change** coverage.
   > The diff coverage is `n/a`.
   
   [![Impacted file tree 
graph](https://codecov.io/gh/apache/airflow/pull/5007/graphs/tree.svg?width=650=WdLKlKHOAU=150=pr)](https://codecov.io/gh/apache/airflow/pull/5007?src=pr=tree)
   
   ```diff
   @@   Coverage Diff   @@
   ##   master#5007   +/-   ##
   ===
 Coverage   75.76%   75.76%   
   ===
 Files 461  461   
 Lines   2996129961   
   ===
 Hits2270122701   
 Misses   7260 7260
   ```
   
   
   
   --
   
   [Continue to review full report at 
Codecov](https://codecov.io/gh/apache/airflow/pull/5007?src=pr=continue).
   > **Legend** - [Click here to learn 
more](https://docs.codecov.io/docs/codecov-delta)
   > `Δ = absolute  (impact)`, `ø = not affected`, `? = missing data`
   > Powered by 
[Codecov](https://codecov.io/gh/apache/airflow/pull/5007?src=pr=footer). 
Last update 
[9d8719e...db9f7db](https://codecov.io/gh/apache/airflow/pull/5007?src=pr=lastupdated).
 Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] gglanzani commented on a change in pull request #4708: [AIRFLOW-3888] HA for metastore connection

2019-03-29 Thread ASF subversion and git services (JIRA)

gglanzani commented on a change in pull request #4708: [AIRFLOW-3888] HA for 
metastore connection
URL: https://github.com/apache/airflow/pull/4708#discussion_r270582868
 
 

 ##
 File path: tests/hooks/test_hive_hook.py
 ##
 @@ -282,6 +283,13 @@ def test_get_max_partition_from_valid_part_specs(self):
 def test_get_metastore_client(self):
 self.assertIsInstance(self.hook.get_metastore_client(), HMSClient)
 
+@mock.patch("airflow.hooks.hive_hooks.HiveMetastoreHook.get_connection",
+**{'return_value': [Connection(host="localhost", port=9802)]})
 
 Review comment:
   > Your scientists were so preoccupied with whether or not they could, they 
didn’t stop to think if they should.
   
   The suggested syntax does not work, indeed, when you have invalid 
identifiers as keyword arguments. `return_value` is not one of them


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[jira] [Commented] (AIRFLOW-3540) Presence of ~/airflow/airflow.cfg shouldn't override config file settings



[ 
https://issues.apache.org/jira/browse/AIRFLOW-3540?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16805458#comment-16805458
 ] 

ASF subversion and git services commented on AIRFLOW-3540:
--

Commit 540612253d2f743ebc1def91fda8c9b3e36ff8bd in airflow's branch 
refs/heads/v1-10-stable from Ash Berlin-Taylor
[ https://gitbox.apache.org/repos/asf?p=airflow.git;h=5406122 ]

[AIRFLOW-3540] Warn if old airflow.cfg file is found (#5006)

We used to read this config file, but we now don't. To make it easier on
users (particularly of puckel's docker container) lets at least issue a
warning

> Presence of ~/airflow/airflow.cfg shouldn't override config file settings
> -
>
> Key: AIRFLOW-3540
> URL: https://issues.apache.org/jira/browse/AIRFLOW-3540
> Project: Apache Airflow
>  Issue Type: Improvement
>Reporter: Josh Carp
>Assignee: Josh Carp
>Priority: Trivial
> Fix For: 1.10.3
>
>
> If `~/airflow/airflow.cfg` exists, airflow uses that file instead of 
> `$AIRFLOW_HOME/airflow.cfg`. This behavior is confusing. If `$AIRFLOW_HOME` 
> is set and `$AIRFLOW_CONFIG` is not, we should only check 
> `$AIRFLOW_HOME/airflow.cfg`.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] [airflow] feng-tao merged pull request #5006: [AIRFLOW-3540] Warn if old airflow.cfg file is found

feng-tao merged pull request #5006: [AIRFLOW-3540] Warn if old airflow.cfg file 
is found
URL: https://github.com/apache/airflow/pull/5006
 
 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] jmcarp commented on issue #4994: [AIRFLOW-4177] Check types in tests.

jmcarp commented on issue #4994: [AIRFLOW-4177] Check types in tests.
URL: https://github.com/apache/airflow/pull/4994#issuecomment-478142304
 
 
   Rebased; tests passing.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] galak75 edited a comment on issue #4743: [AIRFLOW-3871] render Operators template fields recursively

galak75 edited a comment on issue #4743: [AIRFLOW-3871] render Operators 
template fields recursively
URL: https://github.com/apache/airflow/pull/4743#issuecomment-473318089
 
 
   > I'm leaning towards a -1 on this - it seems that there are going to be 
many more case where this won't Just Work leading to confusion on behalf of 
users.
   > 
   > Can you give a concrete example of when you'd need this? In examples 
you've shown above why couldn't you create the `MyAwesomeDataFileTransformer` 
_inside_ the callable instead?
   
   Sure @ashb. Here is the real use case:
   
   We have a lot of different data sources to be imported, transformed and then 
exported to several destinations. So we tried to decouple our business logic 
(importing data, transforming it, and then exporting it) from airflow callable 
functions:
   
   First, we defined a template method to be used as a `PythonOperator` 
callable:
   ```python
   def process_data(dataImporter, dataTransformer, dataExporter):
   data = dataImporter.import_data()
   data = dataTransformer.transform_data(data)
   dataExporter.export_data(data)
   ```
   
   Then we just have to "inject" the proper `data_importer`, `data_transformer` 
and `data_exporter` in our DAG task (without having to write a new callable 
function when the data source is changing or the transformation needs some more 
parameters...)
   let's look at some simple examples:
   ```python
   task1 = PythonOperator(
   task_id='task_1',
   python_callable=process_data,
   op_args=[
   SomeFileDataImporter('/tmp/{{ ds }}/input_data'),
   SomeDataTransformer(some_value, 'path/to/other/file/{{ ds }}/file'),
   SomeFileDataExporter('/data/output/{{ dag.dag_id }}/output_file')
   ],
   dag=dag
   )
   
   task2 = PythonOperator(
   task_id='task_2',
   python_callable=process_data,
   op_args=[
   SomeJoiningCsvFilesDataImporter(
   '/tmp/{{ ds }}/input_data_1',
   '/tmp/{{ ds }}/input_data_2',
   join_on='id'
   ),
   SomeOtherDataTransformer(some_value),
   SomeOtherDataExporter(execution_ts={{ ts }})
   ],
   dag=dag
   )
   ```
   
   This approach has several benefits:
   
   - separation of concerns: the callable function does not need to know how to 
import data (it could come from one file, several files, a database, an API, 
etc...), how to transform it and how to export it.
   - respect single responsibility principle: we separate responsibilities in 
simple classes: a pickle file importer, a json file importer, a sql importer, 
etc... 
   - use DAG declaration as an IOC tool: we just inject proper implementations, 
depending on each use case. When another transformation is required, just 
inject another data_transformer implementation without having to write another 
callable function. We do not want the callable function being responsible to 
instantiate the right implementation.
   
   To make this approach work properly, we would like operators to be able to 
render nested template fields.
   
   The `template_fields` solution may have some limitation (as [pointed 
out](https://github.com/apache/airflow/pull/4743#issuecomment-472735428) by 
@bjoernpollex-sc), but it would answer some needed use cases.
   
   I also suggested [adding another `render_template` hook 
function](https://github.com/apache/airflow/pull/4743#issuecomment-472996201) 
that people could implement to solve custom template rendering issues. Would it 
answer your concerns about this submitted pull request?
   
   Thank you for taking time to read this
   cheers


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] mik-laj opened a new pull request #5007: [AIRFLOW-XXX] Remove duplicate information

mik-laj opened a new pull request #5007: [AIRFLOW-XXX] Remove duplicate 
information
URL: https://github.com/apache/airflow/pull/5007
 
 
   Make sure you have checked _all_ steps below.
   
   ### Jira
   
   - [ ] My PR addresses the following [Airflow 
Jira](https://issues.apache.org/jira/browse/AIRFLOW/) issues and references 
them in the PR title. For example, "\[AIRFLOW-XXX\] My Airflow PR"
 - https://issues.apache.org/jira/browse/AIRFLOW-XXX
 - In case you are fixing a typo in the documentation you can prepend your 
commit with \[AIRFLOW-XXX\], code changes always need a Jira issue.
 - In case you are proposing a fundamental code change, you need to create 
an Airflow Improvement Proposal 
([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvements+Proposals)).
 - In case you are adding a dependency, check if the license complies with 
the [ASF 3rd Party License 
Policy](https://www.apache.org/legal/resolved.html#category-x).
   
   ### Description
   
   Preview: http://female-circle.surge.sh/integration.html
   
   The integration.rst file contains a lot of repeated information from the API 
reference documentation. Now the reference documentation is clearer and it 
makes no sense to repeat the information twice.
   
   ### Tests
   
   - [ ] My PR adds the following unit tests __OR__ does not need testing for 
this extremely good reason:
   
   ### Commits
   
   - [ ] My commits all reference Jira issues in their subject lines, and I 
have squashed multiple commits if they address the same issue. In addition, my 
commits follow the guidelines from "[How to write a good git commit 
message](http://chris.beams.io/posts/git-commit/)":
 1. Subject is separated from body by a blank line
 1. Subject is limited to 50 characters (not including Jira issue reference)
 1. Subject does not end with a period
 1. Subject uses the imperative mood ("add", not "adding")
 1. Body wraps at 72 characters
 1. Body explains "what" and "why", not "how"
   
   ### Documentation
   
   - [ ] In case of new functionality, my PR adds documentation that describes 
how to use it.
 - All the public functions and the classes in the PR contain docstrings 
that explain what it does
 - If you implement backwards incompatible changes, please leave a note in 
the [Updating.md](https://github.com/apache/airflow/blob/master/UPDATING.md) so 
we can assign it to a appropriate release
   
   ### Code Quality
   
   - [ ] Passes `flake8`
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] codecov-io commented on issue #5006: [AIRFLOW-3540] Warn if old airflow.cfg file is found

codecov-io commented on issue #5006: [AIRFLOW-3540] Warn if old airflow.cfg 
file is found
URL: https://github.com/apache/airflow/pull/5006#issuecomment-478130429
 
 
   # [Codecov](https://codecov.io/gh/apache/airflow/pull/5006?src=pr=h1) 
Report
   > Merging 
[#5006](https://codecov.io/gh/apache/airflow/pull/5006?src=pr=desc) into 
[v1-10-stable](https://codecov.io/gh/apache/airflow/commit/691d06927ba1000592c673dfba2e2b417f3101de?src=pr=desc)
 will **decrease** coverage by `<.01%`.
   > The diff coverage is `66.66%`.
   
   [![Impacted file tree 
graph](https://codecov.io/gh/apache/airflow/pull/5006/graphs/tree.svg?width=650=WdLKlKHOAU=150=pr)](https://codecov.io/gh/apache/airflow/pull/5006?src=pr=tree)
   
   ```diff
   @@   Coverage Diff   @@
   ##   v1-10-stable   #5006  +/-   ##
   ===
   - Coverage78% 78%   -0.01% 
   ===
 Files   226 226  
 Lines 17626   17629   +3 
   ===
   + Hits  13749   13751   +2 
   - Misses 38773878   +1
   ```
   
   
   | [Impacted 
Files](https://codecov.io/gh/apache/airflow/pull/5006?src=pr=tree) | 
Coverage Δ | |
   |---|---|---|
   | 
[airflow/configuration.py](https://codecov.io/gh/apache/airflow/pull/5006/diff?src=pr=tree#diff-YWlyZmxvdy9jb25maWd1cmF0aW9uLnB5)
 | `89% <66.66%> (-0.23%)` | :arrow_down: |
   
   --
   
   [Continue to review full report at 
Codecov](https://codecov.io/gh/apache/airflow/pull/5006?src=pr=continue).
   > **Legend** - [Click here to learn 
more](https://docs.codecov.io/docs/codecov-delta)
   > `Δ = absolute  (impact)`, `ø = not affected`, `? = missing data`
   > Powered by 
[Codecov](https://codecov.io/gh/apache/airflow/pull/5006?src=pr=footer). 
Last update 
[691d069...e807498](https://codecov.io/gh/apache/airflow/pull/5006?src=pr=lastupdated).
 Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] codecov-io commented on issue #5005: [AIRFLOW-3623] Fix bugs in Download task logs

codecov-io commented on issue #5005: [AIRFLOW-3623] Fix bugs in Download task 
logs
URL: https://github.com/apache/airflow/pull/5005#issuecomment-478128169
 
 
   # [Codecov](https://codecov.io/gh/apache/airflow/pull/5005?src=pr=h1) 
Report
   > Merging 
[#5005](https://codecov.io/gh/apache/airflow/pull/5005?src=pr=desc) into 
[master](https://codecov.io/gh/apache/airflow/commit/9d8719e17d9b506b01ac2de59e5da26089a93555?src=pr=desc)
 will **decrease** coverage by `<.01%`.
   > The diff coverage is `100%`.
   
   [![Impacted file tree 
graph](https://codecov.io/gh/apache/airflow/pull/5005/graphs/tree.svg?width=650=WdLKlKHOAU=150=pr)](https://codecov.io/gh/apache/airflow/pull/5005?src=pr=tree)
   
   ```diff
   @@Coverage Diff @@
   ##   master#5005  +/-   ##
   ==
   - Coverage   75.76%   75.76%   -0.01% 
   ==
 Files 461  461  
 Lines   2996129955   -6 
   ==
   - Hits2270122695   -6 
 Misses   7260 7260
   ```
   
   
   | [Impacted 
Files](https://codecov.io/gh/apache/airflow/pull/5005?src=pr=tree) | 
Coverage Δ | |
   |---|---|---|
   | 
[airflow/www/views.py](https://codecov.io/gh/apache/airflow/pull/5005/diff?src=pr=tree#diff-YWlyZmxvdy93d3cvdmlld3MucHk=)
 | `76.19% <100%> (-0.11%)` | :arrow_down: |
   | 
[airflow/models/\_\_init\_\_.py](https://codecov.io/gh/apache/airflow/pull/5005/diff?src=pr=tree#diff-YWlyZmxvdy9tb2RlbHMvX19pbml0X18ucHk=)
 | `92.96% <0%> (-0.05%)` | :arrow_down: |
   | 
[airflow/contrib/operators/ssh\_operator.py](https://codecov.io/gh/apache/airflow/pull/5005/diff?src=pr=tree#diff-YWlyZmxvdy9jb250cmliL29wZXJhdG9ycy9zc2hfb3BlcmF0b3IucHk=)
 | `83.54% <0%> (+1.26%)` | :arrow_up: |
   
   --
   
   [Continue to review full report at 
Codecov](https://codecov.io/gh/apache/airflow/pull/5005?src=pr=continue).
   > **Legend** - [Click here to learn 
more](https://docs.codecov.io/docs/codecov-delta)
   > `Δ = absolute  (impact)`, `ø = not affected`, `? = missing data`
   > Powered by 
[Codecov](https://codecov.io/gh/apache/airflow/pull/5005?src=pr=footer). 
Last update 
[9d8719e...2999009](https://codecov.io/gh/apache/airflow/pull/5005?src=pr=lastupdated).
 Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] mik-laj commented on a change in pull request #4708: [AIRFLOW-3888] HA for metastore connection

mik-laj commented on a change in pull request #4708: [AIRFLOW-3888] HA for 
metastore connection
URL: https://github.com/apache/airflow/pull/4708#discussion_r270550384
 
 

 ##
 File path: tests/hooks/test_hive_hook.py
 ##
 @@ -282,6 +283,13 @@ def test_get_max_partition_from_valid_part_specs(self):
 def test_get_metastore_client(self):
 self.assertIsInstance(self.hook.get_metastore_client(), HMSClient)
 
+@mock.patch("airflow.hooks.hive_hooks.HiveMetastoreHook.get_connection",
+**{'return_value': [Connection(host="localhost", port=9802)]})
 
 Review comment:
   Preferred syntax does not work properly in all cases supported by mock. I 
often use only one syntax, so that I do not have to think which is right at the 
moment.
   Reference: 
   
https://github.com/apache/airflow/blob/49b98ab2b5dc37c69598d485d62dd39372ab6aa9/tests/contrib/hooks/test_gcp_natural_language_hook.py#L57-L68
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] gglanzani commented on issue #4708: [AIRFLOW-3888] HA for metastore connection

gglanzani commented on issue #4708: [AIRFLOW-3888] HA for metastore connection
URL: https://github.com/apache/airflow/pull/4708#issuecomment-478124168
 
 
   Hi @GuzikJakub, beside a very minor comment, LGTM.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] gglanzani commented on a change in pull request #4708: [AIRFLOW-3888] HA for metastore connection

gglanzani commented on a change in pull request #4708: [AIRFLOW-3888] HA for 
metastore connection
URL: https://github.com/apache/airflow/pull/4708#discussion_r270548914
 
 

 ##
 File path: tests/hooks/test_hive_hook.py
 ##
 @@ -282,6 +283,13 @@ def test_get_max_partition_from_valid_part_specs(self):
 def test_get_metastore_client(self):
 self.assertIsInstance(self.hook.get_metastore_client(), HMSClient)
 
+@mock.patch("airflow.hooks.hive_hooks.HiveMetastoreHook.get_connection",
+**{'return_value': [Connection(host="localhost", port=9802)]})
 
 Review comment:
   Why not `return_value=[Connection(…)]` instead of using the `**` syntax?


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] codecov-io commented on issue #4994: [AIRFLOW-4177] Check types in tests.

codecov-io commented on issue #4994: [AIRFLOW-4177] Check types in tests.
URL: https://github.com/apache/airflow/pull/4994#issuecomment-478123460
 
 
   # [Codecov](https://codecov.io/gh/apache/airflow/pull/4994?src=pr=h1) 
Report
   > Merging 
[#4994](https://codecov.io/gh/apache/airflow/pull/4994?src=pr=desc) into 
[master](https://codecov.io/gh/apache/airflow/commit/9d8719e17d9b506b01ac2de59e5da26089a93555?src=pr=desc)
 will **decrease** coverage by `<.01%`.
   > The diff coverage is `n/a`.
   
   [![Impacted file tree 
graph](https://codecov.io/gh/apache/airflow/pull/4994/graphs/tree.svg?width=650=WdLKlKHOAU=150=pr)](https://codecov.io/gh/apache/airflow/pull/4994?src=pr=tree)
   
   ```diff
   @@Coverage Diff @@
   ##   master#4994  +/-   ##
   ==
   - Coverage   75.76%   75.76%   -0.01% 
   ==
 Files 461  461  
 Lines   2996129961  
   ==
   - Hits2270122700   -1 
   - Misses   7260 7261   +1
   ```
   
   
   | [Impacted 
Files](https://codecov.io/gh/apache/airflow/pull/4994?src=pr=tree) | 
Coverage Δ | |
   |---|---|---|
   | 
[airflow/models/\_\_init\_\_.py](https://codecov.io/gh/apache/airflow/pull/4994/diff?src=pr=tree#diff-YWlyZmxvdy9tb2RlbHMvX19pbml0X18ucHk=)
 | `92.96% <0%> (-0.05%)` | :arrow_down: |
   
   --
   
   [Continue to review full report at 
Codecov](https://codecov.io/gh/apache/airflow/pull/4994?src=pr=continue).
   > **Legend** - [Click here to learn 
more](https://docs.codecov.io/docs/codecov-delta)
   > `Δ = absolute  (impact)`, `ø = not affected`, `? = missing data`
   > Powered by 
[Codecov](https://codecov.io/gh/apache/airflow/pull/4994?src=pr=footer). 
Last update 
[9d8719e...752981e](https://codecov.io/gh/apache/airflow/pull/4994?src=pr=lastupdated).
 Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] codecov-io edited a comment on issue #4994: [AIRFLOW-4177] Check types in tests.

codecov-io edited a comment on issue #4994: [AIRFLOW-4177] Check types in tests.
URL: https://github.com/apache/airflow/pull/4994#issuecomment-478123460
 
 
   # [Codecov](https://codecov.io/gh/apache/airflow/pull/4994?src=pr=h1) 
Report
   > Merging 
[#4994](https://codecov.io/gh/apache/airflow/pull/4994?src=pr=desc) into 
[master](https://codecov.io/gh/apache/airflow/commit/9d8719e17d9b506b01ac2de59e5da26089a93555?src=pr=desc)
 will **decrease** coverage by `<.01%`.
   > The diff coverage is `n/a`.
   
   [![Impacted file tree 
graph](https://codecov.io/gh/apache/airflow/pull/4994/graphs/tree.svg?width=650=WdLKlKHOAU=150=pr)](https://codecov.io/gh/apache/airflow/pull/4994?src=pr=tree)
   
   ```diff
   @@Coverage Diff @@
   ##   master#4994  +/-   ##
   ==
   - Coverage   75.76%   75.76%   -0.01% 
   ==
 Files 461  461  
 Lines   2996129961  
   ==
   - Hits2270122700   -1 
   - Misses   7260 7261   +1
   ```
   
   
   | [Impacted 
Files](https://codecov.io/gh/apache/airflow/pull/4994?src=pr=tree) | 
Coverage Δ | |
   |---|---|---|
   | 
[airflow/models/\_\_init\_\_.py](https://codecov.io/gh/apache/airflow/pull/4994/diff?src=pr=tree#diff-YWlyZmxvdy9tb2RlbHMvX19pbml0X18ucHk=)
 | `92.96% <0%> (-0.05%)` | :arrow_down: |
   
   --
   
   [Continue to review full report at 
Codecov](https://codecov.io/gh/apache/airflow/pull/4994?src=pr=continue).
   > **Legend** - [Click here to learn 
more](https://docs.codecov.io/docs/codecov-delta)
   > `Δ = absolute  (impact)`, `ø = not affected`, `? = missing data`
   > Powered by 
[Codecov](https://codecov.io/gh/apache/airflow/pull/4994?src=pr=footer). 
Last update 
[9d8719e...752981e](https://codecov.io/gh/apache/airflow/pull/4994?src=pr=lastupdated).
 Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] codecov-io edited a comment on issue #4743: [AIRFLOW-3871] render Operators template fields recursively

codecov-io edited a comment on issue #4743: [AIRFLOW-3871] render Operators 
template fields recursively
URL: https://github.com/apache/airflow/pull/4743#issuecomment-466461316
 
 
   # [Codecov](https://codecov.io/gh/apache/airflow/pull/4743?src=pr=h1) 
Report
   > Merging 
[#4743](https://codecov.io/gh/apache/airflow/pull/4743?src=pr=desc) into 
[master](https://codecov.io/gh/apache/airflow/commit/9d8719e17d9b506b01ac2de59e5da26089a93555?src=pr=desc)
 will **increase** coverage by `<.01%`.
   > The diff coverage is `100%`.
   
   [![Impacted file tree 
graph](https://codecov.io/gh/apache/airflow/pull/4743/graphs/tree.svg?width=650=WdLKlKHOAU=150=pr)](https://codecov.io/gh/apache/airflow/pull/4743?src=pr=tree)
   
   ```diff
   @@Coverage Diff @@
   ##   master#4743  +/-   ##
   ==
   + Coverage   75.76%   75.77%   +<.01% 
   ==
 Files 461  461  
 Lines   2996129973  +12 
   ==
   + Hits2270122713  +12 
 Misses   7260 7260
   ```
   
   
   | [Impacted 
Files](https://codecov.io/gh/apache/airflow/pull/4743?src=pr=tree) | 
Coverage Δ | |
   |---|---|---|
   | 
[airflow/models/\_\_init\_\_.py](https://codecov.io/gh/apache/airflow/pull/4743/diff?src=pr=tree#diff-YWlyZmxvdy9tb2RlbHMvX19pbml0X18ucHk=)
 | `93.05% <100%> (+0.04%)` | :arrow_up: |
   
   --
   
   [Continue to review full report at 
Codecov](https://codecov.io/gh/apache/airflow/pull/4743?src=pr=continue).
   > **Legend** - [Click here to learn 
more](https://docs.codecov.io/docs/codecov-delta)
   > `Δ = absolute  (impact)`, `ø = not affected`, `? = missing data`
   > Powered by 
[Codecov](https://codecov.io/gh/apache/airflow/pull/4743?src=pr=footer). 
Last update 
[9d8719e...0246619](https://codecov.io/gh/apache/airflow/pull/4743?src=pr=lastupdated).
 Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] mik-laj commented on issue #4980: [AIRFLOW-3971] Add Google Cloud Natural Language operators

mik-laj commented on issue #4980: [AIRFLOW-3971] Add Google Cloud Natural 
Language operators
URL: https://github.com/apache/airflow/pull/4980#issuecomment-478115948
 
 
   PTAL @kaxil I considered all the suggestions of the community.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] mik-laj commented on a change in pull request #4980: [AIRFLOW-3971] Add Google Cloud Natural Language operators

mik-laj commented on a change in pull request #4980: [AIRFLOW-3971] Add Google 
Cloud Natural Language operators
URL: https://github.com/apache/airflow/pull/4980#discussion_r270539435
 
 

 ##
 File path: airflow/contrib/hooks/gcp_natural_language_hook.py
 ##
 @@ -0,0 +1,217 @@
+# -*- coding: utf-8 -*-
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+from google.cloud.language_v1 import LanguageServiceClient
+
+
+from airflow.contrib.hooks.gcp_api_base_hook import GoogleCloudBaseHook
+
+
+# noinspection PyAbstractClass
+class CloudNaturalLanguageHook(GoogleCloudBaseHook):
+"""
+Hook for Google Cloud Natural Language Service.
+
+:param gcp_conn_id: The connection ID to use when fetching connection info.
+:type gcp_conn_id: str
+:param delegate_to: The account to impersonate, if any.
+For this to work, the service account making the request must have
+domain-wide delegation enabled.
+:type delegate_to: str
+"""
+
+_conn = None
+
+def __init__(self, gcp_conn_id="google_cloud_default", delegate_to=None):
+super(CloudNaturalLanguageHook, self).__init__(gcp_conn_id, 
delegate_to)
+
+def get_conn(self):
+"""
+Retrieves connection to Cloud Natural Language service.
+
+:return: Cloud Natural Language service object
+:rtype: google.cloud.language_v1.LanguageServiceClient
+"""
+if not self._conn:
+self._conn = 
LanguageServiceClient(credentials=self._get_credentials())
+return self._conn
+
+@GoogleCloudBaseHook.catch_http_exception
+def analyze_entities(self, document, encoding_type=None, retry=None, 
timeout=None, metadata=None):
+"""
+Finds named entities (currently proper names and common nouns) in the 
text along with entity types,
 
 Review comment:
   Thanks. I omit the phrase in parentheses.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] mik-laj commented on a change in pull request #4980: [AIRFLOW-3971] Add Google Cloud Natural Language operators

mik-laj commented on a change in pull request #4980: [AIRFLOW-3971] Add Google 
Cloud Natural Language operators
URL: https://github.com/apache/airflow/pull/4980#discussion_r270538983
 
 

 ##
 File path: docs/howto/operator/gcp/natural_language.rst
 ##
 @@ -0,0 +1,179 @@
+..  Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements.  See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership.  The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License.  You may obtain a copy of the License at
+
+..http://www.apache.org/licenses/LICENSE-2.0
+
+..  Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied.  See the License for the
+specific language governing permissions and limitations
+under the License.
+
+Google Cloud Natural Language Operators
+===
+
+The `Google Cloud Natural Language 
`__
+can be used to reveal the structure and meaning of text via powerful machine
+learning models. You can use it to extract information about people, places,
+events and much more, mentioned in text documents, news articles or blog posts.
+You can use it to understand sentiment about your product on social media or
+parse intent from customer conversations happening in a call center or a
+messaging app.
+
+.. contents::
+  :depth: 1
+  :local:
+
+Prerequisite Tasks
+^^
+
+.. include:: _partials/prerequisite_tasks.rst
+
+
+.. _howto/operator:CloudNaturalLanguageDocuments:
+
+Documents
+^
+
+Each operator use a :class:`~google.cloud.language_v1.types.Document` for
 
 Review comment:
   Github don't support all sphinx directive. Full preview is available: 
https://github.com/apache/airflow/pull/4980#issuecomment-478105100


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] mik-laj commented on a change in pull request #4980: [AIRFLOW-3971] Add Google Cloud Natural Language operators

mik-laj commented on a change in pull request #4980: [AIRFLOW-3971] Add Google 
Cloud Natural Language operators
URL: https://github.com/apache/airflow/pull/4980#discussion_r270539024
 
 

 ##
 File path: docs/howto/operator/gcp/natural_language.rst
 ##
 @@ -0,0 +1,179 @@
+..  Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements.  See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership.  The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License.  You may obtain a copy of the License at
+
+..http://www.apache.org/licenses/LICENSE-2.0
+
+..  Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied.  See the License for the
+specific language governing permissions and limitations
+under the License.
+
+Google Cloud Natural Language Operators
+===
+
+The `Google Cloud Natural Language 
`__
+can be used to reveal the structure and meaning of text via powerful machine
+learning models. You can use it to extract information about people, places,
+events and much more, mentioned in text documents, news articles or blog posts.
+You can use it to understand sentiment about your product on social media or
+parse intent from customer conversations happening in a call center or a
+messaging app.
+
+.. contents::
+  :depth: 1
+  :local:
+
+Prerequisite Tasks
+^^
+
+.. include:: _partials/prerequisite_tasks.rst
+
+
+.. _howto/operator:CloudNaturalLanguageDocuments:
+
+Documents
+^
+
+Each operator use a :class:`~google.cloud.language_v1.types.Document` for
 
 Review comment:
   Fixed.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] mik-laj commented on a change in pull request #4980: [AIRFLOW-3971] Add Google Cloud Natural Language operators

mik-laj commented on a change in pull request #4980: [AIRFLOW-3971] Add Google 
Cloud Natural Language operators
URL: https://github.com/apache/airflow/pull/4980#discussion_r270538955
 
 

 ##
 File path: docs/howto/operator/gcp/natural_language.rst
 ##
 @@ -0,0 +1,179 @@
+..  Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements.  See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership.  The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License.  You may obtain a copy of the License at
+
+..http://www.apache.org/licenses/LICENSE-2.0
+
+..  Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied.  See the License for the
+specific language governing permissions and limitations
+under the License.
+
+Google Cloud Natural Language Operators
+===
+
+The `Google Cloud Natural Language 
`__
+can be used to reveal the structure and meaning of text via powerful machine
+learning models. You can use it to extract information about people, places,
+events and much more, mentioned in text documents, news articles or blog posts.
+You can use it to understand sentiment about your product on social media or
+parse intent from customer conversations happening in a call center or a
+messaging app.
+
+.. contents::
+  :depth: 1
+  :local:
+
+Prerequisite Tasks
+^^
+
+.. include:: _partials/prerequisite_tasks.rst
+
+
+.. _howto/operator:CloudNaturalLanguageDocuments:
+
+Documents
+^
+
+Each operator use a :class:`~google.cloud.language_v1.types.Document` for
+representing text.
+
+Here is an example of document with text provided as a string:
+
+.. literalinclude:: 
../../../../airflow/contrib/example_dags/example_gcp_natural_language.py
 
 Review comment:
   Github don't support all sphinx directive. Full preview is available: 
https://github.com/apache/airflow/pull/4980#issuecomment-478105100


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] mik-laj commented on a change in pull request #4980: [AIRFLOW-3971] Add Google Cloud Natural Language operators

mik-laj commented on a change in pull request #4980: [AIRFLOW-3971] Add Google 
Cloud Natural Language operators
URL: https://github.com/apache/airflow/pull/4980#discussion_r270538079
 
 

 ##
 File path: docs/howto/operator/gcp/_partials/prerequisite_tasks.rst
 ##
 @@ -0,0 +1,31 @@
+..  Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements.  See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership.  The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License.  You may obtain a copy of the License at
+
+..http://www.apache.org/licenses/LICENSE-2.0
+
+..  Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied.  See the License for the
+specific language governing permissions and limitations
+under the License.
+
+To use these operators, you must do a few things:
+
+  * Select or create a Cloud Platform project using `Cloud Console 
`__.
+  * Enable billing for your project, as described in `Google Cloud 
documentation 
`__.
+  * Enable the Google Cloud Language API using `Cloud Console 
`__.
 
 Review comment:
   I will do it.
   
   Do you think it's worth updating the google-cloud-python library to use the 
correct name?
   https://github.com/googleapis/google-cloud-python/blob/master/docs/index.rst


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] mik-laj commented on a change in pull request #4980: [AIRFLOW-3971] Add Google Cloud Natural Language operators

mik-laj commented on a change in pull request #4980: [AIRFLOW-3971] Add Google 
Cloud Natural Language operators
URL: https://github.com/apache/airflow/pull/4980#discussion_r270536584
 
 

 ##
 File path: airflow/contrib/operators/gcp_natural_language_operator.py
 ##
 @@ -0,0 +1,262 @@
+# -*- coding: utf-8 -*-
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+from google.protobuf.json_format import MessageToDict
+
+from airflow.contrib.hooks.gcp_natural_language_hook import 
CloudNaturalLanguageHook
+from airflow.models import BaseOperator
+
+
+class CloudLanguageAnalyzeEntitiesOperator(BaseOperator):
+"""
+Finds named entities (currently proper names and common nouns) in the text 
along with entity types,
+salience, mentions for each entity, and other properties.
+
+.. seealso::
+For more information on how to use this operator, take a look at the 
guide:
+:ref:`howto/operator:CloudLanguageAnalyzeEntitiesOperator`
+
+:param document: Input document.
+If a dict is provided, it must be of the same form as the protobuf 
message Document
+:type document: dict or google.cloud.language_v1.types.Document
+:param encoding_type: The encoding type used by the API to calculate 
offsets.
+:type encoding_type: google.cloud.language_v1.types.EncodingType
+:param retry: A retry object used to retry requests. If None is specified, 
requests will not be
+retried.
+:param timeout: The amount of time, in seconds, to wait for the request to 
complete. Note that if
+retry is specified, the timeout applies to each individual attempt.
+:type timeout: float
+:param metadata: Additional metadata that is provided to the method.
+:type metadata: seq[tuple[str, str]]]
+:param gcp_conn_id: The connection ID to use connecting to Google Cloud 
Platform.
+:type gcp_conn_id: str
+"""
+
+# [START natural_langauge_analyze_entities_template_fields]
+template_fields = ("document", "gcp_conn_id")
+# [END natural_langauge_analyze_entities_template_fields]
+
+def __init__(
+self,
+document,
+encoding_type=None,
+retry=None,
+timeout=None,
+metadata=None,
+gcp_conn_id="google_cloud_default",
+*args,
+**kwargs
+):
+super(self, CloudLanguageAnalyzeEntitiesOperator).__init__(*args, 
**kwargs)
+self.document = document
+self.encoding_type = encoding_type
+self.retry = retry
+self.timeout = timeout
+self.metadata = metadata
+self.gcp_conn_id = gcp_conn_id
+
+def execute(self, context):
+hook = CloudNaturalLanguageHook(gcp_conn_id=self.gcp_conn_id)
+
+self.log.info("Start analyzing entities")
+response = hook.analyze_entities(
+document=self.document, retry=self.retry, timeout=self.timeout, 
metadata=self.metadata
 
 Review comment:
   google-cloud-python library perform client of parameters. The binary 
protocols are internally used, so validation is required.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] mik-laj commented on a change in pull request #4980: [AIRFLOW-3971] Add Google Cloud Natural Language operators

mik-laj commented on a change in pull request #4980: [AIRFLOW-3971] Add Google 
Cloud Natural Language operators
URL: https://github.com/apache/airflow/pull/4980#discussion_r270536584
 
 

 ##
 File path: airflow/contrib/operators/gcp_natural_language_operator.py
 ##
 @@ -0,0 +1,262 @@
+# -*- coding: utf-8 -*-
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+from google.protobuf.json_format import MessageToDict
+
+from airflow.contrib.hooks.gcp_natural_language_hook import 
CloudNaturalLanguageHook
+from airflow.models import BaseOperator
+
+
+class CloudLanguageAnalyzeEntitiesOperator(BaseOperator):
+"""
+Finds named entities (currently proper names and common nouns) in the text 
along with entity types,
+salience, mentions for each entity, and other properties.
+
+.. seealso::
+For more information on how to use this operator, take a look at the 
guide:
+:ref:`howto/operator:CloudLanguageAnalyzeEntitiesOperator`
+
+:param document: Input document.
+If a dict is provided, it must be of the same form as the protobuf 
message Document
+:type document: dict or google.cloud.language_v1.types.Document
+:param encoding_type: The encoding type used by the API to calculate 
offsets.
+:type encoding_type: google.cloud.language_v1.types.EncodingType
+:param retry: A retry object used to retry requests. If None is specified, 
requests will not be
+retried.
+:param timeout: The amount of time, in seconds, to wait for the request to 
complete. Note that if
+retry is specified, the timeout applies to each individual attempt.
+:type timeout: float
+:param metadata: Additional metadata that is provided to the method.
+:type metadata: seq[tuple[str, str]]]
+:param gcp_conn_id: The connection ID to use connecting to Google Cloud 
Platform.
+:type gcp_conn_id: str
+"""
+
+# [START natural_langauge_analyze_entities_template_fields]
+template_fields = ("document", "gcp_conn_id")
+# [END natural_langauge_analyze_entities_template_fields]
+
+def __init__(
+self,
+document,
+encoding_type=None,
+retry=None,
+timeout=None,
+metadata=None,
+gcp_conn_id="google_cloud_default",
+*args,
+**kwargs
+):
+super(self, CloudLanguageAnalyzeEntitiesOperator).__init__(*args, 
**kwargs)
+self.document = document
+self.encoding_type = encoding_type
+self.retry = retry
+self.timeout = timeout
+self.metadata = metadata
+self.gcp_conn_id = gcp_conn_id
+
+def execute(self, context):
+hook = CloudNaturalLanguageHook(gcp_conn_id=self.gcp_conn_id)
+
+self.log.info("Start analyzing entities")
+response = hook.analyze_entities(
+document=self.document, retry=self.retry, timeout=self.timeout, 
metadata=self.metadata
 
 Review comment:
   google-cloud-python library perform client-side validation of parameters. 
The binary protocols are internally used, so validation is required.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[jira] [Commented] (AIRFLOW-3540) Presence of ~/airflow/airflow.cfg shouldn't override config file settings



[ 
https://issues.apache.org/jira/browse/AIRFLOW-3540?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16805324#comment-16805324
 ] 

ASF GitHub Bot commented on AIRFLOW-3540:
-

ashb commented on pull request #5006: [AIRFLOW-3540] Warn if old airflow.cfg 
file is found
URL: https://github.com/apache/airflow/pull/5006
 
 
   Make sure you have checked _all_ steps below.
   
   ### Jira
   
   - [x] https://issues.apache.org/jira/browse/AIRFLOW-3540
   
   ### Description
   
   - [x] People might have been caught by this situation of having (a dual 
config file) without even knowning it. To make it obvious for those people we 
should _say_ if we found the old config file even if we don't read from it.
   
 **Note:** Against v1-10-stable branch, not master.
   
 A DeprecationWarning may not actually be the right thing here, as we 
aren't reading from it. Anyone have any thoughts?
   
   ### Tests
   
   - [x] My PR adds the following unit tests __OR__ does not need testing for 
this extremely good reason: tested manually.
   
   ### Commits
   
   - [x] My commits all reference Jira issues in their subject lines, and I 
have squashed multiple commits if they address the same issue. In addition, my 
commits follow the guidelines from "[How to write a good git commit 
message](http://chris.beams.io/posts/git-commit/)":
 1. Subject is separated from body by a blank line
 1. Subject is limited to 50 characters (not including Jira issue reference)
 1. Subject does not end with a period
 1. Subject uses the imperative mood ("add", not "adding")
 1. Body wraps at 72 characters
 1. Body explains "what" and "why", not "how"
   
   ### Documentation
   
   - [x] In case of new functionality, my PR adds documentation that describes 
how to use it.
 - All the public functions and the classes in the PR contain docstrings 
that explain what it does
 - If you implement backwards incompatible changes, please leave a note in 
the [Updating.md](https://github.com/apache/airflow/blob/master/UPDATING.md) so 
we can assign it to a appropriate release
   
   ### Code Quality
   
   - [x] Passes `flake8`
   
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


> Presence of ~/airflow/airflow.cfg shouldn't override config file settings
> -
>
> Key: AIRFLOW-3540
> URL: https://issues.apache.org/jira/browse/AIRFLOW-3540
> Project: Apache Airflow
>  Issue Type: Improvement
>Reporter: Josh Carp
>Assignee: Josh Carp
>Priority: Trivial
> Fix For: 1.10.3
>
>
> If `~/airflow/airflow.cfg` exists, airflow uses that file instead of 
> `$AIRFLOW_HOME/airflow.cfg`. This behavior is confusing. If `$AIRFLOW_HOME` 
> is set and `$AIRFLOW_CONFIG` is not, we should only check 
> `$AIRFLOW_HOME/airflow.cfg`.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] [airflow] ashb opened a new pull request #5006: [AIRFLOW-3540] Warn if old airflow.cfg file is found

ashb opened a new pull request #5006: [AIRFLOW-3540] Warn if old airflow.cfg 
file is found
URL: https://github.com/apache/airflow/pull/5006
 
 
   Make sure you have checked _all_ steps below.
   
   ### Jira
   
   - [x] https://issues.apache.org/jira/browse/AIRFLOW-3540
   
   ### Description
   
   - [x] People might have been caught by this situation of having (a dual 
config file) without even knowning it. To make it obvious for those people we 
should _say_ if we found the old config file even if we don't read from it.
   
 **Note:** Against v1-10-stable branch, not master.
   
 A DeprecationWarning may not actually be the right thing here, as we 
aren't reading from it. Anyone have any thoughts?
   
   ### Tests
   
   - [x] My PR adds the following unit tests __OR__ does not need testing for 
this extremely good reason: tested manually.
   
   ### Commits
   
   - [x] My commits all reference Jira issues in their subject lines, and I 
have squashed multiple commits if they address the same issue. In addition, my 
commits follow the guidelines from "[How to write a good git commit 
message](http://chris.beams.io/posts/git-commit/)":
 1. Subject is separated from body by a blank line
 1. Subject is limited to 50 characters (not including Jira issue reference)
 1. Subject does not end with a period
 1. Subject uses the imperative mood ("add", not "adding")
 1. Body wraps at 72 characters
 1. Body explains "what" and "why", not "how"
   
   ### Documentation
   
   - [x] In case of new functionality, my PR adds documentation that describes 
how to use it.
 - All the public functions and the classes in the PR contain docstrings 
that explain what it does
 - If you implement backwards incompatible changes, please leave a note in 
the [Updating.md](https://github.com/apache/airflow/blob/master/UPDATING.md) so 
we can assign it to a appropriate release
   
   ### Code Quality
   
   - [x] Passes `flake8`
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] mik-laj commented on a change in pull request #4980: [AIRFLOW-3971] Add Google Cloud Natural Language operators

mik-laj commented on a change in pull request #4980: [AIRFLOW-3971] Add Google 
Cloud Natural Language operators
URL: https://github.com/apache/airflow/pull/4980#discussion_r270532182
 
 

 ##
 File path: airflow/contrib/operators/gcp_natural_language_operator.py
 ##
 @@ -0,0 +1,262 @@
+# -*- coding: utf-8 -*-
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+from google.protobuf.json_format import MessageToDict
+
+from airflow.contrib.hooks.gcp_natural_language_hook import 
CloudNaturalLanguageHook
+from airflow.models import BaseOperator
+
+
+class CloudLanguageAnalyzeEntitiesOperator(BaseOperator):
+"""
+Finds named entities (currently proper names and common nouns) in the text 
along with entity types,
+salience, mentions for each entity, and other properties.
+
+.. seealso::
+For more information on how to use this operator, take a look at the 
guide:
+:ref:`howto/operator:CloudLanguageAnalyzeEntitiesOperator`
+
+:param document: Input document.
+If a dict is provided, it must be of the same form as the protobuf 
message Document
+:type document: dict or google.cloud.language_v1.types.Document
+:param encoding_type: The encoding type used by the API to calculate 
offsets.
+:type encoding_type: google.cloud.language_v1.types.EncodingType
+:param retry: A retry object used to retry requests. If None is specified, 
requests will not be
+retried.
+:param timeout: The amount of time, in seconds, to wait for the request to 
complete. Note that if
+retry is specified, the timeout applies to each individual attempt.
+:type timeout: float
+:param metadata: Additional metadata that is provided to the method.
+:type metadata: seq[tuple[str, str]]]
+:param gcp_conn_id: The connection ID to use connecting to Google Cloud 
Platform.
+:type gcp_conn_id: str
+"""
+
+# [START natural_langauge_analyze_entities_template_fields]
+template_fields = ("document", "gcp_conn_id")
 
 Review comment:
   Other operators do not have this field templated. For consistency, it should 
not.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] mik-laj commented on issue #4980: [AIRFLOW-3971] Add Google Cloud Natural Language operators

mik-laj commented on issue #4980: [AIRFLOW-3971] Add Google Cloud Natural 
Language operators
URL: https://github.com/apache/airflow/pull/4980#issuecomment-478105100
 
 
   Preview: http://classy-fan.surge.sh/howto/operator/gcp/natural_language.html


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[jira] [Resolved] (AIRFLOW-1526) Add dingding opeartor

2019-03-29 Thread Ash Berlin-Taylor (JIRA)



 [ 
https://issues.apache.org/jira/browse/AIRFLOW-1526?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ash Berlin-Taylor resolved AIRFLOW-1526.

Resolution: Resolved

> Add dingding opeartor
> -
>
> Key: AIRFLOW-1526
> URL: https://issues.apache.org/jira/browse/AIRFLOW-1526
> Project: Apache Airflow
>  Issue Type: New Feature
>  Components: operators
>Affects Versions: 1.8.0
>Reporter: cocopc
>Assignee: zhongjiajie
>Priority: Major
>  Labels: features
>
> h4. This operator implements send message to 
> dingding(https://www.dingtalk.com/),like slack.
> h5. usage scenario example:
> * on task fail, send message to dingding group,rather than  email.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] [airflow] ashb opened a new pull request #5005: [AIRFLOW-3623] Fix bugs in Download task logs

2019-03-29 Thread ASF subversion and git services (JIRA)

ashb opened a new pull request #5005: [AIRFLOW-3623] Fix bugs in Download task 
logs
URL: https://github.com/apache/airflow/pull/5005
 
 
   Make sure you have checked _all_ steps below.
   
   ### Jira
   
   - [x] https://issues.apache.org/jira/browse/AIRFLOW-3623

   
   ### Description
   
   - [x] They didn't work in Python 3 (Bytes vs Strings), the graph_view wasn't
   specifying the try_number correctly, and the "All" logs had "None" as
   the suffix instead of "all".
   
 /cc   @zhongjiajie
   
   ### Tests
   
   - [x] My PR adds the following unit tests __OR__ does not need testing for 
this extremely good reason: I maybe should have added better tests for this... 
Hmm
   
   ### Commits
   
   - [x] My commits all reference Jira issues in their subject lines, and I 
have squashed multiple commits if they address the same issue. In addition, my 
commits follow the guidelines from "[How to write a good git commit 
message](http://chris.beams.io/posts/git-commit/)":
 1. Subject is separated from body by a blank line
 1. Subject is limited to 50 characters (not including Jira issue reference)
 1. Subject does not end with a period
 1. Subject uses the imperative mood ("add", not "adding")
 1. Body wraps at 72 characters
 1. Body explains "what" and "why", not "how"
   
   ### Documentation
   
   - [ ] In case of new functionality, my PR adds documentation that describes 
how to use it.
 - All the public functions and the classes in the PR contain docstrings 
that explain what it does
 - If you implement backwards incompatible changes, please leave a note in 
the [Updating.md](https://github.com/apache/airflow/blob/master/UPDATING.md) so 
we can assign it to a appropriate release
   
   ### Code Quality
   
   - [ ] Passes `flake8`
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[jira] [Commented] (AIRFLOW-1526) Add dingding opeartor



[ 
https://issues.apache.org/jira/browse/AIRFLOW-1526?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16805316#comment-16805316
 ] 

ASF subversion and git services commented on AIRFLOW-1526:
--

Commit 691d06927ba1000592c673dfba2e2b417f3101de in airflow's branch 
refs/heads/v1-10-stable from Jiajie Zhong
[ https://gitbox.apache.org/repos/asf?p=airflow.git;h=691d069 ]

[AIRFLOW-1526] Add dingding hook and operator (#4895)

Dingding is popular team collaboration tools talk
just like Slack. This PR is add it to master, could
help us easy send message to Dingding.

Add how to operator dingding section tell users
how to send different kind of dingding message or
how to use dingding as callback funtion in DAG
and add example to show how to use.

Set operator UI color as dingding icon color

> Add dingding opeartor
> -
>
> Key: AIRFLOW-1526
> URL: https://issues.apache.org/jira/browse/AIRFLOW-1526
> Project: Apache Airflow
>  Issue Type: New Feature
>  Components: operators
>Affects Versions: 1.8.0
>Reporter: cocopc
>Assignee: zhongjiajie
>Priority: Major
>  Labels: features
>
> h4. This operator implements send message to 
> dingding(https://www.dingtalk.com/),like slack.
> h5. usage scenario example:
> * on task fail, send message to dingding group,rather than  email.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] [airflow] mik-laj commented on issue #4980: [AIRFLOW-3971] Add Google Cloud Natural Language operators

mik-laj commented on issue #4980: [AIRFLOW-3971] Add Google Cloud Natural 
Language operators
URL: https://github.com/apache/airflow/pull/4980#issuecomment-478099956
 
 
   @wwlian If you are interested, I will prepare you preview of the built-in 
documentation in HTML format.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] mik-laj commented on a change in pull request #4980: [AIRFLOW-3971] Add Google Cloud Natural Language operators

mik-laj commented on a change in pull request #4980: [AIRFLOW-3971] Add Google
Cloud Natural Language operators
URL: https://github.com/apache/airflow/pull/4980#discussion_r270524733

##
File path: docs/code.rst
##
@@ -0,0 +1,574 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+..http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+
+API Reference
+=
+
+.. _api-reference-operators:
+
+Operators
+-
+Operators allow for generation of certain types of tasks that become nodes in
+the DAG when instantiated. All operators derive from ``BaseOperator`` and
+inherit many attributes and methods that way. Refer to the BaseOperator_
+documentation for more details.
+
+There are 3 main types of operators:
+
+- Operators that performs an **action**, or tell another system to
+ perform an action
+- **Transfer** operators move data from one system to another
+- **Sensors** are a certain type of operator that will keep running until a
+ certain criterion is met. Examples include a specific file landing in HDFS or
+ S3, a partition appearing in Hive, or a specific time of the day. Sensors
+ are derived from ``BaseSensorOperator`` and run a poke
+ method at a specified ``poke_interval`` until it returns ``True``.
+
+BaseOperator
+
+All operators are derived from ``BaseOperator`` and acquire much
+functionality through inheritance. Since this is the core of the engine,
+it's worth taking the time to understand the parameters of ``BaseOperator``
+to understand the primitive features that can be leveraged in your
+DAGs.
+
+
+.. autoclass:: airflow.models.BaseOperator
+
+
+BaseSensorOperator
+'''
+All sensors are derived from ``BaseSensorOperator``. All sensors inherit
+the ``timeout`` and ``poke_interval`` on top of the ``BaseOperator``
+attributes.
+
+.. autoclass:: airflow.sensors.base_sensor_operator.BaseSensorOperator
+
+
+Core Operators
+''
+
+Operators
+^
+
+.. autoclass:: airflow.operators.bash_operator.BashOperator

Review comment:
Github does not support additional directives provided by Sphinx. Ever since
I made this PR, the format of the documentation has also changed.

Reference:

https://lists.apache.org/thread.html/7c4a303a9de81c9b49de2f47589c799db12ad4bcfd66530309a09dc1@%3Cdev.airflow.apache.org%3E

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

With regards,
Apache Git Services

[GitHub] [airflow] wwlian commented on a change in pull request #4980: [AIRFLOW-3971] Add Google Cloud Natural Language operators

wwlian commented on a change in pull request #4980: [AIRFLOW-3971] Add Google 
Cloud Natural Language operators
URL: https://github.com/apache/airflow/pull/4980#discussion_r270511164
 
 

 ##
 File path: airflow/contrib/operators/gcp_natural_language_operator.py
 ##
 @@ -0,0 +1,262 @@
+# -*- coding: utf-8 -*-
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+from google.protobuf.json_format import MessageToDict
+
+from airflow.contrib.hooks.gcp_natural_language_hook import 
CloudNaturalLanguageHook
+from airflow.models import BaseOperator
+
+
+class CloudLanguageAnalyzeEntitiesOperator(BaseOperator):
+"""
+Finds named entities (currently proper names and common nouns) in the text 
along with entity types,
+salience, mentions for each entity, and other properties.
+
+.. seealso::
+For more information on how to use this operator, take a look at the 
guide:
+:ref:`howto/operator:CloudLanguageAnalyzeEntitiesOperator`
+
+:param document: Input document.
+If a dict is provided, it must be of the same form as the protobuf 
message Document
+:type document: dict or google.cloud.language_v1.types.Document
+:param encoding_type: The encoding type used by the API to calculate 
offsets.
+:type encoding_type: google.cloud.language_v1.types.EncodingType
+:param retry: A retry object used to retry requests. If None is specified, 
requests will not be
+retried.
+:param timeout: The amount of time, in seconds, to wait for the request to 
complete. Note that if
+retry is specified, the timeout applies to each individual attempt.
+:type timeout: float
+:param metadata: Additional metadata that is provided to the method.
+:type metadata: seq[tuple[str, str]]]
+:param gcp_conn_id: The connection ID to use connecting to Google Cloud 
Platform.
+:type gcp_conn_id: str
+"""
+
+# [START natural_langauge_analyze_entities_template_fields]
+template_fields = ("document", "gcp_conn_id")
+# [END natural_langauge_analyze_entities_template_fields]
+
+def __init__(
+self,
+document,
+encoding_type=None,
+retry=None,
+timeout=None,
+metadata=None,
+gcp_conn_id="google_cloud_default",
+*args,
+**kwargs
+):
+super(self, CloudLanguageAnalyzeEntitiesOperator).__init__(*args, 
**kwargs)
+self.document = document
+self.encoding_type = encoding_type
+self.retry = retry
+self.timeout = timeout
+self.metadata = metadata
+self.gcp_conn_id = gcp_conn_id
+
+def execute(self, context):
+hook = CloudNaturalLanguageHook(gcp_conn_id=self.gcp_conn_id)
+
+self.log.info("Start analyzing entities")
+response = hook.analyze_entities(
+document=self.document, retry=self.retry, timeout=self.timeout, 
metadata=self.metadata
 
 Review comment:
   Does the natural language client library perform client-side validation of 
the document? For example, to check that a dict-type document maps correctly to 
a Document message? If not, might it make sense to use 
[GcpBodyFieldValidator](https://github.com/apache/airflow/blob/master/airflow/contrib/utils/gcp_field_validator.py)?


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] wwlian commented on a change in pull request #4980: [AIRFLOW-3971] Add Google Cloud Natural Language operators

wwlian commented on a change in pull request #4980: [AIRFLOW-3971] Add Google 
Cloud Natural Language operators
URL: https://github.com/apache/airflow/pull/4980#discussion_r270514327
 
 

 ##
 File path: airflow/contrib/operators/gcp_natural_language_operator.py
 ##
 @@ -0,0 +1,262 @@
+# -*- coding: utf-8 -*-
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+from google.protobuf.json_format import MessageToDict
+
+from airflow.contrib.hooks.gcp_natural_language_hook import 
CloudNaturalLanguageHook
+from airflow.models import BaseOperator
+
+
+class CloudLanguageAnalyzeEntitiesOperator(BaseOperator):
+"""
+Finds named entities (currently proper names and common nouns) in the text 
along with entity types,
+salience, mentions for each entity, and other properties.
+
+.. seealso::
+For more information on how to use this operator, take a look at the 
guide:
+:ref:`howto/operator:CloudLanguageAnalyzeEntitiesOperator`
+
+:param document: Input document.
+If a dict is provided, it must be of the same form as the protobuf 
message Document
+:type document: dict or google.cloud.language_v1.types.Document
+:param encoding_type: The encoding type used by the API to calculate 
offsets.
+:type encoding_type: google.cloud.language_v1.types.EncodingType
+:param retry: A retry object used to retry requests. If None is specified, 
requests will not be
+retried.
+:param timeout: The amount of time, in seconds, to wait for the request to 
complete. Note that if
+retry is specified, the timeout applies to each individual attempt.
+:type timeout: float
+:param metadata: Additional metadata that is provided to the method.
+:type metadata: seq[tuple[str, str]]]
+:param gcp_conn_id: The connection ID to use connecting to Google Cloud 
Platform.
+:type gcp_conn_id: str
+"""
+
+# [START natural_langauge_analyze_entities_template_fields]
+template_fields = ("document", "gcp_conn_id")
 
 Review comment:
   Should `metadata` also be templated in all of these operators?


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] wwlian commented on a change in pull request #4980: [AIRFLOW-3971] Add Google Cloud Natural Language operators

wwlian commented on a change in pull request #4980: [AIRFLOW-3971] Add Google 
Cloud Natural Language operators
URL: https://github.com/apache/airflow/pull/4980#discussion_r270519703
 
 

 ##
 File path: docs/howto/operator/gcp/natural_language.rst
 ##
 @@ -0,0 +1,179 @@
+..  Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements.  See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership.  The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License.  You may obtain a copy of the License at
+
+..http://www.apache.org/licenses/LICENSE-2.0
+
+..  Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied.  See the License for the
+specific language governing permissions and limitations
+under the License.
+
+Google Cloud Natural Language Operators
+===
+
+The `Google Cloud Natural Language 
`__
+can be used to reveal the structure and meaning of text via powerful machine
+learning models. You can use it to extract information about people, places,
+events and much more, mentioned in text documents, news articles or blog posts.
+You can use it to understand sentiment about your product on social media or
+parse intent from customer conversations happening in a call center or a
+messaging app.
+
+.. contents::
+  :depth: 1
+  :local:
+
+Prerequisite Tasks
+^^
+
+.. include:: _partials/prerequisite_tasks.rst
+
+
+.. _howto/operator:CloudNaturalLanguageDocuments:
+
+Documents
+^
+
+Each operator use a :class:`~google.cloud.language_v1.types.Document` for
 
 Review comment:
   nit: s/use/uses/


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] wwlian commented on a change in pull request #4980: [AIRFLOW-3971] Add Google Cloud Natural Language operators

wwlian commented on a change in pull request #4980: [AIRFLOW-3971] Add Google
Cloud Natural Language operators
URL: https://github.com/apache/airflow/pull/4980#discussion_r270520554

##
File path: docs/howto/operator/gcp/natural_language.rst
##
@@ -0,0 +1,179 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+..http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+
+Google Cloud Natural Language Operators
+===
+
+The `Google Cloud Natural Language
`__
+can be used to reveal the structure and meaning of text via powerful machine
+learning models. You can use it to extract information about people, places,
+events and much more, mentioned in text documents, news articles or blog posts.
+You can use it to understand sentiment about your product on social media or
+parse intent from customer conversations happening in a call center or a
+messaging app.
+
+.. contents::
+ :depth: 1
+ :local:
+
+Prerequisite Tasks
+^^
+
+.. include:: _partials/prerequisite_tasks.rst
+
+
+.. _howto/operator:CloudNaturalLanguageDocuments:
+
+Documents
+^
+
+Each operator use a :class:`~google.cloud.language_v1.types.Document` for
+representing text.
+
+Here is an example of document with text provided as a string:
+
+.. literalinclude::
../../../../airflow/contrib/example_dags/example_gcp_natural_language.py

Review comment:
This may just be a problem with rendering in the PR, but the literal
includes are not working for me:

https://github.com/apache/airflow/blob/b66ea5b2a1dd87572e2ac5626fce9ea164bd8097/docs/howto/operator/gcp/natural_language.rst#id7

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

With regards,
Apache Git Services

[GitHub] [airflow] wwlian commented on a change in pull request #4980: [AIRFLOW-3971] Add Google Cloud Natural Language operators

wwlian commented on a change in pull request #4980: [AIRFLOW-3971] Add Google 
Cloud Natural Language operators
URL: https://github.com/apache/airflow/pull/4980#discussion_r270516978
 
 

 ##
 File path: docs/howto/operator/gcp/_partials/prerequisite_tasks.rst
 ##
 @@ -0,0 +1,31 @@
+..  Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements.  See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership.  The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License.  You may obtain a copy of the License at
+
+..http://www.apache.org/licenses/LICENSE-2.0
+
+..  Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied.  See the License for the
+specific language governing permissions and limitations
+under the License.
+
+To use these operators, you must do a few things:
+
+  * Select or create a Cloud Platform project using `Cloud Console 
`__.
+  * Enable billing for your project, as described in `Google Cloud 
documentation 
`__.
+  * Enable the Google Cloud Language API using `Cloud Console 
`__.
 
 Review comment:
   Please use the full name of the API: Google Cloud **Natural** Language API.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] wwlian commented on a change in pull request #4980: [AIRFLOW-3971] Add Google Cloud Natural Language operators

wwlian commented on a change in pull request #4980: [AIRFLOW-3971] Add Google
Cloud Natural Language operators
URL: https://github.com/apache/airflow/pull/4980#discussion_r270519489

Review comment:
I'm not sure if this is just a rendering problem during review, but all of
these autoclass:: entries don't seem to be rendering correctly for me here:

https://github.com/apache/airflow/blob/b66ea5b2a1dd87572e2ac5626fce9ea164bd8097/docs/code.rst

They just show up as text literals in a code-style text box.

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

With regards,
Apache Git Services

[GitHub] [airflow] wwlian commented on a change in pull request #4980: [AIRFLOW-3971] Add Google Cloud Natural Language operators

wwlian commented on a change in pull request #4980: [AIRFLOW-3971] Add Google
Cloud Natural Language operators
URL: https://github.com/apache/airflow/pull/4980#discussion_r270519973

Review comment:
:class:`~google.cloud.language_v1.types.Document` isn't rendering correctly
for me. The link text is that entire string, and the link doesn't go anywhere.
This is the case for many links on this page:
https://github.com/apache/airflow/blob/b66ea5b2a1dd87572e2ac5626fce9ea164bd8097/docs/howto/operator/gcp/natural_language.rst#id7

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

With regards,
Apache Git Services

[GitHub] [airflow] feng-tao commented on issue #4983: [AIRFLOW-4163] IntervalCheckOperator supports relative diff.

feng-tao commented on issue #4983: [AIRFLOW-4163] IntervalCheckOperator 
supports relative diff.
URL: https://github.com/apache/airflow/pull/4983#issuecomment-478092832
 
 
   will take a look.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] feng-tao merged pull request #5003: [AIRFLOW-XXX] 1-setup-env.sh should only run in docker