[GitHub] [airflow] mik-laj commented on a change in pull request #8539: JIRA and Github issues explanation

2020-05-15 Thread GitBox
mik-laj commented on a change in pull request #8539: URL: https://github.com/apache/airflow/pull/8539#discussion_r426125154 ## File path: CONTRIBUTING.rst ## @@ -879,7 +879,7 @@ Apache Airflow is a Community within Apache Software Foundation. As the motto of the Apache Softwa

[GitHub] [airflow] mik-laj commented on pull request #8808: Support YAML input for CloudBuildCreateOperator

2020-05-15 Thread GitBox
mik-laj commented on pull request #8808: URL: https://github.com/apache/airflow/pull/8808#issuecomment-629596799 @joppevos Github action is sad. Can you fix it? This is an automated message from the Apache Git Service. To res

[GitHub] [airflow] jhtimmins commented on a change in pull request #8884: Use env_vars to for reload config test.

2020-05-15 Thread GitBox
jhtimmins commented on a change in pull request #8884: URL: https://github.com/apache/airflow/pull/8884#discussion_r426124504 ## File path: tests/test_logging_config.py ## @@ -128,43 +128,41 @@ def settings_context(content, directory=None, name='LOGGING_CONFIG'): :param c

[GitHub] [airflow] mik-laj commented on a change in pull request #8884: Use env_vars to for reload config test.

2020-05-15 Thread GitBox
mik-laj commented on a change in pull request #8884: URL: https://github.com/apache/airflow/pull/8884#discussion_r426124472 ## File path: tests/test_logging_config.py ## @@ -128,43 +128,41 @@ def settings_context(content, directory=None, name='LOGGING_CONFIG'): :param con

[GitHub] [airflow] jhtimmins opened a new pull request #8884: Use env_vars to for reload config test.

2020-05-15 Thread GitBox
jhtimmins opened a new pull request #8884: URL: https://github.com/apache/airflow/pull/8884 Updates the test logging configuration to use the `env_vars` to set the environment variables instead of setting them explicitly within the `settings_context` context manager. --- Make

[GitHub] [airflow] rahulj51 commented on issue #8484: Airflow 1.10.7+ suppresses Operator logs

2020-05-15 Thread GitBox
rahulj51 commented on issue #8484: URL: https://github.com/apache/airflow/issues/8484#issuecomment-629594838 For us, it's happening across all operators. I have seen this with pythonoperator ( see above steps) and also with dataprocoperator and a few others. I have tried this wi

[GitHub] [airflow] mik-laj commented on pull request #8879: [8878] Add the ability for the webserver to sync dags to the DB

2020-05-15 Thread GitBox
mik-laj commented on pull request #8879: URL: https://github.com/apache/airflow/pull/8879#issuecomment-629593316 This change will cause a lot of libraries to be loaded into the webserver process that are unnecessary and will never be used. If we need a method to force synchronization, I t

[GitHub] [airflow] ewjmulder commented on issue #8484: Airflow 1.10.7+ suppresses Operator logs

2020-05-15 Thread GitBox
ewjmulder commented on issue #8484: URL: https://github.com/apache/airflow/issues/8484#issuecomment-629591351 Indeed for us it was a problem for the KubernetesPodOperator but not for the PythonOperator. Looking at the top post it also affects the DataprocOperator, which also lives in the c

[jira] [Commented] (AIRFLOW-6290) Create guide for GKE service operator

2020-05-15 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17108849#comment-17108849 ] ASF GitHub Bot commented on AIRFLOW-6290: - tanjinP opened a new pull request #8

[GitHub] [airflow] tanjinP opened a new pull request #8883: [AIRFLOW-6290] Create guide for GKE operators

2020-05-15 Thread GitBox
tanjinP opened a new pull request #8883: URL: https://github.com/apache/airflow/pull/8883 Closes AIRFLOW-6290 / issue #8206 . Contains a guide that is appropriately linked to on how to use the various operators in GKE Service. --- Make sure to mark the boxes below before cr

[GitHub] [airflow] j-y-matsubara removed a comment on pull request #8867: Add argument to BaseSensorOperator to make respect the trigger_rule in downstream tasks.

2020-05-15 Thread GitBox
j-y-matsubara removed a comment on pull request #8867: URL: https://github.com/apache/airflow/pull/8867#issuecomment-629042421 @potiuk @kaxil Would you mind confirming this PR? This is an automated message from the Apache

[GitHub] [airflow] jhtimmins commented on issue #8484: Airflow 1.10.7+ suppresses Operator logs

2020-05-15 Thread GitBox
jhtimmins commented on issue #8484: URL: https://github.com/apache/airflow/issues/8484#issuecomment-629539238 Could you all share the different operators where you've seen this happen? Looks like it's an issue on KubernetesPodOperator? I was unable to attempted to replicate with Pyth

[GitHub] [airflow] BasPH commented on pull request #8879: [8878] Add the ability for the webserver to sync dags to the DB

2020-05-15 Thread GitBox
BasPH commented on pull request #8879: URL: https://github.com/apache/airflow/pull/8879#issuecomment-629524294 This seems like a change oddly specific to your installation. Ideally I'd work towards some process dedicated to serializing DAGs into the database, and have the webserver _

[GitHub] [airflow] ashb opened a new pull request #8882: Speed up TestAwsLambdaHook by not actually running a function

2020-05-15 Thread GitBox
ashb opened a new pull request #8882: URL: https://github.com/apache/airflow/pull/8882 Moto's mock_lambda _actually runs the code_ in a docker container. This is useful if you are testing a Lambda function but is massively overkill for testing that we make a request to a function -- Ai

[GitHub] [airflow] ashb commented on a change in pull request #8875: [AIRFLOW-8875] fix Dag Run UI execution date with timezone cannot be saved issue

2020-05-15 Thread GitBox
ashb commented on a change in pull request #8875: URL: https://github.com/apache/airflow/pull/8875#discussion_r426064945 ## File path: airflow/www/forms.py ## @@ -82,7 +82,8 @@ class DagRunForm(DynamicForm): widget=Select2Widget()) execution_date = DateTimeField(

[GitHub] [airflow] ashb commented on a change in pull request #8875: [AIRFLOW-8875] fix Dag Run UI execution date with timezone cannot be saved issue

2020-05-15 Thread GitBox
ashb commented on a change in pull request #8875: URL: https://github.com/apache/airflow/pull/8875#discussion_r426065144 ## File path: tests/www/test_views.py ## @@ -2601,14 +2601,14 @@ def test_create_dagrun(self): dr = self.session.query(models.DagRun).one() -

[airflow] branch master updated (92585ca -> 82de6f7)

2020-05-15 Thread ash
This is an automated email from the ASF dual-hosted git repository. ash pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/airflow.git. from 92585ca Added automated release notes generation for backport operators (#8807) add 82de6f7 Spend less time wai

[GitHub] [airflow] ashb merged pull request #8814: Spend less time waiting for DagFileProcessor processes to complete

2020-05-15 Thread GitBox
ashb merged pull request #8814: URL: https://github.com/apache/airflow/pull/8814 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [airflow] fuyi commented on issue #7982: When creating a DagRun from the interface Run Id should be mandatory

2020-05-15 Thread GitBox
fuyi commented on issue #7982: URL: https://github.com/apache/airflow/issues/7982#issuecomment-629456740 I think the issue is labeled wrong. it is an issue for **Scheduler** This is an automated message from the Apache Git S

[GitHub] [airflow] boring-cyborg[bot] commented on issue #8881: parallelism becomes 1/2 of config value after enabling Kubernetes Executor

2020-05-15 Thread GitBox
boring-cyborg[bot] commented on issue #8881: URL: https://github.com/apache/airflow/issues/8881#issuecomment-629449681 Thanks for opening your first issue here! Be sure to follow the issue template! This is an automated

[GitHub] [airflow] fuyi opened a new issue #8881: parallelism becomes 1/2 of config value after enabling Kubernetes Executor

2020-05-15 Thread GitBox
fuyi opened a new issue #8881: URL: https://github.com/apache/airflow/issues/8881 **Apache Airflow version**: 1.10.8 **Kubernetes version (if you are using kubernetes)** (use `kubectl version`): 1.17.5 **Environment**: - **Cloud provider or hardware configu

[GitHub] [airflow] jaketf commented on a change in pull request #8550: Add DataflowStartFlexTemplateOperator

2020-05-15 Thread GitBox
jaketf commented on a change in pull request #8550: URL: https://github.com/apache/airflow/pull/8550#discussion_r419001554 ## File path: airflow/providers/google/cloud/hooks/dataflow.py ## @@ -583,6 +623,49 @@ def start_template_dataflow( jobs_controller.wait_for_done(

[GitHub] [airflow] jaketf commented on pull request #7197: [AIRFLOW-6586] Improvements to gcs sensor

2020-05-15 Thread GitBox
jaketf commented on pull request #7197: URL: https://github.com/apache/airflow/pull/7197#issuecomment-629421104 @turbaszek seems like only quarantined test failing. Any blocker to merge? This is an automated message from the

[jira] [Commented] (AIRFLOW-6586) GCSUploadSessionCompleteSensor breaks in reschedule mode.

2020-05-15 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6586?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17108549#comment-17108549 ] ASF GitHub Bot commented on AIRFLOW-6586: - jaketf commented on pull request #71

[GitHub] [airflow] dimberman commented on pull request #6377: Monitor pods by labels instead of names

2020-05-15 Thread GitBox
dimberman commented on pull request #6377: URL: https://github.com/apache/airflow/pull/6377#issuecomment-629418392 @kaxil added docs This is an automated message from the Apache Git Service. To respond to the message, please

[GitHub] [airflow] potiuk commented on issue #8880: Got ModuleNotFoundError on execute `airflow initdb` command

2020-05-15 Thread GitBox
potiuk commented on issue #8880: URL: https://github.com/apache/airflow/issues/8880#issuecomment-629415851 Python 3.8 is not yet officially supported. There is a PR in progress #8674 This is an automated message from the Apa

[GitHub] [airflow] kaxil commented on a change in pull request #6377: Monitor pods by labels instead of names

2020-05-15 Thread GitBox
kaxil commented on a change in pull request #6377: URL: https://github.com/apache/airflow/pull/6377#discussion_r425980965 ## File path: airflow/kubernetes/pod_launcher.py ## @@ -116,9 +114,15 @@ def run_pod( time.sleep(1) self.log.debug('Pod not ye

[GitHub] [airflow] potiuk closed issue #8880: Got ModuleNotFoundError on execute `airflow initdb` command

2020-05-15 Thread GitBox
potiuk closed issue #8880: URL: https://github.com/apache/airflow/issues/8880 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the sp

[GitHub] [airflow] kaxil commented on a change in pull request #6377: Monitor pods by labels instead of names

2020-05-15 Thread GitBox
kaxil commented on a change in pull request #6377: URL: https://github.com/apache/airflow/pull/6377#discussion_r425980965 ## File path: airflow/kubernetes/pod_launcher.py ## @@ -116,9 +114,15 @@ def run_pod( time.sleep(1) self.log.debug('Pod not ye

[GitHub] [airflow] boring-cyborg[bot] commented on issue #8880: Got ModuleNotFoundError on execute `airflow initdb` command

2020-05-15 Thread GitBox
boring-cyborg[bot] commented on issue #8880: URL: https://github.com/apache/airflow/issues/8880#issuecomment-629413015 Thanks for opening your first issue here! Be sure to follow the issue template! This is an automated

[GitHub] [airflow] kaxil commented on a change in pull request #6377: Monitor pods by labels instead of names

2020-05-15 Thread GitBox
kaxil commented on a change in pull request #6377: URL: https://github.com/apache/airflow/pull/6377#discussion_r425977128 ## File path: airflow/providers/cncf/kubernetes/operators/kubernetes_pod.py ## @@ -324,3 +316,99 @@ def _set_name(self, name): return None

[GitHub] [airflow] rafaelreuber opened a new issue #8880: Got ModuleNotFoundError on execute `airflow initdb` command

2020-05-15 Thread GitBox
rafaelreuber opened a new issue #8880: URL: https://github.com/apache/airflow/issues/8880 I just tried to follow quick start and I got the following error on execute the command `airflow initdb` ``` [reuber@stormy teste]$ airflow initdb Traceback (most recent call last): F

[GitHub] [airflow] kaxil commented on a change in pull request #6377: Monitor pods by labels instead of names

2020-05-15 Thread GitBox
kaxil commented on a change in pull request #6377: URL: https://github.com/apache/airflow/pull/6377#discussion_r425976877 ## File path: airflow/providers/cncf/kubernetes/operators/kubernetes_pod.py ## @@ -324,3 +316,99 @@ def _set_name(self, name): return None

[GitHub] [airflow] kaxil commented on a change in pull request #6377: Monitor pods by labels instead of names

2020-05-15 Thread GitBox
kaxil commented on a change in pull request #6377: URL: https://github.com/apache/airflow/pull/6377#discussion_r425976748 ## File path: airflow/providers/cncf/kubernetes/operators/kubernetes_pod.py ## @@ -324,3 +316,99 @@ def _set_name(self, name): return None

[GitHub] [airflow] kaxil commented on a change in pull request #6377: Monitor pods by labels instead of names

2020-05-15 Thread GitBox
kaxil commented on a change in pull request #6377: URL: https://github.com/apache/airflow/pull/6377#discussion_r425976189 ## File path: airflow/providers/cncf/kubernetes/operators/kubernetes_pod.py ## @@ -225,7 +229,31 @@ def __init__(self, # pylint: disable=too-many-argument

[GitHub] [airflow] kaxil commented on a change in pull request #6377: Monitor pods by labels instead of names

2020-05-15 Thread GitBox
kaxil commented on a change in pull request #6377: URL: https://github.com/apache/airflow/pull/6377#discussion_r425975921 ## File path: airflow/kubernetes/pod_launcher.py ## @@ -116,9 +114,15 @@ def run_pod( time.sleep(1) self.log.debug('Pod not ye

[GitHub] [airflow] potiuk commented on pull request #8807: Added automated release notes generation for backport operators

2020-05-15 Thread GitBox
potiuk commented on pull request #8807: URL: https://github.com/apache/airflow/pull/8807#issuecomment-629372418 I will prepare the release candidates and make the announcement later today with target for end of Tuesday This

[airflow] branch master updated (f82ad45 -> 92585ca)

2020-05-15 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/airflow.git. from f82ad45 Fix KubernetesPodOperator pod name length validation (#8829) add 92585ca Added automated release note

[GitHub] [airflow] potiuk merged pull request #8807: Added automated release notes generation for backport operators

2020-05-15 Thread GitBox
potiuk merged pull request #8807: URL: https://github.com/apache/airflow/pull/8807 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to t

[GitHub] [airflow] potiuk commented on pull request #8807: Added automated release notes generation for backport operators

2020-05-15 Thread GitBox
potiuk commented on pull request #8807: URL: https://github.com/apache/airflow/pull/8807#issuecomment-629371984 And merged :) This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [airflow] ArgentFalcon opened a new pull request #8879: [Airflow-8878] Add the ability for the webserver to sync dags to the DB

2020-05-15 Thread GitBox
ArgentFalcon opened a new pull request #8879: URL: https://github.com/apache/airflow/pull/8879 This is related to https://github.com/apache/airflow/issues/8878 The issue is that sync_to_db is only called from two places: The scheduler (which is not running) initdb (which runs

[GitHub] [airflow] ArgentFalcon opened a new issue #8878: Enable the webserver to sync DAGS to the DB

2020-05-15 Thread GitBox
ArgentFalcon opened a new issue #8878: URL: https://github.com/apache/airflow/issues/8878 Add a config to Airflow to enable the webserver to update the metastore with new/updated dag infromation. At Lyft, we run a special Airflow cluster we use for developing and testing DAGS. This

[GitHub] [airflow] ArgentFalcon commented on issue #8878: Enable the webserver to sync DAGS to the DB

2020-05-15 Thread GitBox
ArgentFalcon commented on issue #8878: URL: https://github.com/apache/airflow/issues/8878#issuecomment-629351878 I plan to have a PR for this shortly This is an automated message from the Apache Git Service. To respond to the

[GitHub] [airflow] boring-cyborg[bot] commented on issue #8878: Enable the webserver to sync DAGS to the DB

2020-05-15 Thread GitBox
boring-cyborg[bot] commented on issue #8878: URL: https://github.com/apache/airflow/issues/8878#issuecomment-629351752 Thanks for opening your first issue here! Be sure to follow the issue template! This is an automated

[GitHub] [airflow] feluelle commented on a change in pull request #8877: [WIP] Add S3ToRedshift example dag and system test

2020-05-15 Thread GitBox
feluelle commented on a change in pull request #8877: URL: https://github.com/apache/airflow/pull/8877#discussion_r425906307 ## File path: tests/providers/amazon/aws/operators/test_s3_to_redshift_system.py ## @@ -0,0 +1,62 @@ +# +# Licensed to the Apache Software Foundation (AS

[GitHub] [airflow] feluelle commented on a change in pull request #8877: [WIP] Add S3ToRedshift example dag and system test

2020-05-15 Thread GitBox
feluelle commented on a change in pull request #8877: URL: https://github.com/apache/airflow/pull/8877#discussion_r425901097 ## File path: tests/test_utils/postgres_system_helpers.py ## @@ -0,0 +1,97 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more co

[GitHub] [airflow] feluelle commented on a change in pull request #8877: [WIP] Add S3ToRedshift example dag and system test

2020-05-15 Thread GitBox
feluelle commented on a change in pull request #8877: URL: https://github.com/apache/airflow/pull/8877#discussion_r425901097 ## File path: tests/test_utils/postgres_system_helpers.py ## @@ -0,0 +1,97 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more co

[GitHub] [airflow] feluelle commented on a change in pull request #8877: [WIP] Add S3ToRedshift example dag and system test

2020-05-15 Thread GitBox
feluelle commented on a change in pull request #8877: URL: https://github.com/apache/airflow/pull/8877#discussion_r425901097 ## File path: tests/test_utils/postgres_system_helpers.py ## @@ -0,0 +1,97 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more co

[GitHub] [airflow] feluelle commented on a change in pull request #8877: [WIP] Add S3ToRedshift example dag and system test

2020-05-15 Thread GitBox
feluelle commented on a change in pull request #8877: URL: https://github.com/apache/airflow/pull/8877#discussion_r425901097 ## File path: tests/test_utils/postgres_system_helpers.py ## @@ -0,0 +1,97 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more co

[GitHub] [airflow] feluelle opened a new pull request #8877: [WIP] Add S3ToRedshift example dag and system test

2020-05-15 Thread GitBox
feluelle opened a new pull request #8877: URL: https://github.com/apache/airflow/pull/8877 This PR / change adds an example dag and tests it (=system test). NOTE: Everything except the Redshift Cluster will be _provided_. So the Redshift Cluster needs to exist and so must the connect

[GitHub] [airflow] kaxil commented on a change in pull request #8739: Test that DagFileProcessor can operator against on a Serialized DAG

2020-05-15 Thread GitBox
kaxil commented on a change in pull request #8739: URL: https://github.com/apache/airflow/pull/8739#discussion_r425893005 ## File path: airflow/models/dagbag.py ## @@ -456,4 +456,5 @@ def sync_to_db(self): from airflow.models.dag import DAG from airflow.models

[GitHub] [airflow] dimberman commented on pull request #8829: Fix KubernetesPodOperator pod name length validation

2020-05-15 Thread GitBox
dimberman commented on pull request #8829: URL: https://github.com/apache/airflow/pull/8829#issuecomment-629323544 Thank you @dsaiztc ! This is an automated message from the Apache Git Service. To respond to the message, plea

[airflow] branch master updated (85bbab2 -> f82ad45)

2020-05-15 Thread dimberman
This is an automated email from the ASF dual-hosted git repository. dimberman pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/airflow.git. from 85bbab2 Add EMR operators howto docs (#8863) add f82ad45 Fix KubernetesPodOperator pod name length validat

[GitHub] [airflow] boring-cyborg[bot] commented on pull request #8829: Fix KubernetesPodOperator pod name length validation

2020-05-15 Thread GitBox
boring-cyborg[bot] commented on pull request #8829: URL: https://github.com/apache/airflow/pull/8829#issuecomment-629323035 Awesome work, congrats on your first merged pull request! This is an automated message from the A

[GitHub] [airflow] dimberman merged pull request #8829: Fix KubernetesPodOperator pod name length validation

2020-05-15 Thread GitBox
dimberman merged pull request #8829: URL: https://github.com/apache/airflow/pull/8829 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go t

[GitHub] [airflow] dsaiztc commented on pull request #8829: Fix KubernetesPodOperator pod name length validation

2020-05-15 Thread GitBox
dsaiztc commented on pull request #8829: URL: https://github.com/apache/airflow/pull/8829#issuecomment-629321636 All tests passed @dimberman =) Any further feedback welcome. This is an automated message from the Apache

[GitHub] [airflow] ashb commented on pull request #8814: Spend less time waiting for DagFileProcessor processes to complete

2020-05-15 Thread GitBox
ashb commented on pull request #8814: URL: https://github.com/apache/airflow/pull/8814#issuecomment-629319138 Double checked before+after timing, still valid. This is an automated message from the Apache Git Service. To respo

[GitHub] [airflow] klsnreddy opened a new issue #8876: Provide option to time delta on upstream tasks in TimeDeltaSensor

2020-05-15 Thread GitBox
klsnreddy opened a new issue #8876: URL: https://github.com/apache/airflow/issues/8876 **Description** Provide option to time delta on upstream tasks in TimeDeltaSensor. Currently it adds delta to the schedule end time, irrespective of where the TimeDeltaSensor is in DAG. **Use c

[GitHub] [airflow] mik-laj commented on a change in pull request #8721: Add OpenAPI specification (II)

2020-05-15 Thread GitBox
mik-laj commented on a change in pull request #8721: URL: https://github.com/apache/airflow/pull/8721#discussion_r425872093 ## File path: openapi.yaml ## @@ -2087,7 +2087,7 @@ components: The value can be repeated to retrieve multiple matching values (OR condition).

[GitHub] [airflow] samuelkhtu commented on issue #8525: SQLBranchOperator

2020-05-15 Thread GitBox
samuelkhtu commented on issue #8525: URL: https://github.com/apache/airflow/issues/8525#issuecomment-629296674 Great. Thank you @jeffolsi ! This is an automated message from the Apache Git Service. To respond to the mess

[GitHub] [airflow] jeffolsi commented on issue #8525: SQLBranchOperator

2020-05-15 Thread GitBox
jeffolsi commented on issue #8525: URL: https://github.com/apache/airflow/issues/8525#issuecomment-629287817 @samuelkhtu exactly what i was thinking about This is an automated message from the Apache Git Service. To respond t

[GitHub] [airflow] PeWu commented on a change in pull request #8721: Add OpenAPI specification (II)

2020-05-15 Thread GitBox
PeWu commented on a change in pull request #8721: URL: https://github.com/apache/airflow/pull/8721#discussion_r425865356 ## File path: openapi.yaml ## @@ -0,0 +1,2411 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. Se

[GitHub] [airflow] potiuk commented on a change in pull request #8721: Add OpenAPI specification (II)

2020-05-15 Thread GitBox
potiuk commented on a change in pull request #8721: URL: https://github.com/apache/airflow/pull/8721#discussion_r425863539 ## File path: openapi.yaml ## @@ -2087,7 +2087,7 @@ components: The value can be repeated to retrieve multiple matching values (OR condition).

[GitHub] [airflow] mik-laj edited a comment on pull request #8721: Add OpenAPI specification (II)

2020-05-15 Thread GitBox
mik-laj edited a comment on pull request #8721: URL: https://github.com/apache/airflow/pull/8721#issuecomment-629282839 @ashb @kaxil @potiuk @ephraimbuddy I think I have already included all mailing list and Github comments. Latest changes: ``` 39b0472f4 Fix typo - response in

[GitHub] [airflow] mik-laj commented on pull request #8721: Add OpenAPI specification (II)

2020-05-15 Thread GitBox
mik-laj commented on pull request #8721: URL: https://github.com/apache/airflow/pull/8721#issuecomment-629282839 @ashb @kaxil @potiuk @ephraimbuddy I think I have already included all mailing list and Github comments. Latest changes: ``` 39b0472f4 Fix typo - response in trigger

[GitHub] [airflow] ashb commented on pull request #8814: Spend less time waiting for DagFileProcessor processes to complete

2020-05-15 Thread GitBox
ashb commented on pull request #8814: URL: https://github.com/apache/airflow/pull/8814#issuecomment-629281193 :heavy_check_mark: Finally! This is an automated message from the Apache Git Service. To respond to the message, p

[GitHub] [airflow] potiuk commented on pull request #8807: Added automated release notes generation for backport operators

2020-05-15 Thread GitBox
potiuk commented on pull request #8807: URL: https://github.com/apache/airflow/pull/8807#issuecomment-629275524 > (You're just after all the + lines to up your stats, aren't you? ) Obviously. That's all I care about :). But I am still faar behind @mik-laj :) -

[GitHub] [airflow] ashb commented on a change in pull request #8739: Test that DagFileProcessor can operator against on a Serialized DAG

2020-05-15 Thread GitBox
ashb commented on a change in pull request #8739: URL: https://github.com/apache/airflow/pull/8739#discussion_r425839302 ## File path: airflow/models/dagbag.py ## @@ -456,4 +456,5 @@ def sync_to_db(self): from airflow.models.dag import DAG from airflow.models.

[GitHub] [airflow] ashb commented on a change in pull request #8739: Test that DagFileProcessor can operator against on a Serialized DAG

2020-05-15 Thread GitBox
ashb commented on a change in pull request #8739: URL: https://github.com/apache/airflow/pull/8739#discussion_r425839031 ## File path: airflow/models/dagbag.py ## @@ -456,4 +456,5 @@ def sync_to_db(self): from airflow.models.dag import DAG from airflow.models.

[GitHub] [airflow] potiuk commented on pull request #8807: Added automated release notes generation for backport operators

2020-05-15 Thread GitBox
potiuk commented on pull request #8807: URL: https://github.com/apache/airflow/pull/8807#issuecomment-629255974 And here without the TOC (it did not work at PyPI): https://test.pypi.org/project/apache-airflow-backport-providers-google/2020.5.19rc4/ - we still have TOC in GitHub and it w

[GitHub] [airflow] potiuk commented on a change in pull request #8807: Added automated release notes generation for backport operators

2020-05-15 Thread GitBox
potiuk commented on a change in pull request #8807: URL: https://github.com/apache/airflow/pull/8807#discussion_r425825902 ## File path: airflow/providers/apache/druid/README.md ## @@ -0,0 +1,125 @@ + + + +# Package apache-airflow-backport-providers-apache-druid + +Release: 202

[GitHub] [airflow] boring-cyborg[bot] commented on pull request #8875: [AIRFLOW-8842] fix Dag Run UI execution date with timezone cannot be saved issue

2020-05-15 Thread GitBox
boring-cyborg[bot] commented on pull request #8875: URL: https://github.com/apache/airflow/pull/8875#issuecomment-629249520 Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our Contr

[GitHub] [airflow] realradical opened a new pull request #8875: [AIRFLOW-8842] fix Dag Run UI execution date with timezone cannot be saved issue

2020-05-15 Thread GitBox
realradical opened a new pull request #8875: URL: https://github.com/apache/airflow/pull/8875 --- Make sure to mark the boxes below before creating PR: [x] - [ ] Description above provides context of the change - [ ] Unit tests coverage for changes (not needed for documentation

[jira] [Commented] (AIRFLOW-5221) Add host alias support to the KubernetesPodOperator

2020-05-15 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-5221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17108280#comment-17108280 ] ASF GitHub Bot commented on AIRFLOW-5221: - dimberman commented on pull request

[GitHub] [airflow] dimberman commented on pull request #7365: [AIRFLOW-5221] add host_aliases to KubernetesPodOperator

2020-05-15 Thread GitBox
dimberman commented on pull request #7365: URL: https://github.com/apache/airflow/pull/7365#issuecomment-629235310 Hi @kuromt could you please rebase with master? When those tests pass I'm good to merge This is an automated

[GitHub] [airflow] aarrtteemmuuss commented on issue #8770: Airflow 1.10.7 logs from S3 won't load, just hanging

2020-05-15 Thread GitBox
aarrtteemmuuss commented on issue #8770: URL: https://github.com/apache/airflow/issues/8770#issuecomment-629233042 @feluelle I am actually using RBAC in k8s cluster, and there I am not even able to write to S3. How are RBAC and S3 logs related? ---

[GitHub] [airflow] dimberman commented on pull request #8829: Fix KubernetesPodOperator pod name length validation

2020-05-15 Thread GitBox
dimberman commented on pull request #8829: URL: https://github.com/apache/airflow/pull/8829#issuecomment-629231778 thank you @ddelange! Once tests pass I'll merge. This is an automated message from the Apache Git Service. To

[GitHub] [airflow] aarrtteemmuuss commented on issue #8770: Airflow 1.10.7 logs from S3 won't load, just hanging

2020-05-15 Thread GitBox
aarrtteemmuuss commented on issue #8770: URL: https://github.com/apache/airflow/issues/8770#issuecomment-629231295 @feluelle Not using RBAC because it is local setup, not in K8S cluster. I have tried adding region and removing quotes - same result, I am actually able to write like I sai

[GitHub] [airflow] samuelkhtu edited a comment on issue #8525: SQLBranchOperator

2020-05-15 Thread GitBox
samuelkhtu edited a comment on issue #8525: URL: https://github.com/apache/airflow/issues/8525#issuecomment-629226528 Thank you @jeffolsi . I see. I guess what you are looking for is sightly different than the Python version. Yes, the SQL query can decide True/False but we still need to le

[GitHub] [airflow] samuelkhtu edited a comment on issue #8525: SQLBranchOperator

2020-05-15 Thread GitBox
samuelkhtu edited a comment on issue #8525: URL: https://github.com/apache/airflow/issues/8525#issuecomment-629226528 Thank you @jeffolsi . I see. I guess what you are looking for is sightly different than the Python version. Yes, the SQL query can decide True/False but we still need to le

[GitHub] [airflow] samuelkhtu edited a comment on issue #8525: SQLBranchOperator

2020-05-15 Thread GitBox
samuelkhtu edited a comment on issue #8525: URL: https://github.com/apache/airflow/issues/8525#issuecomment-629226528 Thank you @jeffolsi . I see. I guess what you are looking for is sightly different than the Python version. Yes, the SQL query can decide True/False but we still need to le

[GitHub] [airflow] samuelkhtu commented on issue #8525: SQLBranchOperator

2020-05-15 Thread GitBox
samuelkhtu commented on issue #8525: URL: https://github.com/apache/airflow/issues/8525#issuecomment-629226528 Thank you @jeffolsi . I see. I guess what you are looking for is sightly different than the Python version. Yes, the SQL query can decide True/False but we still need to let the o

[GitHub] [airflow] elwinarens commented on issue #8258: Implement Azure key vault secrets backend

2020-05-15 Thread GitBox
elwinarens commented on issue #8258: URL: https://github.com/apache/airflow/issues/8258#issuecomment-629216849 ``` from typing import Optional import re from azure.identity import DefaultAzureCredential from azure.keyvault.secrets import SecretClient from cached_property i

[GitHub] [airflow] elwinarens removed a comment on issue #8258: Implement Azure key vault secrets backend

2020-05-15 Thread GitBox
elwinarens removed a comment on issue #8258: URL: https://github.com/apache/airflow/issues/8258#issuecomment-629216849 ``` from typing import Optional import re from azure.identity import DefaultAzureCredential from azure.keyvault.secrets import SecretClient from cached_pr

[GitHub] [airflow] ashb commented on a change in pull request #8807: Added automated release notes generation for backport operators

2020-05-15 Thread GitBox
ashb commented on a change in pull request #8807: URL: https://github.com/apache/airflow/pull/8807#discussion_r425774528 ## File path: airflow/providers/apache/druid/README.md ## @@ -0,0 +1,125 @@ + + + +# Package apache-airflow-backport-providers-apache-druid + +Release: 2020.

[GitHub] [airflow] potiuk commented on pull request #8807: Added automated release notes generation for backport operators

2020-05-15 Thread GitBox
potiuk commented on pull request #8807: URL: https://github.com/apache/airflow/pull/8807#issuecomment-629210654 Just for @ashb @kaxil @feluelle @mik-laj : https://test.pypi.org/project/apache-airflow-backport-providers-google/2020.5.19rc1/ this is a test release in PyPI -

[GitHub] [airflow] potiuk commented on a change in pull request #8807: Added automated release notes generation for backport operators

2020-05-15 Thread GitBox
potiuk commented on a change in pull request #8807: URL: https://github.com/apache/airflow/pull/8807#discussion_r425769883 ## File path: airflow/providers/apache/druid/README.md ## @@ -0,0 +1,125 @@ + + + +# Package apache-airflow-backport-providers-apache-druid + +Release: 202

[GitHub] [airflow] potiuk commented on a change in pull request #8807: Added automated release notes generation for backport operators

2020-05-15 Thread GitBox
potiuk commented on a change in pull request #8807: URL: https://github.com/apache/airflow/pull/8807#discussion_r425769427 ## File path: airflow/providers/apache/druid/README.md ## @@ -0,0 +1,125 @@ + + + +# Package apache-airflow-backport-providers-apache-druid + +Release: 202

[GitHub] [airflow] jeffolsi commented on issue #8525: SQLBranchOperator

2020-05-15 Thread GitBox
jeffolsi commented on issue #8525: URL: https://github.com/apache/airflow/issues/8525#issuecomment-629204930 @samuelkhtu the query return only true / false. Think of the query as the equivalent of the python callable. The callable return only true false and with that the follow branch is

[GitHub] [airflow] zhongjiajie commented on a change in pull request #8828: Check for same task instead of Equality to detect Duplicate Tasks

2020-05-15 Thread GitBox
zhongjiajie commented on a change in pull request #8828: URL: https://github.com/apache/airflow/pull/8828#discussion_r425758194 ## File path: tests/models/test_dag.py ## @@ -979,34 +979,18 @@ def test_duplicate_task_ids_not_allowed_with_dag_context_manager(self): se

[GitHub] [airflow] potiuk commented on a change in pull request #8807: Added automated release notes generation for backport operators

2020-05-15 Thread GitBox
potiuk commented on a change in pull request #8807: URL: https://github.com/apache/airflow/pull/8807#discussion_r425756678 ## File path: airflow/providers/apache/cassandra/README.md ## @@ -0,0 +1,105 @@ + + + +# Package apache-airflow-backport-providers-apache-cassandra + +Rele

[GitHub] [airflow] kaxil commented on a change in pull request #8828: Check for same task instead of Equality to detect Duplicate Tasks

2020-05-15 Thread GitBox
kaxil commented on a change in pull request #8828: URL: https://github.com/apache/airflow/pull/8828#discussion_r425748387 ## File path: tests/models/test_dag.py ## @@ -979,34 +979,18 @@ def test_duplicate_task_ids_not_allowed_with_dag_context_manager(self): self.ass

[GitHub] [airflow] zhongjiajie commented on a change in pull request #8828: Check for same task instead of Equality to detect Duplicate Tasks

2020-05-15 Thread GitBox
zhongjiajie commented on a change in pull request #8828: URL: https://github.com/apache/airflow/pull/8828#discussion_r425743881 ## File path: tests/models/test_dag.py ## @@ -979,34 +979,18 @@ def test_duplicate_task_ids_not_allowed_with_dag_context_manager(self): se

[GitHub] [airflow] stale[bot] commented on pull request #6931: [AIRFLOW-6372] Align WASB remote logging URI scheme

2020-05-15 Thread GitBox
stale[bot] commented on pull request #6931: URL: https://github.com/apache/airflow/pull/6931#issuecomment-629182539 This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[jira] [Commented] (AIRFLOW-6372) Align Azure Blob Storage remote logging URI scheme with S3 and GCS

2020-05-15 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6372?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17108186#comment-17108186 ] ASF GitHub Bot commented on AIRFLOW-6372: - stale[bot] commented on pull request

[GitHub] [airflow] feluelle commented on issue #8770: Airflow 1.10.7 logs from S3 won't load, just hanging

2020-05-15 Thread GitBox
feluelle commented on issue #8770: URL: https://github.com/apache/airflow/issues/8770#issuecomment-629177661 You are not using RBAC, right? This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [airflow] feluelle edited a comment on issue #8770: Airflow 1.10.7 logs from S3 won't load, just hanging

2020-05-15 Thread GitBox
feluelle edited a comment on issue #8770: URL: https://github.com/apache/airflow/issues/8770#issuecomment-629176390 Can you try adding `"region":"your-s3-region"` to the extras and see if this helps? Or remove the quotes from `remote_log_conn_id = "S3Connection"` -> S3Connection. So `S3

[GitHub] [airflow] feluelle commented on issue #8770: Airflow 1.10.7 logs from S3 won't load, just hanging

2020-05-15 Thread GitBox
feluelle commented on issue #8770: URL: https://github.com/apache/airflow/issues/8770#issuecomment-629176390 Can you try adding `"region":"your-s3-region"` to the extras and see if this helps? Or remove the parenthesis from `remote_log_conn_id = "S3Connection"` -> S3Connection. So `S3Co

[GitHub] [airflow] ashb commented on pull request #8814: Spend less time waiting for DagFileProcessor processes to complete

2020-05-15 Thread GitBox
ashb commented on pull request #8814: URL: https://github.com/apache/airflow/pull/8814#issuecomment-629169838 Think I've finally worked out why the test_dags_system_exit was _sometimes_ failing now. I started the processes earlier so it's possible that it will have run and had results coll

[GitHub] [airflow] ashb commented on a change in pull request #8807: Added automated release notes generation for backport operators

2020-05-15 Thread GitBox
ashb commented on a change in pull request #8807: URL: https://github.com/apache/airflow/pull/8807#discussion_r425709206 ## File path: airflow/providers/apache/cassandra/README.md ## @@ -0,0 +1,105 @@ + + + +# Package apache-airflow-backport-providers-apache-cassandra + +Releas

  1   2   >