[airflow] branch master updated (5e8b537 -> b21c3b5)
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/airflow.git. from 5e8b537 Remove the ability to import operators and sensors from plugins (#12072) add b21c3b5 Fix typo in docker-context-files/README.md (#12078) No new revisions were added by this update. Summary of changes: docker-context-files/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
[GitHub] [airflow] potiuk merged pull request #12078: Fix typo in docker-context-files/README.md
potiuk merged pull request #12078: URL: https://github.com/apache/airflow/pull/12078 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [airflow] potiuk commented on issue #11965: Airflow fails to initdb with cattrs 1.1.0
potiuk commented on issue #11965: URL: https://github.com/apache/airflow/issues/11965#issuecomment-721572179 Yeah. Constraints file does not work if you already have higher version installed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [airflow] potiuk edited a comment on pull request #12080: Fixes problem with building a PROD image
potiuk edited a comment on pull request #12080: URL: https://github.com/apache/airflow/pull/12080#issuecomment-721571565 confirmed: https://github.com/potiuk/airflow/runs/1351642857?check_suite_focus=true - the right PROD image is built now in PROD . With the BUG, the CI image tag was was applied to PROD image instead of the PROD one. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [airflow] potiuk commented on pull request #12080: Fixes problem with building a PROD image
potiuk commented on pull request #12080: URL: https://github.com/apache/airflow/pull/12080#issuecomment-721571565 confirmed: https://github.com/potiuk/airflow/runs/1351642857?check_suite_focus=true - the right PROD image is built now in PROD . With the BUG, the CI image was being built instead of PROD one. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [airflow] potiuk commented on pull request #12077: Adding MySql howto-documentation and example DAG
potiuk commented on pull request #12077: URL: https://github.com/apache/airflow/pull/12077#issuecomment-721570717 Dont worry about the failing PROD build. It's being fixe d in master, but should have no effect for the review. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [airflow] potiuk edited a comment on pull request #12077: Adding MySql howto-documentation and example DAG
potiuk edited a comment on pull request #12077: URL: https://github.com/apache/airflow/pull/12077#issuecomment-721570717 Dont worry about the failing PROD build. It's being fixed in master, but should have no effect for the review. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [airflow] potiuk opened a new pull request #12080: Fixes problem with building a PROD image
potiuk opened a new pull request #12080: URL: https://github.com/apache/airflow/pull/12080 The change #12050 that aimed at automation of Docker images building in DockerHub had an undesired effect of overriding the production image tag with the CI one. This is fixed by this PR. --- **^ Add meaningful description above** Read the **[Pull Request Guidelines](https://github.com/apache/airflow/blob/master/CONTRIBUTING.rst#pull-request-guidelines)** for more information. In case of fundamental code change, Airflow Improvement Proposal ([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvements+Proposals)) is needed. In case of a new dependency, check compliance with the [ASF 3rd Party License Policy](https://www.apache.org/legal/resolved.html#category-x). In case of backwards incompatible changes please leave a note in [UPDATING.md](https://github.com/apache/airflow/blob/master/UPDATING.md). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [airflow] github-actions[bot] commented on pull request #10917: Fix race conditions in task callback invocations
github-actions[bot] commented on pull request #10917: URL: https://github.com/apache/airflow/pull/10917#issuecomment-721569962 [The Workflow run](https://github.com/apache/airflow/actions/runs/345079009) is cancelling this PR. It has some failed jobs matching ^Pylint$,^Static checks,^Build docs$,^Spell check docs$,^Backport packages$,^Provider packages,^Checks: Helm tests$,^Test OpenAPI*. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [airflow] akocukcu edited a comment on issue #10992: Transfer table/data from mssql to mssql
akocukcu edited a comment on issue #10992: URL: https://github.com/apache/airflow/issues/10992#issuecomment-721565457 @shizidushu Also since you are planning to use Hive, [`HiveToMySqlOperator`](https://github.com/apache/airflow/blob/master/airflow/providers/apache/hive/transfers/hive_to_mysql.py) is a very elegant example (Which can be adapted to use with MsSql). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [airflow] akocukcu commented on issue #10992: Transfer table/data from mssql to mssql
akocukcu commented on issue #10992: URL: https://github.com/apache/airflow/issues/10992#issuecomment-721565457 @shizidushu Also since you are planning to use Hive, `[HiveToMySqlOperator]`(https://github.com/apache/airflow/blob/master/airflow/providers/apache/hive/transfers/hive_to_mysql.py) is a very elegant example (Which can be adapted to use with MsSql). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [airflow] akocukcu commented on issue #10992: Transfer table/data from mssql to mssql
akocukcu commented on issue #10992: URL: https://github.com/apache/airflow/issues/10992#issuecomment-721562330 @shizidushu for your case again I kindly suggest using "Control Table" pattern with a Staging table. You can insert data directly (maybe bulk load) to **empty** Staging table. After that since staging table and datawarehouse table are in the same server, upsert operation will complete at lightning speed. BTW, I think in the same sense there are `preoperator` parameters inside a lot of transfer operators. For example if you look at the definition of `mysql_preoperator` parameter of [PrestoToMySqlOperator](https://github.com/apache/airflow/blob/fdd9b6f65b608c516b8a062b058972d9a45ec9e3/airflow/providers/mysql/transfers/presto_to_mysql.py), it says: > ["... typically use to truncate of delete in place of the data coming in ...".](https://github.com/apache/airflow/blob/fdd9b6f65b608c516b8a062b058972d9a45ec9e3/airflow/providers/mysql/transfers/presto_to_mysql.py#L42) So basically, first you can truncate the staging table, second insert the (only new or updated) oltp data to staging table, finally you can upsert staging data to datawarehouse table. Just like in the blog post, you can test that approach in your environment and data. Independent from your case I can add upsert functionality by adding `MERGE` to `MsSqlHook`. But I have concerns, if folks use it for upserting large data. Maybe I can add a comment just like they did in `GenericTransfer`: > ["... This is meant to be used on small-ish datasets ..."](https://github.com/apache/airflow/blob/4e8f9cc8d02b29c325b8a5a76b4837671bdf5f68/airflow/operators/generic_transfer.py#L32) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [airflow] potiuk commented on pull request #12077: Adding MySql howto-documentation and example DAG
potiuk commented on pull request #12077: URL: https://github.com/apache/airflow/pull/12077#issuecomment-721561528 I think simply the queue was long due to many rebased PRs. But it should be ok now. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [airflow] yuqian90 commented on pull request #11922: Extend the same keyword arguments callable support in PythonOperator to some other sensors/operators
yuqian90 commented on pull request #11922: URL: https://github.com/apache/airflow/pull/11922#issuecomment-721560299 > Can you please rebased your PR on latest Master since we have applied [Black](https://github.com/apache/airflow/commit/4e8f9cc8d02b29c325b8a5a76b4837671bdf5f68) and [PyUpgrade](https://github.com/apache/airflow/commit/8c42cf1b00c90f0d7f11b8a3a455381de8e003c5) on Master. > > It will help if your squash your commits into single commit first so that there are less conflicts. Thanks. I've rebased. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [airflow] feluelle commented on pull request #11747: S3ToFTPOperator
feluelle commented on pull request #11747: URL: https://github.com/apache/airflow/pull/11747#issuecomment-721557634 > Why don't use `NamedTemporaryFile` to avoid `local_file_path` ? @TobKed, I think you are right. That is better, because it takes more control over the existence of the temporarily file. It is important that this file will be deleted. In the current code (my previous suggestion) the file won't be deleted for example if the ftp_conn_id is missing. The `try`-`finally` is a bad decision if we can do it through context manager of NamedTemporaryFile already - we just cannot use `s3_hook.download_file` and instead must stick to using boto api (`s3_obj.download_fileobj`) directly. ```python s3_obj = self.get_key(key, bucket_name) with NamedTemporaryFile() as local_tmp_file: s3_obj.download_fileobj(local_tmp_file) ftp_hook.store_file(self.ftp_path, local_tmp_file.name) ``` WDYT @JavierLopezT ? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [airflow] pedresnyman commented on a change in pull request #11883: Feature/emrstepsensor
pedresnyman commented on a change in pull request #11883: URL: https://github.com/apache/airflow/pull/11883#discussion_r517137237 ## File path: airflow/providers/amazon/aws/sensors/emr_step.py ## @@ -21,24 +21,34 @@ from airflow.providers.amazon.aws.sensors.emr_base import EmrBaseSensor from airflow.utils.decorators import apply_defaults +from airflow.exceptions import AirflowException +from airflow.models.xcom import XCOM_RETURN_KEY + class EmrStepSensor(EmrBaseSensor): """ Asks for the state of the step until it reaches any of the target states. +If no step Id has been provided the sensor retrieves all steps and current states +and will continue to ask until all steps reaches any of the target states. Review comment: Hi @feluelle, The EmrJobFlowSensor can work yes. Can we add a xcom_push to the EmrJobFlowSensor so that we have a list of all steps? So then we have all the steps and the step states. With that functionality added you can always rerun a failed step instead or re-running all of the steps. Regards This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [airflow] shizidushu commented on issue #10992: Transfer table/data from mssql to mssql
shizidushu commented on issue #10992: URL: https://github.com/apache/airflow/issues/10992#issuecomment-721537142 @akocukcu Thank you for your post. I didn't realize there are problems with `MERGE` operation before. In our case, we have a data warehouse using Sql Server, there are many merge in the current code base (low performance). I was considering refactor the etl with airflow in september. Now our team decide to switch to airflow/hive/spark from scratch. And now I'm struggling with how to do incremental etl with hive (without merge, insert overwrite partition approach). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [airflow] yuqian90 commented on pull request #12058: Treat AirflowSensorTimeout as immediate failure without retrying
yuqian90 commented on pull request #12058: URL: https://github.com/apache/airflow/pull/12058#issuecomment-721526066 This test failure looks unrelated to my change: CI Build / Quarantined tests (mysql) (pull_request) Failing after 4m — Quarantined tests (mysql) ``` tests/cli/commands/test_task_command.py Error: Process completed with exit code 137. ``` This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [airflow] wangze999999 commented on pull request #11997: Modified ajax call for external links
wangze99 commented on pull request #11997: URL: https://github.com/apache/airflow/pull/11997#issuecomment-721522871 unsubscribe This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [airflow] github-actions[bot] commented on pull request #11997: Modified ajax call for external links
github-actions[bot] commented on pull request #11997: URL: https://github.com/apache/airflow/pull/11997#issuecomment-721522161 [The Workflow run](https://github.com/apache/airflow/actions/runs/344924603) is cancelling this PR. It has some failed jobs matching ^Pylint$,^Static checks,^Build docs$,^Spell check docs$,^Backport packages$,^Provider packages,^Checks: Helm tests$,^Test OpenAPI*. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [airflow] mik-laj opened a new pull request #12079: Add authentication to AWS with Google credentials
mik-laj opened a new pull request #12079: URL: https://github.com/apache/airflow/pull/12079 Close: https://github.com/apache/airflow/issues/9873 CC: @vladiceanu --- **^ Add meaningful description above** Read the **[Pull Request Guidelines](https://github.com/apache/airflow/blob/master/CONTRIBUTING.rst#pull-request-guidelines)** for more information. In case of fundamental code change, Airflow Improvement Proposal ([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvements+Proposals)) is needed. In case of a new dependency, check compliance with the [ASF 3rd Party License Policy](https://www.apache.org/legal/resolved.html#category-x). In case of backwards incompatible changes please leave a note in [UPDATING.md](https://github.com/apache/airflow/blob/master/UPDATING.md). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [airflow] ashmeet13 commented on a change in pull request #11241: Create Undefined Jinja Variables Rule
ashmeet13 commented on a change in pull request #11241:
URL: https://github.com/apache/airflow/pull/11241#discussion_r517095762
##
File path: airflow/upgrade/rules/undefined_jinja_varaibles.py
##
@@ -0,0 +1,121 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+from __future__ import absolute_import
+
+import re
+
+import jinja2
+import six
+
+from airflow import conf
+from airflow.models import DagBag, TaskInstance
+from airflow.upgrade.rules.base_rule import BaseRule
+from airflow.utils import timezone
+
+
+class UndefinedJinjaVariablesRule(BaseRule):
+
+title = "Jinja Template Variables cannot be undefined"
+
+description = """\
+Jinja Templates have been updated to the following rule -
jinja2.StrictUndefined
+With this change a task will fail if it recieves any undefined variables.
+"""
+
+def _check_rendered_content(self, rendered_content):
+"""Replicates the logic in BaseOperator.render_template() to
+cover all the cases needed to be checked.
+"""
+if isinstance(rendered_content, six.string_types):
+return set(re.findall(r"{{(.*?)}}", rendered_content))
+
+elif isinstance(rendered_content, (tuple, list, set)):
+debug_error_messages = set()
+for element in rendered_content:
+
debug_error_messages.union(self._check_rendered_content(element))
+return debug_error_messages
+
+elif isinstance(rendered_content, dict):
+debug_error_messages = set()
+for key, value in rendered_content.items():
+
debug_error_messages.union(self._check_rendered_content(str(value)))
+return debug_error_messages
+
+def _render_task_content(self, task, content, context):
+completed_rendering = False
+errors_while_rendering = []
+while not completed_rendering:
+# Catch errors such as {{ object.element }} where
+# object is not defined
+try:
+renderend_content = task.render_template(content, context)
+completed_rendering = True
+except Exception as e:
+undefined_variable = re.sub(" is undefined", "", str(e))
+undefined_variable = re.sub("'", "", undefined_variable)
+context[undefined_variable] = dict()
+message = "Could not find the object
'{}'".format(undefined_variable)
+errors_while_rendering.append(message)
+return renderend_content, errors_while_rendering
+
+def _task_level_(self, task):
+messages = {}
+task_instance = TaskInstance(task=task,
execution_date=timezone.utcnow())
+context = task_instance.get_template_context()
+for attr_name in task.template_fields:
+content = getattr(task, attr_name)
+if content:
+rendered_content, errors_while_rendering =
self._render_task_content(
+task, content, context
+)
+debug_error_messages = list(
+self._check_rendered_content(rendered_content)
+)
+messages[attr_name] = errors_while_rendering +
debug_error_messages
+
+return messages
+
+def _dag_level_(self, dag):
+dag.template_undefined = jinja2.DebugUndefined
+tasks = dag.tasks
+messages = {}
+for task in tasks:
+error_messages = self._task_level_(task)
+messages[task.task_id] = error_messages
+return messages
+
+def check(self, dagbag=None):
+if not dagbag:
+dag_folder = conf.get("core", "dags_folder")
+dagbag = DagBag(dag_folder)
+dags = dagbag.dags
+messages = []
+for dag_id, dag in dags.items():
+dag_messages = self._dag_level_(dag)
Review comment:
Got it. Working on the changes.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[GitHub] [airflow] Google987 commented on pull request #11997: Modified ajax call for external links
Google987 commented on pull request #11997: URL: https://github.com/apache/airflow/pull/11997#issuecomment-721509793 Hi @mjpieters, Thanks for the help so far. I pulled the lastest master and merged it with my branch. there was a conflict. resolved that and now views.py is showing a lot of changes. Does it look right? isn't it supposed to show only my changes? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [airflow] yuqian90 commented on pull request #12058: Treat AirflowSensorTimeout as immediate failure without retrying
yuqian90 commented on pull request #12058: URL: https://github.com/apache/airflow/pull/12058#issuecomment-721496827 > I have rebased your PR on latest Master since we have applied [Black](https://github.com/apache/airflow/commit/4e8f9cc8d02b29c325b8a5a76b4837671bdf5f68) and [PyUpgrade](https://github.com/apache/airflow/commit/8c42cf1b00c90f0d7f11b8a3a455381de8e003c5) on Master Thanks! This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [airflow] github-actions[bot] commented on pull request #11964: Add new datetime branch operator
github-actions[bot] commented on pull request #11964: URL: https://github.com/apache/airflow/pull/11964#issuecomment-721496197 [The Workflow run](https://github.com/apache/airflow/actions/runs/344817513) is cancelling this PR. It has some failed jobs matching ^Pylint$,^Static checks,^Build docs$,^Spell check docs$,^Backport packages$,^Provider packages,^Checks: Helm tests$,^Test OpenAPI*. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [airflow] tomasfarias commented on pull request #11964: Add new datetime branch operator
tomasfarias commented on pull request #11964: URL: https://github.com/apache/airflow/pull/11964#issuecomment-721487330 @eladkal I agree, any unit smaller than hour is probably not something you'd be able to match reliably anyways 😓. I like your idea, and I've also allowed for either the upper bound or the lower bound to be set as `None`, but not both, to allow for unilateral comparisons, for example: if `target_upper` is `None` and the current `datetime` falls above `target_lower` then return true branch. This also reduces the number of arguments, which was getting unnecessarily out of control as @mik-laj suggested. Branch has been rebased as @kaxil asked. I'm was having a bit of trouble getting the docs build to pass: now docs build passes locally, so I'm expecting it to pass with the latest commit. Thanks all for reviews and input 💪 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [airflow] github-actions[bot] commented on pull request #11964: Add new datetime branch operator
github-actions[bot] commented on pull request #11964: URL: https://github.com/apache/airflow/pull/11964#issuecomment-721474221 [The Workflow run](https://github.com/apache/airflow/actions/runs/344729593) is cancelling this PR. It has some failed jobs matching ^Pylint$,^Static checks,^Build docs$,^Spell check docs$,^Backport packages$,^Provider packages,^Checks: Helm tests$,^Test OpenAPI*. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [airflow] jfmolano commented on pull request #12077: Adding MySql howto-documentation and example DAG
jfmolano commented on pull request #12077: URL: https://github.com/apache/airflow/pull/12077#issuecomment-721470692 Dear @kaxil I rebased my repo as you told me but the checking process is still stuck at CI Build / Wait for PROD images (approx 2 hours). What should I do now? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [airflow] github-actions[bot] commented on pull request #12075: Avoid unnecessary IF checks when generate Duration & Landing Time views
github-actions[bot] commented on pull request #12075: URL: https://github.com/apache/airflow/pull/12075#issuecomment-721468750 [The Workflow run](https://github.com/apache/airflow/actions/runs/344705012) is cancelling this PR. It has some failed jobs matching ^Pylint$,^Static checks,^Build docs$,^Spell check docs$,^Backport packages$,^Provider packages,^Checks: Helm tests$,^Test OpenAPI*. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Error while running github feature from .asf.yaml in airflow!
An error occurred while running github feature in .asf.yaml!: [GitHub] Request error with message: "None". (status code: 200)
[airflow] branch master updated: Remove the ability to import operators and sensors from plugins (#12072)
This is an automated email from the ASF dual-hosted git repository.
kaxilnaik pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/airflow.git
The following commit(s) were added to refs/heads/master by this push:
new 5e8b537 Remove the ability to import operators and sensors from
plugins (#12072)
5e8b537 is described below
commit 5e8b537b85ce6aa355966df08df5c1846adb4cd6
Author: Ash Berlin-Taylor
AuthorDate: Wed Nov 4 01:17:18 2020 +
Remove the ability to import operators and sensors from plugins (#12072)
We have deprecated this in #12069 (for inclusion in 1.10.13) and the
docs http://airflow.apache.org/docs/stable/howto/custom-operator.html
already show how to do this without a plugin.
Closes #9498
---
UPDATING.md| 20 +++
airflow/cli/commands/plugins_command.py| 4 ---
airflow/plugins_manager.py | 25 +
airflow/www/views.py | 2 --
docs/howto/custom-operator.rst | 2 +-
docs/plugins.rst | 56 +-
tests/cli/commands/test_plugins_command.py | 1 -
tests/plugins/test_plugins_manager.py | 16 +++--
tests/test_utils/mock_plugins.py | 2 --
9 files changed, 35 insertions(+), 93 deletions(-)
diff --git a/UPDATING.md b/UPDATING.md
index f958831..d84a93a 100644
--- a/UPDATING.md
+++ b/UPDATING.md
@@ -50,6 +50,26 @@ assists users migrating to a new version.
## Airflow Master
+### Adding Operators and Sensors via plugins is no longer supported
+
+Operators and Sensors should no longer be registered or imported via Airflow's
plugin mechanism -- these types of classes are just treated as plain python
classes by Airflow, so there is no need to register them with Airflow.
+
+If you previously had a `plugins/my_plugin.py` and you used it like this in a
DAG:
+
+```
+from airflow.operators.my_plugin import MyOperator
+```
+
+You should instead import it as:
+
+```
+from my_plugin import MyOperator
+```
+
+The name under `airflow.operators.` was the plugin name, where as in the
second example it is the python module name where the operator is defined.
+
+See
http://airflow.apache.org/docs/stable/howto/custom-operator.html#define-an-operator-extra-link
for more info.
+
### The default value for `[core] enable_xcom_pickling` has been changed to
`False`
The pickle type for XCom messages has been replaced to JSON by default to
prevent RCE attacks.
diff --git a/airflow/cli/commands/plugins_command.py
b/airflow/cli/commands/plugins_command.py
index 40d12fe..785b8ed 100644
--- a/airflow/cli/commands/plugins_command.py
+++ b/airflow/cli/commands/plugins_command.py
@@ -25,8 +25,6 @@ from airflow import plugins_manager
PLUGINS_MANAGER_ATTRIBUTES_TO_DUMP = [
"plugins",
"import_errors",
-"operators_modules",
-"sensors_modules",
"hooks_modules",
"macros_modules",
"executors_modules",
@@ -39,8 +37,6 @@ PLUGINS_MANAGER_ATTRIBUTES_TO_DUMP = [
]
# list to maintain the order of items.
PLUGINS_ATTRIBUTES_TO_DUMP = [
-"operators",
-"sensors",
"hooks",
"executors",
"macros",
diff --git a/airflow/plugins_manager.py b/airflow/plugins_manager.py
index 99c05b9..8958742 100644
--- a/airflow/plugins_manager.py
+++ b/airflow/plugins_manager.py
@@ -38,8 +38,6 @@ import_errors: Dict[str, str] = {}
plugins = None # type: Optional[List[AirflowPlugin]]
# Plugin components to integrate as modules
-operators_modules: Optional[List[Any]] = None
-sensors_modules: Optional[List[Any]] = None
hooks_modules: Optional[List[Any]] = None
macros_modules: Optional[List[Any]] = None
executors_modules: Optional[List[Any]] = None
@@ -106,8 +104,6 @@ class AirflowPlugin:
name: Optional[str] = None
source: Optional[AirflowPluginSource] = None
-operators: List[Any] = []
-sensors: List[Any] = []
hooks: List[Any] = []
executors: List[Any] = []
macros: List[Any] = []
@@ -382,18 +378,11 @@ def integrate_dag_plugins() -> None:
"""Integrates operator, sensor, hook, macro plugins."""
# pylint: disable=global-statement
global plugins
-global operators_modules
-global sensors_modules
global hooks_modules
global macros_modules
# pylint: enable=global-statement
-if (
-operators_modules is not None
-and sensors_modules is not None
-and hooks_modules is not None
-and macros_modules is not None
-):
+if hooks_modules is not None and macros_modules is not None:
return
ensure_plugins_loaded()
@@ -403,8 +392,6 @@ def integrate_dag_plugins() -> None:
log.debug("Integrate DAG plugins")
-operators_modules = []
-sensors_modules = []
hooks_modules = []
macros_modules = []
@@ -412,19 +399,9 @@ def integrate_dag_plugins() -> None:
if plugin.name is None:
raise AirflowPluginEx
Error while running github feature from .asf.yaml in airflow!
An error occurred while running github feature in .asf.yaml!: [GitHub] Request error with message: "None". (status code: 200)
[GitHub] [airflow] github-actions[bot] commented on pull request #11964: Add new datetime branch operator
github-actions[bot] commented on pull request #11964: URL: https://github.com/apache/airflow/pull/11964#issuecomment-721458558 [The Workflow run](https://github.com/apache/airflow/actions/runs/34465) is cancelling this PR. It has some failed jobs matching ^Pylint$,^Static checks,^Build docs$,^Spell check docs$,^Backport packages$,^Provider packages,^Checks: Helm tests$,^Test OpenAPI*. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[airflow] branch master updated: Remove the ability to import operators and sensors from plugins (#12072)
This is an automated email from the ASF dual-hosted git repository.
kaxilnaik pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/airflow.git
The following commit(s) were added to refs/heads/master by this push:
new 5e8b537 Remove the ability to import operators and sensors from
plugins (#12072)
5e8b537 is described below
commit 5e8b537b85ce6aa355966df08df5c1846adb4cd6
Author: Ash Berlin-Taylor
AuthorDate: Wed Nov 4 01:17:18 2020 +
Remove the ability to import operators and sensors from plugins (#12072)
We have deprecated this in #12069 (for inclusion in 1.10.13) and the
docs http://airflow.apache.org/docs/stable/howto/custom-operator.html
already show how to do this without a plugin.
Closes #9498
---
UPDATING.md| 20 +++
airflow/cli/commands/plugins_command.py| 4 ---
airflow/plugins_manager.py | 25 +
airflow/www/views.py | 2 --
docs/howto/custom-operator.rst | 2 +-
docs/plugins.rst | 56 +-
tests/cli/commands/test_plugins_command.py | 1 -
tests/plugins/test_plugins_manager.py | 16 +++--
tests/test_utils/mock_plugins.py | 2 --
9 files changed, 35 insertions(+), 93 deletions(-)
diff --git a/UPDATING.md b/UPDATING.md
index f958831..d84a93a 100644
--- a/UPDATING.md
+++ b/UPDATING.md
@@ -50,6 +50,26 @@ assists users migrating to a new version.
## Airflow Master
+### Adding Operators and Sensors via plugins is no longer supported
+
+Operators and Sensors should no longer be registered or imported via Airflow's
plugin mechanism -- these types of classes are just treated as plain python
classes by Airflow, so there is no need to register them with Airflow.
+
+If you previously had a `plugins/my_plugin.py` and you used it like this in a
DAG:
+
+```
+from airflow.operators.my_plugin import MyOperator
+```
+
+You should instead import it as:
+
+```
+from my_plugin import MyOperator
+```
+
+The name under `airflow.operators.` was the plugin name, where as in the
second example it is the python module name where the operator is defined.
+
+See
http://airflow.apache.org/docs/stable/howto/custom-operator.html#define-an-operator-extra-link
for more info.
+
### The default value for `[core] enable_xcom_pickling` has been changed to
`False`
The pickle type for XCom messages has been replaced to JSON by default to
prevent RCE attacks.
diff --git a/airflow/cli/commands/plugins_command.py
b/airflow/cli/commands/plugins_command.py
index 40d12fe..785b8ed 100644
--- a/airflow/cli/commands/plugins_command.py
+++ b/airflow/cli/commands/plugins_command.py
@@ -25,8 +25,6 @@ from airflow import plugins_manager
PLUGINS_MANAGER_ATTRIBUTES_TO_DUMP = [
"plugins",
"import_errors",
-"operators_modules",
-"sensors_modules",
"hooks_modules",
"macros_modules",
"executors_modules",
@@ -39,8 +37,6 @@ PLUGINS_MANAGER_ATTRIBUTES_TO_DUMP = [
]
# list to maintain the order of items.
PLUGINS_ATTRIBUTES_TO_DUMP = [
-"operators",
-"sensors",
"hooks",
"executors",
"macros",
diff --git a/airflow/plugins_manager.py b/airflow/plugins_manager.py
index 99c05b9..8958742 100644
--- a/airflow/plugins_manager.py
+++ b/airflow/plugins_manager.py
@@ -38,8 +38,6 @@ import_errors: Dict[str, str] = {}
plugins = None # type: Optional[List[AirflowPlugin]]
# Plugin components to integrate as modules
-operators_modules: Optional[List[Any]] = None
-sensors_modules: Optional[List[Any]] = None
hooks_modules: Optional[List[Any]] = None
macros_modules: Optional[List[Any]] = None
executors_modules: Optional[List[Any]] = None
@@ -106,8 +104,6 @@ class AirflowPlugin:
name: Optional[str] = None
source: Optional[AirflowPluginSource] = None
-operators: List[Any] = []
-sensors: List[Any] = []
hooks: List[Any] = []
executors: List[Any] = []
macros: List[Any] = []
@@ -382,18 +378,11 @@ def integrate_dag_plugins() -> None:
"""Integrates operator, sensor, hook, macro plugins."""
# pylint: disable=global-statement
global plugins
-global operators_modules
-global sensors_modules
global hooks_modules
global macros_modules
# pylint: enable=global-statement
-if (
-operators_modules is not None
-and sensors_modules is not None
-and hooks_modules is not None
-and macros_modules is not None
-):
+if hooks_modules is not None and macros_modules is not None:
return
ensure_plugins_loaded()
@@ -403,8 +392,6 @@ def integrate_dag_plugins() -> None:
log.debug("Integrate DAG plugins")
-operators_modules = []
-sensors_modules = []
hooks_modules = []
macros_modules = []
@@ -412,19 +399,9 @@ def integrate_dag_plugins() -> None:
if plugin.name is None:
raise AirflowPluginEx
[GitHub] [airflow] kaxil merged pull request #12072: Remove the ability to import operators and sensors from plugins
kaxil merged pull request #12072: URL: https://github.com/apache/airflow/pull/12072 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [airflow] kaxil closed issue #9498: Remove adding Operators via plugins in 2.0
kaxil closed issue #9498: URL: https://github.com/apache/airflow/issues/9498 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [airflow] kaxil closed issue #11543: Scheduler Deadlock in tests for MySQL 5.7
kaxil closed issue #11543: URL: https://github.com/apache/airflow/issues/11543 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [airflow] kaxil opened a new pull request #12078: Fix typo in docker-context-files/README.md
kaxil opened a new pull request #12078: URL: https://github.com/apache/airflow/pull/12078 `par` -> `part` --- **^ Add meaningful description above** Read the **[Pull Request Guidelines](https://github.com/apache/airflow/blob/master/CONTRIBUTING.rst#pull-request-guidelines)** for more information. In case of fundamental code change, Airflow Improvement Proposal ([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvements+Proposals)) is needed. In case of a new dependency, check compliance with the [ASF 3rd Party License Policy](https://www.apache.org/legal/resolved.html#category-x). In case of backwards incompatible changes please leave a note in [UPDATING.md](https://github.com/apache/airflow/blob/master/UPDATING.md). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [airflow] github-actions[bot] commented on pull request #11784: Add pod_template_override to executor_config
github-actions[bot] commented on pull request #11784: URL: https://github.com/apache/airflow/pull/11784#issuecomment-721453771 [The Workflow run](https://github.com/apache/airflow/actions/runs/344642280) is cancelling this PR. It has some failed jobs matching ^Pylint$,^Static checks,^Build docs$,^Spell check docs$,^Backport packages$,^Provider packages,^Checks: Helm tests$,^Test OpenAPI*. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [airflow] github-actions[bot] commented on pull request #12075: Avoid unnecessary IF checks when generate Duration & Landing Time views
github-actions[bot] commented on pull request #12075: URL: https://github.com/apache/airflow/pull/12075#issuecomment-721453776 [The Workflow run](https://github.com/apache/airflow/actions/runs/344642280) is cancelling this PR. It has some failed jobs matching ^Pylint$,^Static checks,^Build docs$,^Spell check docs$,^Backport packages$,^Provider packages,^Checks: Helm tests$,^Test OpenAPI*. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [airflow] kaxil commented on pull request #11213: Created initial guide for Apache Sqoop
kaxil commented on pull request #11213: URL: https://github.com/apache/airflow/pull/11213#issuecomment-721446764 Can you please rebase your PR on latest Master since we have applied [Black](https://github.com/apache/airflow/commit/4e8f9cc8d02b29c325b8a5a76b4837671bdf5f68) and [PyUpgrade](https://github.com/apache/airflow/commit/8c42cf1b00c90f0d7f11b8a3a455381de8e003c5) on Master. It will help if your squash your commits into single commit first so that there are less conflicts. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [airflow] kaxil commented on pull request #11156: Changes to enhance charts with Legend Search functionality
kaxil commented on pull request #11156: URL: https://github.com/apache/airflow/pull/11156#issuecomment-721446923 Can you please rebase your PR on latest Master since we have applied [Black](https://github.com/apache/airflow/commit/4e8f9cc8d02b29c325b8a5a76b4837671bdf5f68) and [PyUpgrade](https://github.com/apache/airflow/commit/8c42cf1b00c90f0d7f11b8a3a455381de8e003c5) on Master. It will help if your squash your commits into single commit first so that there are less conflicts. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [airflow] kaxil commented on pull request #11215: Initial guide for Apache Hive software operators
kaxil commented on pull request #11215: URL: https://github.com/apache/airflow/pull/11215#issuecomment-721446688 Can you please rebase your PR on latest Master since we have applied [Black](https://github.com/apache/airflow/commit/4e8f9cc8d02b29c325b8a5a76b4837671bdf5f68) and [PyUpgrade](https://github.com/apache/airflow/commit/8c42cf1b00c90f0d7f11b8a3a455381de8e003c5) on Master. It will help if your squash your commits into single commit first so that there are less conflicts. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [airflow] kaxil commented on pull request #11199: spark-on-k8s sensor logs - properly pass defined namespace to pod log call
kaxil commented on pull request #11199: URL: https://github.com/apache/airflow/pull/11199#issuecomment-721446845 Can you please rebase your PR on latest Master since we have applied [Black](https://github.com/apache/airflow/commit/4e8f9cc8d02b29c325b8a5a76b4837671bdf5f68) and [PyUpgrade](https://github.com/apache/airflow/commit/8c42cf1b00c90f0d7f11b8a3a455381de8e003c5) on Master. It will help if your squash your commits into single commit first so that there are less conflicts. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [airflow] kaxil commented on pull request #11121: Filter dags by owner
kaxil commented on pull request #11121: URL: https://github.com/apache/airflow/pull/11121#issuecomment-721446887 Can you please rebase your PR on latest Master since we have applied [Black](https://github.com/apache/airflow/commit/4e8f9cc8d02b29c325b8a5a76b4837671bdf5f68) and [PyUpgrade](https://github.com/apache/airflow/commit/8c42cf1b00c90f0d7f11b8a3a455381de8e003c5) on Master. It will help if your squash your commits into single commit first so that there are less conflicts. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [airflow] kaxil commented on pull request #11212: Created initial guide for HDFS operators
kaxil commented on pull request #11212: URL: https://github.com/apache/airflow/pull/11212#issuecomment-721446792 Can you please rebase your PR on latest Master since we have applied [Black](https://github.com/apache/airflow/commit/4e8f9cc8d02b29c325b8a5a76b4837671bdf5f68) and [PyUpgrade](https://github.com/apache/airflow/commit/8c42cf1b00c90f0d7f11b8a3a455381de8e003c5) on Master. It will help if your squash your commits into single commit first so that there are less conflicts. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [airflow] kaxil commented on pull request #11015: Add Azure Data Factory hook
kaxil commented on pull request #11015: URL: https://github.com/apache/airflow/pull/11015#issuecomment-721446950 Can you please rebase your PR on latest Master since we have applied [Black](https://github.com/apache/airflow/commit/4e8f9cc8d02b29c325b8a5a76b4837671bdf5f68) and [PyUpgrade](https://github.com/apache/airflow/commit/8c42cf1b00c90f0d7f11b8a3a455381de8e003c5) on Master. It will help if your squash your commits into single commit first so that there are less conflicts. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [airflow] kaxil commented on pull request #11214: ClickHouse operator
kaxil commented on pull request #11214: URL: https://github.com/apache/airflow/pull/11214#issuecomment-721446735 Can you please rebase your PR on latest Master since we have applied [Black](https://github.com/apache/airflow/commit/4e8f9cc8d02b29c325b8a5a76b4837671bdf5f68) and [PyUpgrade](https://github.com/apache/airflow/commit/8c42cf1b00c90f0d7f11b8a3a455381de8e003c5) on Master. It will help if your squash your commits into single commit first so that there are less conflicts. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [airflow] github-actions[bot] commented on pull request #12057: Correct failure message in sql_sensor.py.
github-actions[bot] commented on pull request #12057: URL: https://github.com/apache/airflow/pull/12057#issuecomment-721446566 [The Workflow run](https://github.com/apache/airflow/actions/runs/344599098) is cancelling this PR. It has some failed jobs matching ^Pylint$,^Static checks,^Build docs$,^Spell check docs$,^Backport packages$,^Provider packages,^Checks: Helm tests$,^Test OpenAPI*. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [airflow] github-actions[bot] commented on pull request #12071: Add session_parameters option to snowflake_hook
github-actions[bot] commented on pull request #12071: URL: https://github.com/apache/airflow/pull/12071#issuecomment-721446573 [The Workflow run](https://github.com/apache/airflow/actions/runs/344599098) is cancelling this PR. It has some failed jobs matching ^Pylint$,^Static checks,^Build docs$,^Spell check docs$,^Backport packages$,^Provider packages,^Checks: Helm tests$,^Test OpenAPI*. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [airflow] kaxil commented on pull request #11227: includes the STS token if STS credentials are used
kaxil commented on pull request #11227: URL: https://github.com/apache/airflow/pull/11227#issuecomment-721446646 Can you please rebase your PR on latest Master since we have applied [Black](https://github.com/apache/airflow/commit/4e8f9cc8d02b29c325b8a5a76b4837671bdf5f68) and [PyUpgrade](https://github.com/apache/airflow/commit/8c42cf1b00c90f0d7f11b8a3a455381de8e003c5) on Master. It will help if your squash your commits into single commit first so that there are less conflicts. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [airflow] kaxil commented on pull request #11220: introduce get_partition_by_name within hive hook
kaxil commented on pull request #11220: URL: https://github.com/apache/airflow/pull/11220#issuecomment-721446669 Can you please rebase your PR on latest Master since we have applied [Black](https://github.com/apache/airflow/commit/4e8f9cc8d02b29c325b8a5a76b4837671bdf5f68) and [PyUpgrade](https://github.com/apache/airflow/commit/8c42cf1b00c90f0d7f11b8a3a455381de8e003c5) on Master. It will help if your squash your commits into single commit first so that there are less conflicts. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [airflow] kaxil commented on pull request #11260: Add DAG permissions based on DAG tags
kaxil commented on pull request #11260: URL: https://github.com/apache/airflow/pull/11260#issuecomment-721446394 Can you please rebase your PR on latest Master since we have applied [Black](https://github.com/apache/airflow/commit/4e8f9cc8d02b29c325b8a5a76b4837671bdf5f68) and [PyUpgrade](https://github.com/apache/airflow/commit/8c42cf1b00c90f0d7f11b8a3a455381de8e003c5) on Master. It will help if your squash your commits into single commit first so that there are less conflicts. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [airflow] kaxil commented on pull request #11316: Re-use paramiko connection in SSHHook instead of recreating
kaxil commented on pull request #11316: URL: https://github.com/apache/airflow/pull/11316#issuecomment-721446266 Can you please rebase your PR on latest Master since we have applied [Black](https://github.com/apache/airflow/commit/4e8f9cc8d02b29c325b8a5a76b4837671bdf5f68) and [PyUpgrade](https://github.com/apache/airflow/commit/8c42cf1b00c90f0d7f11b8a3a455381de8e003c5) on Master. It will help if your squash your commits into single commit first so that there are less conflicts. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [airflow] kaxil commented on pull request #11293: Move dummy_operator.py to dummy.py (#11178)
kaxil commented on pull request #11293: URL: https://github.com/apache/airflow/pull/11293#issuecomment-721446314 Can you please rebase your PR on latest Master since we have applied [Black](https://github.com/apache/airflow/commit/4e8f9cc8d02b29c325b8a5a76b4837671bdf5f68) and [PyUpgrade](https://github.com/apache/airflow/commit/8c42cf1b00c90f0d7f11b8a3a455381de8e003c5) on Master. It will help if your squash your commits into single commit first so that there are less conflicts. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [airflow] kaxil commented on pull request #11307: Moved subdag_operator.py to subdag.py (#11178)
kaxil commented on pull request #11307: URL: https://github.com/apache/airflow/pull/11307#issuecomment-721446281 Can you please rebase your PR on latest Master since we have applied [Black](https://github.com/apache/airflow/commit/4e8f9cc8d02b29c325b8a5a76b4837671bdf5f68) and [PyUpgrade](https://github.com/apache/airflow/commit/8c42cf1b00c90f0d7f11b8a3a455381de8e003c5) on Master. It will help if your squash your commits into single commit first so that there are less conflicts. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [airflow] kaxil commented on pull request #11332: SnowflakeToS3Operator
kaxil commented on pull request #11332: URL: https://github.com/apache/airflow/pull/11332#issuecomment-721446212 Can you please rebase your PR on latest Master since we have applied [Black](https://github.com/apache/airflow/commit/4e8f9cc8d02b29c325b8a5a76b4837671bdf5f68) and [PyUpgrade](https://github.com/apache/airflow/commit/8c42cf1b00c90f0d7f11b8a3a455381de8e003c5) on Master. It will help if your squash your commits into single commit first so that there are less conflicts. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [airflow] kaxil commented on pull request #11277: Move dagrun_operator.py to dagrun.py (#11178)
kaxil commented on pull request #11277: URL: https://github.com/apache/airflow/pull/11277#issuecomment-721446368 Can you please rebase your PR on latest Master since we have applied [Black](https://github.com/apache/airflow/commit/4e8f9cc8d02b29c325b8a5a76b4837671bdf5f68) and [PyUpgrade](https://github.com/apache/airflow/commit/8c42cf1b00c90f0d7f11b8a3a455381de8e003c5) on Master. It will help if your squash your commits into single commit first so that there are less conflicts. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [airflow] kaxil commented on pull request #11392: Change some warning logs to info/debug
kaxil commented on pull request #11392: URL: https://github.com/apache/airflow/pull/11392#issuecomment-721446146 Can you please rebase your PR on latest Master since we have applied [Black](https://github.com/apache/airflow/commit/4e8f9cc8d02b29c325b8a5a76b4837671bdf5f68) and [PyUpgrade](https://github.com/apache/airflow/commit/8c42cf1b00c90f0d7f11b8a3a455381de8e003c5) on Master. It will help if your squash your commits into single commit first so that there are less conflicts. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [airflow] kaxil commented on pull request #11350: Allow execution of multiple sql statements in SnowflakeHook
kaxil commented on pull request #11350: URL: https://github.com/apache/airflow/pull/11350#issuecomment-721446180 Can you please rebase your PR on latest Master since we have applied [Black](https://github.com/apache/airflow/commit/4e8f9cc8d02b29c325b8a5a76b4837671bdf5f68) and [PyUpgrade](https://github.com/apache/airflow/commit/8c42cf1b00c90f0d7f11b8a3a455381de8e003c5) on Master. It will help if your squash your commits into single commit first so that there are less conflicts. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [airflow] kaxil commented on pull request #11284: Improve GCSToSFTPOperator paths handling
kaxil commented on pull request #11284: URL: https://github.com/apache/airflow/pull/11284#issuecomment-721446338 Can you please rebase your PR on latest Master since we have applied [Black](https://github.com/apache/airflow/commit/4e8f9cc8d02b29c325b8a5a76b4837671bdf5f68) and [PyUpgrade](https://github.com/apache/airflow/commit/8c42cf1b00c90f0d7f11b8a3a455381de8e003c5) on Master. It will help if your squash your commits into single commit first so that there are less conflicts. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [airflow] kaxil commented on pull request #11318: Add Vertica to GCS cloud transfer operator
kaxil commented on pull request #11318: URL: https://github.com/apache/airflow/pull/11318#issuecomment-721446242 Can you please rebase your PR on latest Master since we have applied [Black](https://github.com/apache/airflow/commit/4e8f9cc8d02b29c325b8a5a76b4837671bdf5f68) and [PyUpgrade](https://github.com/apache/airflow/commit/8c42cf1b00c90f0d7f11b8a3a455381de8e003c5) on Master. It will help if your squash your commits into single commit first so that there are less conflicts. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [airflow] kaxil commented on pull request #11410: Optionally use iam role for redshift unload to s3.
kaxil commented on pull request #11410: URL: https://github.com/apache/airflow/pull/11410#issuecomment-721446110 Can you please rebase your PR on latest Master since we have applied [Black](https://github.com/apache/airflow/commit/4e8f9cc8d02b29c325b8a5a76b4837671bdf5f68) and [PyUpgrade](https://github.com/apache/airflow/commit/8c42cf1b00c90f0d7f11b8a3a455381de8e003c5) on Master. It will help if your squash your commits into single commit first so that there are less conflicts. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [airflow] kaxil commented on pull request #11452: Enable Pydocstyle D200 check
kaxil commented on pull request #11452: URL: https://github.com/apache/airflow/pull/11452#issuecomment-721446061 Can you please rebase your PR on latest Master since we have applied [Black](https://github.com/apache/airflow/commit/4e8f9cc8d02b29c325b8a5a76b4837671bdf5f68) and [PyUpgrade](https://github.com/apache/airflow/commit/8c42cf1b00c90f0d7f11b8a3a455381de8e003c5) on Master. It will help if your squash your commits into single commit first so that there are less conflicts. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [airflow] kaxil commented on pull request #11437: Update pod-template-file.yaml
kaxil commented on pull request #11437: URL: https://github.com/apache/airflow/pull/11437#issuecomment-721446083 Can you please rebase your PR on latest Master since we have applied [Black](https://github.com/apache/airflow/commit/4e8f9cc8d02b29c325b8a5a76b4837671bdf5f68) and [PyUpgrade](https://github.com/apache/airflow/commit/8c42cf1b00c90f0d7f11b8a3a455381de8e003c5) on Master. It will help if your squash your commits into single commit first so that there are less conflicts. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [airflow] kaxil commented on pull request #11509: Refactor Elasticsearch provider to support 1.10.x
kaxil commented on pull request #11509: URL: https://github.com/apache/airflow/pull/11509#issuecomment-721445877 Can you please rebase your PR on latest Master since we have applied [Black](https://github.com/apache/airflow/commit/4e8f9cc8d02b29c325b8a5a76b4837671bdf5f68) and [PyUpgrade](https://github.com/apache/airflow/commit/8c42cf1b00c90f0d7f11b8a3a455381de8e003c5) on Master. It will help if your squash your commits into single commit first so that there are less conflicts. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [airflow] kaxil commented on pull request #11462: Fix issue with stale dags not being marked inactive by Scheduler
kaxil commented on pull request #11462: URL: https://github.com/apache/airflow/pull/11462#issuecomment-721446033 Can you please rebase your PR on latest Master since we have applied [Black](https://github.com/apache/airflow/commit/4e8f9cc8d02b29c325b8a5a76b4837671bdf5f68) and [PyUpgrade](https://github.com/apache/airflow/commit/8c42cf1b00c90f0d7f11b8a3a455381de8e003c5) on Master. It will help if your squash your commits into single commit first so that there are less conflicts. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [airflow] kaxil commented on pull request #11486: Annotate DagRun methods with return types.
kaxil commented on pull request #11486: URL: https://github.com/apache/airflow/pull/11486#issuecomment-721445984 Can you please rebase your PR on latest Master since we have applied [Black](https://github.com/apache/airflow/commit/4e8f9cc8d02b29c325b8a5a76b4837671bdf5f68) and [PyUpgrade](https://github.com/apache/airflow/commit/8c42cf1b00c90f0d7f11b8a3a455381de8e003c5) on Master. It will help if your squash your commits into single commit first so that there are less conflicts. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [airflow] kaxil commented on pull request #11501: Dataflow - add waiting for successful job cancel
kaxil commented on pull request #11501: URL: https://github.com/apache/airflow/pull/11501#issuecomment-721445943 Can you please rebase your PR on latest Master since we have applied [Black](https://github.com/apache/airflow/commit/4e8f9cc8d02b29c325b8a5a76b4837671bdf5f68) and [PyUpgrade](https://github.com/apache/airflow/commit/8c42cf1b00c90f0d7f11b8a3a455381de8e003c5) on Master. It will help if your squash your commits into single commit first so that there are less conflicts. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (AIRFLOW-6786) Adding KafkaConsumerHook, KafkaProducerHook, and KafkaSensor
[ https://issues.apache.org/jira/browse/AIRFLOW-6786?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17225751#comment-17225751 ] ASF GitHub Bot commented on AIRFLOW-6786: - kaxil commented on pull request #11520: URL: https://github.com/apache/airflow/pull/11520#issuecomment-721445791 Can you please rebase your PR on latest Master since we have applied [Black](https://github.com/apache/airflow/commit/4e8f9cc8d02b29c325b8a5a76b4837671bdf5f68) and [PyUpgrade](https://github.com/apache/airflow/commit/8c42cf1b00c90f0d7f11b8a3a455381de8e003c5) on Master. It will help if your squash your commits into single commit first so that there are less conflicts. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Adding KafkaConsumerHook, KafkaProducerHook, and KafkaSensor > > > Key: AIRFLOW-6786 > URL: https://issues.apache.org/jira/browse/AIRFLOW-6786 > Project: Apache Airflow > Issue Type: New Feature > Components: contrib, hooks >Affects Versions: 1.10.9 >Reporter: Daniel Ferguson >Assignee: Daniel Ferguson >Priority: Minor > > Add the KafkaProducerHook. > Add the KafkaConsumerHook. > Add the KafkaSensor which listens to messages with a specific topic. > Related Issue: > #1311 (Pre-dates Jira Migration) > Reminder to contributors: > You must add an Apache License header to all new files > Please squash your commits when possible and follow the 7 rules of good Git > commits > I am new to the community, I am not sure the files are at the right place or > missing anything. > The sensor could be used as the first node of a dag where the second node can > be a TriggerDagRunOperator. The messages are polled in a batch and the dag > runs are dynamically generated. > Thanks! > Note, as per denied PR [#1415|https://github.com/apache/airflow/pull/1415], > it is important to mention these integrations are not suitable for > low-latency/high-throughput/streaming. For reference, [#1415 > (comment)|https://github.com/apache/airflow/pull/1415#issuecomment-484429806]. > Co-authored-by: Dan Ferguson > [dferguson...@gmail.com|mailto:dferguson...@gmail.com] > Co-authored-by: YuanfΞi Zhu -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [airflow] kaxil commented on pull request #11526: Adds automated installation of dependent packages
kaxil commented on pull request #11526: URL: https://github.com/apache/airflow/pull/11526#issuecomment-721445824 Can you please rebase your PR on latest Master since we have applied [Black](https://github.com/apache/airflow/commit/4e8f9cc8d02b29c325b8a5a76b4837671bdf5f68) and [PyUpgrade](https://github.com/apache/airflow/commit/8c42cf1b00c90f0d7f11b8a3a455381de8e003c5) on Master. It will help if your squash your commits into single commit first so that there are less conflicts. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [airflow] kaxil commented on pull request #11520: [AIRFLOW-6786] Added Kafka components differently
kaxil commented on pull request #11520: URL: https://github.com/apache/airflow/pull/11520#issuecomment-721445791 Can you please rebase your PR on latest Master since we have applied [Black](https://github.com/apache/airflow/commit/4e8f9cc8d02b29c325b8a5a76b4837671bdf5f68) and [PyUpgrade](https://github.com/apache/airflow/commit/8c42cf1b00c90f0d7f11b8a3a455381de8e003c5) on Master. It will help if your squash your commits into single commit first so that there are less conflicts. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [airflow] kaxil commented on pull request #11587: #10472: Add D200 pydocstyle check - part1
kaxil commented on pull request #11587: URL: https://github.com/apache/airflow/pull/11587#issuecomment-721445503 Can you please rebase your PR on latest Master since we have applied [Black](https://github.com/apache/airflow/commit/4e8f9cc8d02b29c325b8a5a76b4837671bdf5f68) and [PyUpgrade](https://github.com/apache/airflow/commit/8c42cf1b00c90f0d7f11b8a3a455381de8e003c5) on Master. It will help if your squash your commits into single commit first so that there are less conflicts. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [airflow] kaxil commented on pull request #11613: fix imports sort
kaxil commented on pull request #11613: URL: https://github.com/apache/airflow/pull/11613#issuecomment-721445477 Can you please rebase your PR on latest Master since we have applied [Black](https://github.com/apache/airflow/commit/4e8f9cc8d02b29c325b8a5a76b4837671bdf5f68) and [PyUpgrade](https://github.com/apache/airflow/commit/8c42cf1b00c90f0d7f11b8a3a455381de8e003c5) on Master. It will help if your squash your commits into single commit first so that there are less conflicts. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [airflow] kaxil commented on pull request #11528: Add upgrade rule to check for mesos executor and flag to change it.
kaxil commented on pull request #11528: URL: https://github.com/apache/airflow/pull/11528#issuecomment-721445539 Can you please rebase your PR on latest Master since we have applied [Black](https://github.com/apache/airflow/commit/4e8f9cc8d02b29c325b8a5a76b4837671bdf5f68) and [PyUpgrade](https://github.com/apache/airflow/commit/8c42cf1b00c90f0d7f11b8a3a455381de8e003c5) on Master. It will help if your squash your commits into single commit first so that there are less conflicts. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [airflow] kaxil commented on pull request #11540: JSON serialization of dates fix
kaxil commented on pull request #11540: URL: https://github.com/apache/airflow/pull/11540#issuecomment-721445522 Can you please rebase your PR on latest Master since we have applied [Black](https://github.com/apache/airflow/commit/4e8f9cc8d02b29c325b8a5a76b4837671bdf5f68) and [PyUpgrade](https://github.com/apache/airflow/commit/8c42cf1b00c90f0d7f11b8a3a455381de8e003c5) on Master. It will help if your squash your commits into single commit first so that there are less conflicts. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [airflow] kaxil removed a comment on pull request #11528: Add upgrade rule to check for mesos executor and flag to change it.
kaxil removed a comment on pull request #11528: URL: https://github.com/apache/airflow/pull/11528#issuecomment-721445539 Can you please rebase your PR on latest Master since we have applied [Black](https://github.com/apache/airflow/commit/4e8f9cc8d02b29c325b8a5a76b4837671bdf5f68) and [PyUpgrade](https://github.com/apache/airflow/commit/8c42cf1b00c90f0d7f11b8a3a455381de8e003c5) on Master. It will help if your squash your commits into single commit first so that there are less conflicts. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [airflow] kaxil commented on pull request #11627: Add D205 pydocstyle check
kaxil commented on pull request #11627: URL: https://github.com/apache/airflow/pull/11627#issuecomment-721445384 Can you please rebase your PR on latest Master since we have applied [Black](https://github.com/apache/airflow/commit/4e8f9cc8d02b29c325b8a5a76b4837671bdf5f68) and [PyUpgrade](https://github.com/apache/airflow/commit/8c42cf1b00c90f0d7f11b8a3a455381de8e003c5) on Master. It will help if your squash your commits into single commit first so that there are less conflicts. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [airflow] kaxil commented on pull request #11631: Show Celery stats in webui
kaxil commented on pull request #11631: URL: https://github.com/apache/airflow/pull/11631#issuecomment-721445341 Can you please rebase your PR on latest Master since we have applied [Black](https://github.com/apache/airflow/commit/4e8f9cc8d02b29c325b8a5a76b4837671bdf5f68) and [PyUpgrade](https://github.com/apache/airflow/commit/8c42cf1b00c90f0d7f11b8a3a455381de8e003c5) on Master. It will help if your squash your commits into single commit first so that there are less conflicts. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [airflow] kaxil commented on pull request #11750: update variables names in _in_container_script_init.sh
kaxil commented on pull request #11750: URL: https://github.com/apache/airflow/pull/11750#issuecomment-721445077 Can you please rebase your PR on latest Master since we have applied [Black](https://github.com/apache/airflow/commit/4e8f9cc8d02b29c325b8a5a76b4837671bdf5f68) and [PyUpgrade](https://github.com/apache/airflow/commit/8c42cf1b00c90f0d7f11b8a3a455381de8e003c5) on Master. It will help if your squash your commits into single commit first so that there are less conflicts. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [airflow] kaxil commented on pull request #11726: Add DataflowJobStatusSensor and support non-blocking execution of jobs
kaxil commented on pull request #11726: URL: https://github.com/apache/airflow/pull/11726#issuecomment-721445181 Can you please rebase your PR on latest Master since we have applied [Black](https://github.com/apache/airflow/commit/4e8f9cc8d02b29c325b8a5a76b4837671bdf5f68) and [PyUpgrade](https://github.com/apache/airflow/commit/8c42cf1b00c90f0d7f11b8a3a455381de8e003c5) on Master. It will help if your squash your commits into single commit first so that there are less conflicts. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [airflow] github-actions[bot] commented on pull request #12075: Avoid unnecessary IF checks when generate Duration & Landing Time views
github-actions[bot] commented on pull request #12075: URL: https://github.com/apache/airflow/pull/12075#issuecomment-721445235 [The Workflow run](https://github.com/apache/airflow/actions/runs/344594169) is cancelling this PR. It has some failed jobs matching ^Pylint$,^Static checks,^Build docs$,^Spell check docs$,^Backport packages$,^Provider packages,^Checks: Helm tests$,^Test OpenAPI*. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [airflow] kaxil commented on pull request #11742: Fix: add support for no-menu plugin views (#10788)
kaxil commented on pull request #11742: URL: https://github.com/apache/airflow/pull/11742#issuecomment-721445126 Can you please rebase your PR on latest Master since we have applied [Black](https://github.com/apache/airflow/commit/4e8f9cc8d02b29c325b8a5a76b4837671bdf5f68) and [PyUpgrade](https://github.com/apache/airflow/commit/8c42cf1b00c90f0d7f11b8a3a455381de8e003c5) on Master. It will help if your squash your commits into single commit first so that there are less conflicts. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [airflow] kaxil commented on pull request #11729: Allow Kubernetes ServiceAccount airflow-webserver to read pod logs
kaxil commented on pull request #11729: URL: https://github.com/apache/airflow/pull/11729#issuecomment-721445153 Can you please rebase your PR on latest Master since we have applied [Black](https://github.com/apache/airflow/commit/4e8f9cc8d02b29c325b8a5a76b4837671bdf5f68) and [PyUpgrade](https://github.com/apache/airflow/commit/8c42cf1b00c90f0d7f11b8a3a455381de8e003c5) on Master. It will help if your squash your commits into single commit first so that there are less conflicts. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [airflow] kaxil commented on pull request #11751: Update pod template - only include subPath for dags mount if gitSync is enabled
kaxil commented on pull request #11751: URL: https://github.com/apache/airflow/pull/11751#issuecomment-721445039 Can you please rebase your PR on latest Master since we have applied [Black](https://github.com/apache/airflow/commit/4e8f9cc8d02b29c325b8a5a76b4837671bdf5f68) and [PyUpgrade](https://github.com/apache/airflow/commit/8c42cf1b00c90f0d7f11b8a3a455381de8e003c5) on Master. It will help if your squash your commits into single commit first so that there are less conflicts. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [airflow] kaxil commented on pull request #11747: S3ToFTPOperator
kaxil commented on pull request #11747: URL: https://github.com/apache/airflow/pull/11747#issuecomment-721445100 Can you please rebase your PR on latest Master since we have applied [Black](https://github.com/apache/airflow/commit/4e8f9cc8d02b29c325b8a5a76b4837671bdf5f68) and [PyUpgrade](https://github.com/apache/airflow/commit/8c42cf1b00c90f0d7f11b8a3a455381de8e003c5) on Master. It will help if your squash your commits into single commit first so that there are less conflicts. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [airflow] kaxil commented on pull request #11699: Enforce ESLint (JavaScript linting) with pre-commit hooks
kaxil commented on pull request #11699: URL: https://github.com/apache/airflow/pull/11699#issuecomment-721445214 Can you please rebase your PR on latest Master since we have applied [Black](https://github.com/apache/airflow/commit/4e8f9cc8d02b29c325b8a5a76b4837671bdf5f68) and [PyUpgrade](https://github.com/apache/airflow/commit/8c42cf1b00c90f0d7f11b8a3a455381de8e003c5) on Master. It will help if your squash your commits into single commit first so that there are less conflicts. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [airflow] kaxil commented on pull request #11652: Support for sorting DAGs in the web UI
kaxil commented on pull request #11652: URL: https://github.com/apache/airflow/pull/11652#issuecomment-721445290 Can you please rebase your PR on latest Master since we have applied [Black](https://github.com/apache/airflow/commit/4e8f9cc8d02b29c325b8a5a76b4837671bdf5f68) and [PyUpgrade](https://github.com/apache/airflow/commit/8c42cf1b00c90f0d7f11b8a3a455381de8e003c5) on Master. It will help if your squash your commits into single commit first so that there are less conflicts. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [airflow] kaxil commented on pull request #11767: Fix: Removed password if HiveServer2Hook in LDAP or CUSTOM mode
kaxil commented on pull request #11767: URL: https://github.com/apache/airflow/pull/11767#issuecomment-721445012 Can you please rebase your PR on latest Master since we have applied [Black](https://github.com/apache/airflow/commit/4e8f9cc8d02b29c325b8a5a76b4837671bdf5f68) and [PyUpgrade](https://github.com/apache/airflow/commit/8c42cf1b00c90f0d7f11b8a3a455381de8e003c5) on Master. It will help if your squash your commits into single commit first so that there are less conflicts. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [airflow] kaxil commented on pull request #11778: Use default view in dr op link
kaxil commented on pull request #11778: URL: https://github.com/apache/airflow/pull/11778#issuecomment-721444944 Can you please rebased your PR on latest Master since we have applied [Black](https://github.com/apache/airflow/commit/4e8f9cc8d02b29c325b8a5a76b4837671bdf5f68) and [PyUpgrade](https://github.com/apache/airflow/commit/8c42cf1b00c90f0d7f11b8a3a455381de8e003c5) on Master. It will help if your squash your commits into single commit first so that there are less conflicts. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [airflow] kaxil commented on pull request #11793: [AIRFLOW-8664] Postgres to GCS operator - use named cursor
kaxil commented on pull request #11793: URL: https://github.com/apache/airflow/pull/11793#issuecomment-721444914 Can you please rebased your PR on latest Master since we have applied [Black](https://github.com/apache/airflow/commit/4e8f9cc8d02b29c325b8a5a76b4837671bdf5f68) and [PyUpgrade](https://github.com/apache/airflow/commit/8c42cf1b00c90f0d7f11b8a3a455381de8e003c5) on Master. It will help if your squash your commits into single commit first so that there are less conflicts. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [airflow] jfmolano opened a new pull request #12077: Adding MySql howto-documentation and example DAG
jfmolano opened a new pull request #12077: URL: https://github.com/apache/airflow/pull/12077 --- **^ Add meaningful description above** Read the **[Pull Request Guidelines](https://github.com/apache/airflow/blob/master/CONTRIBUTING.rst#pull-request-guidelines)** for more information. In case of fundamental code change, Airflow Improvement Proposal ([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvements+Proposals)) is needed. In case of a new dependency, check compliance with the [ASF 3rd Party License Policy](https://www.apache.org/legal/resolved.html#category-x). In case of backwards incompatible changes please leave a note in [UPDATING.md](https://github.com/apache/airflow/blob/master/UPDATING.md). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [airflow] kaxil commented on pull request #11812: Remove whitespace from airflow/utils/process_utils.py
kaxil commented on pull request #11812: URL: https://github.com/apache/airflow/pull/11812#issuecomment-721444760 Can you please rebased your PR on latest Master since we have applied [Black](https://github.com/apache/airflow/commit/4e8f9cc8d02b29c325b8a5a76b4837671bdf5f68) and [PyUpgrade](https://github.com/apache/airflow/commit/8c42cf1b00c90f0d7f11b8a3a455381de8e003c5) on Master. It will help if your squash your commits into single commit first so that there are less conflicts. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [airflow] kaxil commented on pull request #11784: Add pod_template_override to executor_config
kaxil commented on pull request #11784: URL: https://github.com/apache/airflow/pull/11784#issuecomment-721444012 I have rebased your PR on latest Master since we have applied [Black](https://github.com/apache/airflow/commit/4e8f9cc8d02b29c325b8a5a76b4837671bdf5f68) and [PyUpgrade](https://github.com/apache/airflow/commit/8c42cf1b00c90f0d7f11b8a3a455381de8e003c5) on Master. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [airflow] kaxil commented on pull request #11815: Render k8s yaml for tasks via the Airflow UI
kaxil commented on pull request #11815: URL: https://github.com/apache/airflow/pull/11815#issuecomment-721443683 I have rebased your PR on latest Master since we have applied [Black](https://github.com/apache/airflow/commit/4e8f9cc8d02b29c325b8a5a76b4837671bdf5f68) and [PyUpgrade](https://github.com/apache/airflow/commit/8c42cf1b00c90f0d7f11b8a3a455381de8e003c5) on Master. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [airflow] kaxil commented on pull request #11950: Add python 3.9 to supported versions
kaxil commented on pull request #11950: URL: https://github.com/apache/airflow/pull/11950#issuecomment-721440733 Can you please rebased your PR on latest Master since we have applied [Black](https://github.com/apache/airflow/commit/4e8f9cc8d02b29c325b8a5a76b4837671bdf5f68) and [PyUpgrade](https://github.com/apache/airflow/commit/8c42cf1b00c90f0d7f11b8a3a455381de8e003c5) on Master. It will help if your squash your commits into single commit first so that there are less conflicts. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [airflow] kaxil commented on pull request #11922: Extend the same keyword arguments callable support in PythonOperator to some other sensors/operators
kaxil commented on pull request #11922: URL: https://github.com/apache/airflow/pull/11922#issuecomment-721440805 Can you please rebased your PR on latest Master since we have applied [Black](https://github.com/apache/airflow/commit/4e8f9cc8d02b29c325b8a5a76b4837671bdf5f68) and [PyUpgrade](https://github.com/apache/airflow/commit/8c42cf1b00c90f0d7f11b8a3a455381de8e003c5) on Master. It will help if your squash your commits into single commit first so that there are less conflicts. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [airflow] kaxil commented on pull request #11960: Log more types of errors when loading a DAG file to DB (POC)
kaxil commented on pull request #11960: URL: https://github.com/apache/airflow/pull/11960#issuecomment-721440691 Can you please rebased your PR on latest Master since we have applied [Black](https://github.com/apache/airflow/commit/4e8f9cc8d02b29c325b8a5a76b4837671bdf5f68) and [PyUpgrade](https://github.com/apache/airflow/commit/8c42cf1b00c90f0d7f11b8a3a455381de8e003c5) on Master. It will help if your squash your commits into single commit first so that there are less conflicts. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [airflow] kaxil commented on pull request #11970: Unify user session lifetime configuration
kaxil commented on pull request #11970: URL: https://github.com/apache/airflow/pull/11970#issuecomment-721440601 Can you please rebased your PR on latest Master since we have applied [Black](https://github.com/apache/airflow/commit/4e8f9cc8d02b29c325b8a5a76b4837671bdf5f68) and [PyUpgrade](https://github.com/apache/airflow/commit/8c42cf1b00c90f0d7f11b8a3a455381de8e003c5) on Master. It will help if your squash your commits into single commit first so that there are less conflicts. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [airflow] kaxil commented on pull request #11964: Add new datetime branch operator
kaxil commented on pull request #11964: URL: https://github.com/apache/airflow/pull/11964#issuecomment-721440649 Can you please rebased your PR on latest Master since we have applied [Black](https://github.com/apache/airflow/commit/4e8f9cc8d02b29c325b8a5a76b4837671bdf5f68) and [PyUpgrade](https://github.com/apache/airflow/commit/8c42cf1b00c90f0d7f11b8a3a455381de8e003c5) on Master. It will help if your squash your commits into single commit first so that there are less conflicts. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
