[GitHub] [airflow] potiuk commented on issue #19569: The latest docker image is not the "latest"

[GitBox]

potiuk commented on issue #19569:
URL: https://github.com/apache/airflow/issues/19569#issuecomment-967800512


   Good. Spot. I fixed it now in the repo. I will update the process of 
releasing to include refresh of the latest images as it was missing!


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] potiuk commented on pull request #18675: New Tree View

[GitBox]

potiuk commented on pull request #18675:
URL: https://github.com/apache/airflow/pull/18675#issuecomment-967799567


   Yep! Cool!


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] github-actions[bot] commented on pull request #19487: Fix task instance api cannot list task instances with None state

[GitBox]

github-actions[bot] commented on pull request #19487:
URL: https://github.com/apache/airflow/pull/19487#issuecomment-967796463


   The PR is likely OK to be merged with just subset of tests for default 
Python and Database versions without running the full matrix of tests, because 
it does not modify the core of Airflow. If the committers decide that the full 
tests matrix is needed, they will add the label 'full tests needed'. Then you 
should rebase to the latest main or amend the last commit of the PR, and push 
it with --force-with-lease.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] eladkal closed pull request #18929: Do not require all extra params in salesforce hook

[GitBox]

eladkal closed pull request #18929:
URL: https://github.com/apache/airflow/pull/18929


   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] eladkal commented on pull request #18929: Do not require all extra params in salesforce hook

[GitBox]

eladkal commented on pull request #18929:
URL: https://github.com/apache/airflow/pull/18929#issuecomment-967794769


   Closing as issue solved in https://github.com/apache/airflow/pull/19530


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[airflow] branch main updated (1c2dfde -> 9519bf6)

[ephraimanierobi]

This is an automated email from the ASF dual-hosted git repository.

ephraimanierobi pushed a change to branch main
in repository https://gitbox.apache.org/repos/asf/airflow.git.


from 1c2dfde  Minor touch up for async docs (#19539)
 add 9519bf6  Fix IntegrityError in `DagFileProcessor.manage_slas` (#19553)

No new revisions were added by this update.

Summary of changes:
 airflow/dag_processing/processor.py| 11 ++
 tests/dag_processing/test_processor.py | 39 ++
 2 files changed, 50 insertions(+)

[GitHub] [airflow] ephraimbuddy merged pull request #19553: Fix IntegrityError in `DagFileProcessor.manage_slas`

[GitBox]

ephraimbuddy merged pull request #19553:
URL: https://github.com/apache/airflow/pull/19553


   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] david30907d opened a new pull request #19571: :pencil: (providers_google) add a location check

[GitBox]

david30907d opened a new pull request #19571:
URL: https://github.com/apache/airflow/pull/19571


   if you didn't, you'll get this `returned "Not found: Job xxx"` exception
   related: 
   
   Although BigQueryHook says `location` is optional, turns out it's required 
under some scenario
   
[code](https://github.com/apache/airflow/blob/main/airflow/providers/google/cloud/hooks/bigquery.py#L99)
   
   when we invoke `get_records()` 
([here](https://github.com/apache/airflow/blob/main/airflow/hooks/dbapi.py#L155)),
  it would end up go to this `run_query()` to get the `job id` 
([here](https://github.com/apache/airflow/blob/main/airflow/providers/google/cloud/hooks/bigquery.py#L2202)).
   
   another alternative is raise a `missing location exception` in `run_query()`
   
   thoughts on this?
   ```bash
   Traceback (most recent call last):
 File 
"/home/airflow/.local/lib/python3.8/site-packages/airflow/models/taskinstance.py",
 line 1157, in _run_raw_task
   self._prepare_and_execute_task_with_callbacks(context, task)
 File 
"/home/airflow/.local/lib/python3.8/site-packages/airflow/models/taskinstance.py",
 line 1331, in _prepare_and_execute_task_with_callbacks
   result = self._execute_task(context, task_copy)
 File 
"/home/airflow/.local/lib/python3.8/site-packages/airflow/models/taskinstance.py",
 line 1361, in _execute_task
   result = task_copy.execute(context=context)
 File 
"/home/airflow/.local/lib/python3.8/site-packages/airflow/operators/python.py", 
line 150, in execute
   return_value = self.execute_callable()
 File 
"/home/airflow/.local/lib/python3.8/site-packages/airflow/operators/python.py", 
line 161, in execute_callable
   return self.python_callable(*self.op_args, **self.op_kwargs)
 File "/opt/airflow/dags/dags/utils/others/subscription_related.py", line 
112, in wrapper
   return func(*args, **kwargs)
 File 
"/opt/airflow/dags/dags/utils/extractors/platform_data_extractors/shopify_extractor.py",
 line 75, in wrapper
   return func(*args, **kwargs)
 File 
"/opt/airflow/dags/dags/utils/extractors/platform_data_extractors/shopify_extractor.py",
 line 1019, in add_abandoned
   abandoned_checkouts_of_this_page = _parse_this_page(response_json)
 File 
"/opt/airflow/dags/dags/utils/extractors/platform_data_extractors/shopify_extractor.py",
 line 980, in _parse_this_page
   persons_queried_by_checkout_id = db_hook.get_records(
 File 
"/home/airflow/.local/lib/python3.8/site-packages/airflow/hooks/dbapi.py", line 
135, in get_records
   return cur.fetchall()
 File 
"/home/airflow/.local/lib/python3.8/site-packages/airflow/providers/google/cloud/hooks/bigquery.py",
 line 2886, in fetchall
   one = self.fetchone()
 File 
"/home/airflow/.local/lib/python3.8/site-packages/airflow/providers/google/cloud/hooks/bigquery.py",
 line 2811, in fetchone
   return self.next()
 File 
"/home/airflow/.local/lib/python3.8/site-packages/airflow/providers/google/cloud/hooks/bigquery.py",
 line 2827, in next
   self.service.jobs()
 File 
"/home/airflow/.local/lib/python3.8/site-packages/googleapiclient/_helpers.py", 
line 134, in positional_wrapper
   return wrapped(*args, **kwargs)
 File 
"/home/airflow/.local/lib/python3.8/site-packages/googleapiclient/http.py", 
line 915, in execute
   raise HttpError(resp, content, uri=self.uri)
   googleapiclient.errors.HttpError: https://bigquery.googleapis.com/bigquery/v2/projects/xx/queries/airflow_x?alt=json
 returned "Not found: Job x:x
   ```
   
   **^ Add meaningful description above**
   
   Read the **[Pull Request 
Guidelines](https://github.com/apache/airflow/blob/main/CONTRIBUTING.rst#pull-request-guidelines)**
 for more information.
   In case of fundamental code change, Airflow Improvement Proposal 
([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvements+Proposals))
 is needed.
   In case of a new dependency, check compliance with the [ASF 3rd Party 
License Policy](https://www.apache.org/legal/resolved.html#category-x).
   In case of backwards incompatible changes please leave a note in 
[UPDATING.md](https://github.com/apache/airflow/blob/main/UPDATING.md).
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] david30907d opened a new issue #19570: get_records() don't work out for BigQueryHook

[GitBox]

david30907d opened a new issue #19570:
URL: https://github.com/apache/airflow/issues/19570


   ### Apache Airflow Provider(s)
   
   google
   
   ### Versions of Apache Airflow Providers
   
   `apache-airflow-providers-google  4.0.0`
   
   ### Apache Airflow version
   
   2.1.2
   
   ### Operating System
   
   PRETTY_NAME="Debian GNU/Linux 10 (buster)" NAME="Debian GNU/Linux" 
VERSION_ID="10" VERSION="10 (buster)" VERSION_CODENAME=buster ID=debian 
HOME_URL="https://www.debian.org/; SUPPORT_URL="https://www.debian.org/support; 
BUG_REPORT_URL="https://bugs.debian.org/;
   
   ### Deployment
   
   Docker-Compose
   
   ### Deployment details
   
   I'm using official docker-compose
   
   ### What happened
   
   ```bash
   [2021-11-12 09:24:21,409] {logging_mixin.py:104} WARNING - 
/home/***/.local/lib/python3.8/site-packages/***/providers/google/cloud/hooks/bigquery.py:141
 DeprecationWarning: This method will be deprecated. Please use 
`BigQueryHook.get_client` method
   [2021-11-12 09:24:23,033] {logging_mixin.py:104} WARNING - 
/home/***/.local/lib/python3.8/site-packages/***/providers/google/cloud/hooks/bigquery.py:2195
 DeprecationWarning: This method is deprecated. Please use 
`BigQueryHook.insert_job` method.
   [2021-11-12 09:24:23,042] {bigquery.py:1639} INFO - Inserting job 
***_1636709063039439_14741bf3004db91ee4cbb5eb024ac5ba
   [2021-11-12 09:24:27,463] {taskinstance.py:1501} ERROR - Task failed with 
exception
   Traceback (most recent call last):
 File 
"/home/airflow/.local/lib/python3.8/site-packages/airflow/models/taskinstance.py",
 line 1157, in _run_raw_task
   self._prepare_and_execute_task_with_callbacks(context, task)
 File 
"/home/airflow/.local/lib/python3.8/site-packages/airflow/models/taskinstance.py",
 line 1331, in _prepare_and_execute_task_with_callbacks
   result = self._execute_task(context, task_copy)
 File 
"/home/airflow/.local/lib/python3.8/site-packages/airflow/models/taskinstance.py",
 line 1361, in _execute_task
   result = task_copy.execute(context=context)
 File 
"/home/airflow/.local/lib/python3.8/site-packages/airflow/operators/python.py", 
line 150, in execute
   return_value = self.execute_callable()
 File 
"/home/airflow/.local/lib/python3.8/site-packages/airflow/operators/python.py", 
line 161, in execute_callable
   return self.python_callable(*self.op_args, **self.op_kwargs)
 File "/opt/airflow/dags/dags/utils/others/subscription_related.py", line 
112, in wrapper
   return func(*args, **kwargs)
 File 
"/opt/airflow/dags/dags/utils/extractors/platform_data_extractors/shopify_extractor.py",
 line 75, in wrapper
   return func(*args, **kwargs)
 File 
"/opt/airflow/dags/dags/utils/extractors/platform_data_extractors/shopify_extractor.py",
 line 1019, in add_abandoned
   abandoned_checkouts_of_this_page = _parse_this_page(response_json)
 File 
"/opt/airflow/dags/dags/utils/extractors/platform_data_extractors/shopify_extractor.py",
 line 980, in _parse_this_page
   persons_queried_by_checkout_id = db_hook.get_records(
 File 
"/home/airflow/.local/lib/python3.8/site-packages/airflow/hooks/dbapi.py", line 
135, in get_records
   return cur.fetchall()
 File 
"/home/airflow/.local/lib/python3.8/site-packages/airflow/providers/google/cloud/hooks/bigquery.py",
 line 2886, in fetchall
   one = self.fetchone()
 File 
"/home/airflow/.local/lib/python3.8/site-packages/airflow/providers/google/cloud/hooks/bigquery.py",
 line 2811, in fetchone
   return self.next()
 File 
"/home/airflow/.local/lib/python3.8/site-packages/airflow/providers/google/cloud/hooks/bigquery.py",
 line 2827, in next
   self.service.jobs()
 File 
"/home/airflow/.local/lib/python3.8/site-packages/googleapiclient/_helpers.py", 
line 134, in positional_wrapper
   return wrapped(*args, **kwargs)
 File 
"/home/airflow/.local/lib/python3.8/site-packages/googleapiclient/http.py", 
line 915, in execute
   raise HttpError(resp, content, uri=self.uri)
   googleapiclient.errors.HttpError: https://bigquery.googleapis.com/bigquery/v2/projects/tresl-co-001/queries/airflow_1636709063039439_14741bf3004db91ee4cbb5eb024ac5ba?alt=json
 returned "Not found: Job 
tresl-co-001:airflow_1636709063039439_14741bf3004db91ee4cbb5eb024ac5ba". 
Details: "Not found: Job 
tresl-co-001:airflow_1636709063039439_14741bf3004db91ee4cbb5eb024ac5ba">
   ```
   
   ### What you expected to happen
   
   `bq_hook.get_records(sql)` should work anyway
   
   ### How to reproduce
   
   ```python
   from dags.utils.hooks.bigquery import BigQueryHook
   bq_hook = BigQueryHook(gcp_conn_id='', use_legacy_sql=False)
   bq_hook.get_records(sql)
   ```
   
   ### Anything else
   
   _No response_
   
   ### Are you willing to submit PR?
   
   - [X] Yes I am willing to submit a PR!
   
   ### Code of Conduct
   
   - [X] I agree to follow this project's [Code of 

[GitHub] [airflow] dstandish commented on pull request #18675: New Tree View

[GitBox]

dstandish commented on pull request #18675:
URL: https://github.com/apache/airflow/pull/18675#issuecomment-96120


   very cool


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] boring-cyborg[bot] commented on issue #19569: The latest docker image is not the "latest"

[GitBox]

boring-cyborg[bot] commented on issue #19569:
URL: https://github.com/apache/airflow/issues/19569#issuecomment-967769215


   Thanks for opening your first issue here! Be sure to follow the issue 
template!
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] huxuan opened a new issue #19569: The latest docker image is not the "latest"

[GitBox]

huxuan opened a new issue #19569:
URL: https://github.com/apache/airflow/issues/19569


   ### Apache Airflow version
   
   2.2.1 (latest released)
   
   ### Operating System
   
   N/A
   
   ### Versions of Apache Airflow Providers
   
   _No response_
   
   ### Deployment
   
   Other Docker-based deployment
   
   ### Deployment details
   
   _No response_
   
   ### What happened
   
   The image tags with `latest` and `latest-python3.X` is either release two 
months ago or even 5 months ago.
   
   https://hub.docker.com/r/apache/airflow/tags?name=latest
   
   ### What you expected to happen
   
   According to the documentation here [1], seems it should be aligned with the 
latest stable version.
   
   BTW, I am willing to submit a PR, but might need some hints how we manage 
the docker image tags.
   
   [1] https://airflow.apache.org/docs/docker-stack/index.html
   
   
   ### How to reproduce
   
   _No response_
   
   ### Anything else
   
   _No response_
   
   ### Are you willing to submit PR?
   
   - [X] Yes I am willing to submit a PR!
   
   ### Code of Conduct
   
   - [X] I agree to follow this project's [Code of 
Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] dstandish commented on a change in pull request #19546: Track completed triggers until deleted from database

[GitBox]

dstandish commented on a change in pull request #19546:
URL: https://github.com/apache/airflow/pull/19546#discussion_r748671412



##
File path: airflow/jobs/triggerer_job.py
##
@@ -148,6 +149,25 @@ def load_triggers(self):
 ids = Trigger.ids_for_triggerer(self.id)
 self.runner.update_triggers(set(ids))
 
+@provide_session
+def purge_completed_triggers_list(self, session):

Review comment:
   it would be nice if we didn't have to query the database here but i 
don't think it can be avoided.
   
   i explored updating `clean_unused` to return the `ids` deleted.  then we 
could pass those to `purge_completed` and purge those rows.  but this won't 
work because `clean_unused` does not filter w.r.t. triggerer_id, so other 
triggerer instances could delete the rows belonging to _this_ triggerer, so 
`completed_triggers` would grow over time.
   
   on the bright side, we're querying a PK, so it shouldn't be that expensive.
   
   alternatively we could change `clean_unused` to take a triggerer id and 
delete only the rows for that triggerer,  then the above approach would 
probably work.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] dstandish commented on a change in pull request #19546: Track completed triggers until deleted from database

[GitBox]

dstandish commented on a change in pull request #19546:
URL: https://github.com/apache/airflow/pull/19546#discussion_r748671412



##
File path: airflow/jobs/triggerer_job.py
##
@@ -148,6 +149,25 @@ def load_triggers(self):
 ids = Trigger.ids_for_triggerer(self.id)
 self.runner.update_triggers(set(ids))
 
+@provide_session
+def purge_completed_triggers_list(self, session):

Review comment:
   it would be nice if we didn't have to query the database here but i 
don't think it can be avoided.
   
   i explored updating `clean_unused` to return the `ids` deleted.  then we 
could pass those to `purge_completed` and purge those rows.  but this won't 
work because `clean_unused` does not filter w.r.t. triggerer_id, so other 
triggerer instances could delete the rows belonging to _this_ triggerer, so 
`completed_triggers` would grow over time.
   
   on the bright side, we're querying a PK, so it shouldn't be that expensive.
   
   alternatively we could change `clean_unused` to take a triggerer id and 
delete only the rows for that triggerer.  this would probably work.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] dstandish commented on a change in pull request #19546: Track completed triggers until deleted from database

[GitBox]

dstandish commented on a change in pull request #19546:
URL: https://github.com/apache/airflow/pull/19546#discussion_r748671412



##
File path: airflow/jobs/triggerer_job.py
##
@@ -148,6 +149,25 @@ def load_triggers(self):
 ids = Trigger.ids_for_triggerer(self.id)
 self.runner.update_triggers(set(ids))
 
+@provide_session
+def purge_completed_triggers_list(self, session):

Review comment:
   it would be nice if we didn't have to query the database here but i 
don't think it can be avoided.
   
   i explored updating `clean_unused` to return the `ids` deleted.  then we 
could pass those to `purge_completed` and purge those rows.  but this won't 
work because `clean_unused` does not filter w.r.t. triggerer_id, so other 
triggerer instances could delete the rows belonging to _this_ triggerer, so 
`completed_triggers` would grow over time.
   
   on the bright side, we're querying a PK, so it shouldn't be that expensive.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] dstandish commented on a change in pull request #19546: Track completed triggers until deleted from database

[GitBox]

dstandish commented on a change in pull request #19546:
URL: https://github.com/apache/airflow/pull/19546#discussion_r748671412



##
File path: airflow/jobs/triggerer_job.py
##
@@ -148,6 +149,25 @@ def load_triggers(self):
 ids = Trigger.ids_for_triggerer(self.id)
 self.runner.update_triggers(set(ids))
 
+@provide_session
+def purge_completed_triggers_list(self, session):

Review comment:
   it would be nice if we didn't have to query tthe database here but i 
don't think it can be avoided.
   
   i explored updating `clean_unused` to return the `ids` deleted.  then we 
could pass those to `purge_completed` and purge those rows.  but this won't 
work because `clean_unused` does not filter w.r.t. triggerer_id, so other 
triggerer instances could delete the rows belonging to _this_ triggerer, so 
`completed_triggers` would grow over time.
   
   on the bright side, we're querying a PK, so it shouldn't be that expensive.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] dstandish commented on a change in pull request #19546: Track completed triggers until deleted from database

[GitBox]

dstandish commented on a change in pull request #19546:
URL: https://github.com/apache/airflow/pull/19546#discussion_r748671412



##
File path: airflow/jobs/triggerer_job.py
##
@@ -148,6 +149,25 @@ def load_triggers(self):
 ids = Trigger.ids_for_triggerer(self.id)
 self.runner.update_triggers(set(ids))
 
+@provide_session
+def purge_completed_triggers_list(self, session):

Review comment:
   it would be nice if we didn't have to query tthe database here but i 
don't think it can be avoided.
   
   i explored updating `clean_unused` to return the `ids` deleted.  then we 
could pass those to `purge_completed` and purge those rows.  but this won't 
work because `clean_unused` does not filter w.r.t. triggerer_id, so other 
triggerer instances could delete the rows belonging to _this_ triggerer, so 
`completed_triggers` would grow over time.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] josh-fell closed pull request #19431: Update Azure modules to comply with AIP-21

[GitBox]

josh-fell closed pull request #19431:
URL: https://github.com/apache/airflow/pull/19431


   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] josh-fell commented on a change in pull request #19563: Add example SLA DAG

[GitBox]

josh-fell commented on a change in pull request #19563:
URL: https://github.com/apache/airflow/pull/19563#discussion_r748669977



##
File path: airflow/example_dags/example_sla_dag.py
##
@@ -0,0 +1,56 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+import time
+from datetime import datetime, timedelta
+
+from airflow.decorators import dag, task
+
+"""Example DAG demonstrating SLA use in Tasks"""
+
+
+# [START howto_task_sla]
+def sla_callback(*args):

Review comment:
   Not saying it's confusing at all, this is a great example. I was just 
thinking since the Concepts doc has "Examples of ``sla_miss_callback`` function 
signature:..." noted it might be good for users to see the same function in the 
context of this example DAG as well as the other components that are outlined 
in the doc. Again, no strong opinions here.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] karakanb commented on issue #19566: Breeze - The CSS and JS files are not built

[GitBox]

karakanb commented on issue #19566:
URL: https://github.com/apache/airflow/issues/19566#issuecomment-967755882


   I already ran it, both from within the container and outside of it, although 
nothing changed.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] artik00 commented on issue #17255: x_frame_enabled logic broken in Airflow 2

[GitBox]

artik00 commented on issue #17255:
URL: https://github.com/apache/airflow/issues/17255#issuecomment-967755209


   Any idea since when this was broken?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] boring-cyborg[bot] commented on issue #19568: Error running worker after upgrade to 2.2

[GitBox]

boring-cyborg[bot] commented on issue #19568:
URL: https://github.com/apache/airflow/issues/19568#issuecomment-967746331


   Thanks for opening your first issue here! Be sure to follow the issue 
template!
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] jorenby opened a new issue #19568: Error running worker after upgrade to 2.2

[GitBox]

jorenby opened a new issue #19568:
URL: https://github.com/apache/airflow/issues/19568


   ### Apache Airflow version
   
   2.2.1 (latest released)
   
   ### Operating System
   
   Linux
   
   ### Versions of Apache Airflow Providers
   
   apache-airflow-providers-amazon @ 
file:///Users/joshuajorenby/Library/Caches/pypoetry/artifacts/46/cb/dc/5d78f8e7fa74661e7f8ae99d492b60ed2641b4e8872d51ca3f416ef043/apache_airflow_providers_amazon-1.4.0-py3-none-any.whl
   apache-airflow-providers-ftp @ 
file:///Users/joshuajorenby/Library/Caches/pypoetry/artifacts/a5/13/da/bf14abc40193a1ee1b82bbd800e3ac230427d7684b9d40998ac3684bef/apache_airflow_providers_ftp-2.0.1-py3-none-any.whl
   apache-airflow-providers-http @ 
file:///Users/joshuajorenby/Library/Caches/pypoetry/artifacts/22/4d/d8/afbd1e5175f52b087f61e8adba4fb9f3dd775eba59ab1a870dbfd06bbb/apache_airflow_providers_http-2.0.1-py3-none-any.whl
   apache-airflow-providers-imap @ 
file:///Users/joshuajorenby/Library/Caches/pypoetry/artifacts/af/5d/de/21c10bfc7ac076a415dcc3fc909317547e77e38c005487552cf40ddd97/apache_airflow_providers_imap-2.0.1-py3-none-any.whl
   apache-airflow-providers-postgres @ 
file:///Users/joshuajorenby/Library/Caches/pypoetry/artifacts/de/5e/a2/a263b7839be406099d7b464770ab18e4502c638fd37784f69e12b13c5f/apache_airflow_providers_postgres-1.0.2-py3-none-any.whl
   apache-airflow-providers-slack @ 
file:///Users/joshuajorenby/Library/Caches/pypoetry/artifacts/38/a6/58/f0d07dbddf3241cfcb10dcc1b16391e6ba4120e1019d2d1d07b73ed84c/apache_airflow_providers_slack-4.1.0-py3-none-any.whl
   apache-airflow-providers-sqlite @ 
file:///Users/joshuajorenby/Library/Caches/pypoetry/artifacts/ec/e6/a3/e0d81fef662ccf79609e7d2c4e4440839a464771fd2a002d252c9a401d/apache_airflow_providers_sqlite-2.0.1-py3-none-any.whl
   
   ### Deployment
   
   Other Docker-based deployment
   
   ### Deployment details
   
   _No response_
   
   ### What happened
   
   We've been running 2.1.2 using CeleryExecutor without issue. After upgrading 
to 2.2.1, workers started failing with:
   
   ```
   Traceback (most recent call last):
   --
   File "/home/airflow/.local/bin/airflow", line 8, in 
   sys.exit(main())
   File "/home/airflow/.local/lib/python3.8/site-packages/airflow/__main__.py", 
line 48, in main
   args.func(args)
   File 
"/home/airflow/.local/lib/python3.8/site-packages/airflow/cli/cli_parser.py", 
line 48, in command
   return func(*args, **kwargs)
   File 
"/home/airflow/.local/lib/python3.8/site-packages/airflow/utils/cli.py", line 
92, in wrapper
   return f(*args, **kwargs)
   File 
"/home/airflow/.local/lib/python3.8/site-packages/airflow/cli/commands/celery_command.py",
 line 188, in worker
   _run_worker(options=options, skip_serve_logs=skip_serve_logs)
   File 
"/home/airflow/.local/lib/python3.8/site-packages/airflow/cli/commands/celery_command.py",
 line 94, in _run_worker
   celery_app.worker_main(options)
   File "/home/airflow/.local/lib/python3.8/site-packages/celery/app/base.py", 
line 365, in worker_main
   return instantiate(
   File "/home/airflow/.local/lib/python3.8/site-packages/celery/bin/base.py", 
line 283, in execute_from_commandline
   self.maybe_patch_concurrency(argv)
   File "/home/airflow/.local/lib/python3.8/site-packages/celery/bin/base.py", 
line 315, in maybe_patch_concurrency
   maybe_patch_concurrency(argv, *pool_option)
   File "/home/airflow/.local/lib/python3.8/site-packages/celery/__init__.py", 
line 143, in maybe_patch_concurrency
   pool = _find_option_with_arg(argv, short_opts, long_opts)
   File "/home/airflow/.local/lib/python3.8/site-packages/celery/__init__.py", 
line 95, in _find_option_with_arg
   if arg.startswith('-'):
   AttributeError: 'int' object has no attribute 'startswith'
   ```
   
   
   ### What you expected to happen
   
   Workers to be able to function as they did before.
   
   ### How to reproduce
   
   Changed dependency to apache-airflow 2.2.1 and set up the entry point to run 
`airflow db upgrade`. No other functional changes beside swapping out 
`max_active_tasks_per_dag` for `dag_concurrency` in the config.
   
   ### Anything else
   
   _No response_
   
   ### Are you willing to submit PR?
   
   - [X] Yes I am willing to submit a PR!
   
   ### Code of Conduct
   
   - [X] I agree to follow this project's [Code of 
Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[airflow] branch main updated (d185c66 -> 1c2dfde)

[kaxilnaik]

This is an automated email from the ASF dual-hosted git repository.

kaxilnaik pushed a change to branch main
in repository https://gitbox.apache.org/repos/asf/airflow.git.


from d185c66  Small formatting tweak to GCP SSH conn doc (#19562)
 add 1c2dfde  Minor touch up for async docs (#19539)

No new revisions were added by this update.

Summary of changes:
 airflow/triggers/base.py   |  4 ++--
 docs/apache-airflow/concepts/deferring.rst | 13 -
 2 files changed, 10 insertions(+), 7 deletions(-)

[GitHub] [airflow] kaxil merged pull request #19539: Minor touch up for async docs

[GitBox]

kaxil merged pull request #19539:
URL: https://github.com/apache/airflow/pull/19539


   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] github-actions[bot] commented on pull request #19539: Minor touch up for async docs

[GitBox]

github-actions[bot] commented on pull request #19539:
URL: https://github.com/apache/airflow/pull/19539#issuecomment-967737679


   The PR most likely needs to run full matrix of tests because it modifies 
parts of the core of Airflow. However, committers might decide to merge it 
quickly and take the risk. If they don't merge it quickly - please rebase it to 
the latest main at your convenience, or amend the last commit of the PR, and 
push it with --force-with-lease.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] boring-cyborg[bot] commented on issue #19567: Provider installer pages link to incorrect page

[GitBox]

boring-cyborg[bot] commented on issue #19567:
URL: https://github.com/apache/airflow/issues/19567#issuecomment-967737559


   Thanks for opening your first issue here! Be sure to follow the issue 
template!
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] sebbASF opened a new issue #19567: Provider installer pages link to incorrect page

[GitBox]

sebbASF opened a new issue #19567:
URL: https://github.com/apache/airflow/issues/19567


   ### Describe the issue with documentation
   
   
https://airflow.apache.org/docs/apache-airflow-providers-airbyte/stable/installing-providers-from-sources.html
 has the following text:
   
   "The packages are available via the Official Apache Software Foundations 
Downloads"
   
   This links to http://ws.apache.org/mirrors.cgi which is not a valid download 
page, and anyway it belongs to the Web Services project.
   
   I don't think the sentence adds any value, so could just be dropped.
   
   ### How to solve the problem
   
   As per above, just remove the sentence
   
   ### Anything else
   
   Note that this presumably applies to all the provider installation pages.
   
   ### Are you willing to submit PR?
   
   - [ ] Yes I am willing to submit a PR!
   
   ### Code of Conduct
   
   - [X] I agree to follow this project's [Code of 
Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] github-actions[bot] closed pull request #18425: Installation Page

[GitBox]

github-actions[bot] closed pull request #18425:
URL: https://github.com/apache/airflow/pull/18425


   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] github-actions[bot] closed pull request #18309: task logs for dummy operators

[GitBox]

github-actions[bot] closed pull request #18309:
URL: https://github.com/apache/airflow/pull/18309


   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] kaxil edited a comment on issue #11618: Support to turn off the sql echo at logging in dbapi_hook

[GitBox]

kaxil edited a comment on issue #11618:
URL: https://github.com/apache/airflow/issues/11618#issuecomment-967734317


   Yup, just add a feature to add `log_sql` kwarg to `DbApiHook` and then 
https://github.com/apache/airflow/pull/18718 should help in passing this kwargs 
to hook so that we don't need to add it to all Operators or just 
`BaseSQLOperator`
   
   cc @denimalpaca


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] kaxil commented on issue #11618: Support to turn off the sql echo at logging in dbapi_hook

[GitBox]

kaxil commented on issue #11618:
URL: https://github.com/apache/airflow/issues/11618#issuecomment-967734317


   Yup, just add a feature to add `log_sql` kwarg to `DbApiHook` and then 
https://github.com/apache/airflow/pull/18718 should help in passing this kwargs 
to hook so that we don't need to add it to all Operators or just 
`BaseSQLOperator`


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] kaxil commented on pull request #18718: Add hook_params in BaseSqlOperator

[GitBox]

kaxil commented on pull request #18718:
URL: https://github.com/apache/airflow/pull/18718#issuecomment-967732763


   @potiuk @ephraimbuddy  Gentle Ping -- to verify this as you'll have 
requested changes earlier


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] kaxil closed issue #19559: Airflow 2.1.4 to 2.2 upgrade failed

[GitBox]

kaxil closed issue #19559:
URL: https://github.com/apache/airflow/issues/19559


   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] kaxil commented on issue #19559: Airflow 2.1.4 to 2.2 upgrade failed

[GitBox]

kaxil commented on issue #19559:
URL: https://github.com/apache/airflow/issues/19559#issuecomment-967732347


   I am going to close this since it is solved in 2.2.2rc2 which will be 
released on Monday.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] kaxil commented on issue #19566: Breeze - The CSS and JS files are not built

[GitBox]

kaxil commented on issue #19566:
URL: https://github.com/apache/airflow/issues/19566#issuecomment-967732200


   I think this error / delay is common among Mac users.
   
   If this happens, I have generally run `./airflow/www/compile_assets.sh` from 
the repo root. See if that works for you.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[airflow] branch main updated (8d9f2f4 -> d185c66)

[kaxilnaik]

This is an automated email from the ASF dual-hosted git repository.

kaxilnaik pushed a change to branch main
in repository https://gitbox.apache.org/repos/asf/airflow.git.


from 8d9f2f4  Fix typo on ``necessary`` word (#19565)
 add d185c66  Small formatting tweak to GCP SSH conn doc (#19562)

No new revisions were added by this update.

Summary of changes:
 docs/apache-airflow-providers-google/connections/gcp_ssh.rst | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)

[GitHub] [airflow] kaxil merged pull request #19562: Small formatting tweak to GCP SSH conn doc

[GitBox]

kaxil merged pull request #19562:
URL: https://github.com/apache/airflow/pull/19562


   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] mik-laj commented on a change in pull request #19518: Support impersonation_chain parameter in the GKEStartPodOperator

[GitBox]

mik-laj commented on a change in pull request #19518:
URL: https://github.com/apache/airflow/pull/19518#discussion_r748633431



##
File path: airflow/providers/google/cloud/operators/kubernetes_engine.py
##
@@ -285,6 +285,15 @@ class GKEStartPodOperator(KubernetesPodOperator):
 :param gcp_conn_id: The google cloud connection id to use. This allows for
 users to specify a service account.
 :type gcp_conn_id: str
+:param impersonation_chain: Optional service account to impersonate using 
short-term
+credentials, or chained list of accounts required to get the 
access_token

Review comment:
   the chained list of multiple accounts is not supported. We only support 
one element listt.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] mik-laj commented on a change in pull request #19518: Support impersonation_chain parameter in the GKEStartPodOperator

[GitBox]

mik-laj commented on a change in pull request #19518:
URL: https://github.com/apache/airflow/pull/19518#discussion_r748633274



##
File path: airflow/providers/google/cloud/operators/kubernetes_engine.py
##
@@ -350,6 +361,13 @@ def execute(self, context) -> Optional[str]:
 "--project",
 self.project_id,
 ]
+if self.impersonation_chain:

Review comment:
   Since we only support one element, we should throw in an exception when 
someone passes several elements in an array.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] mik-laj commented on a change in pull request #19518: Support impersonation_chain parameter in the GKEStartPodOperator

[GitBox]

mik-laj commented on a change in pull request #19518:
URL: https://github.com/apache/airflow/pull/19518#discussion_r748632907



##
File path: docs/apache-airflow-providers-google/connections/gcp.rst
##
@@ -160,10 +160,13 @@ access token, which will allow to act on its behalf using 
its permissions. ``imp
 does not even need to have a generated key.
 
 .. warning::
-  
:class:`~airflow.providers.google.cloud.operators.kubernetes_engine.GKEStartPodOperator`,
   
:class:`~airflow.providers.google.cloud.operators.dataflow.DataflowCreateJavaJobOperator`
 and
   
:class:`~airflow.providers.google.cloud.operators.dataflow.DataflowCreatePythonJobOperator`
-  do not support direct impersonation as of now.
+  do not support direct impersonation as of now. Both are deprecated and new 
operators
+  
:class:`~airflow.providers.apache.beam.operators.beam.BeamRunJavaPipelineOperator`
 and

Review comment:
   Did you check it? I think we are still missing the piece of code that 
will pass impersonation details to the Apache Beam subprocess. For now, only 
the monitoring process supports, but that might not be enough.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] github-actions[bot] commented on pull request #19562: Small formatting tweak to GCP SSH conn doc

[GitBox]

github-actions[bot] commented on pull request #19562:
URL: https://github.com/apache/airflow/pull/19562#issuecomment-967715379


   The PR is likely ready to be merged. No tests are needed as no important 
environment files, nor python files were modified by it. However, committers 
might decide that full test matrix is needed and add the 'full tests needed' 
label. Then you should rebase it to the latest main or amend the last commit of 
the PR, and push it with --force-with-lease.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] mik-laj commented on issue #19548: Option to include subdir in trigger_dag to not make scheduler scan the whole dag folder

[GitBox]

mik-laj commented on issue #19548:
URL: https://github.com/apache/airflow/issues/19548#issuecomment-967714463


   As I can see in the code, this operator checks the path to DAG File in the 
database and only loads one file. I don't know what you would like to optimize 
here
   
https://github.com/apache/airflow/blob/37a12e9c278209d7e8ea914012a31a91a6c6ccff/airflow/api/common/experimental/trigger_dag.py#L117


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] mik-laj closed issue #19564: `KubernetesExecutor` can't use custom image registry

[GitBox]

mik-laj closed issue #19564:
URL: https://github.com/apache/airflow/issues/19564


   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] mik-laj commented on issue #19564: `KubernetesExecutor` can't use custom image registry

[GitBox]

mik-laj commented on issue #19564:
URL: https://github.com/apache/airflow/issues/19564#issuecomment-967712552


   Duplicate: https://github.com/apache/airflow/issues/10605


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] kaxil commented on pull request #18675: New Tree View

[GitBox]

kaxil commented on pull request #18675:
URL: https://github.com/apache/airflow/pull/18675#issuecomment-967702562


   Wohoo... great work @bbovenzi  
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[airflow] branch main updated (1975161 -> 8d9f2f4)

[kaxilnaik]

This is an automated email from the ASF dual-hosted git repository.

kaxilnaik pushed a change to branch main
in repository https://gitbox.apache.org/repos/asf/airflow.git.


from 1975161  New Tree View (#18675)
 add 8d9f2f4  Fix typo on ``necessary`` word (#19565)

No new revisions were added by this update.

Summary of changes:

[GitHub] [airflow] kaxil merged pull request #19565: Fix type on necessary word

[GitBox]

kaxil merged pull request #19565:
URL: https://github.com/apache/airflow/pull/19565


   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] github-actions[bot] commented on pull request #19565: Fix type on necessary word

[GitBox]

github-actions[bot] commented on pull request #19565:
URL: https://github.com/apache/airflow/pull/19565#issuecomment-967646652


   The PR most likely needs to run full matrix of tests because it modifies 
parts of the core of Airflow. However, committers might decide to merge it 
quickly and take the risk. If they don't merge it quickly - please rebase it to 
the latest main at your convenience, or amend the last commit of the PR, and 
push it with --force-with-lease.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] peloyeje opened a new issue #18632: Auto-refresh doesn't take into account the selected date

[GitBox]

peloyeje opened a new issue #18632:
URL: https://github.com/apache/airflow/issues/18632


   ### Apache Airflow version
   
   2.1.3
   
   ### Operating System
   
   Debian GNU/Linux 9 (stretch)
   
   ### Versions of Apache Airflow Providers
   
   _No response_
   
   ### Deployment
   
   Other Docker-based deployment
   
   ### Deployment details
   
   _No response_
   
   ### What happened
   
   In the DAG tree view, if I select a custom date in the date filter (top left 
corner) and press "update", DAG runs are correctly filtered to the selected 
date and number of runs.

   However, if the "auto-refresh" toggle is on, when the next tick refresh 
happens, the date filter is no longer taken into account and the UI displays 
the actual **latest** x DAG runs status. However, neither the header dates (45° 
angle) nor the date filter reflect this new time window 
   
   I investigated in the network inspector and it seems that the xhr request 
that fetches dag runs status doesn't contain a date parameter and thus always 
fetch the latest DAG run data
   
   ### What you expected to happen
   
   I expect that the auto-refresh feature preserves the selected time window
   
   ### How to reproduce
   
   Open a DAG with at least 20 dag runs
   Make sure autorefresh is disabled
   Select a filter date earlier than the 10th dag run start date and a "number 
of runs" value of 10, press "update"
   The DAG tree view should now be focused on the 10 first dag runs
   Now toggle autorefresh and wait for next tick
   The DAG tree view will now display status of the latest 10 runs but the 
header dates will still reflect the old start dates
   
   ### Anything else
   
   _No response_
   
   ### Are you willing to submit PR?
   
   - [X] Yes I am willing to submit a PR!
   
   ### Code of Conduct
   
   - [X] I agree to follow this project's [Code of 
Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] peloyeje closed issue #18632: Auto-refresh doesn't take into account the selected date

[GitBox]

peloyeje closed issue #18632:
URL: https://github.com/apache/airflow/issues/18632


   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] raphaelauv edited a comment on pull request #19324: Catch AccessDeniedException in AWS Secrets Manager Backend

[GitBox]

raphaelauv edited a comment on pull request #19324:
URL: https://github.com/apache/airflow/pull/19324#issuecomment-967341605


   this is already implemented in the gcp secret_manager -> 
https://github.com/apache/airflow/blob/4f1e66d0227fba7a1378e895b9186711c03ead61/airflow/providers/google/cloud/_internal_client/secret_manager_client.py#L92
   
   It make sence to have the same in the aws secret_manager


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] ryanahamilton commented on issue #18587: vertical spacing between tasks has increased significantly in Graph View

[GitBox]

ryanahamilton commented on issue #18587:
URL: https://github.com/apache/airflow/issues/18587#issuecomment-967566117


   > @ryanahamilton You updated the styling for the graph view a little while 
back. Was the extra spacing for labels? I'm curious to hear your opinion.
   > 
   > From the commit message where vertical spacing was changed: (#15257) `` - 
Slightly increases the vertical node separation (`nodeSep`) to make the graphs 
feel a little less crowded. This also reduces the amount that the task nodes 
overlap with the edges. ``
   
   Yeah [the `nodeSep` 
value](https://github.com/astronomer/airflow/blob/3bfb2312043e51cca4c75ce03fdce25b6a647d35/airflow/www/templates/airflow/graph.html#L162)
 was doubled from `15` to `30` to prevent overlapping of nodes and edges. I 
suppose we could test a smaller number in between? 
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] john-jac commented on pull request #19324: Catch AccessDeniedException in AWS Secrets Manager Backend

[GitBox]

john-jac commented on pull request #19324:
URL: https://github.com/apache/airflow/pull/19324#issuecomment-967559744


   > > this is already implemented in the gcp secret_manager ->
   > > 
https://github.com/apache/airflow/blob/4f1e66d0227fba7a1378e895b9186711c03ead61/airflow/providers/google/cloud/_internal_client/secret_manager_client.py#L92
   > > 
   > > It make sence to have the same in thie aws secret_manager
   > 
   > One difference is, there it does not catch _all_ exceptions but only 
certain ones.
   
   The intent was to future-proof the code, as regardless of the reason for the 
exception it should fall back.  If required the last catch-all can be removed.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] karakanb opened a new issue #19566: Breeze - The CSS and JS files are not built

[GitBox]

karakanb opened a new issue #19566:
URL: https://github.com/apache/airflow/issues/19566


   ### Describe the issue with documentation
   
   I followed the documentation about running Breeze to develop locally. I have 
set up the database properly, and all the containers are properly starting with 
Breeze. I am on a Macbook Pro 13-inch, 2020. This machine has 16GB RAM, and I 
have allocated 6GB RAM and 3CPUs to the Docker VM. I am using Docker for all my 
development work, so I am familiar with it.
   
   The final view I get is here:
   https://user-images.githubusercontent.com/16530606/141528323-2b6327b1-af38-400f-8980-9bc94fab.png;>
   
   The `yarn` step on the top right takes ~630 seconds on my machine, I guess a 
lot of it comes from the fact that the Docker networking is pretty bad on macOS 
due to the VM.
   
   Anyway, when I go to `localhost:28080`, I am getting a lot of 404s with CSS 
and JS files.
   
   https://user-images.githubusercontent.com/16530606/141528396-5036f017-1281-4be4-aade-acc82d0627ab.png;>
   
   Obviously, this breaks the UI. While going through the logs, I notice that 
there's a notice there: 
   ```
   WARNING: It seems that the generated assets files do not match the content 
of the sources
   ```
   
   Okay, cool. I stop the Breeze environment, run the command 
`./airflow/www/compile_assets.sh` and I have my assets generated quite quickly 
for me, nice. I start Breeze again. The logs don't complain about the assets 
not matching.
   
   There's a weird problem now:
   - The yarn command takes a long time, and that's fine.
   - While waiting for that to finish, refreshing the page makes everything 
look fine.
 - Although some icons are not loading, and the multi-select tags like the 
ones in the role create UI are not working, and there are still 404s in the 
network tab.
   - I wait until the yarn command finishes.
   - The UI is back to being completely broken.
   
   I presume there's something wrong with the yarn command that breaks the 
assets.
   
   One of the files that seems to be missing is a file called `ab.js`. I am 
searching my airflow folder for that to find a file with that name using `find 
. -name 'ab.js'`, but nothing matches.
   
   On the top left corner, I try to recompile the assets, this time inside the 
container, using the command `./airflow/www/compile_assets.sh`. This command 
takes ~80 seconds, and finishes. Now when I refresh the UI, it is partially 
back again:
   https://user-images.githubusercontent.com/16530606/141531523-650fdd17-742b-4ba0-891a-369304f37145.png;>
   
   At this point, I have no clue about what to fix or how to fix.
   
   
   
   
   
   
   ### How to solve the problem
   
   - I think the documentation has wrong levels of titles that makes it 
confusing to know what steps are alternatives to each other, and what are "must 
do" steps.
 - Docker Compose and pyenv are stated as alternatives to each other in 
"Note to Starters" section of the Contributers Quick Start guide, but then the 
prerequisites section for Ubuntu tells them in order as if both are required. 
   - It might be the case that pyenv is still required for Docker-based 
setup, and that's also fine, but it is not clear if I have to do that, so I 
skipped that because I assumed everything regarding Airflow and its 
dependencies would be covered by Docker containers instead of having a local 
environment.
   - Similarly, the same document is instructing to install `jq` without 
any usage of it. I agree that jq is an amazing tool and a must-have for many 
people, but if it is not relevant for this setup, why have it?
 - It might be the case that the installation script or Breeze requires 
it for some reason, but it'd be good to see this mentioned.
 - Also, if Breeze depends on it, why does it not check if it exists, 
just like it checks for Docker versions?
   - The quick start guide has pretty similar steps for VS Code and PyCharm 
separately, which makes the document confusing.
 -  I love the fact that there are specific instructions for both of the 
IDEs in the document, but are they really relevant for setting up the 
environment?
 - I think the primary focus of the document should be to get a contributor 
having their first local change and being able to see it over localhost.
 - Once this is achieved, the suggestions and instructions for each IDE 
could focus on improving the developer ergonomics.
 - As simple as it is, as a new contributor, I should be able to run 
Breeze, pick up `vim` or `nano` or whatever editor I want, edit an HTML file or 
a Python file somewhere, and immediately see the result of this in my browser 
over localhost.
 - After this, I'd obviously love to see what debugging opportunities does 
VS Code or PyCharm offer, but not before I have stuff running.
 - Also, the same confusion about Docker vs pyenv continues under these 
sections as well.
   -  For example, why 

[GitHub] [airflow] boring-cyborg[bot] commented on issue #19566: Breeze - The CSS and JS files are not built

[GitBox]

boring-cyborg[bot] commented on issue #19566:
URL: https://github.com/apache/airflow/issues/19566#issuecomment-967549963


   Thanks for opening your first issue here! Be sure to follow the issue 
template!
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] bbovenzi commented on issue #18632: Auto-refresh doesn't take into account the selected date

[GitBox]

bbovenzi commented on issue #18632:
URL: https://github.com/apache/airflow/issues/18632#issuecomment-967548594


   @peloyeje do you still want to take this on or should I finish this out? 
Quite a bit changed with #18675


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] ephraimbuddy commented on a change in pull request #19563: Add example SLA DAG

[GitBox]

ephraimbuddy commented on a change in pull request #19563:
URL: https://github.com/apache/airflow/pull/19563#discussion_r748583052



##
File path: airflow/example_dags/example_sla_dag.py
##
@@ -0,0 +1,56 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+import time
+from datetime import datetime, timedelta
+
+from airflow.decorators import dag, task
+
+"""Example DAG demonstrating SLA use in Tasks"""
+
+
+# [START howto_task_sla]
+def sla_callback(*args):
+arguments = ["dag", "task_list", "blocking_task_list", "slas", 
"blocking_tis"]
+print(*zip(arguments, args))
+
+
+@dag(
+"example_sla_dag",

Review comment:
   You're right. I'll remove that




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] ephraimbuddy commented on a change in pull request #19563: Add example SLA DAG

[GitBox]

ephraimbuddy commented on a change in pull request #19563:
URL: https://github.com/apache/airflow/pull/19563#discussion_r748582792



##
File path: airflow/example_dags/example_sla_dag.py
##
@@ -0,0 +1,56 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+import time
+from datetime import datetime, timedelta
+
+from airflow.decorators import dag, task
+
+"""Example DAG demonstrating SLA use in Tasks"""
+
+
+# [START howto_task_sla]
+def sla_callback(*args):

Review comment:
   The print function prints all the args, matching it correctly for the 
user to see. I am of the opinion that it won't be confusing. Is it?




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] dstandish commented on pull request #19324: Catch AccessDeniedException in AWS Secrets Manager Backend

[GitBox]

dstandish commented on pull request #19324:
URL: https://github.com/apache/airflow/pull/19324#issuecomment-967546660


   > this is already implemented in the gcp secret_manager ->
   > 
   > 
https://github.com/apache/airflow/blob/4f1e66d0227fba7a1378e895b9186711c03ead61/airflow/providers/google/cloud/_internal_client/secret_manager_client.py#L92
   > 
   > It make sence to have the same in thie aws secret_manager
   
   One difference is, there it does not catch _all_ exceptions but only certain 
ones.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] dstandish commented on a change in pull request #19356: Add influxdb operator

[GitBox]

dstandish commented on a change in pull request #19356:
URL: https://github.com/apache/airflow/pull/19356#discussion_r748570356



##
File path: airflow/providers/influxdb/operators/influxdb.py
##
@@ -0,0 +1,54 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+from typing import Dict
+
+from airflow.models import BaseOperator
+from airflow.providers.influxdb.hooks.influxdb import InfluxDBHook
+
+
+class InfluxDBOperator(BaseOperator):
+"""
+Executes sql code in a specific InfluxDB database
+
+.. seealso::
+For more information on how to use this operator, take a look at the 
guide:
+:ref:`howto/operator:InfluxDBOperator`
+
+:param sql: the sql code to be executed. Can receive a str representing a
+sql statement
+:type sql: str
+:param influxdb_conn_id: Reference to :ref:`Influxdb connection id 
`.
+:type influxdb_conn_id: str
+"""
+
+def __init__(
+self,
+*,
+sql: str,

Review comment:
   `sql` should probably be added as a templated field




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] josh-fell commented on a change in pull request #19356: Add influxdb operator

[GitBox]

josh-fell commented on a change in pull request #19356:
URL: https://github.com/apache/airflow/pull/19356#discussion_r748570846



##
File path: airflow/providers/influxdb/operators/influxdb.py
##
@@ -0,0 +1,54 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+from typing import Dict
+
+from airflow.models import BaseOperator
+from airflow.providers.influxdb.hooks.influxdb import InfluxDBHook
+
+
+class InfluxDBOperator(BaseOperator):
+"""
+Executes sql code in a specific InfluxDB database
+
+.. seealso::
+For more information on how to use this operator, take a look at the 
guide:
+:ref:`howto/operator:InfluxDBOperator`
+
+:param sql: the sql code to be executed. Can receive a str representing a
+sql statement
+:type sql: str
+:param influxdb_conn_id: Reference to :ref:`Influxdb connection id 
`.
+:type influxdb_conn_id: str
+"""
+
+def __init__(
+self,
+*,
+sql: str,
+influxdb_conn_id: str = 'influxdb_default',
+**kwargs,
+) -> None:
+super().__init__(**kwargs)
+self.influxdb_conn_id = influxdb_conn_id
+self.sql = sql
+self.hook = None

Review comment:
   Yes, I could have been clearer about that. The `self.hook` attr is not 
necessary here. Thanks for keeping me honest.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] dstandish commented on a change in pull request #19356: Add influxdb operator

[GitBox]

dstandish commented on a change in pull request #19356:
URL: https://github.com/apache/airflow/pull/19356#discussion_r748569737



##
File path: airflow/providers/influxdb/operators/influxdb.py
##
@@ -0,0 +1,54 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+from typing import Dict
+
+from airflow.models import BaseOperator
+from airflow.providers.influxdb.hooks.influxdb import InfluxDBHook
+
+
+class InfluxDBOperator(BaseOperator):
+"""
+Executes sql code in a specific InfluxDB database
+
+.. seealso::
+For more information on how to use this operator, take a look at the 
guide:
+:ref:`howto/operator:InfluxDBOperator`
+
+:param sql: the sql code to be executed. Can receive a str representing a
+sql statement
+:type sql: str
+:param influxdb_conn_id: Reference to :ref:`Influxdb connection id 
`.
+:type influxdb_conn_id: str
+"""
+
+def __init__(
+self,
+*,
+sql: str,
+influxdb_conn_id: str = 'influxdb_default',
+**kwargs,
+) -> None:
+super().__init__(**kwargs)
+self.influxdb_conn_id = influxdb_conn_id
+self.sql = sql
+self.hook = None

Review comment:
   pycharm grumbles when you create an attribute that isn't defined in init
   
   but if you're suggesting "don't bother creating a hook attr at all", i 
second the motion




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] josh-fell commented on a change in pull request #19356: Add influxdb operator

[GitBox]

josh-fell commented on a change in pull request #19356:
URL: https://github.com/apache/airflow/pull/19356#discussion_r748548197



##
File path: airflow/providers/influxdb/operators/influxdb.py
##
@@ -0,0 +1,54 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+from typing import Dict
+
+from airflow.models import BaseOperator
+from airflow.providers.influxdb.hooks.influxdb import InfluxDBHook
+
+
+class InfluxDBOperator(BaseOperator):
+"""
+Executes sql code in a specific InfluxDB database
+
+.. seealso::
+For more information on how to use this operator, take a look at the 
guide:
+:ref:`howto/operator:InfluxDBOperator`
+
+:param sql: the sql code to be executed. Can receive a str representing a
+sql statement
+:type sql: str
+:param influxdb_conn_id: Reference to :ref:`Influxdb connection id 
`.
+:type influxdb_conn_id: str
+"""
+
+def __init__(
+self,
+*,
+sql: str,
+influxdb_conn_id: str = 'influxdb_default',
+**kwargs,
+) -> None:
+super().__init__(**kwargs)
+self.influxdb_conn_id = influxdb_conn_id
+self.sql = sql
+self.hook = None

Review comment:
   ```suggestion
   ```
   This can be removed since the hook object is being instantiated only in the 
`execute()` method.

##
File path: airflow/providers/influxdb/example_dags/example_influxdb_query.py
##
@@ -0,0 +1,39 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+from datetime import datetime
+
+from airflow.models.dag import DAG
+from airflow.providers.influxdb.operators.influxdb import InfluxDBOperator
+
+dag = DAG(
+'example_influxdb_operator',
+start_date=datetime(2021, 1, 1),
+tags=['example'],
+catchup=False,
+)
+
+# [START howto_operator_influxdb]

Review comment:
   These `# [START/END howto_...]` tags are for including code snippets in 
docs using the `.. exampleinclude::` directive. This reference should be in the 
operator doc too to provide some more context for the `InfluxDBOperator`.
   
   Here is an example of how these tags are typically used in operator docs: 
https://github.com/apache/airflow/blob/main/docs/apache-airflow-providers-airbyte/operators/airbyte.rst




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] bbovenzi closed issue #16824: Initiating `mark success/failed` operation causes UI border to disappear for task instances with no status

[GitBox]

bbovenzi closed issue #16824:
URL: https://github.com/apache/airflow/issues/16824


   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] bbovenzi commented on issue #16824: Initiating `mark success/failed` operation causes UI border to disappear for task instances with no status

[GitBox]

bbovenzi commented on issue #16824:
URL: https://github.com/apache/airflow/issues/16824#issuecomment-967438026


   I believe this is now fixed by #18675


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] josh-fell commented on a change in pull request #19563: Add example SLA DAG

[GitBox]

josh-fell commented on a change in pull request #19563:
URL: https://github.com/apache/airflow/pull/19563#discussion_r748543962



##
File path: airflow/example_dags/example_sla_dag.py
##
@@ -0,0 +1,56 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+import time
+from datetime import datetime, timedelta
+
+from airflow.decorators import dag, task
+
+"""Example DAG demonstrating SLA use in Tasks"""
+
+
+# [START howto_task_sla]
+def sla_callback(*args):

Review comment:
   WDYT about having the same function name in the SLA Concepts guide and 
this example DAG? I'm thinking the SLA Concepts guide is a detailed breakdown 
of SLAs and this example DAG pulls it all together at the end. Having the 
functions named the same seems like a great way to bring some continuity to the 
doc.

##
File path: airflow/example_dags/example_sla_dag.py
##
@@ -0,0 +1,56 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+import time
+from datetime import datetime, timedelta
+
+from airflow.decorators import dag, task
+
+"""Example DAG demonstrating SLA use in Tasks"""
+
+
+# [START howto_task_sla]
+def sla_callback(*args):
+arguments = ["dag", "task_list", "blocking_task_list", "slas", 
"blocking_tis"]
+print(*zip(arguments, args))
+
+
+@dag(
+"example_sla_dag",

Review comment:
   Including the `dag_id` in the decorator isn't necessary since the 
`dag_id` will default to the function name being decorated. No strong opinion 
here though. It's more of a stylistic comment.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] bbovenzi closed issue #17148: DAGs with many Nodes Cannot be Rendered due to Python Recursion Limit

[GitBox]

bbovenzi closed issue #17148:
URL: https://github.com/apache/airflow/issues/17148


   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] bbovenzi closed issue #17717: Task names get covered by the tree view

[GitBox]

bbovenzi closed issue #17717:
URL: https://github.com/apache/airflow/issues/17717


   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] bbovenzi closed issue #16112: List only a single level of TaskGroup in tree view

[GitBox]

bbovenzi closed issue #16112:
URL: https://github.com/apache/airflow/issues/16112


   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] bbovenzi closed issue #13242: [ui] Tree view provides inconsistent representation of dependent task views

[GitBox]

bbovenzi closed issue #13242:
URL: https://github.com/apache/airflow/issues/13242


   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[airflow] branch main updated (4f1e66d -> 1975161)

[bbovenzi]

This is an automated email from the ASF dual-hosted git repository.

bbovenzi pushed a change to branch main
in repository https://gitbox.apache.org/repos/asf/airflow.git.


from 4f1e66d  Upload KinD logs on cancell too (#19554)
 add 1975161  New Tree View (#18675)

No new revisions were added by this update.

Summary of changes:
 .github/workflows/ci.yml   |   24 +
 .gitignore |3 +
 CI.rst |2 +
 CONTRIBUTING.rst   |   18 +-
 LICENSE|3 +
 airflow/www/.babelrc   |8 +
 airflow/www/.eslintrc  |8 +-
 airflow/{ui/.eslintrc.js => www/jest-setup.js} |   32 +-
 .../{static/js/variable_edit.js => jest.config.js} |   13 +-
 airflow/www/package.json   |   34 +-
 airflow/www/static/js/datetime_utils.js|   12 +
 airflow/www/static/js/graph.js |1 +
 airflow/www/static/js/tree.js  |  496 ---
 airflow/www/static/js/tree/InstanceTooltip.jsx |  145 +
 airflow/www/static/js/tree/StatusBox.jsx   |   72 +
 airflow/www/static/js/tree/Tree.jsx|   75 +
 airflow/www/static/js/tree/dagRuns/Bar.jsx |   86 +
 airflow/www/static/js/tree/dagRuns/Tooltip.jsx |   99 +
 airflow/www/static/js/tree/dagRuns/index.jsx   |  108 +
 airflow/www/static/js/tree/dagRuns/index.test.jsx  |  158 +
 .../src/index.tsx => www/static/js/tree/index.jsx} |   50 +-
 airflow/www/static/js/tree/renderTaskRows.jsx  |  172 +
 airflow/www/static/js/tree/renderTaskRows.test.jsx |  169 +
 airflow/www/static/js/tree/useTreeData.js  |  103 +
 airflow/www/static/js/tree/useTreeData.test.js |  100 +
 airflow/www/templates/airflow/dag.html |4 +-
 airflow/www/templates/airflow/tree.html|   44 +-
 airflow/www/utils.py   |   45 +-
 airflow/www/views.py   |  233 +-
 airflow/www/webpack.config.js  |   30 +-
 airflow/www/yarn.lock  | 4266 +++-
 scripts/ci/selective_ci_checks.sh  |   26 +
 setup.cfg  |1 +
 tests/www/views/test_views_tasks.py|   23 -
 34 files changed, 5842 insertions(+), 821 deletions(-)
 create mode 100644 airflow/www/.babelrc
 copy airflow/{ui/.eslintrc.js => www/jest-setup.js} (67%)
 copy airflow/www/{static/js/variable_edit.js => jest.config.js} (81%)
 delete mode 100644 airflow/www/static/js/tree.js
 create mode 100644 airflow/www/static/js/tree/InstanceTooltip.jsx
 create mode 100644 airflow/www/static/js/tree/StatusBox.jsx
 create mode 100644 airflow/www/static/js/tree/Tree.jsx
 create mode 100644 airflow/www/static/js/tree/dagRuns/Bar.jsx
 create mode 100644 airflow/www/static/js/tree/dagRuns/Tooltip.jsx
 create mode 100644 airflow/www/static/js/tree/dagRuns/index.jsx
 create mode 100644 airflow/www/static/js/tree/dagRuns/index.test.jsx
 copy airflow/{ui/src/index.tsx => www/static/js/tree/index.jsx} (55%)
 create mode 100644 airflow/www/static/js/tree/renderTaskRows.jsx
 create mode 100644 airflow/www/static/js/tree/renderTaskRows.test.jsx
 create mode 100644 airflow/www/static/js/tree/useTreeData.js
 create mode 100644 airflow/www/static/js/tree/useTreeData.test.js

[GitHub] [airflow] bbovenzi merged pull request #18675: New Tree View

[GitBox]

bbovenzi merged pull request #18675:
URL: https://github.com/apache/airflow/pull/18675


   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] bbovenzi closed issue #18009: Task groups are not stable in the UI

[GitBox]

bbovenzi closed issue #18009:
URL: https://github.com/apache/airflow/issues/18009


   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] john-jac commented on pull request #19137: Add RedshiftDataHook

[GitBox]

john-jac commented on pull request #19137:
URL: https://github.com/apache/airflow/pull/19137#issuecomment-967357361


   > > @ephraimbuddy I don't see other similar operators, such as sagemaker or 
glue, using system tests. Are we sure this is necessary for redshift-data?
   > 
   > Agree that some other operators don't have it but this will really help us 
to validate your example dag and the operator. No strong opinion though
   
   OK as a best practice I will add it.  Thanks!


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] ephraimbuddy commented on pull request #19137: Add RedshiftDataHook

[GitBox]

ephraimbuddy commented on pull request #19137:
URL: https://github.com/apache/airflow/pull/19137#issuecomment-967347839


   > @ephraimbuddy I don't see other similar operators, such as sagemaker or 
glue, using system tests. Are we sure this is necessary for redshift-data?
   
   Agree that some other operators don't have it but this will really help us 
to validate your example dag and the operator. No strong opinion though
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] john-jac commented on a change in pull request #19137: Add RedshiftDataHook

[GitBox]

john-jac commented on a change in pull request #19137:
URL: https://github.com/apache/airflow/pull/19137#discussion_r748519054



##
File path: tests/providers/amazon/aws/hooks/test_redshift_data.py
##
@@ -0,0 +1,76 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+#
+
+import unittest
+from unittest import mock
+
+from airflow.providers.amazon.aws.hooks.redshift_data import RedshiftDataHook
+
+

Review comment:
   Working on adding a system test and improving the unit test...will 
update PR when ready.  Thanks!




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] khalidmammadov opened a new pull request #19565: Fix type on necessary word

[GitBox]

khalidmammadov opened a new pull request #19565:
URL: https://github.com/apache/airflow/pull/19565


   This is to fix typo on private function parameter name. 
   
   ---
   **^ Add meaningful description above**
   
   Read the **[Pull Request 
Guidelines](https://github.com/apache/airflow/blob/main/CONTRIBUTING.rst#pull-request-guidelines)**
 for more information.
   In case of fundamental code change, Airflow Improvement Proposal 
([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvements+Proposals))
 is needed.
   In case of a new dependency, check compliance with the [ASF 3rd Party 
License Policy](https://www.apache.org/legal/resolved.html#category-x).
   In case of backwards incompatible changes please leave a note in 
[UPDATING.md](https://github.com/apache/airflow/blob/main/UPDATING.md).
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] raphaelauv edited a comment on pull request #19324: Catch AccessDeniedException in AWS Secrets Manager Backend

[GitBox]

raphaelauv edited a comment on pull request #19324:
URL: https://github.com/apache/airflow/pull/19324#issuecomment-967341605


   this is already implemented in the gcp secret_manager -> 
https://github.com/apache/airflow/blob/4f1e66d0227fba7a1378e895b9186711c03ead61/airflow/providers/google/cloud/_internal_client/secret_manager_client.py#L92
   
   It make sence to have the same in thie aws secret_manager


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] raphaelauv commented on pull request #19324: Catch AccessDeniedException in AWS Secrets Manager Backend

[GitBox]

raphaelauv commented on pull request #19324:
URL: https://github.com/apache/airflow/pull/19324#issuecomment-967341605


   this already implemented in the gcp secret_manager -> 
https://github.com/apache/airflow/blob/4f1e66d0227fba7a1378e895b9186711c03ead61/airflow/providers/google/cloud/_internal_client/secret_manager_client.py#L92
   
   It make sence to have the same in thie aws secret_manager


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] lshw42 opened a new issue #19564: `KubernetesExecutor` can't use custom image registry

[GitBox]

lshw42 opened a new issue #19564:
URL: https://github.com/apache/airflow/issues/19564


   ### Apache Airflow version
   
   2.1.4
   
   ### Operating System
   
   Debian
   
   ### Versions of Apache Airflow Providers
   
   apache-airflow==2.1.4
   apache-airflow-providers-amazon==2.2.0
   apache-airflow-providers-celery==2.0.0
   apache-airflow-providers-cncf-kubernetes==2.0.2
   apache-airflow-providers-docker==2.1.1
   apache-airflow-providers-elasticsearch==2.0.3
   apache-airflow-providers-ftp==2.0.1
   apache-airflow-providers-google==5.1.0
   apache-airflow-providers-grpc==2.0.1
   apache-airflow-providers-hashicorp==2.1.0
   apache-airflow-providers-http==2.0.1
   apache-airflow-providers-imap==2.0.1
   apache-airflow-providers-microsoft-azure==3.1.1
   apache-airflow-providers-mysql==2.1.1
   apache-airflow-providers-postgres==2.2.0
   apache-airflow-providers-redis==2.0.1
   apache-airflow-providers-sendgrid==2.0.1
   apache-airflow-providers-sftp==2.1.1
   apache-airflow-providers-slack==4.0.1
   apache-airflow-providers-sqlite==2.0.1
   apache-airflow-providers-ssh==2.1.1
   
   ### Deployment
   
   Official Apache Airflow Helm Chart
   
   ### Deployment details
   
   - Kubernetes 1.17 (managed)
   - kubectl 1.19
   - Airflow 2.1.4
   - Airflow Helm chart 8.5.2 (Community)
   - Custom registry to pull `alpine` from, configured through 
`airflow_local_settings.py` provided as k8s Secret which is configured as 
`extraVolumeMounts`in `values.yaml` for helm chart
   - KubernetesExecutor with `AIRFLOW__KUBERNETES__WORKER_CONTAINER_REPOSITORY` 
set
   
   ### What happened
   
   In the DAG the tasks are using `KubernetesPodOperator` like the following:
   ```python
   test1 = KubernetesPodOperator(
   task_id = 'test1',
   name = 'tester_test1',
   namespace = 'airflow',
   image = 'custom path to image',
   image_pull_policy = 'IfNotPresent',
   image_pull_secrets = [k8s.V1LocalObjectReference('...')],
   do_xcom_push = True,
   get_logs = True,
   in_cluster = True,
   is_delete_operator_pod = True,
   init_containers = [init_container],
   dag = dag
   )
   ```
   When triggering the DAG and starting a task the output of `kubectl describe 
pod  -n airflow` is like:
   
   > Failed to pull image "alpine": rpc error: code = Unknown desc = Error 
response from daemon: toomanyrequests: You have reached your pull rate limit. 
You may increase the limit by authenticating and upgrading: 
https://www.docker.com/increase-rate-limit
   
   ### What you expected to happen
   
   When triggering the DAG and starting a task the expected output of `kubectl 
describe pod  -n airflow` is like:
   
   > Successfully pulled image "alpine"
   
   This problem is well known and common solution is setting 
`airflow_local_settings.py` as described in the [Airflow Helm chart (Community) 
FAQ](https://artifacthub.io/packages/helm/airflow-helm/airflow#how-to-set-airflow-configs).
   
   As far as I understand, this is related to the `KubernetesExecutor`, because 
it is not possible to configure the executor to inject a configuration into the 
used image for the worker Pod. 
   
   It is possible to configure the `KubernetesExecutor` to use a custom image 
from a custom registry by setting 
`AIRFLOW__KUBERNETES__WORKER_CONTAINER_REPOSITORY` and 
`AIRFLOW__KUBERNETES__WORKER_CONTAINER_TAG`. But it is not possible to build a 
custom Airflow 2.1.4 image which is already configured to use the custom 
registry for pulling `alpine` from.
   
   ### How to reproduce
   
   1. Setup Airflow 2.1.4 on Kubernetes using Helm chart 8.5.2 (community)
   2. Configure `airflow_local_settings.py` as described in Helm chart 8.5.2 
(community) FAQ
   3. Use `KubernetesPodOperator` with `do_xcom_push=True`
   4. Trigger DAG that launch the task
   
   ### Anything else
   
   This problem occures every time the Docker pull limit is hit.
   
   `kubectl describe pod tester-test1 -n airflow` returns at end of output:
   
   **Events:**
   | Type | Reason | Age | From | Message |
   ||--|||---|
   |  Normal | Scheduled | 21s | default-scheduler | Successfully assigned 
tester-test1 | 
   |  Normal | Pulled | 20s | kubelet | Container image 
"custom/repo/alpine:3.14.0" already present on machine | 
   |  Normal | Created | 20s | kubelet | Created container init-container | 
   |  Normal | Started | 19s | kubelet | Started container init-container | 
   |  Normal | Pulled | 19s | kubelet | Container image 
"custom/repo/xcom_test:latest" already present on machine | 
   |  Normal | Created | 19s | kubelet | Created container base | 
   |  Normal | Started | 19s | kubelet | Started container base | 
   |  Normal | BackOff | 16s (x2 over 17s) | kubelet | Back-off pulling image 
"alpine" | 
   |  Warning | Failed | 16s (x2 over 17s) | kubelet | Error: ImagePullBackOff 
| 
   |  Normal | Pulling | 4s (x2 over 19s) | kubelet | Pulling image "alpine" | 
   |  Warning | Failed 

[GitHub] [airflow] boring-cyborg[bot] commented on issue #19564: `KubernetesExecutor` can't use custom image registry

[GitBox]

boring-cyborg[bot] commented on issue #19564:
URL: https://github.com/apache/airflow/issues/19564#issuecomment-967333434


   Thanks for opening your first issue here! Be sure to follow the issue 
template!
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] nediGit commented on issue #19556: Checking roles and users list in Web UI, give me an error

[GitBox]

nediGit commented on issue #19556:
URL: https://github.com/apache/airflow/issues/19556#issuecomment-967307928


   Sure, selecting List Users
   
![image](https://user-images.githubusercontent.com/29753976/141512709-2790d0f4-da48-403f-b19e-777ba5ae7df7.png)
   
   And the error:
   
   Something bad has happened.
   Please consider letting us know by creating a bug report using GitHub.
   
   Python version: 3.6.8
   Airflow version: 2.1.2
   Node: robots-kube003.nosis.interno
   
---
   Traceback (most recent call last):
 File 
"/home/dlabrador@nosis.interno/.local/lib/python3.6/site-packages/flask/app.py",
 line 2447, in wsgi_app
   response = self.full_dispatch_request()
 File 
"/home/dlabrador@nosis.interno/.local/lib/python3.6/site-packages/flask/app.py",
 line 1952, in full_dispatch_request
   rv = self.handle_user_exception(e)
 File 
"/home/dlabrador@nosis.interno/.local/lib/python3.6/site-packages/flask/app.py",
 line 1821, in handle_user_exception
   reraise(exc_type, exc_value, tb)
 File 
"/home/dlabrador@nosis.interno/.local/lib/python3.6/site-packages/flask/_compat.py",
 line 39, in reraise
   raise value
 File 
"/home/dlabrador@nosis.interno/.local/lib/python3.6/site-packages/flask/app.py",
 line 1950, in full_dispatch_request
   rv = self.dispatch_request()
 File 
"/home/dlabrador@nosis.interno/.local/lib/python3.6/site-packages/flask/app.py",
 line 1936, in dispatch_request
   return self.view_functions[rule.endpoint](**req.view_args)
 File 
"/home/dlabrador@nosis.interno/.local/lib/python3.6/site-packages/flask_appbuilder/security/decorators.py",
 line 109, in wraps
   return f(self, *args, **kwargs)
 File 
"/home/dlabrador@nosis.interno/.local/lib/python3.6/site-packages/flask_appbuilder/views.py",
 line 551, in list
   widgets = self._list()
 File 
"/home/dlabrador@nosis.interno/.local/lib/python3.6/site-packages/flask_appbuilder/baseviews.py",
 line 1136, in _list
   form = self.search_form.refresh()
 File 
"/home/dlabrador@nosis.interno/.local/lib/python3.6/site-packages/flask_appbuilder/forms.py",
 line 329, in refresh
   form = self(obj=obj)
 File "/usr/local/lib/python3.6/site-packages/wtforms/form.py", line 208, 
in __call__
   return type.__call__(cls, *args, **kwargs)
 File 
"/home/dlabrador@nosis.interno/.local/lib/python3.6/site-packages/flask_wtf/form.py",
 line 87, in __init__
   super(FlaskForm, self).__init__(formdata=formdata, **kwargs)
 File "/usr/local/lib/python3.6/site-packages/wtforms/form.py", line 268, 
in __init__
   super(Form, self).__init__(self._unbound_fields, meta=meta_obj, 
prefix=prefix)
 File "/usr/local/lib/python3.6/site-packages/wtforms/form.py", line 51, in 
__init__
   field = meta.bind_field(self, unbound_field, options)
 File "/usr/local/lib/python3.6/site-packages/wtforms/meta.py", line 27, in 
bind_field
   return unbound_field.bind(form=form, **options)
 File "/usr/local/lib/python3.6/site-packages/wtforms/fields/core.py", line 
376, in bind
   return self.field_class(*self.args, **kw)
 File 
"/home/dlabrador@nosis.interno/.local/lib/python3.6/site-packages/flask_appbuilder/fields.py",
 line 182, in __init__
   "allow_blank=True does not do anything for QuerySelectMultipleField."
 File "/usr/lib64/python3.6/warnings.py", line 99, in _showwarnmsg
   msg.file, msg.line)
 File 
"/home/dlabrador@nosis.interno/.local/lib/python3.6/site-packages/airflow/settings.py",
 line 114, in custom_show_warning
   write_console.print(msg, soft_wrap=True)
 File 
"/home/dlabrador@nosis.interno/.local/lib/python3.6/site-packages/rich/console.py",
 line 1601, in print
   self._buffer.extend(new_segments)
 File 
"/home/dlabrador@nosis.interno/.local/lib/python3.6/site-packages/rich/console.py",
 line 815, in __exit__
   self._exit_buffer()
 File 
"/home/dlabrador@nosis.interno/.local/lib/python3.6/site-packages/rich/console.py",
 line 774, in _exit_buffer
   self._check_buffer()
 File 
"/home/dlabrador@nosis.interno/.local/lib/python3.6/site-packages/rich/console.py",
 line 1824, in _check_buffer
   self.file.write(text)
   OSError: [Errno 5] Input/output error


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] AlekseiSaff commented on issue #19559: Airflow 2.1.4 to 2.2 upgrade failed

[GitBox]

AlekseiSaff commented on issue #19559:
URL: https://github.com/apache/airflow/issues/19559#issuecomment-967275903


   2.2.2rc2 works, but I'm going to rollback to 2.1.4 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] theister commented on issue #18501: Scheduler overloaded when backfilling by clearing DAG history

[GitBox]

theister commented on issue #18501:
URL: https://github.com/apache/airflow/issues/18501#issuecomment-967271227


   Well, I actually meant the latest release in the `2.2.*` line, I didn't 
check what the latest one was.
   
   I'll see if I can find a way to reproduce the behaviour on a local setup on 
our current images next week, that will also help triaging if it's fixed on the 
latest stable 2.2.1 or RCs of 2.2.2.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] ephraimbuddy opened a new pull request #19563: Add example SLA DAG

[GitBox]

ephraimbuddy opened a new pull request #19563:
URL: https://github.com/apache/airflow/pull/19563


   This PR adds example SLA DAG
   
   ---
   **^ Add meaningful description above**
   
   Read the **[Pull Request 
Guidelines](https://github.com/apache/airflow/blob/main/CONTRIBUTING.rst#pull-request-guidelines)**
 for more information.
   In case of fundamental code change, Airflow Improvement Proposal 
([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvements+Proposals))
 is needed.
   In case of a new dependency, check compliance with the [ASF 3rd Party 
License Policy](https://www.apache.org/legal/resolved.html#category-x).
   In case of backwards incompatible changes please leave a note in 
[UPDATING.md](https://github.com/apache/airflow/blob/main/UPDATING.md).
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] dstandish commented on issue #18501: Scheduler overloaded when backfilling by clearing DAG history

[GitBox]

dstandish commented on issue #18501:
URL: https://github.com/apache/airflow/issues/18501#issuecomment-967259369


   @theister  re
   
   > @leonsmith did you manage to find out if the issue is still present on 
2.2.0?
   
   There _are_ still a few issues with 2.2.0; I think that you're going to want 
to go straight to 2.2.2, and there's an RC available to try:
   ```pip install 'apache-airflow==2.2.2rc2'```


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] AlekseiSaff commented on issue #19559: Airflow 2.1.4 to 2.2 upgrade failed

[GitBox]

AlekseiSaff commented on issue #19559:
URL: https://github.com/apache/airflow/issues/19559#issuecomment-967257077


   MariaDB 10.5.11
   Let me check 2.2.2rc2


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] theister edited a comment on issue #18501: Scheduler overloaded when backfilling by clearing DAG history

[GitBox]

theister edited a comment on issue #18501:
URL: https://github.com/apache/airflow/issues/18501#issuecomment-967255792


   We came across the same problem, but on `2.0.2`.
   
   We recently migrated from 1.10.15 and we're also regularly doing backfills 
by clearing history via the UI and making the airflow catchup re-run them, 
which puts 100s of dag runs to `running` state at the same time.
   While the reprocessing the data eventually succeeded, it completely starved 
out all other dags in the meanwhile, printing many lines of  
   `DAG XYZ already has 10 active runs, not queuing any tasks for run 
2021-06-30 00:00:00+00:00` .
   to the scheduler logs. These were printed for the for the 10 next execution 
dates where no tasks were scheduled for just yet, which corresponds to our 
scheduler setting of `max_dagruns_per_loop_to_schedule=10`.
   
   After digging into the code a little, my undertanding is that if the 
`max_active_runs` limit is hit, the 2.0.2 scheduler prints the above message, 
but returns from `_schedule_dag_run()` without actually updating the 
`last_scheduling_decision` timestamp of the DagRun (See 
https://github.com/apache/airflow/blob/2.0.2/airflow/jobs/scheduler_job.py#L1776),
 which to my understanding only happens in `update_state()`.
   
   Since the `DagRun.next_dagruns_to_examine()` method returns the next DagRuns 
to check sorted ascending by last_scheduling_decision, this will effectively 
block any other dag runs from being scheduled, as long as the max_active_runs 
limit of the dag is being hit.
   
   @leonsmith did you manage to find out if the issue is still present on 2.2.0?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] theister commented on issue #18501: Scheduler overloaded when backfilling by clearing DAG history

[GitBox]

theister commented on issue #18501:
URL: https://github.com/apache/airflow/issues/18501#issuecomment-967255792


   We came across the same problem, but on `2.0.2`.
   
   We recently migrated from 1.10.15 and we're also regularly doing backfills 
by clearing history via the UI and making the airflow catchup re-run them, 
which puts 100s of dag runs to `running` state.
   While the reprocessing the data eventually succeeded, it completely starved 
out all other dags in the meanwhile, printing many lines of  
   `DAG XYZ already has 10 active runs, not queuing any tasks for run 
2021-06-30 00:00:00+00:00` .
   to the scheduler logs. These were printed for the for the 10 next execution 
dates where no tasks were scheduled for just yet, which corresponds to our 
scheduler setting of `max_dagruns_per_loop_to_schedule=10`.
   
   After digging into the code a little, my undertanding is that if the 
`max_active_runs` limit is hit, the 2.0.2 scheduler prints the above message, 
but returns from `_schedule_dag_run()` without actually updating the 
`last_scheduling_decision` timestamp of the DagRun (See 
https://github.com/apache/airflow/blob/2.0.2/airflow/jobs/scheduler_job.py#L1776),
 which to my understanding only happens in `update_state()`.
   
   Since the `DagRun.next_dagruns_to_examine()` method returns the next DagRuns 
to check sorted ascending by last_scheduling_decision, this will effectively 
block any other dag runs from being scheduled, as long as the max_active_runs 
limit of the dag is being hit.
   
   @leonsmith did you manage to find out if the issue is still present on 2.2.0?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] PApostol commented on issue #19558: Status of testing of Apache Airflow 2.2.2rc2

[GitBox]

PApostol commented on issue #19558:
URL: https://github.com/apache/airflow/issues/19558#issuecomment-967255326


   #19211 works with 2.2.2rc2


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] subkanthi commented on pull request #19356: Add influxdb operator

[GitBox]

subkanthi commented on pull request #19356:
URL: https://github.com/apache/airflow/pull/19356#issuecomment-967247205


   Fixed all doc errors.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] leonsmith commented on issue #19558: Status of testing of Apache Airflow 2.2.2rc2

[GitBox]

leonsmith commented on issue #19558:
URL: https://github.com/apache/airflow/issues/19558#issuecomment-967246892


   We have 2.2.2rc2 running in one of our envs for over the weekend initial 
testing looks good.
   
   Confirmed #19528 and #19258 fixed


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] ephraimbuddy closed pull request #19553: Fix IntegrityError in `DagFileProcessor.manage_slas`

[GitBox]

ephraimbuddy closed pull request #19553:
URL: https://github.com/apache/airflow/pull/19553


   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] josh-fell opened a new pull request #19562: Small formatting tweak to GCP SSH conn doc

[GitBox]

josh-fell opened a new pull request #19562:
URL: https://github.com/apache/airflow/pull/19562


   The newline in the bulleted list was causing a line item to become a bolded 
blockquote.
   
   ---
   **^ Add meaningful description above**
   
   Read the **[Pull Request 
Guidelines](https://github.com/apache/airflow/blob/main/CONTRIBUTING.rst#pull-request-guidelines)**
 for more information.
   In case of fundamental code change, Airflow Improvement Proposal 
([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvements+Proposals))
 is needed.
   In case of a new dependency, check compliance with the [ASF 3rd Party 
License Policy](https://www.apache.org/legal/resolved.html#category-x).
   In case of backwards incompatible changes please leave a note in 
[UPDATING.md](https://github.com/apache/airflow/blob/main/UPDATING.md).
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] kaxil commented on issue #19559: Airflow 2.1.4 to 2.2 upgrade failed

[GitBox]

kaxil commented on issue #19559:
URL: https://github.com/apache/airflow/issues/19559#issuecomment-967227018


   What version of MySQL are you using, can you test it with `2.2.2rc` too 
please.
   
   ```
   pip install -U apache-airflow==2.2.2rc2
   ```


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] boring-cyborg[bot] commented on issue #19559: Airflow 2.1.4 to 2.2 upgrade failed

[GitBox]

boring-cyborg[bot] commented on issue #19559:
URL: https://github.com/apache/airflow/issues/19559#issuecomment-967218761


   Thanks for opening your first issue here! Be sure to follow the issue 
template!
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] AlekseiSaff opened a new issue #19559: Airflow 2.1.4 to 2.2 upgrade failed

[GitBox]

AlekseiSaff opened a new issue #19559:
URL: https://github.com/apache/airflow/issues/19559


   ### Apache Airflow version
   
   2.1.4
   
   ### Operating System
   
   CentOS 7.9
   
   ### Versions of Apache Airflow Providers
   
   during aurflow db upgrade receiving an error
   
   > INFO  [alembic.runtime.migration] Running upgrade ccde3e26fe78 -> 
83f031fd9f1c, improve mssql compatibility
   INFO  [alembic.runtime.migration] Running upgrade 83f031fd9f1c -> 
e9304a3141f0, make xcom pkey columns non-nullable
   INFO  [alembic.runtime.migration] Running upgrade e9304a3141f0 -> 
30867afad44a, Rename concurrency column in dag table to max_active_tasks
   INFO  [alembic.runtime.migration] Running upgrade 30867afad44a -> 
54bebd308c5f, Add trigger table and task info
   INFO  [alembic.runtime.migration] Running upgrade 54bebd308c5f -> 
142555e44c17, Add data_interval_[start|end] to DagModel and DagRun.
   INFO  [alembic.runtime.migration] Running upgrade 142555e44c17 -> 
7b2661a43ba3, TaskInstance keyed to DagRun
   /usr/local/lib64/python3.6/site-packages/sqlalchemy/engine/default.py:608 
Warning: (1280, "Name 'task_instance_pkey' ignored for PRIMARY key.")
   Traceback (most recent call last):
 File "/usr/local/lib64/python3.6/site-packages/sqlalchemy/engine/base.py", 
line 1277, in _execute_context
   cursor, statement, parameters, context
 File 
"/usr/local/lib64/python3.6/site-packages/sqlalchemy/engine/default.py", line 
608, in do_execute
   cursor.execute(statement, parameters)
 File "/usr/local/lib64/python3.6/site-packages/MySQLdb/cursors.py", line 
255, in execute
   self.errorhandler(self, exc, value)
 File "/usr/local/lib64/python3.6/site-packages/MySQLdb/connections.py", 
line 50, in defaulterrorhandler
   raise errorvalue
 File "/usr/local/lib64/python3.6/site-packages/MySQLdb/cursors.py", line 
252, in execute
   res = self._query(query)
 File "/usr/local/lib64/python3.6/site-packages/MySQLdb/cursors.py", line 
378, in _query
   db.query(q)
 File "/usr/local/lib64/python3.6/site-packages/MySQLdb/connections.py", 
line 280, in query
   _mysql.connection.query(self, query)
   _mysql_exceptions.OperationalError: (1005, 'Can\'t create table 
`airflow`.`task_reschedule` (errno: 150 "Foreign key constraint is incorrectly 
formed")')
   
   The above exception was the direct cause of the following exception:
   
   Traceback (most recent call last):
 File "/usr/local/bin/airflow", line 8, in 
   sys.exit(main())
 File "/usr/local/lib/python3.6/site-packages/airflow/__main__.py", line 
48, in main
   args.func(args)
 File "/usr/local/lib/python3.6/site-packages/airflow/cli/cli_parser.py", 
line 48, in command
   return func(*args, **kwargs)
 File "/usr/local/lib/python3.6/site-packages/airflow/utils/cli.py", line 
92, in wrapper
   return f(*args, **kwargs)
 File 
"/usr/local/lib/python3.6/site-packages/airflow/cli/commands/db_command.py", 
line 48, in upgradedb
   db.upgradedb()
 File "/usr/local/lib/python3.6/site-packages/airflow/utils/session.py", 
line 70, in wrapper
   return func(*args, session=session, **kwargs)
 File "/usr/local/lib/python3.6/site-packages/airflow/utils/db.py", line 
868, in upgradedb
   command.upgrade(config, 'heads')
 File "/usr/local/lib/python3.6/site-packages/alembic/command.py", line 
298, in upgrade
   script.run_env()
 File "/usr/local/lib/python3.6/site-packages/alembic/script/base.py", line 
489, in run_env
   util.load_python_file(self.dir, "env.py")
 File "/usr/local/lib/python3.6/site-packages/alembic/util/pyfiles.py", 
line 98, in load_python_file
   module = load_module_py(module_id, path)
 File "/usr/local/lib/python3.6/site-packages/alembic/util/compat.py", line 
184, in load_module_py
   spec.loader.exec_module(module)
 File "", line 678, in exec_module
 File "", line 219, in 
_call_with_frames_removed
 File "/usr/local/lib/python3.6/site-packages/airflow/migrations/env.py", 
line 107, in 
   run_migrations_online()
 File "/usr/local/lib/python3.6/site-packages/airflow/migrations/env.py", 
line 101, in run_migrations_online
   context.run_migrations()
 File "", line 8, in run_migrations
 File 
"/usr/local/lib/python3.6/site-packages/alembic/runtime/environment.py", line 
846, in run_migrations
   self.get_context().run_migrations(**kw)
 File 
"/usr/local/lib/python3.6/site-packages/alembic/runtime/migration.py", line 
520, in run_migrations
   step.migration_fn(**kw)
 File 
"/usr/local/lib/python3.6/site-packages/airflow/migrations/versions/7b2661a43ba3_taskinstance_keyed_to_dagrun.py",
 line 321, in upgrade
   ondelete=ondelete,
 File "/usr/lib64/python3.6/contextlib.py", line 88, in __exit__
   next(self.gen)
 File "/usr/local/lib/python3.6/site-packages/alembic/operations/base.py", 
line 354, in batch_alter_table
   impl.flush()
 File 

[GitHub] [airflow] potiuk commented on pull request #19557: add kubernetes 1.21 support

[GitBox]

potiuk commented on pull request #19557:
URL: https://github.com/apache/airflow/pull/19557#issuecomment-967218646


   We should also update those version in README and docs - there are couple of 
places to do so. 
   
   Also we should remove both 1.18 and 1.19: 
   
   Here is our poluicy: 
https://github.com/apache/airflow/blob/main/README.md#support-for-python-and-kubernetes-versions
   
   K8S 1.18 EOL was 4 months ago, 1.19 EOL was 2 weeks ago - via: 
https://endoflife.date/kubernetes


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] potiuk commented on a change in pull request #18675: New Tree View

[GitBox]

potiuk commented on a change in pull request #18675:
URL: https://github.com/apache/airflow/pull/18675#discussion_r748385017



##
File path: airflow/www/webpack.config.js
##
@@ -35,6 +36,22 @@ const JS_DIR = path.resolve(__dirname, './static/js');
 // noinspection JSUnresolvedVariable
 const BUILD_DIR = path.resolve(__dirname, './static/dist');
 
+// Convert licenses json into a standard format for LICENSES.txt
+const formatLicenses = (packages) => {
+  let text = `Apache Airflow
+Copyright 2016-2021 The Apache Software Foundation

Review comment:
   I believe it's not allowed.
   
   My interpretation is that you have to state what was the year your 
contributions were made and it should not be 
 a) future, because for example it would limit you so that you could not 
change licencing scheme until it expires 
 b) `present` verbatim - because it is meaningless in the context of 
"writer". The `present` only makes sense in the context of "reader" because you 
do not keep in the file the date when "present" was put. However, the licence 
file needs the context of "writer", so you need to specify when was the time 
the licence was last "expected" to be granted by those who have the rights to 
grant the licence.
   
   We usually perform a bulk update of all licence dates at the beginning of 
the year. This is also clear expression of intent "yes we want to licence our 
code under the ASF licence also in 2022". So do not worry about it now.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org