date:20221219



uranusjr closed issue #28486: DAG Run conflict with dag_id and logical date
URL: https://github.com/apache/airflow/issues/28486


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] uranusjr commented on issue #28486: DAG Run conflict with dag_id and logical date



uranusjr commented on issue #28486:
URL: https://github.com/apache/airflow/issues/28486#issuecomment-1358964733

   Unfortunately there are still too many things in the ecosystem (including 
some parts of Airflow itself) that depends on the logical/execution date being 
unique that it is unlikely for us to lift this restriction. I’m going to close 
this since there is no clear goal we can work toward, but feel free to open 
discussions (in GitHub Discussions or on the dev mailing list) to propose a 
more concrete plan to eventually migrate away from this constraint.
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] alexott commented on issue #27830: DatabricksSubmitRunDeferrableOperator runs in timeout when using AAD Token for SPN



alexott commented on issue #27830:
URL: https://github.com/apache/airflow/issues/27830#issuecomment-1358946544

   Such error usually means that workspace:
   
   * either has IP access lists enabled and access happens from the network 
that isn’t in list
   * or it has private link enabled with public access disabled and access 
happens via internet, not via azure backbone 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow-site] amoghrajesh opened a new pull request, #715: Adding correct reference to globaltoc.js



amoghrajesh opened a new pull request, #715:
URL: https://github.com/apache/airflow-site/pull/715

   Fixes #713 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] VladaZakharova commented on a diff in pull request #28284: Fix for issue with reading schema fields for JSON files in GCSToBigQueryOperator



VladaZakharova commented on code in PR #28284:
URL: https://github.com/apache/airflow/pull/28284#discussion_r1052979311


##
airflow/providers/google/cloud/transfers/gcs_to_bigquery.py:
##
@@ -322,101 +345,29 @@ def execute(self, context: Context):
 if self.schema_object and self.source_format != "DATASTORE_BACKUP":
 schema_fields = json.loads(gcs_hook.download(self.bucket, 
self.schema_object).decode("utf-8"))
 self.log.info("Autodetected fields from schema object: %s", 
schema_fields)

Review Comment:
   Sure, I have changed the output



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] VladaZakharova commented on a diff in pull request #28284: Fix for issue with reading schema fields for JSON files in GCSToBigQueryOperator



VladaZakharova commented on code in PR #28284:
URL: https://github.com/apache/airflow/pull/28284#discussion_r1052979125


##
tests/system/providers/google/cloud/gcs/example_gcs_to_bigquery_async.py:
##
@@ -81,6 +81,20 @@
 max_id_key=MAX_ID_DATE,
 deferrable=True,
 )
+

Review Comment:
   Done



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] koroder commented on pull request #27145: Add support for DAG display name different than dag_id



koroder commented on PR #27145:
URL: https://github.com/apache/airflow/pull/27145#issuecomment-1358924000

   > Hey @koroder - are you raising the discussion about it as advised ? I 
think it is an important topic and would be great if it is discussed in the 
devlist.
   
   Sorry for the delay in coming back on this. I will be raising it on the dev 
list next week.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] boring-cyborg[bot] commented on issue #28486: DAG Run conflict with dag_id and logical date



boring-cyborg[bot] commented on issue #28486:
URL: https://github.com/apache/airflow/issues/28486#issuecomment-1358917319

   Thanks for opening your first issue here! Be sure to follow the issue 
template!
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] NonlinearNimesh opened a new issue, #28486: DAG Run conflict with dag_id and logical date



NonlinearNimesh opened a new issue, #28486:
URL: https://github.com/apache/airflow/issues/28486

   ### Apache Airflow version
   
   Other Airflow 2 version (please specify below)
   
   ### What happened
   
   I am triggering an airflow dag with an api but it is throwing me this 
response
   ```
   {
 "detail": "DAGRun with DAG ID: 'working_1' and DAGRun logical date: 
'2022-11-03 06:35:14+00:00' already exists",
 "status": 409,
 "title": "Conflict",
 "type": 
"https://airflow.apache.org/docs/apache-airflow/2.3.3/stable-rest-api-ref.html#section/Errors/AlreadyExists;
   }
   ```
   and this is becasue i am triggering multiple dag runs in 1 seconds with same 
DAG ID. Below is my code
   
   ```
   utc_date = datetime.datetime.utcnow()
   formatted_date = utc_date.strftime('%Y-%m-%dT%H:%M:%SZ')
   
   data = {
   "conf": {},
   "replace_microseconds":"false"
   "dag_run_id": str(uuid.uuid4()),
   "logical_date": str(formatted_date),
   }
   headers={
   'Content-type':'application/json',
   'Accept':'application/json'
   }
   json_payload = json.dumps(data)
   r = requests.post("http://localhost:8080/api/v1/dags/working_1/dagRuns;, 
auth=HTTPBasicAuth("airflow", "airflow"), data=json_payload, headers=headers)
   print(r.status_code)
   print(r.text)
   ```
   
   ### What you think should happen instead
   
   In "https://github.com/apache/airflow/pull/15174;  @potiuk  mentioned that 
**"In Airflow 2.2 the primary key is run_id and execution_date (Now 
logical_date)"** and as you can see in the code dag_run_id is always unique so 
even if logical date is same between two api request then also there will be 
any conflict. yes i am getting the conflict error .
   
   Like i want to hit airflow api multiple time in a single seconds and 
according to my understanding of what @potiuk mentioned in the above mentioned 
link with (dag_run_id + logical_date) will always be unique.
   
   ### How to reproduce
   
   You can use the above code and hit api multiple times (For example 20-30 
times) then you will get this conflict error.
   
   ### Operating System
   
   Linux
   
   ### Versions of Apache Airflow Providers
   
   2.3.3
   
   ### Deployment
   
   Docker-Compose
   
   ### Deployment details
   
   _No response_
   
   ### Anything else
   
   _No response_
   
   ### Are you willing to submit PR?
   
   - [X] Yes I am willing to submit a PR!
   
   ### Code of Conduct
   
   - [X] I agree to follow this project's [Code of 
Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] uranusjr commented on a diff in pull request #28484: Add execution role parameter to AddStepsOperator



uranusjr commented on code in PR #28484:
URL: https://github.com/apache/airflow/pull/28484#discussion_r1052965967


##
airflow/providers/amazon/aws/operators/emr.py:
##
@@ -113,8 +122,14 @@ def execute(self, context: Context) -> list[str]:
 steps = self.steps
 if isinstance(steps, str):
 steps = ast.literal_eval(steps)
-
-return emr_hook.add_job_flow_steps(job_flow_id=job_flow_id, 
steps=steps, wait_for_completion=True)
+# from pdb import set_trace
+# set_trace()

Review Comment:
   Debugging code you forgot to delete?
   
   (Also fyi you can use `breakpoint()` for the same effect without importing 
pdb)



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] Dinghang commented on pull request #26639: Multi-threads support for processing diff queues in Kubernetes Executor



Dinghang commented on PR #26639:
URL: https://github.com/apache/airflow/pull/26639#issuecomment-1358912756

   > Just checking if performance is still an issue after #26778 (2.4.2) ? It 
improved scheduler blockage from KubernetesExecutor significantly for us. 
Without the increased complexity of multithreading.
   > 
   > I have another suggestion in #28071 that will do similar type of avoidance 
during task adoptation.
   
   Hi @hterik , the one you shared can enhance the performance. However, it 
might still not be able to resolve the performance when there are really a lot 
of workloads. Multithreads can help with that in this case.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] bharanidharan14 commented on pull request #28262: Hook for managing directories and files in Azure Data Lake Storage Gen2



bharanidharan14 commented on PR #28262:
URL: https://github.com/apache/airflow/pull/28262#issuecomment-1358906988

   @kaxil Need your review on this PR


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] NonlinearNimesh commented on issue #15150: "duplicate key value violates unique constraint "dag_run_dag_id_execution_date_key" when triggering a DAG



NonlinearNimesh commented on issue #15150:
URL: https://github.com/apache/airflow/issues/15150#issuecomment-1358890695

   Hi, 
   i am using airflow version 2.3.3 with docker, but i am getting 409 conflict 
error. 
   
   {
 "detail": "DAGRun with DAG ID: 'working_1' and DAGRun logical date: 
'2022-11-03 06:35:14+00:00' already exists",
 "status": 409,
 "title": "Conflict",
 "type": 
"https://airflow.apache.org/docs/apache-airflow/2.3.3/stable-rest-api-ref.html#section/Errors/AlreadyExists;
   }
   
   can anyone help me in this, am i using a wrong version
   
   Thanks


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] boring-cyborg[bot] commented on pull request #28444: Fix GCSToBigQueryOperator not respecting schema_obj



boring-cyborg[bot] commented on PR #28444:
URL: https://github.com/apache/airflow/pull/28444#issuecomment-1358889857

   Awesome work, congrats on your first merged pull request!
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] eladkal merged pull request #28444: Fix GCSToBigQueryOperator not respecting schema_obj



eladkal merged PR #28444:
URL: https://github.com/apache/airflow/pull/28444


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] eladkal commented on pull request #28444: Fix GCSToBigQueryOperator not respecting schema_obj



eladkal commented on PR #28444:
URL: https://github.com/apache/airflow/pull/28444#issuecomment-135647

   Thanks @vchiapaikeo !
   If you have time I'd appreciate also helping to resolve 
https://github.com/apache/airflow/issues/12329


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[airflow] branch main updated (032a542fee -> 9eacf607be)

This is an automated email from the ASF dual-hosted git repository.

eladkal pushed a change to branch main
in repository https://gitbox.apache.org/repos/asf/airflow.git


from 032a542fee Allow Users to disable SwaggerUI via configuration (#28354)
 add 9eacf607be Fix GCSToBigQueryOperator not respecting schema_obj (#28444)

No new revisions were added by this update.

Summary of changes:
 .../google/cloud/transfers/gcs_to_bigquery.py  |  10 +-
 .../google/cloud/transfers/test_gcs_to_bigquery.py | 114 +
 2 files changed, 120 insertions(+), 4 deletions(-)

[GitHub] [airflow] eladkal closed issue #28441: GCSToBigQueryOperator fails when schema_object is specified without schema_fields



eladkal closed issue #28441: GCSToBigQueryOperator fails when schema_object is 
specified without schema_fields
URL: https://github.com/apache/airflow/issues/28441


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] Dinghang commented on a diff in pull request #26639: Multi-threads support for processing diff queues in Kubernetes Executor



Dinghang commented on code in PR #26639:
URL: https://github.com/apache/airflow/pull/26639#discussion_r1052929714


##
airflow/executors/kubernetes_executor.py:
##
@@ -62,6 +64,50 @@
 KubernetesWatchType = Tuple[str, str, Optional[str], Dict[str, str], str]
 
 
+def multi_threads_queue_process(
+queue_size: int,
+queue_type: str,
+process_method: Callable,
+max_threads: int,
+log: Logger,
+batch_size: Optional[int] = None,
+) -> None:
+"""
+Helper method to enable multi-threads for processing queues used with 
kubernetes executor
+:param queue_size: the size of the queue getting processed
+:param queue_type: the type of the queue
+:param process_method: the real method processing the queue
+:param max_threads: the max num of threads to be used
+:param log: log
+:param batch_size: the max num of items we want to process in this round.
+   If it's not set, the current queue size will be used.
+"""
+if queue_size == 0:
+log.info(f'There is no item to process in the {queue_type} queue.')
+return
+
+start_time = time.time()
+log.info(f'Start processing {queue_type} queue with at most {max_threads} 
threads.')
+
+batch_size = min(batch_size or queue_size, queue_size)
+max_threads = min(max_threads, queue_size)
+
+threads = []
+quotient, remainder = divmod(batch_size, max_threads)
+for i in range(max_threads):
+sub_batch_size = quotient + 1 if i < remainder else quotient
+t = Thread(target=process_method, args=[sub_batch_size])
+threads.append(t)
+t.start()
+for t in threads:
+t.join()

Review Comment:
   Hi @dstandish , thanks for the reply. The idea is the same. IIRC, I was 
using ThreadPoolExecutor at the very beginning and then met some issues in 
production with more workloads. Unfortunately, did not record the issue. But 
with the current implementation, there was no issue and it has more flexibility 
for us to do updates. So would like to still go with that.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] bharanidharan14 opened a new pull request, #28485: [WIP] Add Azure functions operator and hook



bharanidharan14 opened a new pull request, #28485:
URL: https://github.com/apache/airflow/pull/28485

   Added Azure functions operator and hooks


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] hterik commented on pull request #26639: Multi-threads support for processing diff queues in Kubernetes Executor



hterik commented on PR #26639:
URL: https://github.com/apache/airflow/pull/26639#issuecomment-1358861246

   Just checking if performance is still an issue after 
https://github.com/apache/airflow/pull/26778 (2.4.2) ? It improved scheduler 
blockage from KubernetesExecutor significantly for us. Without the increased 
complexity of multithreading.
   
   I have another suggestion in https://github.com/apache/airflow/issues/28071 
that will do similar type of avoidance during task adoptation.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] hterik commented on a diff in pull request #26639: Multi-threads support for processing diff queues in Kubernetes Executor



hterik commented on code in PR #26639:
URL: https://github.com/apache/airflow/pull/26639#discussion_r1052919293


##
airflow/executors/kubernetes_executor.py:
##
@@ -599,8 +655,31 @@ def sync(self) -> None:
 raise AirflowException(NOT_STARTED_MESSAGE)
 self.kube_scheduler.sync()
 
-last_resource_version = None
-while True:
+"""processing result queue"""
+multi_threads_queue_process(
+queue_size=self.result_queue.qsize(),
+queue_type='result',
+process_method=self.process_result_queue,

Review Comment:
   I don't think `process_result_queue` is thread-safe. It goes into 
`self._change_state` where it mutates `self.running` and `self.event_buffer`



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow-site] amoghrajesh commented on issue #713: Docs site making request to globaltoc.js but it doesn't exist



amoghrajesh commented on issue #713:
URL: https://github.com/apache/airflow-site/issues/713#issuecomment-1358854271

   While going through the build process, I think we have a bigger problem. 
   The documentation page doesn't load on a dev setup. 
   https://user-images.githubusercontent.com/35884252/208589373-2419dae2-be07-4beb-9226-d0034a87c423.png;>
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[airflow] branch main updated: Allow Users to disable SwaggerUI via configuration (#28354)

2022-12-19 Thread msumit

This is an automated email from the ASF dual-hosted git repository.

msumit pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/airflow.git


The following commit(s) were added to refs/heads/main by this push:
 new 032a542fee Allow Users to disable SwaggerUI via configuration (#28354)
032a542fee is described below

commit 032a542feeb617d1f92580b97fa0ad3cdca09d63
Author: Glenn Schuurman 
AuthorDate: Tue Dec 20 06:06:21 2022 +0100

Allow Users to disable SwaggerUI via configuration (#28354)

Due to potential issues, we want to give teams the opportunity to
disable the swaggerUI. This can now be done via the configuration
key: webserver.enable_swagger_ui. For backwards compatibility,
this has been set to true by default, but for teams willing to disable
this it can be set to false.
---
 airflow/config_templates/config.yml |  7 +++
 airflow/config_templates/default_airflow.cfg|  3 +++
 airflow/www/extensions/init_appbuilder_links.py | 15 +--
 airflow/www/extensions/init_views.py|  6 --
 4 files changed, 23 insertions(+), 8 deletions(-)

diff --git a/airflow/config_templates/config.yml 
b/airflow/config_templates/config.yml
index 21d787cb56..9005a54885 100644
--- a/airflow/config_templates/config.yml
+++ b/airflow/config_templates/config.yml
@@ -1513,6 +1513,13 @@
   type: string
   example: "dagrun_cleared,failed"
   default: ~
+- name: enable_swagger_ui
+  description: |
+Boolean for running SwaggerUI in the webserver.
+  version_added: 2.6.0
+  type: boolean
+  example: ~
+  default: "True"
 - name: run_internal_api
   description: |
 Boolean for running Internal API in the webserver.
diff --git a/airflow/config_templates/default_airflow.cfg 
b/airflow/config_templates/default_airflow.cfg
index a3aff78e95..4bd2883563 100644
--- a/airflow/config_templates/default_airflow.cfg
+++ b/airflow/config_templates/default_airflow.cfg
@@ -771,6 +771,9 @@ audit_view_excluded_events = 
gantt,landing_times,tries,duration,calendar,graph,g
 # Example: audit_view_included_events = dagrun_cleared,failed
 # audit_view_included_events =
 
+# Boolean for running SwaggerUI in the webserver.
+enable_swagger_ui = True
+
 # Boolean for running Internal API in the webserver.
 run_internal_api = False
 
diff --git a/airflow/www/extensions/init_appbuilder_links.py 
b/airflow/www/extensions/init_appbuilder_links.py
index 619d302fb3..67bd1f6bb4 100644
--- a/airflow/www/extensions/init_appbuilder_links.py
+++ b/airflow/www/extensions/init_appbuilder_links.py
@@ -16,6 +16,7 @@
 # under the License.
 from __future__ import annotations
 
+from airflow.configuration import conf
 from airflow.utils.docs import get_docs_url
 
 
@@ -36,12 +37,14 @@ def init_appbuilder_links(app):
 appbuilder.add_link(
 name="Documentation", label="GitHub Repo", 
href="https://github.com/apache/airflow;, category="Docs"
 )
-appbuilder.add_link(
-name="Documentation",
-label="REST API Reference (Swagger UI)",
-href="/api/v1./api/v1_swagger_ui_index",
-category="Docs",
-)
+
+if conf.getboolean("webserver", "enable_swagger_ui", fallback=True):
+appbuilder.add_link(
+name="Documentation",
+label="REST API Reference (Swagger UI)",
+href="/api/v1./api/v1_swagger_ui_index",
+category="Docs",
+)
 appbuilder.add_link(
 name="Documentation", label="REST API Reference (Redoc)", 
href="RedocView.redoc", category="Docs"
 )
diff --git a/airflow/www/extensions/init_views.py 
b/airflow/www/extensions/init_views.py
index 86f94d2f22..ca4ef6cd5e 100644
--- a/airflow/www/extensions/init_views.py
+++ b/airflow/www/extensions/init_views.py
@@ -208,7 +208,8 @@ def init_api_connexion(app: Flask) -> None:
 return views.method_not_allowed(ex)
 
 spec_dir = path.join(ROOT_APP_DIR, "api_connexion", "openapi")
-connexion_app = App(__name__, specification_dir=spec_dir, 
skip_error_handlers=True)
+options = {"swagger_ui": conf.getboolean("webserver", "enable_swagger_ui", 
fallback=True)}
+connexion_app = App(__name__, specification_dir=spec_dir, 
skip_error_handlers=True, options=options)
 connexion_app.app = app
 api_bp = connexion_app.add_api(
 specification="v1.yaml", base_path=base_path, validate_responses=True, 
strict_validation=True
@@ -227,7 +228,8 @@ def init_api_internal(app: Flask) -> None:
 base_path = "/internal_api/v1"
 
 spec_dir = path.join(ROOT_APP_DIR, "api_internal", "openapi")
-internal_app = App(__name__, specification_dir=spec_dir, 
skip_error_handlers=True)
+options = {"swagger_ui": conf.getboolean("webserver", "enable_swagger_ui", 
fallback=True)}
+internal_app = App(__name__, specification_dir=spec_dir, 
skip_error_handlers=True, options=options)
 internal_app.app = app
 api_bp =

[GitHub] [airflow] boring-cyborg[bot] commented on pull request #28354: Allow Users to disable SwaggerUI via configuration



boring-cyborg[bot] commented on PR #28354:
URL: https://github.com/apache/airflow/pull/28354#issuecomment-1358846858

   Awesome work, congrats on your first merged pull request!
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] msumit merged pull request #28354: Allow Users to disable SwaggerUI via configuration



msumit merged PR #28354:
URL: https://github.com/apache/airflow/pull/28354


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] ChandelAnushka closed issue #28373: Webserver IntegrityError When Loaded



ChandelAnushka closed issue #28373: Webserver IntegrityError When Loaded 
URL: https://github.com/apache/airflow/issues/28373


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] ChandelAnushka commented on issue #28373: Webserver IntegrityError When Loaded



ChandelAnushka commented on issue #28373:
URL: https://github.com/apache/airflow/issues/28373#issuecomment-1358825133

   Update


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] NickYadance commented on issue #28071: Kubernetes logging errors - attempting to adopt taskinstance which was not specified by database



NickYadance commented on issue #28071:
URL: https://github.com/apache/airflow/issues/28071#issuecomment-1358818031

   related to this #27983


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] clairezhuang commented on issue #27830: DatabricksSubmitRunDeferrableOperator runs in timeout when using AAD Token for SPN



clairezhuang commented on issue #27830:
URL: https://github.com/apache/airflow/issues/27830#issuecomment-1358767741

   Hi,
   
   This is Claire from Azure Databricks Support team. I want to share the same 
error:
   INFO - Using AAD Token for SPN.
   INFO - Existing AAD token is expired, or going to expire soon. Refreshing...
   ..
   requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: 
***
   ..
   airflow.exceptions.AirflowException: Response: 
b'{"error_code":"403","message":"Unauthorized access to Org: 
"}', Status Code: 403
   
   since the mentioned job url in the provided logs was executed successfully 
from Databricks side, may I know if there would be more information shared from 
Airflow?
   Thanks.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] syedahsn opened a new pull request, #28484: Add execution role parameter to AddStepsOperator



syedahsn opened a new pull request, #28484:
URL: https://github.com/apache/airflow/pull/28484

   EMR launched a new 
[feature](https://docs.aws.amazon.com/emr/latest/ManagementGuide/emr-steps-runtime-roles.html)
 which allows configuring runtime roles for EMR steps. This PR allows users to 
use this feature with the `EmrAddStepsOperator`. System test has been updated 
to use this feature.
   
   
   
   ---
   **^ Add meaningful description above**
   
   Read the **[Pull Request 
Guidelines](https://github.com/apache/airflow/blob/main/CONTRIBUTING.rst#pull-request-guidelines)**
 for more information.
   In case of fundamental code changes, an Airflow Improvement Proposal 
([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvement+Proposals))
 is needed.
   In case of a new dependency, check compliance with the [ASF 3rd Party 
License Policy](https://www.apache.org/legal/resolved.html#category-x).
   In case of backwards incompatible changes please leave a note in a 
newsfragment file, named `{pr_number}.significant.rst` or 
`{issue_number}.significant.rst`, in 
[newsfragments](https://github.com/apache/airflow/tree/main/newsfragments).
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] github-actions[bot] closed pull request #25714: Adding ClickHouse Provider



github-actions[bot] closed pull request #25714: Adding ClickHouse Provider
URL: https://github.com/apache/airflow/pull/25714


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] github-actions[bot] commented on pull request #26730: Correct the description of "schema"



github-actions[bot] commented on PR #26730:
URL: https://github.com/apache/airflow/pull/26730#issuecomment-1358661722

   This pull request has been automatically marked as stale because it has not 
had recent activity. It will be closed in 5 days if no further activity occurs. 
Thank you for your contributions.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] michaelmicheal opened a new issue, #28483: Issues with Custom Menu Items on Smaller Windows

michaelmicheal opened a new issue, #28483:
URL: https://github.com/apache/airflow/issues/28483

### Apache Airflow version

Other Airflow 2 version (please specify below)

### What happened

We take advantage of the custom menu items with flask appbuilder offer a
variety of dropdown menus with custom DAG filters. We've notice two things:

1. When you have too many dropdown menu items in a single category, several
menu items are unreachable when using the Airflow UI on a small screen:
https://user-images.githubusercontent.com/40223998/208548419-f9d1ff57-6cad-4a40-bc58-dbf20148a92a.png;>

2. When you have too many menu categories, multiple rows of dropdown menus
are displayed, but cover some other components.
https://user-images.githubusercontent.com/40223998/208548222-44e50717-9040-4899-be06-d503a8c0f69a.png;>

### What you think should happen instead

1. When you have too many dropdown menu items in a single category, there
should be a scrollbar.
2. When you have too many menu categories, multiple rows of dropdown menus
are displayed, the menu shouldn't cover the dag import errors or any part of
the UI

### How to reproduce

1. Add a bunch of menu items under the same category in a custom plugin and
resize your window smaller
2. Add a large number of menu item categories in a custom plugin and resize
your window smaller.

### Operating System

Debian GNU/Linux 10 (buster)

### Versions of Apache Airflow Providers

2.4.3

### Deployment

Official Apache Airflow Helm Chart

### Deployment details

_No response_

### Anything else

I'm happy to make a PR for this. I just don't have the frontend context. If
someone can point me in the right direction that'd be great

### Are you willing to submit PR?

- [X] Yes I am willing to submit a PR!

### Code of Conduct

- [X] I agree to follow this project's [Code of
Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)

--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] pingzh opened a new issue, #28482: Execution dates are truncated on the UI



pingzh opened a new issue, #28482:
URL: https://github.com/apache/airflow/issues/28482

   ### Apache Airflow version
   
   main (development)
   
   ### What happened
   
   The microsecond part of the execution is truncated on the UI on `main` 
branch with commit: `6e3cee1f6e407d5f505410863c3f73fe1bfcc7cf`
   
   On the `Task Instance Details` Page:
   
   
![image](https://user-images.githubusercontent.com/8662365/208546376-c1a9b735-98fa-4997-bbfa-e7c2c36e88e3.png)
   and
   
![image](https://user-images.githubusercontent.com/8662365/208546394-91db295a-0822-4fc5-90b4-fa03109cf7cb.png)
   
   However the run_id is: `manual__2022-12-19T23:10:50.991738+00:00`
   
   
   
   
   ### What you think should happen instead
   
   I think the date should include microseconds.
   
   ### How to reproduce
   
   _No response_
   
   ### Operating System
   
   Mac M1, version: 13.0.1
   
   ### Versions of Apache Airflow Providers
   
   _No response_
   
   ### Deployment
   
   Official Apache Airflow Helm Chart
   
   ### Deployment details
   
   _No response_
   
   ### Anything else
   
   _No response_
   
   ### Are you willing to submit PR?
   
   - [X] Yes I am willing to submit a PR!
   
   ### Code of Conduct
   
   - [X] I agree to follow this project's [Code of 
Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] syedahsn commented on a diff in pull request #28180: Add link for EMR Steps Sensor logs



syedahsn commented on code in PR #28180:
URL: https://github.com/apache/airflow/pull/28180#discussion_r1052720275


##
airflow/providers/amazon/aws/links/emr_logs.py:
##
@@ -0,0 +1,27 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+from __future__ import annotations
+
+from airflow.providers.amazon.aws.links.base_aws import BASE_AWS_CONSOLE_LINK, 
BaseAwsLink
+
+
+class EmrLogsLink(BaseAwsLink):
+"""Helper class for constructing AWS EMR Logs Link"""
+
+name = "EMR Cluster Logs"
+key = "emr_logs"
+format_str = BASE_AWS_CONSOLE_LINK + 
"/s3/buckets/{log_uri}?region={region_name}={job_flow_id}/"

Review Comment:
   I don't see any links to cloudwatch in the batch links, but it wouldn't be a 
problem if it did I think. As long as the logs are relevant to the batch 
service.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[airflow] branch main updated: Re-enable azure service bus on ARM as it now builds cleanly (#28442)

This is an automated email from the ASF dual-hosted git repository.

potiuk pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/airflow.git


The following commit(s) were added to refs/heads/main by this push:
 new 6e3cee1f6e Re-enable azure service bus on ARM as it now builds cleanly 
(#28442)
6e3cee1f6e is described below

commit 6e3cee1f6e407d5f505410863c3f73fe1bfcc7cf
Author: Jarek Potiuk 
AuthorDate: Mon Dec 19 23:36:07 2022 +0100

Re-enable azure service bus on ARM as it now builds cleanly (#28442)

Previously, the Azure Service Bus had to be disabled in order
to get ARM compatibility (it failed to build cleanly as uampq did
not have binary wheels released and they failed to compile cleanly
on debian). But the last problem is fixed now, so we can re-enable
it for ARM.
---
 airflow/providers/microsoft/azure/provider.yaml | 3 +--
 generated/provider_dependencies.json| 2 +-
 2 files changed, 2 insertions(+), 3 deletions(-)

diff --git a/airflow/providers/microsoft/azure/provider.yaml 
b/airflow/providers/microsoft/azure/provider.yaml
index 43929ea747..4127791f52 100644
--- a/airflow/providers/microsoft/azure/provider.yaml
+++ b/airflow/providers/microsoft/azure/provider.yaml
@@ -63,8 +63,7 @@ dependencies:
   - azure-storage-blob>=12.14.0
   - azure-storage-common>=2.1.0
   - azure-storage-file>=2.1.0
-  # Limited due to https://github.com/Azure/azure-uamqp-python/issues/191
-  - azure-servicebus>=7.6.1; platform_machine != "aarch64"
+  - azure-servicebus>=7.6.1
   - azure-synapse-spark
   - adal>=1.2.7
 
diff --git a/generated/provider_dependencies.json 
b/generated/provider_dependencies.json
index db33ad1505..ce549a6d07 100644
--- a/generated/provider_dependencies.json
+++ b/generated/provider_dependencies.json
@@ -440,7 +440,7 @@
   "azure-mgmt-datafactory>=1.0.0,<2.0",
   "azure-mgmt-datalake-store>=0.5.0",
   "azure-mgmt-resource>=2.2.0",
-  "azure-servicebus>=7.6.1; platform_machine != \"aarch64\"",
+  "azure-servicebus>=7.6.1",
   "azure-storage-blob>=12.14.0",
   "azure-storage-common>=2.1.0",
   "azure-storage-file>=2.1.0",

[GitHub] [airflow] potiuk merged pull request #28442: Re-enable azure service bus on ARM as it now builds cleanly



potiuk merged PR #28442:
URL: https://github.com/apache/airflow/pull/28442


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] Taragolis commented on a diff in pull request #28180: Add link for EMR Steps Sensor logs



Taragolis commented on code in PR #28180:
URL: https://github.com/apache/airflow/pull/28180#discussion_r1052679855


##
airflow/providers/amazon/aws/links/emr_logs.py:
##
@@ -0,0 +1,27 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+from __future__ import annotations
+
+from airflow.providers.amazon.aws.links.base_aws import BASE_AWS_CONSOLE_LINK, 
BaseAwsLink
+
+
+class EmrLogsLink(BaseAwsLink):
+"""Helper class for constructing AWS EMR Logs Link"""
+
+name = "EMR Cluster Logs"
+key = "emr_logs"
+format_str = BASE_AWS_CONSOLE_LINK + 
"/s3/buckets/{log_uri}?region={region_name}={job_flow_id}/"

Review Comment:
   I thought one of link for Batch is actually to CloudWatch service.
   In theory it also could use into another Operators/Sensors.
   
   I also try to improve some communications with links but unfortunetly it 
still some local poc



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] VladaZakharova commented on pull request #27405: Add deferrable mode to MLEngineStartTrainingJobOperator



VladaZakharova commented on PR #27405:
URL: https://github.com/apache/airflow/pull/27405#issuecomment-1358475180

   @potiuk @XD-DENG 
   Team, thanks a lot for so quick response and helping with this problem :)


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[airflow] branch main updated (0311ac9cb1 -> 7ed3cbd74e)

This is an automated email from the ASF dual-hosted git repository.

potiuk pushed a change to branch main
in repository https://gitbox.apache.org/repos/asf/airflow.git


from 0311ac9cb1 Fix flaky test_recover_from_resource_too_old exception 
(#28475)
 add 7ed3cbd74e Update README_RELEASE_PROVIDER_PACKAGES.md (#28479)

No new revisions were added by this update.

Summary of changes:
 dev/README_RELEASE_PROVIDER_PACKAGES.md | 11 +--
 1 file changed, 9 insertions(+), 2 deletions(-)

[GitHub] [airflow] potiuk merged pull request #28479: Update README_RELEASE_PROVIDER_PACKAGES.md



potiuk merged PR #28479:
URL: https://github.com/apache/airflow/pull/28479


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] potiuk commented on pull request #28479: Update README_RELEASE_PROVIDER_PACKAGES.md



potiuk commented on PR #28479:
URL: https://github.com/apache/airflow/pull/28479#issuecomment-1358473044

   Nice. Actually using ponymail for sending that announcement might be a good 
idea :)


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] michaelmicheal opened a new pull request, #28481: dagwarning, add error handling to purge_inactive_dag_warnings



michaelmicheal opened a new pull request, #28481:
URL: https://github.com/apache/airflow/pull/28481

   This PR safely catches exceptions to the `purge_inactive_dag_warnings`. This 
prevents unhandled exceptions from causing the DagFileProcessorManager to fail.
   related: #28480 
   
   ---
   **^ Add meaningful description above**
   
   Read the **[Pull Request 
Guidelines](https://github.com/apache/airflow/blob/main/CONTRIBUTING.rst#pull-request-guidelines)**
 for more information.
   In case of fundamental code changes, an Airflow Improvement Proposal 
([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvement+Proposals))
 is needed.
   In case of a new dependency, check compliance with the [ASF 3rd Party 
License Policy](https://www.apache.org/legal/resolved.html#category-x).
   In case of backwards incompatible changes please leave a note in a 
newsfragment file, named `{pr_number}.significant.rst` or 
`{issue_number}.significant.rst`, in 
[newsfragments](https://github.com/apache/airflow/tree/main/newsfragments).
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] michaelmicheal opened a new issue, #28480: `purge_inactive_dag_warnings` Unhandled Exception in DagFileProcessorManager



michaelmicheal opened a new issue, #28480:
URL: https://github.com/apache/airflow/issues/28480

   ### Apache Airflow version
   
   2.5.0
   
   ### What happened
   
   Airflow Version: 2.4.3
   
   In one of our instances, after upgrading to Airflow 2.4.3, we started seeing 
the 
[`purge_inactive_dag_warnings`](https://github.com/apache/airflow/blob/main/airflow/dag_processing/manager.py#L598)
 query failing a subset of parsing loops, but consistently happening within 500 
parsing loops. 
   
   ```python
   self._deactivate_stale_dags()
   DagWarning.purge_inactive_dag_warnings()
   refreshed_dag_dir = self._refresh_dag_dir()
   ```
   This caused the DagFileProcessorManager to exit consistently before 
processing all DAG files.
   
   ```python
   Traceback (most recent call last):
 File "/usr/local/lib/python3.9/multiprocessing/process.py", line 315, in 
_bootstrap
   self.run()
 File "/usr/local/lib/python3.9/multiprocessing/process.py", line 108, in 
run
   self._target(*self._args, **self._kwargs)
 File 
"/usr/local/lib/python3.9/site-packages/airflow/dag_processing/manager.py", 
line 257, in _run_processor_manager
   processor_manager.start()
 File 
"/usr/local/lib/python3.9/site-packages/airflow/dag_processing/manager.py", 
line 489, in start
   return self._run_parsing_loop()
 File 
"/usr/local/lib/python3.9/site-packages/airflow/dag_processing/manager.py", 
line 609, in _run_parsing_loop
   DagWarning.purge_inactive_dag_warnings()
 File "/usr/local/lib/python3.9/site-packages/airflow/utils/session.py", 
line 75, in wrapper
   return func(*args, session=session, **kwargs)
 File 
"/usr/local/lib/python3.9/site-packages/airflow/models/dagwarning.py", line 82, 
in purge_inactive_dag_warnings
   query.delete(synchronize_session=False)
 File "/usr/local/lib/python3.9/site-packages/sqlalchemy/orm/query.py", 
line 3191, in delete
   result = self.session.execute(
 File "/usr/local/lib/python3.9/site-packages/sqlalchemy/orm/session.py", 
line 1689, in execute
   result = conn._execute_20(statement, params or {}, execution_options)
 File "/usr/local/lib/python3.9/site-packages/sqlalchemy/engine/base.py", 
line 1614, in _execute_20
   return meth(self, args_10style, kwargs_10style, execution_options)
 File "/usr/local/lib/python3.9/site-packages/sqlalchemy/sql/elements.py", 
line 325, in _execute_on_connection
   return connection._execute_clauseelement(
 File "/usr/local/lib/python3.9/site-packages/sqlalchemy/engine/base.py", 
line 1481, in _execute_clauseelement
   ret = self._execute_context(
 File "/usr/local/lib/python3.9/site-packages/sqlalchemy/engine/base.py", 
line 1845, in _execute_context
   self._handle_dbapi_exception(
 File "/usr/local/lib/python3.9/site-packages/sqlalchemy/engine/base.py", 
line 2026, in _handle_dbapi_exception
   util.raise_(
 File "/usr/local/lib/python3.9/site-packages/sqlalchemy/util/compat.py", 
line 207, in raise_
   raise exception
 File "/usr/local/lib/python3.9/site-packages/sqlalchemy/engine/base.py", 
line 1802, in _execute_context
   self.dialect.do_execute(
 File 
"/usr/local/lib/python3.9/site-packages/sqlalchemy/engine/default.py", line 
719, in do_execute
   cursor.execute(statement, parameters)
 File "/usr/local/lib/python3.9/site-packages/MySQLdb/cursors.py", line 
206, in execute
   res = self._query(query)
 File "/usr/local/lib/python3.9/site-packages/MySQLdb/cursors.py", line 
319, in _query
   db.query(q)
 File "/usr/local/lib/python3.9/site-packages/MySQLdb/connections.py", line 
254, in query
   _mysql.connection.query(self, query)
   sqlalchemy.exc.OperationalError: (MySQLdb.OperationalError) (2013, 'Lost 
connection to MySQL server during query')
   [SQL: DELETE FROM dag_warning USING dag_warning, dag WHERE 
dag_warning.dag_id = dag.dag_id AND dag.is_active = false]
   ```
   
   
   ### What you think should happen instead
   
   To resolve this, we wrapped the query in a try-except block to avoid the 
unhandled exception. I think we should 
   
   1. Wrap the `purge_inactive_dag_warnings` in a try-except block.
   2. Consider not calling the `purge_inactive_dag_warnings` method every 
parsing loop. Maybe we could do it when we've parsed all the DAG files.
   
   ### How to reproduce
   
   Hard to reproduce, but will happen if the `purge_inactive_dag_warnings` 
query throws an exception
   
   ### Operating System
   
   Debian GNU/Linux 10 (buster)
   
   ### Versions of Apache Airflow Providers
   
   _No response_
   
   ### Deployment
   
   Other 3rd-party Helm chart
   
   ### Deployment details
   
   Other 3rd-party Helm chart
   
   ### Anything else
   
   Airflow 2.4.3 on Kubernetes
   MySQL Version: 8.0.18
   
   ### Are you willing to submit PR?
   
   - [X] Yes I am willing to submit a PR!
   
   ### Code of Conduct
   
   - [X] I agree to follow this project's [Code of

[GitHub] [airflow] Taragolis commented on pull request #28442: Re-enable azure service bus on ARM as it now builds cleanly



Taragolis commented on PR #28442:
URL: https://github.com/apache/airflow/pull/28442#issuecomment-1358432899

   > A better option will be to add links to installation instructions for 
those packages that we know might be problematic. For example for Plyvel 
linking https://plyvel.readthedocs.io/en/latest/installation.html would be 
fine. NOTE - even Plyvel developer limited those to "Ubuntu and Debian".
   
   Yep initially i tried to use that doc, and unfortunately they refers to 
outdated name of package, from Debian 9 I guess :man_facepalming: 
   
https://packages.debian.org/search?suite=bullseye=any=names=libleveldb
   
   Ok, lets better see if any kind of issue appear, it is possible that no one 
actually affected or found solution by their own.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] eladkal opened a new pull request, #28479: Update README_RELEASE_PROVIDER_PACKAGES.md



eladkal opened a new pull request, #28479:
URL: https://github.com/apache/airflow/pull/28479

   explain how to avoid failure in sending announcement
   
   copied from helm release readme
   
https://github.com/apache/airflow/blob/f4a3c64dd0d8996382dec813288e0995fb0a1e0c/dev/README_RELEASE_HELM_CHART.md#L684-L686
   
   
   
   
   ---
   **^ Add meaningful description above**
   
   Read the **[Pull Request 
Guidelines](https://github.com/apache/airflow/blob/main/CONTRIBUTING.rst#pull-request-guidelines)**
 for more information.
   In case of fundamental code changes, an Airflow Improvement Proposal 
([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvement+Proposals))
 is needed.
   In case of a new dependency, check compliance with the [ASF 3rd Party 
License Policy](https://www.apache.org/legal/resolved.html#category-x).
   In case of backwards incompatible changes please leave a note in a 
newsfragment file, named `{pr_number}.significant.rst` or 
`{issue_number}.significant.rst`, in 
[newsfragments](https://github.com/apache/airflow/tree/main/newsfragments).
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[airflow-site] branch gh-pages updated: Deploying to gh-pages from @ b2a8d6907120e095b9a100643bab3e2df60a4490 

2022-12-19 Thread github-bot

This is an automated email from the ASF dual-hosted git repository.

github-bot pushed a commit to branch gh-pages
in repository https://gitbox.apache.org/repos/asf/airflow-site.git


The following commit(s) were added to refs/heads/gh-pages by this push:
 new 767f0db87a Deploying to gh-pages from  @ 
b2a8d6907120e095b9a100643bab3e2df60a4490 
767f0db87a is described below

commit 767f0db87a45f60d1a91b2aa6e00e25a8e13d604
Author: eladkal <45845474+elad...@users.noreply.github.com>
AuthorDate: Mon Dec 19 21:40:17 2022 +

Deploying to gh-pages from  @ b2a8d6907120e095b9a100643bab3e2df60a4490 
---
 announcements/index.html   |  14 +--
 blog/airflow-1.10.10/index.html|   4 +-
 blog/airflow-1.10.12/index.html|   4 +-
 blog/airflow-1.10.8-1.10.9/index.html  |   4 +-
 blog/airflow-2.2.0/index.html  |   4 +-
 blog/airflow-2.3.0/index.html  |   4 +-
 blog/airflow-2.4.0/index.html  |   4 +-
 blog/airflow-2.5.0/index.html  |   4 +-
 blog/airflow-survey-2020/index.html|   4 +-
 blog/airflow-survey-2022/index.html|   4 +-
 blog/airflow-survey/index.html |   4 +-
 blog/airflow-two-point-oh-is-here/index.html   |   4 +-
 blog/airflow_summit_2021/index.html|   4 +-
 blog/airflow_summit_2022/index.html|   4 +-
 blog/announcing-new-website/index.html |   4 +-
 blog/apache-airflow-for-newcomers/index.html   |   4 +-
 .../index.html |   4 +-
 .../index.html |   4 +-
 .../index.html |   4 +-
 .../index.html |   4 +-
 .../index.html |   4 +-
 .../index.html |   4 +-
 community/index.html   |  30 +++
 index.html |  32 +++
 search/index.html  |   4 +-
 sitemap.xml| 100 ++---
 use-cases/adobe/index.html |   4 +-
 use-cases/adyen/index.html |   4 +-
 use-cases/big-fish-games/index.html|   4 +-
 use-cases/dish/index.html  |   4 +-
 use-cases/experity/index.html  |   4 +-
 use-cases/onefootball/index.html   |   4 +-
 use-cases/plarium-krasnodar/index.html |   4 +-
 use-cases/seniorlink/index.html|   4 +-
 use-cases/sift/index.html  |   4 +-
 35 files changed, 167 insertions(+), 133 deletions(-)

diff --git a/announcements/index.html b/announcements/index.html
index 58a95ce033..24c2fa71b4 100644
--- a/announcements/index.html
+++ b/announcements/index.html
@@ -399,6 +399,11 @@
  
  
 Note: Follow https://twitter.com/ApacheAirflow;>@ApacheAirflow on Twitter for the 
latest news and announcements!
+December 19, 2022
+Airflow PMC welcomes new Airflow Committer:
+
+Niko Oliveira (https://github.com/o-nikolas;>@o-nikolas
+
 December 2, 2022
 We’ve just released Apache Airflow 2.5.0. You can read all 
about it in our whats new in Apache 
Airflow 2.5.0 blog post.
  PyPI: https://pypi.org/project/apache-airflow/2.5.0/;>https://pypi.org/project/apache-airflow/2.5.0/
 
@@ -406,11 +411,10 @@
 ️ Release Notes: https://airflow.apache.org/docs/apache-airflow/.5.0/release_notes.html;>https://airflow.apache.org/docs/apache-airflow/.5.0/release_notes.html
 
 襁 Sources: https://airflow.apache.org/docs/apache-airflow/2.5.0/installation/installing-from-sources.html;>https://airflow.apache.org/docs/apache-airflow/2.5.0/installation/installing-from-sources.html
 December 2, 2022
-We’ve just released Apache Airflow 2.5.0.
- PyPI: https://pypi.org/project/apache-airflow/2.5.0/;>https://pypi.org/project/apache-airflow/2.5.0/
 
- Docs: https://airflow.apache.org/docs/apache-airflow/2.5.0;>https://airflow.apache.org/docs/apache-airflow/2.5.0
 
-️ Release Notes: https://airflow.apache.org/docs/apache-airflow/2.5.0/release_notes.html;>https://airflow.apache.org/docs/apache-airflow/2.5.0/release_notes.html
 
-襁 Sources: https://airflow.apache.org/docs/apache-airflow/2.5.0/installation/installing-from-sources.html;>https://airflow.apache.org/docs/apache-airflow/2.5.0/installation/installing-from-sources.html
+Airflow PMC welcomes new Airflow Committer:
+
+Andrey Anshin (https://https://github.com/Taragolis;>@Taragolis
+
 November 14, 2022
 We’ve just released Apache Airflow 2.4.3.
  PyPI: https://pypi.org/project/apache-airflow/2.4.3/;>https://pypi.org/project/apache-airflow/2.4.3/
 
diff --git a/blog/airflow-1.10.10/index.html b/blog/airflow-1.10.10/index.html
index aba53ade21..be43451426 100644
--- a/blog/airflow-1.10.10/index.html
+++

[GitHub] [airflow] vchiapaikeo commented on pull request #28444: Fix GCSToBigQueryOperator not respecting schema_obj



vchiapaikeo commented on PR #28444:
URL: https://github.com/apache/airflow/pull/28444#issuecomment-1358403044

   cc @turbaszek, I believe you are codeowner. Can you approve this as well?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow-site] eladkal closed pull request #710: Update _index.md



eladkal closed pull request #710: Update _index.md
URL: https://github.com/apache/airflow-site/pull/710


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow-site] eladkal commented on pull request #710: Update _index.md



eladkal commented on PR #710:
URL: https://github.com/apache/airflow-site/pull/710#issuecomment-1358396735

   fixed in https://github.com/apache/airflow-site/pull/714


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[airflow-site] branch main updated: Announce Niko as committer (#714)

This is an automated email from the ASF dual-hosted git repository.

eladkal pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/airflow-site.git


The following commit(s) were added to refs/heads/main by this push:
 new b2a8d69071 Announce Niko as committer (#714)
b2a8d69071 is described below

commit b2a8d6907120e095b9a100643bab3e2df60a4490
Author: eladkal <45845474+elad...@users.noreply.github.com>
AuthorDate: Mon Dec 19 23:28:32 2022 +0200

Announce Niko as committer (#714)
---
 landing-pages/site/content/en/announcements/_index.md | 13 -
 landing-pages/site/data/committers.json   |  6 ++
 2 files changed, 14 insertions(+), 5 deletions(-)

diff --git a/landing-pages/site/content/en/announcements/_index.md 
b/landing-pages/site/content/en/announcements/_index.md
index 85cddc121a..f897b24401 100644
--- a/landing-pages/site/content/en/announcements/_index.md
+++ b/landing-pages/site/content/en/announcements/_index.md
@@ -13,6 +13,12 @@ menu:
 
 **Note:** Follow [@ApacheAirflow](https://twitter.com/ApacheAirflow) on 
Twitter for the latest news and announcements!
 
+# December 19, 2022
+
+Airflow PMC welcomes new Airflow Committer:
+
+* **Niko Oliveira** ([@o-nikolas](https://github.com/o-nikolas)
+
 # December 2, 2022
 
 We’ve just released Apache **Airflow 2.5.0**. You can read all about it in our 
[what's new in Apache Airflow 2.5.0]({{< ref "blog/airflow-2.5.0/index.md" >}}) 
blog post.
@@ -25,12 +31,9 @@ We’ve just released Apache **Airflow 2.5.0**. You can read 
all about it in our
 
 # December 2, 2022
 
-We’ve just released Apache **Airflow 2.5.0**.
+Airflow PMC welcomes new Airflow Committer:
 
- PyPI: https://pypi.org/project/apache-airflow/2.5.0/ \
- Docs: https://airflow.apache.org/docs/apache-airflow/2.5.0 \
-️ Release Notes: 
https://airflow.apache.org/docs/apache-airflow/2.5.0/release_notes.html \
-襁 Sources: 
https://airflow.apache.org/docs/apache-airflow/2.5.0/installation/installing-from-sources.html
+* **Andrey Anshin** ([@Taragolis](https://https://github.com/Taragolis)
 
 # November 14, 2022
 
diff --git a/landing-pages/site/data/committers.json 
b/landing-pages/site/data/committers.json
index 3955a79a7e..4ef4778acb 100644
--- a/landing-pages/site/data/committers.json
+++ b/landing-pages/site/data/committers.json
@@ -59,6 +59,12 @@
 "image": "https://github.com/zhongjiajie.png;,
 "nick": "zhongjiajie"
   },
+  {
+"name": "Niko Oliveira",
+"github": "https://github.com/o-nikolas;,
+"image": "https://github.com/o-nikolas.png;,
+"nick": "o-nikolas"
+  },
   {
 "name": "Qian Yu",
 "github": "https://github.com/yuqian90;,

[GitHub] [airflow-site] eladkal merged pull request #714: Announce Niko as committer



eladkal merged PR #714:
URL: https://github.com/apache/airflow-site/pull/714


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] potiuk commented on pull request #28442: Re-enable azure service bus on ARM as it now builds cleanly



potiuk commented on PR #28442:
URL: https://github.com/apache/airflow/pull/28442#issuecomment-1358388757

   > We could try to do with simple cases. It is not mandatory I'd rather say 
it nice to have, if we know that package could be install on Linux (glib-based, 
not musl) but required install from sources then would be nice to have this 
information which save some time.
   
   A better option will be to add links to installation instructions for those 
packages that we know might be problematic. For example for Plyvel linking 
https://plyvel.readthedocs.io/en/latest/installation.html would be fine. NOTE - 
even Plyvel developer limited those to "Ubuntu and Debian". 
   
   > I thought you have some kind template about how many contributors we 
already have and would be nice if someone who find become a contributor and 
improve this part 藍
   
   Well. I do.. But this one is tricky :). Any contribution there will at most 
explain what is needed for the OS/distribution of that particular user. Which 
might be even more misleading for other distros/MacOS/ARM. It's super-hard to 
write a generic installation instructions even if we limit to apt/yum 
(debian/RedHat). Because the same OS packages are often even named differently. 
This is a true rabbit hole we want to avoid. For some reason even creators of 
the libraries are super vague sometimes and only limit it to some distros.
   
   > I think this would be nice if it not required a lot of effort from our 
side.
   > BTW, do we have some statistic about downloads particular image from 
Docker Hub?
   
   Very little. We know the total for apache/airflow:
   
   ```
   curl -s https://hub.docker.com/v2/repositories/apache/airflow | jq -r 
".pull_count"
   ```
   
   Result: 95575880
   
   Also we can see "per tag" last pull - and usually it is between few seconds 
and 2 hours (8-10 hrs sometimes) last when I checked. But this is unscientific 
really. We do not pull the images during the CI (we only use ghcr.io) so we 
know it's not skewed by our CI though.
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[airflow-site] 01/01: Announce Niko as committer

This is an automated email from the ASF dual-hosted git repository.

eladkal pushed a commit to branch announce
in repository https://gitbox.apache.org/repos/asf/airflow-site.git

commit 6e53b6ad6dee93c0cf2834b16bb6933981fff715
Author: Elad Kalif <45845474+elad...@users.noreply.github.com>
AuthorDate: Mon Dec 19 23:09:01 2022 +0200

Announce Niko as committer
---
 landing-pages/site/content/en/announcements/_index.md | 13 -
 landing-pages/site/data/committers.json   |  6 ++
 2 files changed, 14 insertions(+), 5 deletions(-)

diff --git a/landing-pages/site/content/en/announcements/_index.md 
b/landing-pages/site/content/en/announcements/_index.md
index 85cddc121a..f897b24401 100644
--- a/landing-pages/site/content/en/announcements/_index.md
+++ b/landing-pages/site/content/en/announcements/_index.md
@@ -13,6 +13,12 @@ menu:
 
 **Note:** Follow [@ApacheAirflow](https://twitter.com/ApacheAirflow) on 
Twitter for the latest news and announcements!
 
+# December 19, 2022
+
+Airflow PMC welcomes new Airflow Committer:
+
+* **Niko Oliveira** ([@o-nikolas](https://github.com/o-nikolas)
+
 # December 2, 2022
 
 We’ve just released Apache **Airflow 2.5.0**. You can read all about it in our 
[what's new in Apache Airflow 2.5.0]({{< ref "blog/airflow-2.5.0/index.md" >}}) 
blog post.
@@ -25,12 +31,9 @@ We’ve just released Apache **Airflow 2.5.0**. You can read 
all about it in our
 
 # December 2, 2022
 
-We’ve just released Apache **Airflow 2.5.0**.
+Airflow PMC welcomes new Airflow Committer:
 
- PyPI: https://pypi.org/project/apache-airflow/2.5.0/ \
- Docs: https://airflow.apache.org/docs/apache-airflow/2.5.0 \
-️ Release Notes: 
https://airflow.apache.org/docs/apache-airflow/2.5.0/release_notes.html \
-襁 Sources: 
https://airflow.apache.org/docs/apache-airflow/2.5.0/installation/installing-from-sources.html
+* **Andrey Anshin** ([@Taragolis](https://https://github.com/Taragolis)
 
 # November 14, 2022
 
diff --git a/landing-pages/site/data/committers.json 
b/landing-pages/site/data/committers.json
index 3955a79a7e..4ef4778acb 100644
--- a/landing-pages/site/data/committers.json
+++ b/landing-pages/site/data/committers.json
@@ -59,6 +59,12 @@
 "image": "https://github.com/zhongjiajie.png;,
 "nick": "zhongjiajie"
   },
+  {
+"name": "Niko Oliveira",
+"github": "https://github.com/o-nikolas;,
+"image": "https://github.com/o-nikolas.png;,
+"nick": "o-nikolas"
+  },
   {
 "name": "Qian Yu",
 "github": "https://github.com/yuqian90;,

[airflow-site] branch announce created (now 6e53b6ad6d)

This is an automated email from the ASF dual-hosted git repository.

eladkal pushed a change to branch announce
in repository https://gitbox.apache.org/repos/asf/airflow-site.git


  at 6e53b6ad6d Announce Niko as committer

This branch includes the following new commits:

 new 6e53b6ad6d Announce Niko as committer

The 1 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails.  The revisions
listed as "add" were already present in the repository and have only
been added to this reference.

[GitHub] [airflow] syedahsn commented on a diff in pull request #28180: Add link for EMR Steps Sensor logs



syedahsn commented on code in PR #28180:
URL: https://github.com/apache/airflow/pull/28180#discussion_r1052645834


##
airflow/providers/amazon/aws/links/emr_logs.py:
##
@@ -0,0 +1,27 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+from __future__ import annotations
+
+from airflow.providers.amazon.aws.links.base_aws import BASE_AWS_CONSOLE_LINK, 
BaseAwsLink
+
+
+class EmrLogsLink(BaseAwsLink):
+"""Helper class for constructing AWS EMR Logs Link"""
+
+name = "EMR Cluster Logs"
+key = "emr_logs"
+format_str = BASE_AWS_CONSOLE_LINK + 
"/s3/buckets/{log_uri}?region={region_name}={job_flow_id}/"

Review Comment:
   That's a good point. I didn't notice it before, but that's how its done for 
batch links. I'll make the change.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] potiuk commented on a diff in pull request #28442: Re-enable azure service bus on ARM as it now builds cleanly



potiuk commented on code in PR #28442:
URL: https://github.com/apache/airflow/pull/28442#discussion_r1052629532


##
airflow/providers/microsoft/azure/provider.yaml:
##
@@ -64,7 +64,7 @@ dependencies:
   - azure-storage-common>=2.1.0
   - azure-storage-file>=2.1.0
   # Limited due to https://github.com/Azure/azure-uamqp-python/issues/191
-  - azure-servicebus>=7.6.1; platform_machine != "aarch64"
+  - azure-servicebus>=7.6.1

Review Comment:
   Fixed.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] potiuk commented on a diff in pull request #28442: Re-enable azure service bus on ARM as it now builds cleanly



potiuk commented on code in PR #28442:
URL: https://github.com/apache/airflow/pull/28442#discussion_r1052628776


##
airflow/providers/microsoft/azure/provider.yaml:
##
@@ -64,7 +64,7 @@ dependencies:
   - azure-storage-common>=2.1.0
   - azure-storage-file>=2.1.0
   # Limited due to https://github.com/Azure/azure-uamqp-python/issues/191
-  - azure-servicebus>=7.6.1; platform_machine != "aarch64"
+  - azure-servicebus>=7.6.1

Review Comment:
   臘 of course



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[airflow] branch main updated: Fix flaky test_recover_from_resource_too_old exception (#28475)

This is an automated email from the ASF dual-hosted git repository.

potiuk pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/airflow.git


The following commit(s) were added to refs/heads/main by this push:
 new 0311ac9cb1 Fix flaky test_recover_from_resource_too_old exception 
(#28475)
0311ac9cb1 is described below

commit 0311ac9cb19a2c16131118c935d7973db89baaba
Author: Jarek Potiuk 
AuthorDate: Mon Dec 19 21:56:10 2022 +0100

Fix flaky test_recover_from_resource_too_old exception (#28475)

After #28047 the test_recover_from_resource_too_old started to
fail in a flaky way. Turned out that - depend on some other test
run the Singleton ResourceVersion could containt not one but
two namespaces (including default namespace).

Also while fixing the tests it's been noticed that the test
missed an assert - it did not assert that the Exception was in fact
thrown, so the test could have succeeded even if the exception was
not really thrown (there was assert in "except" clause but if the
exception was not thrown, it would not have been called at all).
---
 tests/executors/test_kubernetes_executor.py | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/tests/executors/test_kubernetes_executor.py 
b/tests/executors/test_kubernetes_executor.py
index 9f4fe76bac..e48d2087cb 100644
--- a/tests/executors/test_kubernetes_executor.py
+++ b/tests/executors/test_kubernetes_executor.py
@@ -1249,12 +1249,13 @@ class TestKubernetesJobWatcher:
 try:
 # self.watcher._run() is mocked and return "500" as last 
resource_version
 self.watcher.run()
+assert False, "Should have raised Exception"
 except Exception as e:
 assert e.args == ("sentinel",)
 
 # both resource_version should be 0 after _run raises an exception
 assert self.watcher.resource_version == "0"
-assert ResourceVersion().resource_version == {self.test_namespace: 
"0"}
+assert ResourceVersion().resource_version[self.test_namespace] == 
"0"
 
 # check that in the next run, _run is invoked with 
resource_version = 0
 mock_underscore_run.reset_mock()

[GitHub] [airflow] potiuk merged pull request #28475: Fix flaky test_recover_from_resource_too_old exception



potiuk merged PR #28475:
URL: https://github.com/apache/airflow/pull/28475


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] vincbeck commented on a diff in pull request #28425: AIP-44: Add command for running standalone Internal API



vincbeck commented on code in PR #28425:
URL: https://github.com/apache/airflow/pull/28425#discussion_r1052614980


##
airflow/cli/commands/internal_api_command.py:
##
@@ -0,0 +1,268 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""Internal API command."""
+from __future__ import annotations
+
+import logging
+import os
+import signal
+import subprocess
+import sys
+import textwrap
+from contextlib import suppress
+from tempfile import gettempdir
+from time import sleep
+
+import daemon
+import psutil
+from daemon.pidfile import TimeoutPIDLockFile
+from flask import Flask
+from flask_appbuilder import SQLA
+from flask_caching import Cache
+from flask_wtf.csrf import CSRFProtect
+from lockfile.pidlockfile import read_pid_from_pidfile
+from sqlalchemy.engine.url import make_url
+
+from airflow import settings
+from airflow.cli.commands.webserver_command import GunicornMonitor
+from airflow.configuration import conf
+from airflow.exceptions import AirflowConfigException
+from airflow.logging_config import configure_logging
+from airflow.models import import_all_models
+from airflow.utils import cli as cli_utils
+from airflow.utils.cli import setup_locations, setup_logging
+from airflow.utils.process_utils import check_if_pidfile_process_is_running
+from airflow.www.extensions.init_dagbag import init_dagbag
+from airflow.www.extensions.init_jinja_globals import init_jinja_globals
+from airflow.www.extensions.init_manifest_files import configure_manifest_files
+from airflow.www.extensions.init_security import init_xframe_protection
+from airflow.www.extensions.init_views import init_api_internal, 
init_error_handlers
+
+log = logging.getLogger(__name__)
+app: Flask | None = None
+
+
+@cli_utils.action_cli
+def internal_api(args):
+"""Starts Airflow Internal API."""
+print(settings.HEADER)
+
+access_logfile = args.access_logfile if args.access_logfile is not None 
else "-"
+error_logfile = args.error_logfile if args.error_logfile is not None else 
"-"

Review Comment:
   I think I rather have `*_logfile` with "-" as value than empty string (since 
I have no idea, what's the consequence of having logs file referenced as empty 
string)



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] potiuk commented on pull request #28300: Add Public Interface description to Airflow documentation



potiuk commented on PR #28300:
URL: https://github.com/apache/airflow/pull/28300#issuecomment-1358275835

   Any more comments to my explanation ? 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[airflow] branch main updated (672264b0af -> 290fc462e2)

This is an automated email from the ASF dual-hosted git repository.

potiuk pushed a change to branch main
in repository https://gitbox.apache.org/repos/asf/airflow.git


from 672264b0af Remove extra H1 & improve formatting of Listeners docs page 
(#28450)
 add 290fc462e2 Update release process for providers after Elad's release 
(#28473)

No new revisions were added by this update.

Summary of changes:
 dev/README_RELEASE_PROVIDER_PACKAGES.md | 27 ---
 1 file changed, 16 insertions(+), 11 deletions(-)

[GitHub] [airflow] potiuk merged pull request #28473: Update release process for providers after Elad's release



potiuk merged PR #28473:
URL: https://github.com/apache/airflow/pull/28473


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] vincbeck commented on pull request #28282: Mark stepIds for cancel in EmrAddStepsOperator



vincbeck commented on PR #28282:
URL: https://github.com/apache/airflow/pull/28282#issuecomment-1358271382

   I agree with @eladkal. It feels to me you are modifying the operator 
`EmrAddStepsOperator` to handle a very niche use case. You want to cancel some 
EMR steps while adding some through `EmrAddStepsOperator` if they have the same 
name. I dont think it should be handled on the operator side but more a 
separate task in your DAG, or as a separate DAG. This is just my opinion and 
dont want to discourage you to contribute to Airflow. Overall I am not saying 
your code or even your idea is bad, I only think this is too specific (again, 
to me) to be handled on the operator. If all users would handle their business 
requirements in the operators, it would be a big mess :D Happy to hear others 
feedback but to me this not generic enough to be handled by the operator. On 
the other side I am not a big user of EMR, so if others think this is a good 
feature they'd like to have in `EmrAddStepsOperator`, happy to move forward 
with this :)


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] potiuk commented on a diff in pull request #28475: Fix flaky test_recover_from_resource_too_old exception



potiuk commented on code in PR #28475:
URL: https://github.com/apache/airflow/pull/28475#discussion_r1052611348


##
tests/executors/test_kubernetes_executor.py:
##
@@ -1249,12 +1249,13 @@ def effect():
 try:
 # self.watcher._run() is mocked and return "500" as last 
resource_version
 self.watcher.run()
+assert False, "Should have raised Exception"

Review Comment:
   This is just consistency change.  The author actually expected the sentinel 
exception (and it's the one that is raised form the watcher to break the loop) 
but the path when (potentially) no exception is thrown would not run this 
assert:
   
   ```
   except Exception as e:
   assert e.args == ("sentinel",)
   ```
   
   Sure, we know what happens inside the run method and by inspecting it, we 
know it is not going to happen most likely - because exception is the only way 
to get out of the loop. But ... this might change in the future.
   
   Not likely and stupid example but if someonedoes fast "return 0" in the 
`watcher.run()` - the test would have succeeded as well (no exception and 
resource = 0, `assert e.args == ("sentinel", )` would have not been called). 
   
   Adding the assert simply makes absolutely sure that the exception was thrown 
(because otherwise assert would fail). That's the usual pattern :)



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] mhenc commented on a diff in pull request #28425: AIP-44: Add command for running standalone Internal API



mhenc commented on code in PR #28425:
URL: https://github.com/apache/airflow/pull/28425#discussion_r1052610890


##
airflow/cli/commands/internal_api_command.py:
##
@@ -0,0 +1,268 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""Internal API command."""
+from __future__ import annotations
+
+import logging
+import os
+import signal
+import subprocess
+import sys
+import textwrap
+from contextlib import suppress
+from tempfile import gettempdir
+from time import sleep
+
+import daemon
+import psutil
+from daemon.pidfile import TimeoutPIDLockFile
+from flask import Flask
+from flask_appbuilder import SQLA
+from flask_caching import Cache
+from flask_wtf.csrf import CSRFProtect
+from lockfile.pidlockfile import read_pid_from_pidfile
+from sqlalchemy.engine.url import make_url
+
+from airflow import settings
+from airflow.cli.commands.webserver_command import GunicornMonitor
+from airflow.configuration import conf
+from airflow.exceptions import AirflowConfigException
+from airflow.logging_config import configure_logging
+from airflow.models import import_all_models
+from airflow.utils import cli as cli_utils
+from airflow.utils.cli import setup_locations, setup_logging
+from airflow.utils.process_utils import check_if_pidfile_process_is_running
+from airflow.www.extensions.init_dagbag import init_dagbag
+from airflow.www.extensions.init_jinja_globals import init_jinja_globals
+from airflow.www.extensions.init_manifest_files import configure_manifest_files
+from airflow.www.extensions.init_security import init_xframe_protection
+from airflow.www.extensions.init_views import init_api_internal, 
init_error_handlers
+
+log = logging.getLogger(__name__)
+app: Flask | None = None
+
+
+@cli_utils.action_cli
+def internal_api(args):
+"""Starts Airflow Internal API."""
+print(settings.HEADER)
+
+access_logfile = args.access_logfile if args.access_logfile is not None 
else "-"
+error_logfile = args.error_logfile if args.error_logfile is not None else 
"-"
+access_logformat = args.access_logformat
+num_workers = args.workers
+worker_timeout = args.worker_timeout
+
+if args.debug:
+log.info(f"Starting the Internal API server on port {args.port} and 
host {args.hostname}.")
+app = create_app(testing=conf.getboolean("core", "unit_test_mode"))
+app.run(
+debug=True,
+use_reloader=not app.config["TESTING"],
+port=args.port,
+host=args.hostname,
+)
+else:
+pid_file, stdout, stderr, log_file = setup_locations(
+"internal-api", args.pid, args.stdout, args.stderr, args.log_file
+)
+
+# Check if Internal APi is already running if not, remove old pidfile
+check_if_pidfile_process_is_running(pid_file=pid_file, 
process_name="internal-api")
+
+log.info(
+textwrap.dedent(
+f"""\
+Running the Gunicorn Server with:
+Workers: {num_workers} {args.workerclass}
+Host: {args.hostname}:{args.port}
+Timeout: {worker_timeout}
+Logfiles: {access_logfile} {error_logfile}
+Access Logformat: {access_logformat}
+
="""
+)
+)
+
+run_args = [
+sys.executable,
+"-m",
+"gunicorn",
+"--workers",
+str(num_workers),
+"--worker-class",
+str(args.workerclass),
+"--timeout",
+str(worker_timeout),
+"--bind",
+args.hostname + ":" + str(args.port),
+"--name",
+"airflow-internal-api",
+"--pid",
+pid_file,
+"--access-logfile",
+str(access_logfile),
+"--error-logfile",
+str(error_logfile),
+]
+
+if args.access_logformat and args.access_logformat.strip():
+run_args += ["--access-logformat", str(args.access_logformat)]
+
+if args.daemon:
+run_args += ["--daemon"]
+
+run_args +=

[GitHub] [airflow] mhenc commented on a diff in pull request #28425: AIP-44: Add command for running standalone Internal API



mhenc commented on code in PR #28425:
URL: https://github.com/apache/airflow/pull/28425#discussion_r1052610349


##
airflow/cli/commands/internal_api_command.py:
##
@@ -0,0 +1,268 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""Internal API command."""
+from __future__ import annotations
+
+import logging
+import os
+import signal
+import subprocess
+import sys
+import textwrap
+from contextlib import suppress
+from tempfile import gettempdir
+from time import sleep
+
+import daemon
+import psutil
+from daemon.pidfile import TimeoutPIDLockFile
+from flask import Flask
+from flask_appbuilder import SQLA
+from flask_caching import Cache
+from flask_wtf.csrf import CSRFProtect
+from lockfile.pidlockfile import read_pid_from_pidfile
+from sqlalchemy.engine.url import make_url
+
+from airflow import settings
+from airflow.cli.commands.webserver_command import GunicornMonitor
+from airflow.configuration import conf
+from airflow.exceptions import AirflowConfigException
+from airflow.logging_config import configure_logging
+from airflow.models import import_all_models
+from airflow.utils import cli as cli_utils
+from airflow.utils.cli import setup_locations, setup_logging
+from airflow.utils.process_utils import check_if_pidfile_process_is_running
+from airflow.www.extensions.init_dagbag import init_dagbag
+from airflow.www.extensions.init_jinja_globals import init_jinja_globals
+from airflow.www.extensions.init_manifest_files import configure_manifest_files
+from airflow.www.extensions.init_security import init_xframe_protection
+from airflow.www.extensions.init_views import init_api_internal, 
init_error_handlers
+
+log = logging.getLogger(__name__)
+app: Flask | None = None
+
+
+@cli_utils.action_cli
+def internal_api(args):
+"""Starts Airflow Internal API."""
+print(settings.HEADER)
+
+access_logfile = args.access_logfile if args.access_logfile is not None 
else "-"
+error_logfile = args.error_logfile if args.error_logfile is not None else 
"-"
+access_logformat = args.access_logformat
+num_workers = args.workers
+worker_timeout = args.worker_timeout
+
+if args.debug:
+log.info(f"Starting the Internal API server on port {args.port} and 
host {args.hostname}.")
+app = create_app(testing=conf.getboolean("core", "unit_test_mode"))
+app.run(
+debug=True,
+use_reloader=not app.config["TESTING"],
+port=args.port,
+host=args.hostname,
+)
+else:
+pid_file, stdout, stderr, log_file = setup_locations(
+"internal-api", args.pid, args.stdout, args.stderr, args.log_file
+)
+
+# Check if Internal APi is already running if not, remove old pidfile
+check_if_pidfile_process_is_running(pid_file=pid_file, 
process_name="internal-api")
+
+log.info(
+textwrap.dedent(
+f"""\
+Running the Gunicorn Server with:
+Workers: {num_workers} {args.workerclass}
+Host: {args.hostname}:{args.port}
+Timeout: {worker_timeout}
+Logfiles: {access_logfile} {error_logfile}
+Access Logformat: {access_logformat}
+
="""
+)
+)
+
+run_args = [
+sys.executable,
+"-m",
+"gunicorn",
+"--workers",
+str(num_workers),
+"--worker-class",
+str(args.workerclass),
+"--timeout",
+str(worker_timeout),
+"--bind",
+args.hostname + ":" + str(args.port),
+"--name",
+"airflow-internal-api",
+"--pid",
+pid_file,
+"--access-logfile",
+str(access_logfile),
+"--error-logfile",
+str(error_logfile),
+]
+
+if args.access_logformat and args.access_logformat.strip():
+run_args += ["--access-logformat", str(args.access_logformat)]
+
+if args.daemon:
+run_args += ["--daemon"]
+
+run_args +=

[GitHub] [airflow] eladkal commented on issue #28349: Status of testing Providers that were prepared on December 14, 2022



eladkal commented on issue #28349:
URL: https://github.com/apache/airflow/issues/28349#issuecomment-1358248369

   providers releases. thank you everyone for testing!


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] pingzh closed pull request #28367: mkdirs should set mode correctly



pingzh closed pull request #28367: mkdirs should set mode correctly
URL: https://github.com/apache/airflow/pull/28367


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] mhenc commented on a diff in pull request #28425: AIP-44: Add command for running standalone Internal API



mhenc commented on code in PR #28425:
URL: https://github.com/apache/airflow/pull/28425#discussion_r1052603868


##
airflow/cli/commands/internal_api_command.py:
##
@@ -0,0 +1,268 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""Internal API command."""
+from __future__ import annotations
+
+import logging
+import os
+import signal
+import subprocess
+import sys
+import textwrap
+from contextlib import suppress
+from tempfile import gettempdir
+from time import sleep
+
+import daemon
+import psutil
+from daemon.pidfile import TimeoutPIDLockFile
+from flask import Flask
+from flask_appbuilder import SQLA
+from flask_caching import Cache
+from flask_wtf.csrf import CSRFProtect
+from lockfile.pidlockfile import read_pid_from_pidfile
+from sqlalchemy.engine.url import make_url
+
+from airflow import settings
+from airflow.cli.commands.webserver_command import GunicornMonitor
+from airflow.configuration import conf
+from airflow.exceptions import AirflowConfigException
+from airflow.logging_config import configure_logging
+from airflow.models import import_all_models
+from airflow.utils import cli as cli_utils
+from airflow.utils.cli import setup_locations, setup_logging
+from airflow.utils.process_utils import check_if_pidfile_process_is_running
+from airflow.www.extensions.init_dagbag import init_dagbag
+from airflow.www.extensions.init_jinja_globals import init_jinja_globals
+from airflow.www.extensions.init_manifest_files import configure_manifest_files
+from airflow.www.extensions.init_security import init_xframe_protection
+from airflow.www.extensions.init_views import init_api_internal, 
init_error_handlers
+
+log = logging.getLogger(__name__)
+app: Flask | None = None
+
+
+@cli_utils.action_cli
+def internal_api(args):
+"""Starts Airflow Internal API."""
+print(settings.HEADER)
+
+access_logfile = args.access_logfile if args.access_logfile is not None 
else "-"
+error_logfile = args.error_logfile if args.error_logfile is not None else 
"-"

Review Comment:
   But this will default to "-" if *_logfile is empty string - not sure if this 
is intended ? WDYT?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] vincbeck commented on pull request #28476: Migrate DagFileProcessorManager._deactivate_stale_dags to Internal API



vincbeck commented on PR #28476:
URL: https://github.com/apache/airflow/pull/28476#issuecomment-1358236031

   cc @potiuk , @mhenc 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] Taragolis closed pull request #28471: [POC] Reset ResourceVersion.resource_version to default during tests



Taragolis closed pull request #28471: [POC] Reset 
ResourceVersion.resource_version to default during tests
URL: https://github.com/apache/airflow/pull/28471


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] pingzh opened a new pull request, #28477: Ensure correct log dir in file task handler



pingzh opened a new pull request, #28477:
URL: https://github.com/apache/airflow/pull/28477

   since Path.mkdir combines with the process’ umask value to determine the 
file mode and access flags, thus the newly created folder isn't 0o777
   
   
   
   ---
   **^ Add meaningful description above**
   
   Read the **[Pull Request 
Guidelines](https://github.com/apache/airflow/blob/main/CONTRIBUTING.rst#pull-request-guidelines)**
 for more information.
   In case of fundamental code changes, an Airflow Improvement Proposal 
([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvement+Proposals))
 is needed.
   In case of a new dependency, check compliance with the [ASF 3rd Party 
License Policy](https://www.apache.org/legal/resolved.html#category-x).
   In case of backwards incompatible changes please leave a note in a 
newsfragment file, named `{pr_number}.significant.rst` or 
`{issue_number}.significant.rst`, in 
[newsfragments](https://github.com/apache/airflow/tree/main/newsfragments).
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] eladkal closed issue #28349: Status of testing Providers that were prepared on December 14, 2022



eladkal closed issue #28349: Status of testing Providers that were prepared on 
December 14, 2022
URL: https://github.com/apache/airflow/issues/28349


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] vincbeck commented on issue #28269: AIP-44 Migrate DagFileProcessor.execute_callbacks to Internal API.



vincbeck commented on issue #28269:
URL: https://github.com/apache/airflow/issues/28269#issuecomment-1358238349

   I'd like to work on this one
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] vincbeck opened a new pull request, #28476: Migrate DagFileProcessorManager._deactivate_stale_dags to Internal API



vincbeck opened a new pull request, #28476:
URL: https://github.com/apache/airflow/pull/28476

   Migrate `DagFileProcessorManager._deactivate_stale_dags` to Internal API. 
Please while reviewing it to pay double attention to how I handle the logs. 
When discussing with @mhenc, we decided:
   - If a function is NOT executed though internal API, the logger should stay 
as is
   - If a function is executed through internal API, the logger should be 
separate
   
   Very happy to hear your feedback and/or suggestions since, to be very 
honest, I am not a big fan of passing the logger as parameter
   
   Closes #28270 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] pingzh commented on pull request #28367: mkdirs should set mode correctly



pingzh commented on PR #28367:
URL: https://github.com/apache/airflow/pull/28367#issuecomment-1358242305

   close this in favor of https://github.com/apache/airflow/pull/28477


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[airflow] annotated tag providers-apache-hive/5.0.0 updated (1976ff72c5 -> abe0dd487a)

This is an automated email from the ASF dual-hosted git repository.

eladkal pushed a change to annotated tag providers-apache-hive/5.0.0
in repository https://gitbox.apache.org/repos/asf/airflow.git


*** WARNING: tag providers-apache-hive/5.0.0 was modified! ***

from 1976ff72c5 (commit)
  to abe0dd487a (tag)
 tagging 1976ff72c52c3f5bf9f3e16329b55ab3888c3406 (commit)
 replaces providers-exasol/4.1.2
  by Elad Kalif
  on Mon Dec 19 21:51:57 2022 +0200

- Log -
Release 2022-12-19 of providers
---


No new revisions were added by this update.

Summary of changes:

[airflow] annotated tag providers-microsoft-azure/5.0.1 updated (1976ff72c5 -> e1ac0a6cfa)

This is an automated email from the ASF dual-hosted git repository.

eladkal pushed a change to annotated tag providers-microsoft-azure/5.0.1
in repository https://gitbox.apache.org/repos/asf/airflow.git


*** WARNING: tag providers-microsoft-azure/5.0.1 was modified! ***

from 1976ff72c5 (commit)
  to e1ac0a6cfa (tag)
 tagging 1976ff72c52c3f5bf9f3e16329b55ab3888c3406 (commit)
 replaces providers-exasol/4.1.2
  by Elad Kalif
  on Mon Dec 19 21:51:57 2022 +0200

- Log -
Release 2022-12-19 of providers
---


No new revisions were added by this update.

Summary of changes:

[GitHub] [airflow-site] eladkal merged pull request #711: Add documentation for packages - 2022-12-14



eladkal merged PR #711:
URL: https://github.com/apache/airflow-site/pull/711


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] XD-DENG commented on a diff in pull request #28475: Fix flaky test_recover_from_resource_too_old exception



XD-DENG commented on code in PR #28475:
URL: https://github.com/apache/airflow/pull/28475#discussion_r1052576617


##
tests/executors/test_kubernetes_executor.py:
##
@@ -1249,12 +1249,13 @@ def effect():
 try:
 # self.watcher._run() is mocked and return "500" as last 
resource_version
 self.watcher.run()
+assert False, "Should have raised Exception"

Review Comment:
   The change in line 1258 is all good to me.
   But is this change at line 1252 necessary? I don't fully get it yet . The 
watcher is already mocked and will fail



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] potiuk commented on pull request #27405: Add deferrable mode to MLEngineStartTrainingJobOperator



potiuk commented on PR #27405:
URL: https://github.com/apache/airflow/pull/27405#issuecomment-1358166398

   Fix to the flaky exception here: https://github.com/apache/airflow/pull/28475


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] potiuk opened a new pull request, #28475: Fixed flaky Resource exception



potiuk opened a new pull request, #28475:
URL: https://github.com/apache/airflow/pull/28475

   After #28047 the test_recover_from_resource_too_old started to fail in a 
flaky way. Turned out that - depend on some other test run the Singleton 
ResourceVersion could containt not one but two namespaces (including default 
namespace).
   
   Also while fixing the tests it's been noticed that the test missed an assert 
- it did not assert that the Exception was in fact thrown, so the test could 
have succeeded even if the exception was not really thrown (there was assert in 
"except" clause but if the exception was not thrown, it would not have been 
called at all).
   
   
   
   ---
   **^ Add meaningful description above**
   
   Read the **[Pull Request 
Guidelines](https://github.com/apache/airflow/blob/main/CONTRIBUTING.rst#pull-request-guidelines)**
 for more information.
   In case of fundamental code changes, an Airflow Improvement Proposal 
([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvement+Proposals))
 is needed.
   In case of a new dependency, check compliance with the [ASF 3rd Party 
License Policy](https://www.apache.org/legal/resolved.html#category-x).
   In case of backwards incompatible changes please leave a note in a 
newsfragment file, named `{pr_number}.significant.rst` or 
`{issue_number}.significant.rst`, in 
[newsfragments](https://github.com/apache/airflow/tree/main/newsfragments).
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] ibrahimdurmus commented on issue #28412: Make `--executor` flag works for `breeze start-airflow` command



ibrahimdurmus commented on issue #28412:
URL: https://github.com/apache/airflow/issues/28412#issuecomment-1358164311

   yes i am working on it thanks @victorjourne 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] potiuk commented on pull request #27405: Add deferrable mode to MLEngineStartTrainingJobOperator



potiuk commented on PR #27405:
URL: https://github.com/apache/airflow/pull/27405#issuecomment-1358158556

   OK. I fixed it (and I found that the test missed one assert). PR is coming.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] maxdml opened a new issue, #28474: varimport payloads are not logged in audit logs



maxdml opened a new issue, #28474:
URL: https://github.com/apache/airflow/issues/28474

   ### Description
   
   The content of the payload of the /variable/varimport  is not reported to 
audit logs. 
   
   ### Use case/motivation
   
   We should audit this payload in the same way we audit 
creating/updating/deleting individual variables through the UI
   
   ### Related issues
   
   _No response_
   
   ### Are you willing to submit a PR?
   
   - [X] Yes I am willing to submit a PR!
   
   ### Code of Conduct
   
   - [X] I agree to follow this project's [Code of 
Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] boring-cyborg[bot] commented on issue #28474: varimport payloads are not logged in audit logs



boring-cyborg[bot] commented on issue #28474:
URL: https://github.com/apache/airflow/issues/28474#issuecomment-1358152338

   Thanks for opening your first issue here! Be sure to follow the issue 
template!
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] eladkal commented on pull request #28282: Mark stepIds for cancel in EmrAddStepsOperator



eladkal commented on PR #28282:
URL: https://github.com/apache/airflow/pull/28282#issuecomment-1358151116

   @o-nikolas @ferruzzi @vincbeck can you please share thoughts on my concern 
above?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] Taragolis closed pull request #28471: [POC] Reset ResourceVersion.resource_version to default during tests



Taragolis closed pull request #28471: [POC] Reset 
ResourceVersion.resource_version to default during tests
URL: https://github.com/apache/airflow/pull/28471


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] potiuk opened a new pull request, #28473: Update release process for providers after Elad's release



potiuk opened a new pull request, #28473:
URL: https://github.com/apache/airflow/pull/28473

   While running the release several issues have been spotted when run by 
first-time release manager.
   
   
   
   ---
   **^ Add meaningful description above**
   
   Read the **[Pull Request 
Guidelines](https://github.com/apache/airflow/blob/main/CONTRIBUTING.rst#pull-request-guidelines)**
 for more information.
   In case of fundamental code changes, an Airflow Improvement Proposal 
([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvement+Proposals))
 is needed.
   In case of a new dependency, check compliance with the [ASF 3rd Party 
License Policy](https://www.apache.org/legal/resolved.html#category-x).
   In case of backwards incompatible changes please leave a note in a 
newsfragment file, named `{pr_number}.significant.rst` or 
`{issue_number}.significant.rst`, in 
[newsfragments](https://github.com/apache/airflow/tree/main/newsfragments).
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

svn commit: r58810 - /dev/airflow/providers/ /release/airflow/providers/

Author: potiuk
Date: Mon Dec 19 18:45:14 2022
New Revision: 58810

Log:
Release Airflow Providers on 2022-12-19

Added:
release/airflow/providers/apache-airflow-providers-apache-hive-5.0.0.tar.gz
  - copied unchanged from r58809, 
dev/airflow/providers/apache-airflow-providers-apache-hive-5.0.0.tar.gz

release/airflow/providers/apache-airflow-providers-apache-hive-5.0.0.tar.gz.asc
  - copied unchanged from r58809, 
dev/airflow/providers/apache-airflow-providers-apache-hive-5.0.0.tar.gz.asc

release/airflow/providers/apache-airflow-providers-apache-hive-5.0.0.tar.gz.sha512
  - copied unchanged from r58809, 
dev/airflow/providers/apache-airflow-providers-apache-hive-5.0.0.tar.gz.sha512

release/airflow/providers/apache-airflow-providers-microsoft-azure-5.0.1.tar.gz
  - copied unchanged from r58809, 
dev/airflow/providers/apache-airflow-providers-microsoft-azure-5.0.1.tar.gz

release/airflow/providers/apache-airflow-providers-microsoft-azure-5.0.1.tar.gz.asc
  - copied unchanged from r58809, 
dev/airflow/providers/apache-airflow-providers-microsoft-azure-5.0.1.tar.gz.asc

release/airflow/providers/apache-airflow-providers-microsoft-azure-5.0.1.tar.gz.sha512
  - copied unchanged from r58809, 
dev/airflow/providers/apache-airflow-providers-microsoft-azure-5.0.1.tar.gz.sha512

release/airflow/providers/apache_airflow_providers_apache_hive-5.0.0-py3-none-any.whl
  - copied unchanged from r58809, 
dev/airflow/providers/apache_airflow_providers_apache_hive-5.0.0-py3-none-any.whl

release/airflow/providers/apache_airflow_providers_apache_hive-5.0.0-py3-none-any.whl.asc
  - copied unchanged from r58809, 
dev/airflow/providers/apache_airflow_providers_apache_hive-5.0.0-py3-none-any.whl.asc

release/airflow/providers/apache_airflow_providers_apache_hive-5.0.0-py3-none-any.whl.sha512
  - copied unchanged from r58809, 
dev/airflow/providers/apache_airflow_providers_apache_hive-5.0.0-py3-none-any.whl.sha512

release/airflow/providers/apache_airflow_providers_microsoft_azure-5.0.1-py3-none-any.whl
  - copied unchanged from r58809, 
dev/airflow/providers/apache_airflow_providers_microsoft_azure-5.0.1-py3-none-any.whl

release/airflow/providers/apache_airflow_providers_microsoft_azure-5.0.1-py3-none-any.whl.asc
  - copied unchanged from r58809, 
dev/airflow/providers/apache_airflow_providers_microsoft_azure-5.0.1-py3-none-any.whl.asc

release/airflow/providers/apache_airflow_providers_microsoft_azure-5.0.1-py3-none-any.whl.sha512
  - copied unchanged from r58809, 
dev/airflow/providers/apache_airflow_providers_microsoft_azure-5.0.1-py3-none-any.whl.sha512
Removed:
dev/airflow/providers/apache-airflow-providers-apache-hive-5.0.0.tar.gz
dev/airflow/providers/apache-airflow-providers-apache-hive-5.0.0.tar.gz.asc

dev/airflow/providers/apache-airflow-providers-apache-hive-5.0.0.tar.gz.sha512
dev/airflow/providers/apache-airflow-providers-microsoft-azure-5.0.1.tar.gz

dev/airflow/providers/apache-airflow-providers-microsoft-azure-5.0.1.tar.gz.asc

dev/airflow/providers/apache-airflow-providers-microsoft-azure-5.0.1.tar.gz.sha512

dev/airflow/providers/apache_airflow_providers_apache_hive-5.0.0-py3-none-any.whl

dev/airflow/providers/apache_airflow_providers_apache_hive-5.0.0-py3-none-any.whl.asc

dev/airflow/providers/apache_airflow_providers_apache_hive-5.0.0-py3-none-any.whl.sha512

dev/airflow/providers/apache_airflow_providers_microsoft_azure-5.0.1-py3-none-any.whl

dev/airflow/providers/apache_airflow_providers_microsoft_azure-5.0.1-py3-none-any.whl.asc

dev/airflow/providers/apache_airflow_providers_microsoft_azure-5.0.1-py3-none-any.whl.sha512
release/airflow/providers/apache-airflow-providers-apache-hive-4.1.1.tar.gz

release/airflow/providers/apache-airflow-providers-apache-hive-4.1.1.tar.gz.asc

release/airflow/providers/apache-airflow-providers-apache-hive-4.1.1.tar.gz.sha512

release/airflow/providers/apache-airflow-providers-microsoft-azure-5.0.0.tar.gz

release/airflow/providers/apache-airflow-providers-microsoft-azure-5.0.0.tar.gz.asc

release/airflow/providers/apache-airflow-providers-microsoft-azure-5.0.0.tar.gz.sha512

release/airflow/providers/apache_airflow_providers_apache_hive-4.1.1-py3-none-any.whl

release/airflow/providers/apache_airflow_providers_apache_hive-4.1.1-py3-none-any.whl.asc

release/airflow/providers/apache_airflow_providers_apache_hive-4.1.1-py3-none-any.whl.sha512

release/airflow/providers/apache_airflow_providers_microsoft_azure-5.0.0-py3-none-any.whl

release/airflow/providers/apache_airflow_providers_microsoft_azure-5.0.0-py3-none-any.whl.asc

release/airflow/providers/apache_airflow_providers_microsoft_azure-5.0.0-py3-none-any.whl.sha512

[GitHub] [airflow] snjypl commented on issue #28470: Microsoft Operators has Azure Blob Storage Transfer Operator listed twice



snjypl commented on issue #28470:
URL: https://github.com/apache/airflow/issues/28470#issuecomment-1358084577

   @dnarain  they are both links to  different operators.  one is 
AzureBlobStorageToGCSOperator and  other is SFTPToWasbOperator . the link needs 
to be fixed. 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] Taragolis commented on a diff in pull request #28464: Add waiter config params to emr.add_job_flow_steps



Taragolis commented on code in PR #28464:
URL: https://github.com/apache/airflow/pull/28464#discussion_r1052506327


##
airflow/providers/amazon/aws/hooks/emr.py:
##
@@ -126,14 +126,21 @@ def create_job_flow(self, job_flow_overrides: dict[str, 
Any]) -> dict[str, Any]:
 return response
 
 def add_job_flow_steps(
-self, job_flow_id: str, steps: list[dict] | str | None = None, 
wait_for_completion: bool = False
+self,
+job_flow_id: str,
+steps: list[dict] | str | None = None,
+wait_for_completion: bool = False,
+waiter_delay: int = 5,
+waiter_max_attempts: int = 100,

Review Comment:
   Yep, in this case we reduce number of parameters.
   
   But we need to be sure that we send as a dictionary (otherwise it raise an 
error) and only `Delay` and `MaxAttempts` are acceptable parameters.
   
   Unfortunetly waiters not well documented or better say `botocore` sources 
are only the documentation of waiters internals:
   
   
https://github.com/boto/botocore/blob/6af5c29d9c71deb325eca99f24698303fc9e3dea/botocore/waiter.py#L336-L338



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] vandonr-amz commented on pull request #28472: Add AWS Sagemaker Auto ML operator and sensor



vandonr-amz commented on PR #28472:
URL: https://github.com/apache/airflow/pull/28472#issuecomment-1358066883

   sorry, making this as draft as it's based on top of 
https://github.com/apache/airflow/pull/28024
   It's not a strict dependency, but having this merged second would avoid a 
lot of conflict solving. I'll mark it as ready for review once the other PR is 
merged.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] VladaZakharova closed pull request #28284: Fix for issue with reading schema fields for JSON files in GCSToBigQueryOperator



VladaZakharova closed pull request #28284: Fix for issue with reading schema 
fields for JSON files in GCSToBigQueryOperator
URL: https://github.com/apache/airflow/pull/28284


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] vandonr-amz opened a new pull request, #28472: Add AWS Sagemaker Auto ML operator and sensor



vandonr-amz opened a new pull request, #28472:
URL: https://github.com/apache/airflow/pull/28472

   AutoML is a way to do machine learning experiments with minimal human 
oversight, just pushing training data and getting a model at the other end.
   
   The operator allows triggering an Auto ML pipeline, consuming training data 
from S3, with optional auto-deployment of the model.
   The sensor watches the job until it finishes.
   
   Most of the code is in the hook, this time it's the operator that's thin.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] XD-DENG commented on pull request #27405: Add deferrable mode to MLEngineStartTrainingJobOperator



XD-DENG commented on PR #27405:
URL: https://github.com/apache/airflow/pull/27405#issuecomment-1358058017

   Sure I will take a look @potiuk . Please ping me for reminding if you don't 
hear from me later


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] potiuk commented on pull request #27405: Add deferrable mode to MLEngineStartTrainingJobOperator