[GitHub] [airflow] uranusjr commented on a diff in pull request #29094: Add `max_active_tis_per_dagrun` for Dynamic Task Mapping

2023-04-13 Thread via GitHub
uranusjr commented on code in PR #29094: URL: https://github.com/apache/airflow/pull/29094#discussion_r1166300450 ## airflow/models/dag.py: ## @@ -2789,7 +2789,9 @@ def bulk_write_to_db( orm_dag.description = dag.description orm_dag.max_active_tasks =

[GitHub] [airflow] uranusjr commented on a diff in pull request #29094: Add `max_active_tis_per_dagrun` for Dynamic Task Mapping

2023-04-13 Thread via GitHub
uranusjr commented on code in PR #29094: URL: https://github.com/apache/airflow/pull/29094#discussion_r1166297704 ## airflow/jobs/scheduler_job_runner.py: ## @@ -304,26 +323,24 @@ def _executable_task_instances_to_queued(self, max_tis: int, session: Session) -

[GitHub] [airflow] uranusjr commented on a diff in pull request #29094: Add `max_active_tis_per_dagrun` for Dynamic Task Mapping

2023-04-13 Thread via GitHub
uranusjr commented on code in PR #29094: URL: https://github.com/apache/airflow/pull/29094#discussion_r1166296913 ## airflow/jobs/scheduler_job_runner.py: ## @@ -231,28 +255,21 @@ def is_alive(self, grace_multiplier: float | None = None) -> bool: <

[GitHub] [airflow] syedahsn commented on pull request #30032: Add support for deferrable operators in AMPP

2023-04-13 Thread via GitHub
syedahsn commented on PR #30032: URL: https://github.com/apache/airflow/pull/30032#issuecomment-1507915614 I've resolved all the failing tests. Please have another look and let me know what you think @potiuk @Taragolis @pankajastro -- This is an automated message from the Apache Git

[GitHub] [airflow] boring-cyborg[bot] commented on pull request #30637: Update quicksight.py / QuickSightHook bug on wait_for_state() function: #30626

2023-04-13 Thread via GitHub
boring-cyborg[bot] commented on PR #30637: URL: https://github.com/apache/airflow/pull/30637#issuecomment-1507914999 Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our Contribution

[GitHub] [airflow] AkshatJoshi-18 opened a new pull request, #30637: Update quicksight.py / QuickSightHook bug on wait_for_state() function: #30626

2023-04-13 Thread via GitHub
AkshatJoshi-18 opened a new pull request, #30637: URL: https://github.com/apache/airflow/pull/30637 --- **^ Add meaningful description above** Read the **[Pull Request Guidelines](https://github.com/apache/airflow/blob/main/CONTRIBUTING.rst#pull-request-guidelines)** for

[GitHub] [airflow] mabrikan commented on issue #30585: Status of testing of Apache Airflow Helm Chart 1.9.0rc1

2023-04-13 Thread via GitHub
mabrikan commented on issue #30585: URL: https://github.com/apache/airflow/issues/30585#issuecomment-1507912035 Can confirm that #29838 is working as expected. Default

[GitHub] [airflow] o-nikolas commented on pull request #28338: New AWS sensor — DynamoDBValueSensor

2023-04-13 Thread via GitHub
o-nikolas commented on PR #28338: URL: https://github.com/apache/airflow/pull/28338#issuecomment-1507883671 Just approved another build on the latest changes! :crossed_fingers: -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

[GitHub] [airflow] utkarsharma2 commented on a diff in pull request #30595: DynamoDBHook - waiter_path() to consider `resource_type` or `client_type`

2023-04-13 Thread via GitHub
utkarsharma2 commented on code in PR #30595: URL: https://github.com/apache/airflow/pull/30595#discussion_r1166247645 ## tests/providers/amazon/aws/hooks/test_dynamodb.py: ## @@ -55,3 +56,9 @@ def test_insert_batch_items_dynamodb_table(self):

[GitHub] [airflow] hussein-awala commented on a diff in pull request #29094: Add `max_active_tis_per_dagrun` for Dynamic Task Mapping

2023-04-13 Thread via GitHub
hussein-awala commented on code in PR #29094: URL: https://github.com/apache/airflow/pull/29094#discussion_r1166193199 ## airflow/jobs/scheduler_job.py: ## @@ -83,6 +84,17 @@ DM = DagModel +@dataclass +class ConcurrencyMap: +"""Dataclass to represent concurrency

[GitHub] [airflow] boring-cyborg[bot] commented on issue #30636: In "random" time when execute "./airflow.sh dags reserialize" occurs error like "duplicate key value violates unique constraint seriali

2023-04-13 Thread via GitHub
boring-cyborg[bot] commented on issue #30636: URL: https://github.com/apache/airflow/issues/30636#issuecomment-1507806803 Thanks for opening your first issue here! Be sure to follow the issue template! If you are willing to raise PR to address this issue please do so, no need to wait for

[GitHub] [airflow] fuechter opened a new issue, #30636: In "random" time when execute "./airflow.sh dags reserialize" occurs error like "duplicate key value violates unique constraint serialized_dag_p

2023-04-13 Thread via GitHub
fuechter opened a new issue, #30636: URL: https://github.com/apache/airflow/issues/30636 ### Apache Airflow version Other Airflow 2 version (please specify below) ### What happened Version Airflow: 2.5.1 I have a file in python that generate dynamic DAG, and all

[GitHub] [airflow] mrichman commented on pull request #28338: New AWS sensor — DynamoDBValueSensor

2023-04-13 Thread via GitHub
mrichman commented on PR #28338: URL: https://github.com/apache/airflow/pull/28338#issuecomment-1507805127 ![image](https://user-images.githubusercontent.com/11228/231917937-aa35e90c-f76d-42ba-92a3-f2a4286565cc.png) -- This is an automated message from the Apache Git Service. To

[airflow] 01/01: Merge branch 'main' into feat/max_active_tis_per_dagrun

2023-04-13 Thread husseinawala
This is an automated email from the ASF dual-hosted git repository. husseinawala pushed a commit to branch feat/max_active_tis_per_dagrun in repository https://gitbox.apache.org/repos/asf/airflow.git commit 04cb0da4b57255376c7c66276d7ab4eb8094034f Merge: 4081117d06 058b6ebdd8 Author: Hussein

[airflow] branch feat/max_active_tis_per_dagrun created (now 04cb0da4b5)

2023-04-13 Thread husseinawala
This is an automated email from the ASF dual-hosted git repository. husseinawala pushed a change to branch feat/max_active_tis_per_dagrun in repository https://gitbox.apache.org/repos/asf/airflow.git at 04cb0da4b5 Merge branch 'main' into feat/max_active_tis_per_dagrun This branch

[GitHub] [airflow] ying-w opened a new issue, #30635: `BigQueryGetDataOperator` does not respect project_id parameter

2023-04-13 Thread via GitHub
ying-w opened a new issue, #30635: URL: https://github.com/apache/airflow/issues/30635 ### Apache Airflow Provider(s) google ### Versions of Apache Airflow Providers apache-airflow-providers-google==8.11.0 google-cloud-bigquery==2.34.4 ### Apache Airflow

[GitHub] [airflow] hussein-awala commented on pull request #30558: [WIP] Clean microsoft azure provider by deleting the custom prefix from conn extra fields

2023-04-13 Thread via GitHub
hussein-awala commented on PR #30558: URL: https://github.com/apache/airflow/pull/30558#issuecomment-1507770774 The method `_ensure_prefixes` was added in #27041 by @dstandish:

[GitHub] [airflow] shubham22 commented on pull request #30612: Create a metric for task adoption time

2023-04-13 Thread via GitHub
shubham22 commented on PR #30612: URL: https://github.com/apache/airflow/pull/30612#issuecomment-1507766218 I am personally inclined towards one metric. Users generally care about the lag between when there task was ready for execution and when the task actually started executing, not

[GitHub] [airflow] hussein-awala commented on pull request #30300: Fix Redshift unload query when select query contains single quotes

2023-04-13 Thread via GitHub
hussein-awala commented on PR #30300: URL: https://github.com/apache/airflow/pull/30300#issuecomment-1507763678 @vincbeck I tested it and I had a problem, so I added a check as you proposed with a warning, we can remove this check in the next major release, can you check it now? -- This

[GitHub] [airflow] jlaneve opened a new pull request, #30634: Remove duplicate param docstring in EksPodOperator

2023-04-13 Thread via GitHub
jlaneve opened a new pull request, #30634: URL: https://github.com/apache/airflow/pull/30634 This PR removes an old param tag in the EksPodOperator's docstring. Looks like it was left in by accident! --- **^ Add meaningful description above** Read the **[Pull

[GitHub] [airflow] jedcunningham commented on a diff in pull request #30375: Simplify logic to resolve tasks stuck in queued despite stalled_task_timeout

2023-04-13 Thread via GitHub
jedcunningham commented on code in PR #30375: URL: https://github.com/apache/airflow/pull/30375#discussion_r1166109290 ## airflow/jobs/scheduler_job_runner.py: ## @@ -1425,6 +1461,47 @@ def _send_sla_callbacks_to_processor(self, dag: DAG) -> None: )

[GitHub] [airflow] pierrejeambrun opened a new pull request, #30633: Table for affected tasks in actions

2023-04-13 Thread via GitHub
pierrejeambrun opened a new pull request, #30633: URL: https://github.com/apache/airflow/pull/30633 Follow up of https://github.com/apache/airflow/pull/30373, you can only look at the last commit that allows to display the affected task instances in a table for more visibility when

[GitHub] [airflow] jedcunningham commented on a diff in pull request #30375: Simplify logic to resolve tasks stuck in queued despite stalled_task_timeout

2023-04-13 Thread via GitHub
jedcunningham commented on code in PR #30375: URL: https://github.com/apache/airflow/pull/30375#discussion_r1166100533 ## airflow/executors/kubernetes_executor.py: ## @@ -865,6 +826,41 @@ def try_adopt_task_instances(self, tis: Sequence[TaskInstance]) -> Sequence[Task

[GitHub] [airflow] hussein-awala commented on a diff in pull request #30603: Support minAvailable property for PodDisruptionBudget

2023-04-13 Thread via GitHub
hussein-awala commented on code in PR #30603: URL: https://github.com/apache/airflow/pull/30603#discussion_r1166091872 ## chart/values.schema.json: ## @@ -1864,6 +1864,14 @@ "string" ],

[GitHub] [airflow] jedcunningham commented on a diff in pull request #30375: Simplify logic to resolve tasks stuck in queued despite stalled_task_timeout

2023-04-13 Thread via GitHub
jedcunningham commented on code in PR #30375: URL: https://github.com/apache/airflow/pull/30375#discussion_r1166086367 ## airflow/executors/base_executor.py: ## @@ -376,6 +378,22 @@ def terminate(self): """This method is called when the daemon receives a SIGTERM."""

[GitHub] [airflow] jedcunningham commented on a diff in pull request #30375: Simplify logic to resolve tasks stuck in queued despite stalled_task_timeout

2023-04-13 Thread via GitHub
jedcunningham commented on code in PR #30375: URL: https://github.com/apache/airflow/pull/30375#discussion_r1166085371 ## airflow/executors/base_executor.py: ## @@ -297,8 +297,10 @@ def change_state(self, key: TaskInstanceKey, state: str, info=None) -> None: """

[GitHub] [airflow] hussein-awala commented on a diff in pull request #29809: Adding configuration to control retry parameters for k8s api client

2023-04-13 Thread via GitHub
hussein-awala commented on code in PR #29809: URL: https://github.com/apache/airflow/pull/29809#discussion_r1166074222 ## airflow/kubernetes/kube_client.py: ## @@ -125,5 +138,5 @@ def get_kube_client( if ssl_ca_cert: configuration.ssl_ca_cert = ssl_ca_cert -

[GitHub] [airflow] pierrejeambrun commented on pull request #30373: Improve task & run actions ux in grid view

2023-04-13 Thread via GitHub
pierrejeambrun commented on PR #30373: URL: https://github.com/apache/airflow/pull/30373#issuecomment-1507655221 I think you are right @uranusjr. I am opening a follow up PR to display a table there for the task detail, I feel like it is really hard to see the task. Also I just

[GitHub] [airflow] hussein-awala commented on a diff in pull request #29809: Adding configuration to control retry parameters for k8s api client

2023-04-13 Thread via GitHub
hussein-awala commented on code in PR #29809: URL: https://github.com/apache/airflow/pull/29809#discussion_r1166065617 ## tests/kubernetes/test_client.py: ## @@ -81,3 +84,11 @@ def test_disable_verify_ssl(self): else: configuration = Configuration()

[GitHub] [airflow] uranusjr commented on a diff in pull request #30380: Replace prints by logging

2023-04-13 Thread via GitHub
uranusjr commented on code in PR #30380: URL: https://github.com/apache/airflow/pull/30380#discussion_r1166060230 ## airflow/cli/commands/task_command.py: ## @@ -450,13 +448,12 @@ def task_failed_deps(args): dep_context = DepContext(deps=SCHEDULER_QUEUED_DEPS)

[GitHub] [airflow] jedcunningham commented on a diff in pull request #30375: Simplify logic to resolve tasks stuck in queued despite stalled_task_timeout

2023-04-13 Thread via GitHub
jedcunningham commented on code in PR #30375: URL: https://github.com/apache/airflow/pull/30375#discussion_r1166052103 ## newsfragments/30375.significant.rst: ## @@ -0,0 +1,9 @@ +Simplify logic to resolve tasks stuck in queued despite stalled_task_timeout Review Comment:

[GitHub] [airflow] vandonr-amz commented on a diff in pull request #30495: preload airflow imports before dag parsing to save time

2023-04-13 Thread via GitHub
vandonr-amz commented on code in PR #30495: URL: https://github.com/apache/airflow/pull/30495#discussion_r1166056851 ## airflow/dag_processing/processor.py: ## @@ -187,6 +188,18 @@ def _handle_dag_file_processing(): def start(self) -> None: """Launch the process

[GitHub] [airflow] uranusjr commented on a diff in pull request #30495: preload airflow imports before dag parsing to save time

2023-04-13 Thread via GitHub
uranusjr commented on code in PR #30495: URL: https://github.com/apache/airflow/pull/30495#discussion_r1166056086 ## airflow/dag_processing/processor.py: ## @@ -187,6 +188,18 @@ def _handle_dag_file_processing(): def start(self) -> None: """Launch the process

[GitHub] [airflow] uranusjr commented on a diff in pull request #30367: Fix airflow module version check when using ExternalPythonOperator and debug logging level

2023-04-13 Thread via GitHub
uranusjr commented on code in PR #30367: URL: https://github.com/apache/airflow/pull/30367#discussion_r1166055102 ## airflow/operators/python.py: ## @@ -695,7 +695,7 @@ def _get_airflow_version_from_target_env(self) -> str | None: result = subprocess.check_output(

[GitHub] [airflow] uranusjr commented on a diff in pull request #30367: Fix airflow module version check when using ExternalPythonOperator and debug logging level

2023-04-13 Thread via GitHub
uranusjr commented on code in PR #30367: URL: https://github.com/apache/airflow/pull/30367#discussion_r1166054484 ## airflow/operators/python.py: ## @@ -695,7 +695,7 @@ def _get_airflow_version_from_target_env(self) -> str | None: result = subprocess.check_output(

[GitHub] [airflow] uranusjr commented on a diff in pull request #30367: Fix airflow module version check when using ExternalPythonOperator and debug logging level

2023-04-13 Thread via GitHub
uranusjr commented on code in PR #30367: URL: https://github.com/apache/airflow/pull/30367#discussion_r1166053752 ## airflow/__init__.py: ## @@ -61,7 +61,8 @@ # Perform side-effects unless someone has explicitly opted out before import -# WARNING: DO NOT USE THIS UNLESS

[airflow] branch constraints-main updated: Updating constraints. Github run id:4693629049

2023-04-13 Thread github-bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch constraints-main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/constraints-main by this push: new c7abfb4fb1 Updating

[GitHub] [airflow] uranusjr commented on a diff in pull request #30485: Split Celery logs into stdout/stderr

2023-04-13 Thread via GitHub
uranusjr commented on code in PR #30485: URL: https://github.com/apache/airflow/pull/30485#discussion_r1166043127 ## airflow/cli/commands/celery_command.py: ## @@ -96,6 +100,27 @@ def _serve_logs(skip_serve_logs: bool = False): sub_proc.terminate()

[GitHub] [airflow] uranusjr commented on a diff in pull request #30485: Split Celery logs into stdout/stderr

2023-04-13 Thread via GitHub
uranusjr commented on code in PR #30485: URL: https://github.com/apache/airflow/pull/30485#discussion_r1166046760 ## airflow/cli/commands/celery_command.py: ## @@ -96,6 +100,32 @@ def _serve_logs(skip_serve_logs: bool = False): sub_proc.terminate()

[GitHub] [airflow] potiuk commented on pull request #30367: Fix airflow module version check when using ExternalPythonOperator and debug logging level

2023-04-13 Thread via GitHub
potiuk commented on PR #30367: URL: https://github.com/apache/airflow/pull/30367#issuecomment-1507631339 > @uranusjr I added some comments at both places, are they clear enough ? Good enough for me. -- This is an automated message from the Apache Git Service. To respond to the

[GitHub] [airflow] uranusjr commented on a diff in pull request #30485: Split Celery logs into stdout/stderr

2023-04-13 Thread via GitHub
uranusjr commented on code in PR #30485: URL: https://github.com/apache/airflow/pull/30485#discussion_r1166046035 ## airflow/cli/commands/celery_command.py: ## @@ -96,6 +100,27 @@ def _serve_logs(skip_serve_logs: bool = False): sub_proc.terminate()

[GitHub] [airflow] uranusjr commented on a diff in pull request #30485: Split Celery logs into stdout/stderr

2023-04-13 Thread via GitHub
uranusjr commented on code in PR #30485: URL: https://github.com/apache/airflow/pull/30485#discussion_r1166044816 ## airflow/cli/commands/celery_command.py: ## @@ -96,6 +100,32 @@ def _serve_logs(skip_serve_logs: bool = False): sub_proc.terminate()

[GitHub] [airflow] SamWheating closed pull request #30113: Allow setting the DagRun state in create DAGRun API

2023-04-13 Thread via GitHub
SamWheating closed pull request #30113: Allow setting the DagRun state in create DAGRun API URL: https://github.com/apache/airflow/pull/30113 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [airflow] uranusjr commented on a diff in pull request #30485: Split Celery logs into stdout/stderr

2023-04-13 Thread via GitHub
uranusjr commented on code in PR #30485: URL: https://github.com/apache/airflow/pull/30485#discussion_r1166043127 ## airflow/cli/commands/celery_command.py: ## @@ -96,6 +100,27 @@ def _serve_logs(skip_serve_logs: bool = False): sub_proc.terminate()

[GitHub] [airflow] uranusjr commented on a diff in pull request #30485: Split Celery logs into stdout/stderr

2023-04-13 Thread via GitHub
uranusjr commented on code in PR #30485: URL: https://github.com/apache/airflow/pull/30485#discussion_r1166042132 ## airflow/cli/commands/celery_command.py: ## @@ -96,6 +100,32 @@ def _serve_logs(skip_serve_logs: bool = False): sub_proc.terminate()

[GitHub] [airflow] mathbou commented on pull request #30367: Fix airflow module version check when using ExternalPythonOperator and debug logging level

2023-04-13 Thread via GitHub
mathbou commented on PR #30367: URL: https://github.com/apache/airflow/pull/30367#issuecomment-1507625890 @uranusjr I added some comments at both places, are they clear enough ? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [airflow] hussein-awala commented on a diff in pull request #30310: In `DockerOperator`, adding an attribute `tls_verify` to choose whether to validate certificate (#30309)

2023-04-13 Thread via GitHub
hussein-awala commented on code in PR #30310: URL: https://github.com/apache/airflow/pull/30310#discussion_r1166036989 ## tests/providers/docker/operators/test_docker.py: ## @@ -65,6 +66,7 @@ def test_hook_usage(docker_hook_patcher, docker_conn_id, tls_params: dict):

[GitHub] [airflow] uranusjr commented on pull request #30373: Improve task & run actions ux in grid view

2023-04-13 Thread via GitHub
uranusjr commented on PR #30373: URL: https://github.com/apache/airflow/pull/30373#issuecomment-1507617863 It seems the popup got narrower, or is it simply a difference in local setup? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [airflow] hussein-awala commented on a diff in pull request #30432: FEAT : Add command to get DAG Details via CLI

2023-04-13 Thread via GitHub
hussein-awala commented on code in PR #30432: URL: https://github.com/apache/airflow/pull/30432#discussion_r1166027156 ## tests/cli/commands/test_dag_command.py: ## @@ -470,6 +471,28 @@ def test_cli_report(self): assert "airflow/example_dags/example_complex.py" in out

[GitHub] [airflow] o-nikolas commented on a diff in pull request #30541: Organize Amazon provider docs index

2023-04-13 Thread via GitHub
o-nikolas commented on code in PR #30541: URL: https://github.com/apache/airflow/pull/30541#discussion_r1166020767 ## docs/apache-airflow-providers-amazon/operators/emr/emr.rst: ## @@ -15,9 +15,9 @@ specific language governing permissions and limitations under the

[GitHub] [airflow] pierrejeambrun commented on a diff in pull request #30373: Improve task & run actions ux in grid view

2023-04-13 Thread via GitHub
pierrejeambrun commented on code in PR #30373: URL: https://github.com/apache/airflow/pull/30373#discussion_r1166018068 ## airflow/www/static/js/dag/details/dagRun/MarkRunAs.tsx: ## @@ -0,0 +1,91 @@ +/*! + * Licensed to the Apache Software Foundation (ASF) under one + * or more

[airflow] branch main updated (916dd96691 -> 058b6ebdd8)

2023-04-13 Thread eladkal
This is an automated email from the ASF dual-hosted git repository. eladkal pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git from 916dd96691 Better error message where non-compatible providers are not excluded (#30629) add 058b6ebdd8

[GitHub] [airflow] eladkal merged pull request #30631: Update dataform.rst

2023-04-13 Thread via GitHub
eladkal merged PR #30631: URL: https://github.com/apache/airflow/pull/30631 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [airflow] boring-cyborg[bot] commented on pull request #30631: Update dataform.rst

2023-04-13 Thread via GitHub
boring-cyborg[bot] commented on PR #30631: URL: https://github.com/apache/airflow/pull/30631#issuecomment-1507585363 Awesome work, congrats on your first merged pull request! You are invited to check our [Issue Tracker](https://github.com/apache/airflow/issues) for additional

[GitHub] [airflow] ferruzzi commented on pull request #30496: [OTel Integration] Add tagging to existing stats

2023-04-13 Thread via GitHub
ferruzzi commented on PR #30496: URL: https://github.com/apache/airflow/pull/30496#issuecomment-1507566256 Looks like it is all passing. Going to rebase from main, run it all again locally to make sure that doesn't break anything, then I think I've addressed any current concerns. --

[GitHub] [airflow] boring-cyborg[bot] commented on pull request #30631: Update dataform.rst

2023-04-13 Thread via GitHub
boring-cyborg[bot] commented on PR #30631: URL: https://github.com/apache/airflow/pull/30631#issuecomment-1507523347 Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our Contribution

[GitHub] [airflow] shanecglass opened a new pull request, #30631: Update dataform.rst

2023-04-13 Thread via GitHub
shanecglass opened a new pull request, #30631: URL: https://github.com/apache/airflow/pull/30631 Updating the links to the Dataform product documentation to fix 404 redirect error --- **^ Add meaningful description above** Read the **[Pull Request

[GitHub] [airflow] howardyoo commented on pull request #30496: [OTel Integration] Add tagging to existing stats

2023-04-13 Thread via GitHub
howardyoo commented on PR #30496: URL: https://github.com/apache/airflow/pull/30496#issuecomment-1507483775 Awesome, thank you!!Sent from my iPhoneOn Apr 13, 2023, at 2:05 PM, D. Ferruzzi ***@***.***> wrote: Alright, the last two commits should remove the email and run_id tags and

[GitHub] [airflow] eladkal commented on a diff in pull request #30541: Organize Amazon provider docs index

2023-04-13 Thread via GitHub
eladkal commented on code in PR #30541: URL: https://github.com/apache/airflow/pull/30541#discussion_r1165928428 ## docs/apache-airflow-providers-amazon/redirects.txt: ## @@ -15,8 +15,17 @@ # specific language governing permissions and limitations # under the License.

[GitHub] [airflow] ferruzzi commented on a diff in pull request #30541: Organize Amazon provider docs index

2023-04-13 Thread via GitHub
ferruzzi commented on code in PR #30541: URL: https://github.com/apache/airflow/pull/30541#discussion_r1165926734 ## docs/apache-airflow-providers-amazon/redirects.txt: ## @@ -15,8 +15,17 @@ # specific language governing permissions and limitations # under the License.

[GitHub] [airflow] ferruzzi commented on pull request #30496: [OTel Integration] Add tagging to existing stats

2023-04-13 Thread via GitHub
ferruzzi commented on PR #30496: URL: https://github.com/apache/airflow/pull/30496#issuecomment-1507478512 Alright, the last two commits should remove the email and run_id tags and update the unit tests accordingly. -- This is an automated message from the Apache Git Service. To respond

[GitHub] [airflow] eladkal commented on a diff in pull request #30541: Organize Amazon provider docs index

2023-04-13 Thread via GitHub
eladkal commented on code in PR #30541: URL: https://github.com/apache/airflow/pull/30541#discussion_r1165922307 ## docs/apache-airflow-providers-amazon/redirects.txt: ## @@ -15,8 +15,17 @@ # specific language governing permissions and limitations # under the License.

[GitHub] [airflow] eladkal closed issue #30607: Status of testing Providers that were prepared on April 12, 2023

2023-04-13 Thread via GitHub
eladkal closed issue #30607: Status of testing Providers that were prepared on April 12, 2023 URL: https://github.com/apache/airflow/issues/30607 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [airflow] eladkal commented on issue #30607: Status of testing Providers that were prepared on April 12, 2023

2023-04-13 Thread via GitHub
eladkal commented on issue #30607: URL: https://github.com/apache/airflow/issues/30607#issuecomment-1507472424 Thank you everyone. Providers are released I invite everyone to help improve providers for the next release, a list of open issues can be found

[airflow] annotated tag providers-google/9.0.0 updated (0f3b6579cb -> 8af0c1256b)

2023-04-13 Thread eladkal
This is an automated email from the ASF dual-hosted git repository. eladkal pushed a change to annotated tag providers-google/9.0.0 in repository https://gitbox.apache.org/repos/asf/airflow.git *** WARNING: tag providers-google/9.0.0 was modified! *** from 0f3b6579cb (commit) to

[airflow] annotated tag providers-microsoft-azure/5.3.1 updated (0f3b6579cb -> db71c9ca96)

2023-04-13 Thread eladkal
This is an automated email from the ASF dual-hosted git repository. eladkal pushed a change to annotated tag providers-microsoft-azure/5.3.1 in repository https://gitbox.apache.org/repos/asf/airflow.git *** WARNING: tag providers-microsoft-azure/5.3.1 was modified! *** from 0f3b6579cb

[GitHub] [airflow-site] eladkal merged pull request #764: Add documentation for packages - 2023-04-12

2023-04-13 Thread via GitHub
eladkal merged PR #764: URL: https://github.com/apache/airflow-site/pull/764 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [airflow] ferruzzi commented on pull request #30541: Organize Amazon provider docs index

2023-04-13 Thread via GitHub
ferruzzi commented on PR #30541: URL: https://github.com/apache/airflow/pull/30541#issuecomment-1507466846 Sorry, I accidentally clicked... something... that wasn't intentional. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

svn commit: r61234 - /dev/airflow/providers/ /release/airflow/providers/

2023-04-13 Thread eladkal
Author: eladkal Date: Thu Apr 13 18:52:48 2023 New Revision: 61234 Log: Release Airflow Providers on 2023-04-13 Added: release/airflow/providers/apache-airflow-providers-google-9.0.0.tar.gz - copied unchanged from r61233,

[GitHub] [airflow] eladkal commented on a diff in pull request #30541: Organize Amazon provider docs index

2023-04-13 Thread via GitHub
eladkal commented on code in PR #30541: URL: https://github.com/apache/airflow/pull/30541#discussion_r1165907974 ## docs/apache-airflow-providers-amazon/redirects.txt: ## @@ -15,8 +15,17 @@ # specific language governing permissions and limitations # under the License.

[GitHub] [airflow-site] potiuk commented on pull request #761: Update robots.txt to crawl only latest version of documentation

2023-04-13 Thread via GitHub
potiuk commented on PR #761: URL: https://github.com/apache/airflow-site/pull/761#issuecomment-1507447346 BTW., The `--dry-run` command shows copy-pasteable command that you can run manually to see what breeze does. - then you can apply any modification to those or try to mimic it

[GitHub] [airflow] jsnb-devoted closed issue #30630: Unable to clear or rerun tasks

2023-04-13 Thread via GitHub
jsnb-devoted closed issue #30630: Unable to clear or rerun tasks URL: https://github.com/apache/airflow/issues/30630 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

[GitHub] [airflow-site] potiuk commented on pull request #761: Update robots.txt to crawl only latest version of documentation

2023-04-13 Thread via GitHub
potiuk commented on PR #761: URL: https://github.com/apache/airflow-site/pull/761#issuecomment-1507443433 > Followed 1. and 2. only. The script docs/build.sh does not exist anymore in Airflow repository so I ran these commands instead to build the doc and visualize them: Look what

[airflow] branch constraints-main updated: Updating constraints. Github run id:4691994078

2023-04-13 Thread github-bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch constraints-main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/constraints-main by this push: new 56adcc3158 Updating

[GitHub] [airflow] ferruzzi commented on a diff in pull request #30496: [OTel Integration] Add tagging to existing stats

2023-04-13 Thread via GitHub
ferruzzi commented on code in PR #30496: URL: https://github.com/apache/airflow/pull/30496#discussion_r1165875681 ## airflow/dag_processing/processor.py: ## @@ -534,7 +535,9 @@ def manage_slas(cls, dag_folder, dag_id: str, session: Session = NEW_SESSION) ->

[GitHub] [airflow] jsnb-devoted opened a new issue, #30630: Unable to clear or rerun tasks

2023-04-13 Thread via GitHub
jsnb-devoted opened a new issue, #30630: URL: https://github.com/apache/airflow/issues/30630 ### Apache Airflow version Other Airflow 2 version (please specify below) ### What happened In the UI when I want to rerun a task I'll clear the task and downstream dependencies

[GitHub] [airflow] potiuk commented on a diff in pull request #30495: preload airflow imports before dag parsing to save time

2023-04-13 Thread via GitHub
potiuk commented on code in PR #30495: URL: https://github.com/apache/airflow/pull/30495#discussion_r1165868292 ## tests/dags/test_imports.nopy: ## @@ -0,0 +1,26 @@ +# using .nopy extension so that the file won't get reformatted +# because we want to assert that we can parse

[GitHub] [airflow] apilaskowski commented on a diff in pull request #30485: Split Celery logs into stdout/stderr

2023-04-13 Thread via GitHub
apilaskowski commented on code in PR #30485: URL: https://github.com/apache/airflow/pull/30485#discussion_r1165867929 ## airflow/cli/commands/celery_command.py: ## @@ -96,6 +100,27 @@ def _serve_logs(skip_serve_logs: bool = False): sub_proc.terminate()

[GitHub] [airflow] vandonr-amz commented on a diff in pull request #30495: preload airflow imports before dag parsing to save time

2023-04-13 Thread via GitHub
vandonr-amz commented on code in PR #30495: URL: https://github.com/apache/airflow/pull/30495#discussion_r1165866037 ## tests/dags/test_imports.nopy: ## @@ -0,0 +1,26 @@ +# using .nopy extension so that the file won't get reformatted +# because we want to assert that we can

[GitHub] [airflow] o-nikolas commented on a diff in pull request #30595: DynamoDBHook - waiter_path() to consider `resource_type` or `client_type`

2023-04-13 Thread via GitHub
o-nikolas commented on code in PR #30595: URL: https://github.com/apache/airflow/pull/30595#discussion_r1165858957 ## tests/providers/amazon/aws/hooks/test_dynamodb.py: ## @@ -55,3 +56,9 @@ def test_insert_batch_items_dynamodb_table(self):

[GitHub] [airflow] vandonr-amz commented on a diff in pull request #30495: preload airflow imports before dag parsing to save time

2023-04-13 Thread via GitHub
vandonr-amz commented on code in PR #30495: URL: https://github.com/apache/airflow/pull/30495#discussion_r1165857348 ## airflow/dag_processing/processor.py: ## @@ -187,6 +188,18 @@ def _handle_dag_file_processing(): def start(self) -> None: """Launch the process

[GitHub] [airflow] vandonr-amz commented on a diff in pull request #30495: preload airflow imports before dag parsing to save time

2023-04-13 Thread via GitHub
vandonr-amz commented on code in PR #30495: URL: https://github.com/apache/airflow/pull/30495#discussion_r1165856163 ## tests/dags/test_imports.nopy: ## @@ -0,0 +1,26 @@ +# using .nopy extension so that the file won't get reformatted +# because we want to assert that we can

[GitHub] [airflow] vandonr-amz commented on a diff in pull request #30612: Create a metric for task adoption time

2023-04-13 Thread via GitHub
vandonr-amz commented on code in PR #30612: URL: https://github.com/apache/airflow/pull/30612#discussion_r1165849905 ## airflow/models/taskinstance.py: ## @@ -1330,6 +1330,18 @@ def check_and_change_state_before_execution( if not test_mode:

[GitHub] [airflow] eladkal commented on pull request #30541: Organize Amazon provider docs index

2023-04-13 Thread via GitHub
eladkal commented on PR #30541: URL: https://github.com/apache/airflow/pull/30541#issuecomment-1507373391 > but I think we need to stick with the proper names so that there is brand recognition for our users. NP. will modify accordingly -- This is an automated message from the

[GitHub] [airflow] vandonr-amz commented on pull request #30612: Create a metric for task adoption time

2023-04-13 Thread via GitHub
vandonr-amz commented on PR #30612: URL: https://github.com/apache/airflow/pull/30612#issuecomment-1507372122 I need to check, but I think splitting this into 2 metrics might make the job a tad harder... I'm not really sure where the scheduled -> queued transition takes place, I need to

[GitHub] [airflow] eladkal commented on a diff in pull request #30380: Replace prints by logging

2023-04-13 Thread via GitHub
eladkal commented on code in PR #30380: URL: https://github.com/apache/airflow/pull/30380#discussion_r1165845484 ## airflow/cli/commands/task_command.py: ## @@ -450,13 +448,12 @@ def task_failed_deps(args): dep_context = DepContext(deps=SCHEDULER_QUEUED_DEPS)

[GitHub] [airflow] eladkal commented on pull request #30528: Remove email provider test module

2023-04-13 Thread via GitHub
eladkal commented on PR #30528: URL: https://github.com/apache/airflow/pull/30528#issuecomment-1507363968 closing this one in favor of the planned PR -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [airflow] eladkal closed pull request #30528: Remove email provider test module

2023-04-13 Thread via GitHub
eladkal closed pull request #30528: Remove email provider test module URL: https://github.com/apache/airflow/pull/30528 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

[GitHub] [airflow] potiuk commented on pull request #30612: Create a metric for task adoption time

2023-04-13 Thread via GitHub
potiuk commented on PR #30612: URL: https://github.com/apache/airflow/pull/30612#issuecomment-1507349206 > Absolutely! I've been involved with discussions recently around caching and now naming. Makes me think of [two things](https://martinfowler.com/bliki/TwoHardThings.html). I

[GitHub] [airflow] o-nikolas commented on pull request #30541: Organize Amazon provider docs index

2023-04-13 Thread via GitHub
o-nikolas commented on PR #30541: URL: https://github.com/apache/airflow/pull/30541#issuecomment-1507330562 > > Need to come up with convention for prefix Amazon / AWS / Amazon AWS ? > > As I mentioned in one of the comments above, I think we should call it what it is. Some services

[GitHub] [airflow] ferruzzi commented on pull request #30541: Organize Amazon provider docs index

2023-04-13 Thread via GitHub
ferruzzi commented on PR #30541: URL: https://github.com/apache/airflow/pull/30541#issuecomment-1507324542 > MWAA is not an official name :) Nope, it's not. Just the unofficial abbreviation because who wants to say/type that whole thing out every time :P -- This is an

[airflow] branch main updated (312f7e873d -> 916dd96691)

2023-04-13 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git from 312f7e873d Remove mysql-connector-python (#30487) add 916dd96691 Better error message where non-compatible

[GitHub] [airflow] potiuk merged pull request #30629: Better error message where non-compatible providers are not excluded

2023-04-13 Thread via GitHub
potiuk merged PR #30629: URL: https://github.com/apache/airflow/pull/30629 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [airflow] eladkal commented on pull request #30541: Organize Amazon provider docs index

2023-04-13 Thread via GitHub
eladkal commented on PR #30541: URL: https://github.com/apache/airflow/pull/30541#issuecomment-1507308089 @ferruzzi according to that link MWAA is not an official name :) IMHO it's very very annoying to go through a non sorted list so I try to minimize it as much as I can but I don't

[GitHub] [airflow] ferruzzi commented on pull request #30541: Organize Amazon provider docs index

2023-04-13 Thread via GitHub
ferruzzi commented on PR #30541: URL: https://github.com/apache/airflow/pull/30541#issuecomment-1507302646 > Need to come up with convention for prefix Amazon / AWS / Amazon AWS ? As I mentioned in one of the comments above, I think we should call it what it is. Some services are

[airflow] branch constraints-main updated: Updating constraints. Github run id:4691362347

2023-04-13 Thread github-bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch constraints-main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/constraints-main by this push: new b52cc07846 Updating

[GitHub] [airflow] jedcunningham commented on pull request #30612: Create a metric for task adoption time

2023-04-13 Thread via GitHub
jedcunningham commented on PR #30612: URL: https://github.com/apache/airflow/pull/30612#issuecomment-1507292942 Absolutely! I've been involved with discussions recently around caching and now naming. Makes me think of [two things](https://martinfowler.com/bliki/TwoHardThings.html).

[GitHub] [airflow] ferruzzi commented on a diff in pull request #30541: Organize Amazon provider docs index

2023-04-13 Thread via GitHub
ferruzzi commented on code in PR #30541: URL: https://github.com/apache/airflow/pull/30541#discussion_r1165797230 ## docs/apache-airflow-providers-amazon/operators/glacier.rst: ## @@ -15,17 +15,17 @@ specific language governing permissions and limitations under the

[GitHub] [airflow] ferruzzi commented on pull request #30612: Create a metric for task adoption time

2023-04-13 Thread via GitHub
ferruzzi commented on PR #30612: URL: https://github.com/apache/airflow/pull/30612#issuecomment-1507279154 > However, that's not "adoption" time. Adoption refers to tasks a scheduler adopts from a no-longer-running scheduler. Sometimes it feels like naming things is the hardest part

[GitHub] [airflow] potiuk opened a new pull request, #30629: Better error message where non-compatible providers are not excluded

2023-04-13 Thread via GitHub
potiuk opened a new pull request, #30629: URL: https://github.com/apache/airflow/pull/30629 When compatibility check is performed for old version of Airflow, we attempt to install all providers for the old version. However if one of the providers has >= limit on Airflow for newer version

  1   2   >