Re: [PR] Add Open Search Provider [airflow]

2023-10-12 Thread via GitHub
cjames23 commented on PR #34705: URL: https://github.com/apache/airflow/pull/34705#issuecomment-1760864875 > Yeah, I saw that, but the only changes in the Amazon package were some whitespace and punctuation in a docstring. Better done separate, but I didn't think it was worth blocking over

Re: [PR] Add Open Search Provider [airflow]

2023-10-12 Thread via GitHub
cjames23 commented on PR #34705: URL: https://github.com/apache/airflow/pull/34705#issuecomment-1760851926 @eladkal any advice on the failure that is happening with the CI image build step and the static checks? I ran breeze static checks against all files locally and do not receive the

[I] Clear old Triggers when Triggerer is not running [airflow]

2023-10-12 Thread via GitHub
syedahsn opened a new issue, #34906: URL: https://github.com/apache/airflow/issues/34906 ### Apache Airflow version main (development) ### What happened When a deferrable operator or sensor is run without a Triggerer process, the task gets stuck in the deferred state,

Re: [PR] Adding property files option in the Spark Submit command [airflow]

2023-10-12 Thread via GitHub
ghostp13409 commented on code in PR #34894: URL: https://github.com/apache/airflow/pull/34894#discussion_r1357751380 ## airflow/providers/apache/spark/operators/spark_submit.py: ## @@ -160,6 +160,11 @@ def on_kill(self) -> None: self._hook = self._get_hook()

Re: [PR] Introduce a generic export for containerized executor logging [airflow]

2023-10-12 Thread via GitHub
uranusjr commented on code in PR #34903: URL: https://github.com/apache/airflow/pull/34903#discussion_r1357747434 ## airflow/cli/commands/task_command.py: ## @@ -326,7 +326,7 @@ def _move_task_handlers_to_root(ti: TaskInstance) -> Generator[None, None, None]

Re: [PR] Update documentation to enable test connection [airflow]

2023-10-12 Thread via GitHub
uranusjr commented on code in PR #34905: URL: https://github.com/apache/airflow/pull/34905#discussion_r1357745431 ## airflow/www/static/js/types/api-generated.ts: ## Review Comment: This file is generated from `openapi/v1.yaml` and you should not modify it correctly.

Re: [PR] Feature: Allow description to be passed in when using variables CLI [airflow]

2023-10-12 Thread via GitHub
uranusjr commented on code in PR #34791: URL: https://github.com/apache/airflow/pull/34791#discussion_r1357733068 ## tests/cli/commands/test_variable_command.py: ## @@ -49,6 +49,15 @@ def test_variables_set(self): with pytest.raises(KeyError):

Re: [PR] Update documentation to enable test connection [airflow]

2023-10-12 Thread via GitHub
boring-cyborg[bot] commented on PR #34905: URL: https://github.com/apache/airflow/pull/34905#issuecomment-1760719883 Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our Contribution

[PR] Update documentation to enable test connection [airflow]

2023-10-12 Thread via GitHub
vatsrahul1001 opened a new pull request, #34905: URL: https://github.com/apache/airflow/pull/34905 with [PR](https://github.com/apache/airflow/pull/32052) now testing connections is disabled by default, few docs were not updated with that information this PR is to update docs **^

Re: [PR] Add description to pools view [airflow]

2023-10-12 Thread via GitHub
uranusjr commented on PR #34862: URL: https://github.com/apache/airflow/pull/34862#issuecomment-1760710502 No, I’m thinking if it’d be posible to apply `substr` on the column when the rows are selected from the database. That’s where the long strings get loaded. When you reach the

Re: [PR] Add description to pools view [airflow]

2023-10-12 Thread via GitHub
vchiapaikeo commented on PR #34862: URL: https://github.com/apache/airflow/pull/34862#issuecomment-1760648504 Ah yes, `VariableModelView` uses a custom FAB Model [list_template](https://flask-appbuilder.readthedocs.io/en/latest/templates.html#widgets) and injects additional styling there.

Re: [PR] Add description to pools view [airflow]

2023-10-12 Thread via GitHub
vchiapaikeo commented on code in PR #34862: URL: https://github.com/apache/airflow/pull/34862#discussion_r1357641912 ## airflow/www/static/css/bootstrap-theme.css: ## @@ -5945,6 +5945,12 @@ button.close { .affix { position: fixed; } +.truncate-text { Review Comment:

Re: [PR] Feature: Allow description to be passed in when using variables CLI [airflow]

2023-10-12 Thread via GitHub
Zhenye-Na commented on code in PR #34791: URL: https://github.com/apache/airflow/pull/34791#discussion_r1357598523 ## airflow/cli/cli_config.py: ## @@ -543,6 +543,15 @@ def string_lower_type(val): ARG_DEFAULT = Arg( ("-d", "--default"), metavar="VAL", default=None,

Re: [PR] Add Open Search Provider [airflow]

2023-10-12 Thread via GitHub
cjames23 commented on code in PR #34705: URL: https://github.com/apache/airflow/pull/34705#discussion_r1357592192 ## airflow/providers/opensearch/hooks/opensearch.py: ## @@ -0,0 +1,122 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor

Re: [I] Missing log file when there's more than one attempt [airflow]

2023-10-12 Thread via GitHub
github-actions[bot] closed issue #34393: Missing log file when there's more than one attempt URL: https://github.com/apache/airflow/issues/34393 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

Re: [PR] Try fixing non direct upstream -- separate dep approach [airflow]

2023-10-12 Thread via GitHub
github-actions[bot] commented on PR #33820: URL: https://github.com/apache/airflow/pull/33820#issuecomment-1760563569 This pull request has been automatically marked as stale because it has not had recent activity. It will be closed in 5 days if no further activity occurs. Thank you for

Re: [PR] Re-implement client codegen with Python [airflow]

2023-10-12 Thread via GitHub
github-actions[bot] closed pull request #33365: Re-implement client codegen with Python URL: https://github.com/apache/airflow/pull/33365 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

Re: [I] Missing log file when there's more than one attempt [airflow]

2023-10-12 Thread via GitHub
github-actions[bot] commented on issue #34393: URL: https://github.com/apache/airflow/issues/34393#issuecomment-1760563530 This issue has been closed because it has not received response from the issue author. -- This is an automated message from the Apache Git Service. To respond to the

Re: [PR] Feature: Allow description to be passed in when using variables CLI [airflow]

2023-10-12 Thread via GitHub
Zhenye-Na commented on code in PR #34791: URL: https://github.com/apache/airflow/pull/34791#discussion_r1357583319 ## airflow/cli/commands/variable_command.py: ## @@ -61,8 +61,8 @@ def variables_get(args): @cli_utils.action_cli @providers_configuration_loaded def

[PR] Fixes copy bug [airflow]

2023-10-12 Thread via GitHub
Adaverse opened a new pull request, #34904: URL: https://github.com/apache/airflow/pull/34904 Fixes: #34846 Sets the value that needs to be copied whenever it changes. The previous implementation had a bug since the component will only render once with the page.

Re: [PR] Introduce a generic export for containerized executor logging [airflow]

2023-10-12 Thread via GitHub
ferruzzi commented on code in PR #34903: URL: https://github.com/apache/airflow/pull/34903#discussion_r1357507527 ## airflow/cli/commands/task_command.py: ## @@ -326,7 +326,7 @@ def _move_task_handlers_to_root(ti: TaskInstance) -> Generator[None, None, None]

Re: [PR] Use base classes for AWS Lambda Operators/Sensors [airflow]

2023-10-12 Thread via GitHub
Taragolis commented on code in PR #34890: URL: https://github.com/apache/airflow/pull/34890#discussion_r1357500864 ## airflow/providers/amazon/aws/operators/lambda_function.py: ## @@ -62,13 +61,15 @@ class LambdaCreateFunctionOperator(BaseOperator): :param aws_conn_id: The

Re: [I] Task ID copy button copying wrong ID [airflow]

2023-10-12 Thread via GitHub
Adaverse commented on issue #34846: URL: https://github.com/apache/airflow/issues/34846#issuecomment-1760449991 Nice catch. The problem is that the component responsible for copying won't re-render (which is the expectation looking at the current code). Hence we need to explicitly set the

Re: [PR] Use base classes for AWS Lambda Operators/Sensors [airflow]

2023-10-12 Thread via GitHub
o-nikolas commented on code in PR #34890: URL: https://github.com/apache/airflow/pull/34890#discussion_r1357444134 ## airflow/providers/amazon/aws/operators/lambda_function.py: ## @@ -62,13 +61,15 @@ class LambdaCreateFunctionOperator(BaseOperator): :param aws_conn_id: The

Re: [PR] Update dags.rst [airflow]

2023-10-12 Thread via GitHub
url54 commented on code in PR #34898: URL: https://github.com/apache/airflow/pull/34898#discussion_r1357451634 ## docs/apache-airflow/core-concepts/dags.rst: ## @@ -514,6 +514,12 @@ For example, here is a DAG that uses a ``for`` loop to define some tasks: In general, we

Re: [PR] Update dags.rst [airflow]

2023-10-12 Thread via GitHub
url54 commented on code in PR #34898: URL: https://github.com/apache/airflow/pull/34898#discussion_r1357438953 ## docs/apache-airflow/core-concepts/dags.rst: ## @@ -514,6 +514,12 @@ For example, here is a DAG that uses a ``for`` loop to define some tasks: In general, we

[PR] Introduce a generic export for containerized executor logging [airflow]

2023-10-12 Thread via GitHub
o-nikolas opened a new pull request, #34903: URL: https://github.com/apache/airflow/pull/34903 ### Context Pulling this change out of #34381 into a separate PR as requested [here](https://github.com/apache/airflow/pull/34381#discussion_r1327980137) by @eladkal ### Overview A

Re: [PR] Glue DataBrew operator [airflow]

2023-10-12 Thread via GitHub
vincbeck commented on code in PR #34807: URL: https://github.com/apache/airflow/pull/34807#discussion_r1357419745 ## airflow/providers/amazon/aws/hooks/databrew.py: ## @@ -0,0 +1,68 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor

Re: [PR] Glue DataBrew operator [airflow]

2023-10-12 Thread via GitHub
ellisms commented on code in PR #34807: URL: https://github.com/apache/airflow/pull/34807#discussion_r1357406099 ## airflow/providers/amazon/aws/hooks/databrew.py: ## @@ -0,0 +1,68 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor

Re: [I] Support multi-select state filtering on grid view [airflow]

2023-10-12 Thread via GitHub
Adaverse commented on issue #34795: URL: https://github.com/apache/airflow/issues/34795#issuecomment-1760332757 Yup, it would be nice to have. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

Re: [PR] Fix hardcoded container name in remote logging option for Azure Blob Storage [airflow]

2023-10-12 Thread via GitHub
Adaverse commented on PR #32779: URL: https://github.com/apache/airflow/pull/32779#issuecomment-1760323534 > @Adaverse what is the status of this PR? Is it ready for review after @potiuk comments? Yes, the changes are ready for the review. ++@potiuk -- This is an

Re: [PR] Glue DataBrew operator [airflow]

2023-10-12 Thread via GitHub
eladkal commented on code in PR #34807: URL: https://github.com/apache/airflow/pull/34807#discussion_r1357381623 ## airflow/providers/amazon/aws/hooks/databrew.py: ## @@ -0,0 +1,68 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor

Re: [PR] Update dags.rst [airflow]

2023-10-12 Thread via GitHub
eladkal commented on code in PR #34898: URL: https://github.com/apache/airflow/pull/34898#discussion_r1357305417 ## docs/apache-airflow/core-concepts/dags.rst: ## @@ -514,6 +514,12 @@ For example, here is a DAG that uses a ``for`` loop to define some tasks: In general, we

Re: [PR] Update dags.rst [airflow]

2023-10-12 Thread via GitHub
hussein-awala commented on code in PR #34898: URL: https://github.com/apache/airflow/pull/34898#discussion_r1357371674 ## docs/apache-airflow/core-concepts/dags.rst: ## @@ -514,6 +514,12 @@ For example, here is a DAG that uses a ``for`` loop to define some tasks: In

Re: [I] Add AthenaSqlHook based on PyAthena [airflow]

2023-10-12 Thread via GitHub
treyyi commented on issue #34823: URL: https://github.com/apache/airflow/issues/34823#issuecomment-1760285593 Hi @eladkal, I would like to take a look and work on this new `AthenaSqlHook` implementation. Could you assign this task to me? Thanks! -- This is an automated

[PR] Just testing if it builds [airflow]

2023-10-12 Thread via GitHub
potiuk opened a new pull request, #34902: URL: https://github.com/apache/airflow/pull/34902 --- **^ Add meaningful description above** Read the **[Pull Request

[airflow] branch main updated: Inline restoring target workflow scripts in the build-images workflow (#34901)

2023-10-12 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 1f63199351 Inline restoring target workflow

[airflow] branch inline-workflow-actions deleted (was aa05a5b097)

2023-10-12 Thread jedcunningham
This is an automated email from the ASF dual-hosted git repository. jedcunningham pushed a change to branch inline-workflow-actions in repository https://gitbox.apache.org/repos/asf/airflow.git was aa05a5b097 Inline restoring target workflow scripts in the build-images workflow The

Re: [PR] Inline restoring target workflow scripts in the build-images workflow [airflow]

2023-10-12 Thread via GitHub
potiuk merged PR #34901: URL: https://github.com/apache/airflow/pull/34901 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [I] Logs using @task.virtualenv in Kubernetes [airflow]

2023-10-12 Thread via GitHub
jens-scheffler-bosch commented on issue #34865: URL: https://github.com/apache/airflow/issues/34865#issuecomment-1760275108 > No, I'm not. :/ Feel free to provide more notes or re-open if you re-tested with a different setup. -- This is an automated message from the Apache Git

Re: [I] Logs using @task.virtualenv in Kubernetes [airflow]

2023-10-12 Thread via GitHub
jens-scheffler-bosch closed issue #34865: Logs using @task.virtualenv in Kubernetes URL: https://github.com/apache/airflow/issues/34865 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

Re: [I] Logs using @task.virtualenv in Kubernetes [airflow]

2023-10-12 Thread via GitHub
jens-scheffler-bosch commented on issue #34865: URL: https://github.com/apache/airflow/issues/34865#issuecomment-1760273902 @leoneuwald Okay, I did a regression in my local Airflow test env with version 2.7.2 in Celery as well as deployed your example DAG and used the standard

[airflow] branch inline-workflow-actions created (now aa05a5b097)

2023-10-12 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a change to branch inline-workflow-actions in repository https://gitbox.apache.org/repos/asf/airflow.git at aa05a5b097 Inline restoring target workflow scripts in the build-images workflow No new revisions

[PR] Inline restoring target workflow scripts in the build-images workflow [airflow]

2023-10-12 Thread via GitHub
potiuk opened a new pull request, #34901: URL: https://github.com/apache/airflow/pull/34901 --- **^ Add meaningful description above** Read the **[Pull Request

Re: [PR] Inline restoring target workflow scripts in the build-images workflow [airflow]

2023-10-12 Thread via GitHub
potiuk closed pull request #34899: Inline restoring target workflow scripts in the build-images workflow URL: https://github.com/apache/airflow/pull/34899 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[PR] Inline restoring target workflow scripts in the build-images workflow [airflow]

2023-10-12 Thread via GitHub
potiuk opened a new pull request, #34899: URL: https://github.com/apache/airflow/pull/34899 --- **^ Add meaningful description above** Read the **[Pull Request

Re: [PR] Adding payload arg as a templated field for Synapse SparkBatchOperator [airflow]

2023-10-12 Thread via GitHub
Taragolis commented on code in PR #34896: URL: https://github.com/apache/airflow/pull/34896#discussion_r1357314685 ## airflow/providers/microsoft/azure/operators/synapse.py: ## @@ -49,6 +49,7 @@ class AzureSynapseRunSparkBatchOperator(BaseOperator): template_fields:

Re: [PR] Update dags.rst [airflow]

2023-10-12 Thread via GitHub
eladkal commented on code in PR #34898: URL: https://github.com/apache/airflow/pull/34898#discussion_r1357305417 ## docs/apache-airflow/core-concepts/dags.rst: ## @@ -514,6 +514,12 @@ For example, here is a DAG that uses a ``for`` loop to define some tasks: In general, we

Re: [PR] Update dags.rst [airflow]

2023-10-12 Thread via GitHub
eladkal commented on code in PR #34898: URL: https://github.com/apache/airflow/pull/34898#discussion_r1357305417 ## docs/apache-airflow/core-concepts/dags.rst: ## @@ -514,6 +514,12 @@ For example, here is a DAG that uses a ``for`` loop to define some tasks: In general, we

Re: [PR] Update dags.rst [airflow]

2023-10-12 Thread via GitHub
eladkal commented on PR #34898: URL: https://github.com/apache/airflow/pull/34898#issuecomment-1760247545 > Hello, first time ever doing a pull request, so I apologize if I firstly am not doing it correctly. Welcome! Contributions are always appreciated! -- This is an automated

Re: [PR] Update dags.rst [airflow]

2023-10-12 Thread via GitHub
eladkal commented on code in PR #34898: URL: https://github.com/apache/airflow/pull/34898#discussion_r1357305417 ## docs/apache-airflow/core-concepts/dags.rst: ## @@ -514,6 +514,12 @@ For example, here is a DAG that uses a ``for`` loop to define some tasks: In general, we

Re: [I] Expose description columns of Pools in the UI [airflow]

2023-10-12 Thread via GitHub
Zhenye-Na commented on issue #34751: URL: https://github.com/apache/airflow/issues/34751#issuecomment-1760242841 @vchiapaikeo Ack, I was working on the other issue so let me prioritize this one as well and publish the fix for a quick review -- This is an automated message from

Re: [PR] Feature: Allow description to be passed in when using variables CLI [airflow]

2023-10-12 Thread via GitHub
Zhenye-Na commented on code in PR #34791: URL: https://github.com/apache/airflow/pull/34791#discussion_r1357301454 ## airflow/cli/commands/variable_command.py: ## @@ -61,8 +61,8 @@ def variables_get(args): @cli_utils.action_cli @providers_configuration_loaded def

Re: [PR] Feature: Allow description to be passed in when using variables CLI [airflow]

2023-10-12 Thread via GitHub
Zhenye-Na commented on code in PR #34791: URL: https://github.com/apache/airflow/pull/34791#discussion_r1357300825 ## airflow/cli/cli_config.py: ## @@ -543,6 +543,15 @@ def string_lower_type(val): ARG_DEFAULT = Arg( ("-d", "--default"), metavar="VAL", default=None,

[airflow] branch main updated: Extend hooks arguments into `AwsBaseWaiterTrigger` (#34884)

2023-10-12 Thread taragolis
This is an automated email from the ASF dual-hosted git repository. taragolis pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 545e4d505e Extend hooks arguments into

Re: [PR] Extend hooks arguments into `AwsBaseWaiterTrigger` [airflow]

2023-10-12 Thread via GitHub
Taragolis merged PR #34884: URL: https://github.com/apache/airflow/pull/34884 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[PR] Update dags.rst [airflow]

2023-10-12 Thread via GitHub
url54 opened a new pull request, #34898: URL: https://github.com/apache/airflow/pull/34898 Hello, first time ever doing a pull request, so I apologize if I firstly am not doing it correctly. I work for a cloud provider, and one of our services is a managed Airflow instance. I have

Re: [PR] Update dags.rst [airflow]

2023-10-12 Thread via GitHub
boring-cyborg[bot] commented on PR #34898: URL: https://github.com/apache/airflow/pull/34898#issuecomment-1760181730 Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our Contribution

Re: [I] Incorrect description on the implementation of parametrized timetables [airflow]

2023-10-12 Thread via GitHub
boring-cyborg[bot] commented on issue #34897: URL: https://github.com/apache/airflow/issues/34897#issuecomment-1760109885 Thanks for opening your first issue here! Be sure to follow the issue template! If you are willing to raise PR to address this issue please do so, no need to wait for

[I] Incorrect description on the implementation of parametrized timetables [airflow]

2023-10-12 Thread via GitHub
MemphisMeng opened a new issue, #34897: URL: https://github.com/apache/airflow/issues/34897 ### What do you see as an issue? I was learning how to customize a timetable by following the example on the

Re: [PR] Adding payload arg as a templated field for Synapse SparkBatchOperator [airflow]

2023-10-12 Thread via GitHub
boring-cyborg[bot] commented on PR #34896: URL: https://github.com/apache/airflow/pull/34896#issuecomment-1760091789 Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our Contribution

[PR] Adding payload arg as a templated field for Synapse SparkBatchOperator [airflow]

2023-10-12 Thread via GitHub
narayave opened a new pull request, #34896: URL: https://github.com/apache/airflow/pull/34896 This change adds the "payload" argument as a templated field. This helps to send airflow macros as part of the payload and have them render before the rest api call is made.

Re: [PR] Use auth manager `is_authorized_` APIs to check user permissions in Rest API [airflow]

2023-10-12 Thread via GitHub
vincbeck commented on code in PR #34317: URL: https://github.com/apache/airflow/pull/34317#discussion_r1357189276 ## airflow/api_connexion/security.py: ## @@ -48,10 +61,194 @@ def requires_access_decorator(func: T): @wraps(func) def decorated(*args, **kwargs):

Re: [PR] Use auth manager `is_authorized_` APIs to check user permissions in Rest API [airflow]

2023-10-12 Thread via GitHub
vincbeck commented on code in PR #34317: URL: https://github.com/apache/airflow/pull/34317#discussion_r1357188838 ## airflow/api_connexion/endpoints/task_instance_endpoint.py: ## @@ -527,13 +497,8 @@ def post_clear_task_instances(*, dag_id: str, session: Session = NEW_SESSION)

Re: [PR] Use auth manager `is_authorized_` APIs to check user permissions in Rest API [airflow]

2023-10-12 Thread via GitHub
vincbeck commented on code in PR #34317: URL: https://github.com/apache/airflow/pull/34317#discussion_r1357188653 ## airflow/api_connexion/endpoints/task_instance_endpoint.py: ## @@ -461,13 +436,8 @@ def get_task_instances_batch(session: Session = NEW_SESSION) -> APIResponse:

Re: [PR] Chart: Update default airflow version to 2.7.2 [airflow]

2023-10-12 Thread via GitHub
ephraimbuddy merged PR #34886: URL: https://github.com/apache/airflow/pull/34886 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [PR] Use auth manager `is_authorized_` APIs to check user permissions in Rest API [airflow]

2023-10-12 Thread via GitHub
vincbeck commented on code in PR #34317: URL: https://github.com/apache/airflow/pull/34317#discussion_r1357188155 ## airflow/api_connexion/endpoints/task_instance_endpoint.py: ## @@ -61,13 +61,8 @@ T = TypeVar("T") -@security.requires_access( -[ -

[airflow] branch main updated: Chart: Update default airflow version to 2.7.2 (#34886)

2023-10-12 Thread ephraimanierobi
This is an automated email from the ASF dual-hosted git repository. ephraimanierobi pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 8e06897d1d Chart: Update default airflow

Re: [PR] AWS ECS Executor [airflow]

2023-10-12 Thread via GitHub
o-nikolas commented on code in PR #34381: URL: https://github.com/apache/airflow/pull/34381#discussion_r1357183739 ## airflow/utils/log/logging_mixin.py: ## @@ -209,9 +209,9 @@ def __init__(self, stream): @property def stream(self): """Returns current

Re: [PR] Use auth manager `is_authorized_` APIs to check user permissions in Rest API [airflow]

2023-10-12 Thread via GitHub
vincbeck commented on code in PR #34317: URL: https://github.com/apache/airflow/pull/34317#discussion_r1357158265 ## airflow/www/security_manager.py: ## @@ -269,125 +272,47 @@ def get_user_roles(user=None): user = g.user return user.roles -def

Re: [PR] Use auth manager `is_authorized_` APIs to check user permissions in Rest API [airflow]

2023-10-12 Thread via GitHub
vincbeck commented on code in PR #34317: URL: https://github.com/apache/airflow/pull/34317#discussion_r1357158265 ## airflow/www/security_manager.py: ## @@ -269,125 +272,47 @@ def get_user_roles(user=None): user = g.user return user.roles -def

Re: [PR] Use auth manager `is_authorized_` APIs to check user permissions in Rest API [airflow]

2023-10-12 Thread via GitHub
vincbeck commented on code in PR #34317: URL: https://github.com/apache/airflow/pull/34317#discussion_r1357150789 ## airflow/api_connexion/security.py: ## @@ -48,10 +61,194 @@ def requires_access_decorator(func: T): @wraps(func) def decorated(*args, **kwargs):

Re: [PR] Use auth manager `is_authorized_` APIs to check user permissions in Rest API [airflow]

2023-10-12 Thread via GitHub
vincbeck commented on code in PR #34317: URL: https://github.com/apache/airflow/pull/34317#discussion_r1357126074 ## airflow/api_connexion/security.py: ## @@ -48,10 +61,194 @@ def requires_access_decorator(func: T): @wraps(func) def decorated(*args, **kwargs):

[airflow] branch main updated: Fix GCSToGoogleDriveOperator and gdrive system tests (#34545)

2023-10-12 Thread pankaj
This is an automated email from the ASF dual-hosted git repository. pankaj pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 4dc2c40dfe Fix GCSToGoogleDriveOperator and gdrive

Re: [PR] Fix GCSToGoogleDriveOperator and gdrive system tests [airflow]

2023-10-12 Thread via GitHub
pankajastro merged PR #34545: URL: https://github.com/apache/airflow/pull/34545 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [I] Invalid stat name using opentelemetry [airflow]

2023-10-12 Thread via GitHub
ferruzzi commented on issue #34845: URL: https://github.com/apache/airflow/issues/34845#issuecomment-1759996079 2.7.2 was released this morning! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

Re: [I] Logs using @task.virtualenv in Kubernetes [airflow]

2023-10-12 Thread via GitHub
leoneuwald commented on issue #34865: URL: https://github.com/apache/airflow/issues/34865#issuecomment-1759990139 > Just to double-check, are you able to make a regression on Airflow 2.7.1? No, I'm not. :/ -- This is an automated message from the Apache Git Service. To respond to

Re: [PR] Add `DatabricksJobsCreateOperator` [airflow]

2023-10-12 Thread via GitHub
pankajastro commented on PR #32221: URL: https://github.com/apache/airflow/pull/32221#issuecomment-1759989756 Wondering why the job `Build Images / Build CI images ` is getting cancelled and looks like it persisting on this PR. -- This is an automated message from the Apache Git

Re: [PR] Enable pgvector support for Postgres provider [airflow]

2023-10-12 Thread via GitHub
sunank200 commented on code in PR #34891: URL: https://github.com/apache/airflow/pull/34891#discussion_r1357077212 ## airflow/providers/postgres/hooks/postgres.py: ## @@ -320,6 +320,29 @@ def _generate_insert_sql( return sql +def ingest_embedding( +

Re: [PR] Adding property files option in the Spark Submit command [airflow]

2023-10-12 Thread via GitHub
Bowrna commented on PR #34894: URL: https://github.com/apache/airflow/pull/34894#issuecomment-1759958599 > > can this be tested with unit test? > > Hi @eladkal, > > As I mentioned above, I am a newcomer to open source and a also the development as a whole. So I haven't done

Re: [PR] Adding property files option in the Spark Submit command [airflow]

2023-10-12 Thread via GitHub
Bowrna commented on code in PR #34894: URL: https://github.com/apache/airflow/pull/34894#discussion_r1357074531 ## airflow/providers/apache/spark/operators/spark_submit.py: ## @@ -160,6 +160,11 @@ def on_kill(self) -> None: self._hook = self._get_hook()

Re: [I] Scheduler crashes with OverflowError: date value out of range when retries are added to ShortCircuitOperator [airflow]

2023-10-12 Thread via GitHub
Taragolis commented on issue #34869: URL: https://github.com/apache/airflow/issues/34869#issuecomment-1759951396 I can't reproduce, this sample retries without any issues ```python from datetime import timedelta from airflow.decorators import task from airflow.models.dag

Re: [PR] Let auth managers provide their own API endpoints [airflow]

2023-10-12 Thread via GitHub
vincbeck commented on code in PR #34349: URL: https://github.com/apache/airflow/pull/34349#discussion_r1357065387 ## airflow/www/extensions/init_views.py: ## @@ -308,5 +318,16 @@ def init_api_experimental(app): "The authenticated user has full access.",

Re: [PR] Enhancement: Added Code from Image as Text [airflow]

2023-10-12 Thread via GitHub
theaadya commented on code in PR #34868: URL: https://github.com/apache/airflow/pull/34868#discussion_r1357062430 ## CONTRIBUTORS_QUICK_START_VSCODE.rst: ## @@ -92,6 +92,12 @@ Setting up debugging - Change ``"program"`` to point to an example dag and add ``"env"`` and

Re: [I] Scheduler is spending most of its time in clear_not_launched_queued_tasks function [airflow]

2023-10-12 Thread via GitHub
ROVAN1220 commented on issue #34877: URL: https://github.com/apache/airflow/issues/34877#issuecomment-1759924519 It seems like you've identified a performance bottleneck in your Airflow setup when running on a large Kubernetes cluster with a high number of queued tasks. Your proposed

Re: [PR] Use auth manager `is_authorized_` APIs to check user permissions in Rest API [airflow]

2023-10-12 Thread via GitHub
vincbeck commented on code in PR #34317: URL: https://github.com/apache/airflow/pull/34317#discussion_r1357045707 ## airflow/api_connexion/endpoints/task_instance_endpoint.py: ## @@ -61,13 +61,8 @@ T = TypeVar("T") -@security.requires_access( -[ -

Re: [PR] Adding property files option in the Spark Submit command [airflow]

2023-10-12 Thread via GitHub
ghostp13409 commented on PR #34894: URL: https://github.com/apache/airflow/pull/34894#issuecomment-1759904842 > can this be tested with unit test? Hi @eladkal, As I mentioned above, I am a newcomer to open source and a also the development as a whole. So I haven't done any

Re: [PR] Glue DataBrew operator [airflow]

2023-10-12 Thread via GitHub
ellisms commented on PR #34807: URL: https://github.com/apache/airflow/pull/34807#issuecomment-1759901155 > Static check failures and unit tests. Could you please take a look? I don't think all changes were committed/pushed. Just rebased and pushed all changes. Local tests passed.

Re: [PR] Extend hooks arguments into `AwsBaseWaiterTrigger` [airflow]

2023-10-12 Thread via GitHub
Taragolis commented on code in PR #34884: URL: https://github.com/apache/airflow/pull/34884#discussion_r1357028144 ## airflow/providers/amazon/aws/triggers/base.py: ## @@ -72,6 +76,8 @@ def __init__( waiter_max_attempts: int, aws_conn_id: str | None,

Re: [PR] Added "overrides" parameter for CloudRunExecuteJobOperator [airflow]

2023-10-12 Thread via GitHub
ChloeSheasby commented on PR #34874: URL: https://github.com/apache/airflow/pull/34874#issuecomment-1759883361 Looks like google-cloud-run pypi (repo is called googleapis/python-run) doesn't even have the overrides variable available in their API yet. I made an issue for adding this

Re: [PR] Extend hooks arguments into `AwsBaseWaiterTrigger` [airflow]

2023-10-12 Thread via GitHub
Taragolis commented on code in PR #34884: URL: https://github.com/apache/airflow/pull/34884#discussion_r1357014301 ## airflow/providers/amazon/aws/triggers/base.py: ## @@ -72,6 +76,8 @@ def __init__( waiter_max_attempts: int, aws_conn_id: str | None,

Re: [PR] Extend hooks arguments into `AwsBaseWaiterTrigger` [airflow]

2023-10-12 Thread via GitHub
uranusjr commented on code in PR #34884: URL: https://github.com/apache/airflow/pull/34884#discussion_r1357002772 ## airflow/providers/amazon/aws/triggers/base.py: ## @@ -72,6 +76,8 @@ def __init__( waiter_max_attempts: int, aws_conn_id: str | None,

Re: [PR] Use auth manager `is_authorized_` APIs to check user permissions in Rest API [airflow]

2023-10-12 Thread via GitHub
vincbeck commented on code in PR #34317: URL: https://github.com/apache/airflow/pull/34317#discussion_r1356999376 ## airflow/www/extensions/init_jinja_globals.py: ## @@ -69,10 +70,13 @@ def prepare_jinja_globals(): "git_version": git_version,

Re: [PR] Adding property files option in the Spark Submit command [airflow]

2023-10-12 Thread via GitHub
eladkal commented on PR #34894: URL: https://github.com/apache/airflow/pull/34894#issuecomment-1759809180 can this be tested with unit test? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

Re: [I] Adding property files option in the Spark Submit command [airflow]

2023-10-12 Thread via GitHub
ghostp13409 commented on issue #34838: URL: https://github.com/apache/airflow/issues/34838#issuecomment-1759808697 Hi @Bowrna , I've created the PR. kindly review it. I've added property-files operator in the spark submit operator. As I'm a newcomer to this industry I'm not sure if

[PR] 34838 adding propertyfiles option [airflow]

2023-10-12 Thread via GitHub
ghostp13409 opened a new pull request, #34894: URL: https://github.com/apache/airflow/pull/34894 --- Hi @Bowrna closes: #34838 I've added property-files operator in the spark submit operator. As I'm a newcomer to this industry I'm not sure if

Re: [PR] Refactor: Remove trailing whitespace from strings [airflow]

2023-10-12 Thread via GitHub
eladkal commented on PR #34121: URL: https://github.com/apache/airflow/pull/34121#issuecomment-1759803969 @eumiro can you handle it? I'd like to merge this one -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

Re: [PR] property-files command added to SparkSubmitOperator [airflow]

2023-10-12 Thread via GitHub
ghostp13409 closed pull request #34892: property-files command added to SparkSubmitOperator URL: https://github.com/apache/airflow/pull/34892 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

Re: [PR] Extend hooks arguments into `AwsBaseWaiterTrigger` [airflow]

2023-10-12 Thread via GitHub
Taragolis commented on code in PR #34884: URL: https://github.com/apache/airflow/pull/34884#discussion_r1356950506 ## airflow/providers/amazon/aws/triggers/base.py: ## @@ -102,9 +109,15 @@ def serialize(self) -> tuple[str, dict[str, Any]]: },

Re: [PR] Extend hooks arguments into `AwsBaseWaiterTrigger` [airflow]

2023-10-12 Thread via GitHub
Taragolis commented on code in PR #34884: URL: https://github.com/apache/airflow/pull/34884#discussion_r1356922790 ## airflow/providers/amazon/aws/triggers/base.py: ## @@ -102,9 +109,15 @@ def serialize(self) -> tuple[str, dict[str, Any]]: },

  1   2   3   >