[GitHub] [airflow] csp33 commented on issue #29273: Status of testing of Apache Airflow Helm Chart 1.8.0rc1

2023-02-02 Thread via GitHub
csp33 commented on issue #29273: URL: https://github.com/apache/airflow/issues/29273#issuecomment-1415276663 All good in #27625 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [airflow] csp33 commented on issue #29273: Status of testing of Apache Airflow Helm Chart 1.8.0rc1

2023-02-02 Thread via GitHub
csp33 commented on issue #29273: URL: https://github.com/apache/airflow/issues/29273#issuecomment-1415273070 #27443 is also OK -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [airflow] csp33 commented on issue #29273: Status of testing of Apache Airflow Helm Chart 1.8.0rc1

2023-02-02 Thread via GitHub
csp33 commented on issue #29273: URL: https://github.com/apache/airflow/issues/29273#issuecomment-1415270745 #27178 works as expected -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [airflow] csp33 commented on issue #29273: Status of testing of Apache Airflow Helm Chart 1.8.0rc1

2023-02-02 Thread via GitHub
csp33 commented on issue #29273: URL: https://github.com/apache/airflow/issues/29273#issuecomment-1415266941 Guess we can confirm that #27230, #27668, #28074 and #29074 are working✌

[GitHub] [airflow] Bowrna commented on issue #29304: Mismatch of taskinstances executed for dagrun result in API vs web UI

2023-02-02 Thread via GitHub
Bowrna commented on issue #29304: URL: https://github.com/apache/airflow/issues/29304#issuecomment-1415237521 @potiuk thanks for clarifying this part to me. yes it makes sense to me now. i have wrongly assumed that the api response is incorrect. -- This is an automated message from the

[GitHub] [airflow] ecerulm commented on pull request #29016: Fix leak sensitive field via V1EnvVar on exception

2023-02-02 Thread via GitHub
ecerulm commented on PR #29016: URL: https://github.com/apache/airflow/pull/29016#issuecomment-1415206145 I had to fix some formatting issues and some stuff from the rebase from main but now "all checks have passed". -- This is an automated message from the Apache Git Service. To

[GitHub] [airflow] vksunilk opened a new pull request, #29346: Check Absence of files or objects via GCSObjectExistenceSensor

2023-02-02 Thread via GitHub
vksunilk opened a new pull request, #29346: URL: https://github.com/apache/airflow/pull/29346 --- **^ Add meaningful description above** Read the **[Pull Request Guidelines](https://github.com/apache/airflow/blob/main/CONTRIBUTING.rst#pull-request-guidelines)** for more

[GitHub] [airflow] rajaths010494 commented on a diff in pull request #29300: Add deferrable BatchOperator

2023-02-02 Thread via GitHub
rajaths010494 commented on code in PR #29300: URL: https://github.com/apache/airflow/pull/29300#discussion_r1095392580 ## generated/provider_dependencies.json: ## @@ -17,6 +17,7 @@ }, "amazon": { "deps": [ + "aiobotocore>=2.1.1", Review Comment: Sure will

[GitHub] [airflow] potiuk closed issue #29281: (psycopg2.errors.FeatureNotSupported) This ALTER TABLE command is not yet supported .While deploying airflow with Postgres

2023-02-02 Thread via GitHub
potiuk closed issue #29281: (psycopg2.errors.FeatureNotSupported) This ALTER TABLE command is not yet supported .While deploying airflow with Postgres URL: https://github.com/apache/airflow/issues/29281 -- This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [airflow] potiuk commented on issue #29281: (psycopg2.errors.FeatureNotSupported) This ALTER TABLE command is not yet supported .While deploying airflow with Postgres

2023-02-02 Thread via GitHub
potiuk commented on issue #29281: URL: https://github.com/apache/airflow/issues/29281#issuecomment-1415078837 I think you failed to mention that you are not using Postgres but something else that claims that it is Postgres compatible (Yugabyte?). Apparent kk y this database is - as opposed

[GitHub] [airflow] vksunilk opened a new issue, #29344: Check Absence of files or objects via GCSObjectExistenceSensor

2023-02-02 Thread via GitHub
vksunilk opened a new issue, #29344: URL: https://github.com/apache/airflow/issues/29344 ### Description Currently GCSObjectExistenceSensor fails if the object is not present. We can add a parameter `check_for_absence`, which when true, should check for absence and should fail if

[airflow] branch main updated: Fixup some punctuation and grammar (#29342)

2023-02-02 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 70a174c47f Fixup some punctuation and grammar

[GitHub] [airflow] potiuk merged pull request #29342: Fixup some punctuation and grammar

2023-02-02 Thread via GitHub
potiuk merged PR #29342: URL: https://github.com/apache/airflow/pull/29342 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [airflow] uranusjr commented on pull request #29034: Add functionality for operators to template all eligible fields (apac…

2023-02-02 Thread via GitHub
uranusjr commented on PR #29034: URL: https://github.com/apache/airflow/pull/29034#issuecomment-1414988919 Same way as it’s determined now (if `template_fields` is `__all__` on the class we calculate it in `__init__`). -- This is an automated message from the Apache Git Service. To

[GitHub] [airflow] rkarish commented on pull request #29034: Add functionality for operators to template all eligible fields (apac…

2023-02-02 Thread via GitHub
rkarish commented on PR #29034: URL: https://github.com/apache/airflow/pull/29034#issuecomment-1414944253 > I have doubts making this an operator argument is the right design. I’d want to make this designed at the operator class level, e.g. > > ```python > class

[GitHub] [airflow] rkarish commented on a diff in pull request #29034: Add functionality for operators to template all eligible fields (apac…

2023-02-02 Thread via GitHub
rkarish commented on code in PR #29034: URL: https://github.com/apache/airflow/pull/29034#discussion_r1095351968 ## airflow/models/baseoperator.py: ## @@ -1184,7 +1199,23 @@ def render_template_fields( """ if not jinja_env: jinja_env =

[GitHub] [airflow] uranusjr commented on pull request #29034: Add functionality for operators to template all eligible fields (apac…

2023-02-02 Thread via GitHub
uranusjr commented on PR #29034: URL: https://github.com/apache/airflow/pull/29034#issuecomment-1414934990 I have doubts making this an operator argument is the right design. I’d want to make this designed at the operator class level, e.g. ```python class MyOperator(BaseOperator):

[GitHub] [airflow] uranusjr commented on a diff in pull request #29034: Add functionality for operators to template all eligible fields (apac…

2023-02-02 Thread via GitHub
uranusjr commented on code in PR #29034: URL: https://github.com/apache/airflow/pull/29034#discussion_r1095347948 ## airflow/models/baseoperator.py: ## @@ -1184,7 +1199,23 @@ def render_template_fields( """ if not jinja_env: jinja_env =

[GitHub] [airflow] rkarish commented on pull request #29034: Add functionality for operators to template all eligible fields (apac…

2023-02-02 Thread via GitHub
rkarish commented on PR #29034: URL: https://github.com/apache/airflow/pull/29034#issuecomment-1414870638 This one is good to go unless anyone has any objections  -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

[GitHub] [airflow] Bhawana-Barfa commented on issue #29281: (psycopg2.errors.FeatureNotSupported) This ALTER TABLE command is not yet supported .While deploying airflow with Postgres

2023-02-02 Thread via GitHub
Bhawana-Barfa commented on issue #29281: URL: https://github.com/apache/airflow/issues/29281#issuecomment-1414844046 > Postgres DB not library. Look for postgres support in Airflow. It start from postgres 11. (Find it in docs). No idea what the number you posted is Postgres DB

[GitHub] [airflow] amoghrajesh commented on pull request #29270: Adding possibility for annotations in logs pvc

2023-02-02 Thread via GitHub
amoghrajesh commented on PR #29270: URL: https://github.com/apache/airflow/pull/29270#issuecomment-1414830897 @potiuk / @jedcunningham could you help in a review here? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

[GitHub] [airflow] Adaverse commented on pull request #29338: Fixed link to airflow variable in docs

2023-02-02 Thread via GitHub
Adaverse commented on PR #29338: URL: https://github.com/apache/airflow/pull/29338#issuecomment-1414765628 > Thanks for the PR @Adaverse! While we are at it, we might as well fix up the grammar. Thanks for the suggestions @jedcunningham. It's done. -- This is an automated message

[GitHub] [airflow] hussein-awala commented on issue #29339: Failing to execute successive Dynamic Task Group Mapping

2023-02-02 Thread via GitHub
hussein-awala commented on issue #29339: URL: https://github.com/apache/airflow/issues/29339#issuecomment-1414704413 Operator and task group expansion in an expanded task group is not supported yet. We raise

[GitHub] [airflow] yehoshuadimarsky commented on pull request #27947: refactored Amazon Redshift-data functionality into the hook

2023-02-02 Thread via GitHub
yehoshuadimarsky commented on PR #27947: URL: https://github.com/apache/airflow/pull/27947#issuecomment-1414627041 @Taragolis @o-nikolas @dstandish @eladkal anyone able to review the new changes? For the full PR of adding Redshift data api to all the transfer operators... Thanks -- This

[GitHub] [airflow] dadonnelly316 commented on issue #28836: Airflow Scheduler Hangs After Failed K8 API Call

2023-02-02 Thread via GitHub
dadonnelly316 commented on issue #28836: URL: https://github.com/apache/airflow/issues/28836#issuecomment-1414562576 @xiankgx we ended up having to make changes in the way our K8 cluster was configured. You might need to review any plugins/security configurations that might cause pod

[GitHub] [airflow] o-nikolas commented on pull request #29343: Remove executor coupling in conf validation

2023-02-02 Thread via GitHub
o-nikolas commented on PR #29343: URL: https://github.com/apache/airflow/pull/29343#issuecomment-1414558243 Cc: @pierrejeambrun @utkarsharma2 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [airflow] o-nikolas opened a new pull request, #29343: Remove executor coupling in conf validation

2023-02-02 Thread via GitHub
o-nikolas opened a new pull request, #29343: URL: https://github.com/apache/airflow/pull/29343 Use a generic approach to check for single threaded executors that can support DBs like sqlite. An `is_single_threaded` property was added to the base executor interface (with a default of

[GitHub] [airflow] dadonnelly316 commented on issue #28836: Airflow Scheduler Hangs After Failed K8 API Call

2023-02-02 Thread via GitHub
dadonnelly316 commented on issue #28836: URL: https://github.com/apache/airflow/issues/28836#issuecomment-141486 @xi -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

[GitHub] [airflow] jedcunningham commented on a diff in pull request #29338: Fixed link to airflow variable in docs

2023-02-02 Thread via GitHub
jedcunningham commented on code in PR #29338: URL: https://github.com/apache/airflow/pull/29338#discussion_r1095220401 ## docs/apache-airflow/howto/dynamic-dag-generation.rst: ## @@ -40,7 +40,7 @@ If you want to use variables to configure your code, you should always use

[GitHub] [airflow] Dearkano commented on pull request #26269: Added Feature - Glue job continuously log printing in Airflow task logs

2023-02-02 Thread via GitHub
Dearkano commented on PR #26269: URL: https://github.com/apache/airflow/pull/26269#issuecomment-1414537107 Hi is there any update on this PR, thanks! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [airflow] github-actions[bot] commented on pull request #28455: WIP: Add dataset trigger rule engine

2023-02-02 Thread via GitHub
github-actions[bot] commented on PR #28455: URL: https://github.com/apache/airflow/pull/28455#issuecomment-1414536955 This pull request has been automatically marked as stale because it has not had recent activity. It will be closed in 5 days if no further activity occurs. Thank you for

[GitHub] [airflow] xiankgx commented on issue #28836: Airflow Scheduler Hangs After Failed K8 API Call

2023-02-02 Thread via GitHub
xiankgx commented on issue #28836: URL: https://github.com/apache/airflow/issues/28836#issuecomment-1414513094 Hi there, I'm having the same issue with Airflow version 2.2.2 deployed in Kubernetes in AWS. I'm getting the following lines: - ERROR - Exception when attempting to

[GitHub] [airflow] vandonr-amz commented on pull request #29245: fix code checking job names in sagemaker

2023-02-02 Thread via GitHub
vandonr-amz commented on PR #29245: URL: https://github.com/apache/airflow/pull/29245#issuecomment-1414503882 sorry for the force-push, there was a clandestine commit that made its way into this PR :/ -- This is an automated message from the Apache Git Service. To respond to the message,

[airflow] branch constraints-main updated: Updating constraints. Github run id:4079111476

2023-02-02 Thread github-bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch constraints-main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/constraints-main by this push: new f4bf508950 Updating

[GitHub] [airflow] vandonr-amz commented on a diff in pull request #29245: fix code checking job names in sagemaker

2023-02-02 Thread via GitHub
vandonr-amz commented on code in PR #29245: URL: https://github.com/apache/airflow/pull/29245#discussion_r1095192951 ## airflow/providers/amazon/aws/operators/sagemaker.py: ## @@ -654,8 +673,8 @@ class SageMakerTrainingOperator(SageMakerBaseOperator): the operation

[GitHub] [airflow] jedcunningham opened a new pull request, #29342: Fixup some punctuation and grammar

2023-02-02 Thread via GitHub
jedcunningham opened a new pull request, #29342: URL: https://github.com/apache/airflow/pull/29342 After noticing one punctuation mistake around "however", I figured there must be more. So I fixed them, and some other stuff adjacent to them. -- This is an automated message from the

[airflow] branch main updated (04c7210a44 -> 70d00bc8c8)

2023-02-02 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git from 04c7210a44 rewrite invoke lamba operator tests (#29144) add 70d00bc8c8 Add interactive testing in Breeze option

[GitHub] [airflow] potiuk merged pull request #29340: Add interactive testing in Breeze option in testing docs

2023-02-02 Thread via GitHub
potiuk merged PR #29340: URL: https://github.com/apache/airflow/pull/29340 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [airflow] o-nikolas commented on pull request #27185: Handle AzureBatch task result correctly

2023-02-02 Thread via GitHub
o-nikolas commented on PR #27185: URL: https://github.com/apache/airflow/pull/27185#issuecomment-1414442210 Looks like both static checks and tests are failing. To catch and fix static checks early, you can enable pre-commit on your laptop. See steps on how to do that

[GitHub] [airflow] dstandish commented on a diff in pull request #27758: Enable trigger logging in webserver

2023-02-02 Thread via GitHub
dstandish commented on code in PR #27758: URL: https://github.com/apache/airflow/pull/27758#discussion_r1095134849 ## chart/templates/triggerer/triggerer-deployment.yaml: ## @@ -44,6 +45,9 @@ metadata: {{- toYaml .Values.triggerer.annotations | nindent 4 }} {{- end }}

[GitHub] [airflow] o-nikolas commented on a diff in pull request #28161: AIP-51 - Executor Coupling in Logging

2023-02-02 Thread via GitHub
o-nikolas commented on code in PR #28161: URL: https://github.com/apache/airflow/pull/28161#discussion_r1095134144 ## airflow/executors/kubernetes_executor.py: ## @@ -773,6 +775,57 @@ def _change_state(self, key: TaskInstanceKey, state: str | None, pod_id: str, na

[GitHub] [airflow] jedcunningham commented on a diff in pull request #27758: Enable trigger logging in webserver

2023-02-02 Thread via GitHub
jedcunningham commented on code in PR #27758: URL: https://github.com/apache/airflow/pull/27758#discussion_r1095092478 ## chart/templates/triggerer/triggerer-deployment.yaml: ## @@ -44,6 +45,9 @@ metadata: {{- toYaml .Values.triggerer.annotations | nindent 4 }} {{- end

[GitHub] [airflow] jedcunningham commented on a diff in pull request #27758: Enable trigger logging in webserver

2023-02-02 Thread via GitHub
jedcunningham commented on code in PR #27758: URL: https://github.com/apache/airflow/pull/27758#discussion_r1095092261 ## chart/templates/triggerer/triggerer-deployment.yaml: ## @@ -44,6 +45,9 @@ metadata: {{- toYaml .Values.triggerer.annotations | nindent 4 }} {{- end

[GitHub] [airflow] dstandish commented on a diff in pull request #27758: Enable trigger logging in webserver

2023-02-02 Thread via GitHub
dstandish commented on code in PR #27758: URL: https://github.com/apache/airflow/pull/27758#discussion_r1095080223 ## chart/templates/triggerer/triggerer-service.yaml: ## @@ -0,0 +1,48 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor

[GitHub] [airflow] jedcunningham commented on a diff in pull request #27758: Enable trigger logging in webserver

2023-02-02 Thread via GitHub
jedcunningham commented on code in PR #27758: URL: https://github.com/apache/airflow/pull/27758#discussion_r1095023715 ## docs/apache-airflow/administration-and-deployment/logging-monitoring/logging-tasks.rst: ## @@ -122,15 +122,33 @@ When using remote logging, you can

[GitHub] [airflow] bbovenzi commented on issue #29287: Empty Graph view for Consecutive Dynamic Task Group Mapping

2023-02-02 Thread via GitHub
bbovenzi commented on issue #29287: URL: https://github.com/apache/airflow/issues/29287#issuecomment-1414348616 I wonder if this is fixed in https://github.com/apache/airflow/pull/29042, it sounds similar. -- This is an automated message from the Apache Git Service. To respond to the

[GitHub] [airflow] vincbeck commented on a diff in pull request #28900: [WIP] Convert DagFileProcessor.execute_callbacks to Internal API

2023-02-02 Thread via GitHub
vincbeck commented on code in PR #28900: URL: https://github.com/apache/airflow/pull/28900#discussion_r1095066168 ## airflow/models/dag.py: ## @@ -1298,6 +1298,48 @@ def normalized_schedule_interval(self) -> ScheduleInterval: _schedule_interval =

[GitHub] [airflow] bbovenzi commented on issue #29323: DAG dependencies graph not updating when deleting a DAG

2023-02-02 Thread via GitHub
bbovenzi commented on issue #29323: URL: https://github.com/apache/airflow/issues/29323#issuecomment-1414344653 We generate that graph based on `SerializedDagModel.get_dag_dependencies()`. I'm not too sure as to when that is updated after deleting a dag. Maybe someone with more knowledge

[GitHub] [airflow] o-nikolas commented on a diff in pull request #29341: Add clarification about AWS Connection behaviour

2023-02-02 Thread via GitHub
o-nikolas commented on code in PR #29341: URL: https://github.com/apache/airflow/pull/29341#discussion_r1095054455 ## docs/apache-airflow-providers-amazon/connections/aws.rst: ## @@ -20,8 +20,21 @@ Amazon Web Services Connection == -The Amazon

[GitHub] [airflow] morningcloud commented on issue #16313: cryptography.fernet.InvalidToken when attempting to access connections

2023-02-02 Thread via GitHub
morningcloud commented on issue #16313: URL: https://github.com/apache/airflow/issues/16313#issuecomment-1414341541 Thanks @jedcunningham and @danielmorales72! I resolved this with a long and probably unnecessary workaround before seeing those messages! I'll write it here in case

[airflow] branch main updated (8ce8c9500c -> 04c7210a44)

2023-02-02 Thread onikolas
This is an automated email from the ASF dual-hosted git repository. onikolas pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git from 8ce8c9500c Add a param for get_dags endpoint to list only unpaused dags (#28713) add 04c7210a44 rewrite invoke

[GitHub] [airflow] o-nikolas merged pull request #29144: rewrite invoke lamba operator tests

2023-02-02 Thread via GitHub
o-nikolas merged PR #29144: URL: https://github.com/apache/airflow/pull/29144 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [airflow] eladkal commented on a diff in pull request #29341: Add clarification about AWS Connection behaviour

2023-02-02 Thread via GitHub
eladkal commented on code in PR #29341: URL: https://github.com/apache/airflow/pull/29341#discussion_r1095044908 ## docs/apache-airflow-providers-amazon/connections/aws.rst: ## @@ -20,8 +20,21 @@ Amazon Web Services Connection == -The Amazon Web

[GitHub] [airflow] Taragolis opened a new pull request, #29341: Add clarification about AWS Connection behaviour

2023-02-02 Thread via GitHub
Taragolis opened a new pull request, #29341: URL: https://github.com/apache/airflow/pull/29341 It is not uncommon when user tried to use Amazon provider for interact with AWS API compatible services (MinIO or others). Even if we do not officially support them better provide additional

[GitHub] [airflow] ferruzzi commented on pull request #28338: New AWS sensor — DynamoDBValueSensor

2023-02-02 Thread via GitHub
ferruzzi commented on PR #28338: URL: https://github.com/apache/airflow/pull/28338#issuecomment-1414301032 What Elad said :smile: Basically just replace the `airflow` in the file's path with `tests` so tests for `airflow/providers/amazon/aws/sensors/test_dynamodb.py` go in

[GitHub] [airflow] mhenc commented on a diff in pull request #29306: AIP-44 Fix support for classmethods and method mapping

2023-02-02 Thread via GitHub
mhenc commented on code in PR #29306: URL: https://github.com/apache/airflow/pull/29306#discussion_r1095031574 ## airflow/api_internal/endpoints/rpc_api_endpoint.py: ## @@ -81,6 +81,7 @@ def internal_airflow_api(body: dict[str, Any]) -> APIResponse: log.debug("Calling

[GitHub] [airflow] jedcunningham commented on pull request #27758: Enable trigger logging in webserver

2023-02-02 Thread via GitHub
jedcunningham commented on PR #27758: URL: https://github.com/apache/airflow/pull/27758#issuecomment-1414290164 I think this might be a candidate for #protm! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

[GitHub] [airflow] jedcunningham commented on a diff in pull request #27758: Enable trigger logging in webserver

2023-02-02 Thread via GitHub
jedcunningham commented on code in PR #27758: URL: https://github.com/apache/airflow/pull/27758#discussion_r1095013539 ## chart/templates/triggerer/triggerer-service.yaml: ## @@ -0,0 +1,48 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor

[airflow] branch main updated (cb0c90bd66 -> 8ce8c9500c)

2023-02-02 Thread bbovenzi
This is an automated email from the ASF dual-hosted git repository. bbovenzi pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git from cb0c90bd66 Decrypt SecureString value obtained by SsmHook (#29142) add 8ce8c9500c Add a param for get_dags

[GitHub] [airflow] bbovenzi merged pull request #28713: Add a param for get_dags endpoint to list only unpaused dags

2023-02-02 Thread via GitHub
bbovenzi merged PR #28713: URL: https://github.com/apache/airflow/pull/28713 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [airflow] potiuk commented on pull request #29340: Add interactive testing in Breeze option in testing docs

2023-02-02 Thread via GitHub
potiuk commented on PR #29340: URL: https://github.com/apache/airflow/pull/29340#issuecomment-1414276583 See https://apache-airflow.slack.com/archives/CQ9QHSFQX/p1675363984122639 cc: @Taragolis -- This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [airflow] potiuk commented on pull request #29340: Add interactive testing in Breeze option in testing docs

2023-02-02 Thread via GitHub
potiuk commented on PR #29340: URL: https://github.com/apache/airflow/pull/29340#issuecomment-1414276217 Also added the option to ask to rebuild the image automatically for `breeze testing test` command - seems that this is the way users are running tests (just copying the commands from

[GitHub] [airflow] potiuk opened a new pull request, #29340: Add interactive testing in Breeze option in testing docs

2023-02-02 Thread via GitHub
potiuk opened a new pull request, #29340: URL: https://github.com/apache/airflow/pull/29340 The interactive testing is one of the most efficient ways of running and re-running tests in Breeze. This option is now explicitly documented and comment is added that for testing command one might

[GitHub] [airflow] gmontero06 commented on issue #29199: TaskFlow AirflowSkipException causes downstream step to fail when multiple_outputs is true

2023-02-02 Thread via GitHub
gmontero06 commented on issue #29199: URL: https://github.com/apache/airflow/issues/29199#issuecomment-1414256700 Same here. We ran into this issue when trying to upgrade from v2.2.3 to v2.5.1 -- This is an automated message from the Apache Git Service. To respond to the message, please

[GitHub] [airflow] eladkal commented on pull request #28338: New AWS sensor — DynamoDBValueSensor

2023-02-02 Thread via GitHub
eladkal commented on PR #28338: URL: https://github.com/apache/airflow/pull/28338#issuecomment-1414243753 > @ferruzzi Am I to use `tests/providers/amazon/aws/providers` for my unit test? It should be in `tests/providers/amazon/aws/sensors/test_dynamodb.py` If file doesnt

[GitHub] [airflow] mrichman commented on pull request #28338: New AWS sensor — DynamoDBValueSensor

2023-02-02 Thread via GitHub
mrichman commented on PR #28338: URL: https://github.com/apache/airflow/pull/28338#issuecomment-1414228569 @ferruzzi Am I to use `tests/providers/amazon/aws/providers` for my unit test? -- This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [airflow] eladkal commented on issue #29307: Bigquery Operators in defferable mode fail if the location is not US.

2023-02-02 Thread via GitHub
eladkal commented on issue #29307: URL: https://github.com/apache/airflow/issues/29307#issuecomment-1414215349 @lwyszomi can you take a look? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [airflow] hussein-awala commented on issue #29084: Add `max_active_tis_per_dagrun` for Dynamic Task Mapping

2023-02-02 Thread via GitHub
hussein-awala commented on issue #29084: URL: https://github.com/apache/airflow/issues/29084#issuecomment-1414209722 I have two different use cases for this feature: - I have an API used by the users to submit ML processing requests for some data by a list of models (they choose a subset

[airflow] branch main updated (478fd82652 -> cb0c90bd66)

2023-02-02 Thread eladkal
This is an automated email from the ASF dual-hosted git repository. eladkal pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git from 478fd82652 Expose updated_at filter for dag run and task instance endpoints (#28636) add cb0c90bd66 Decrypt

[GitHub] [airflow] eladkal merged pull request #29142: Decrypt SecureString value obtained by SsmHook

2023-02-02 Thread via GitHub
eladkal merged PR #29142: URL: https://github.com/apache/airflow/pull/29142 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [airflow] potiuk commented on pull request #27758: Enable trigger logging in webserver

2023-02-02 Thread via GitHub
potiuk commented on PR #27758: URL: https://github.com/apache/airflow/pull/27758#issuecomment-1414176150 Given the various way we do logging and various level of support in native logging, coming up with a simpler solution is not possible. I think :). -- This is an automated message

[GitHub] [airflow] potiuk commented on a diff in pull request #27758: Enable trigger logging in webserver

2023-02-02 Thread via GitHub
potiuk commented on code in PR #27758: URL: https://github.com/apache/airflow/pull/27758#discussion_r1094910327 ## docs/apache-airflow/administration-and-deployment/logging-monitoring/logging-tasks.rst: ## @@ -47,11 +44,14 @@ These patterns can be adjusted by

[GitHub] [airflow] potiuk commented on a diff in pull request #27758: Enable trigger logging in webserver

2023-02-02 Thread via GitHub
potiuk commented on code in PR #27758: URL: https://github.com/apache/airflow/pull/27758#discussion_r1094896920 ## airflow/jobs/triggerer_job.py: ## @@ -245,11 +410,10 @@ async def arun(self): await self.cleanup_finished_triggers() # Sleep for a bit

[GitHub] [airflow] sfc-gh-syeager commented on issue #24734: ExternalTask: Check if the most recent task succeeds

2023-02-02 Thread via GitHub
sfc-gh-syeager commented on issue #24734: URL: https://github.com/apache/airflow/issues/24734#issuecomment-1414141911 @uranusjr First, estimate the usecase for the most recent run, as opposed to ExternalTaskSensor, which looks for a run at a specific logical time. The usecase in

[GitHub] [airflow] JulesTriomphe commented on issue #29273: Status of testing of Apache Airflow Helm Chart 1.8.0rc1

2023-02-02 Thread via GitHub
JulesTriomphe commented on issue #29273: URL: https://github.com/apache/airflow/issues/29273#issuecomment-1414140194 Very cool, thanks for the clarification @jedcunningham ! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

[GitHub] [airflow] vincbeck commented on a diff in pull request #28900: [WIP] Convert DagFileProcessor.execute_callbacks to Internal API

2023-02-02 Thread via GitHub
vincbeck commented on code in PR #28900: URL: https://github.com/apache/airflow/pull/28900#discussion_r1094867921 ## airflow/models/dag.py: ## @@ -1298,6 +1298,48 @@ def normalized_schedule_interval(self) -> ScheduleInterval: _schedule_interval =

[GitHub] [airflow] hussein-awala commented on pull request #28713: Add a param for get_dags endpoint to list only unpaused dags

2023-02-02 Thread via GitHub
hussein-awala commented on PR #28713: URL: https://github.com/apache/airflow/pull/28713#issuecomment-1414122546 @bbovenzi done -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [airflow] pmercatoris opened a new issue, #29339: Failing to execute successive Dynamic Task Group Mapping

2023-02-02 Thread via GitHub
pmercatoris opened a new issue, #29339: URL: https://github.com/apache/airflow/issues/29339 ### Apache Airflow version 2.5.1 ### What happened I am trying to map a task group within another task group which is already mapped. However, when launching the dag, all tasks

[GitHub] [airflow] bbovenzi commented on pull request #28713: Add a param for get_dags endpoint to list only unpaused dags

2023-02-02 Thread via GitHub
bbovenzi commented on PR #28713: URL: https://github.com/apache/airflow/pull/28713#issuecomment-1414100779 @hussein-awala Could you rebase and regenerate the yaml and typescript files please? -- This is an automated message from the Apache Git Service. To respond to the message, please

[airflow] branch main updated (56175e4afa -> 478fd82652)

2023-02-02 Thread bbovenzi
This is an automated email from the ASF dual-hosted git repository. bbovenzi pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git from 56175e4afa Helm deployment fails when postgresql.nameOverride is used (#29214) add 478fd82652 Expose

[GitHub] [airflow] ephraimbuddy commented on a diff in pull request #29306: AIP-44 Fix support for classmethods and method mapping

2023-02-02 Thread via GitHub
ephraimbuddy commented on code in PR #29306: URL: https://github.com/apache/airflow/pull/29306#discussion_r1094836563 ## airflow/api_internal/endpoints/rpc_api_endpoint.py: ## @@ -81,6 +81,7 @@ def internal_airflow_api(body: dict[str, Any]) -> APIResponse:

[GitHub] [airflow] bbovenzi merged pull request #28636: Expose updated_at filter for dag run and task instance endpoints

2023-02-02 Thread via GitHub
bbovenzi merged PR #28636: URL: https://github.com/apache/airflow/pull/28636 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [airflow] vincbeck commented on a diff in pull request #29306: AIP-44 Fix support for classmethods and method mapping

2023-02-02 Thread via GitHub
vincbeck commented on code in PR #29306: URL: https://github.com/apache/airflow/pull/29306#discussion_r1094834580 ## airflow/api_internal/endpoints/rpc_api_endpoint.py: ## @@ -81,6 +81,7 @@ def internal_airflow_api(body: dict[str, Any]) -> APIResponse: log.debug("Calling

[GitHub] [airflow] boring-cyborg[bot] commented on pull request #29338: Fixed link to airflow variable in docs

2023-02-02 Thread via GitHub
boring-cyborg[bot] commented on PR #29338: URL: https://github.com/apache/airflow/pull/29338#issuecomment-1414094278 Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our Contribution

[GitHub] [airflow] Adaverse opened a new pull request, #29338: Fixed link to airflow variable in docs

2023-02-02 Thread via GitHub
Adaverse opened a new pull request, #29338: URL: https://github.com/apache/airflow/pull/29338 Background: While going through the Dynamic DAG Generation doc -> https://airflow.apache.org/docs/apache-airflow/stable/howto/dynamic-dag-generation.html, found out one of the links to best

[GitHub] [airflow] morkot commented on issue #29273: Status of testing of Apache Airflow Helm Chart 1.8.0rc1

2023-02-02 Thread via GitHub
morkot commented on issue #29273: URL: https://github.com/apache/airflow/issues/29273#issuecomment-1414076234 #27671 has been checked. Changes are there, as expected. You can tick that one off. -- This is an automated message from the Apache Git Service. To respond to the message, please

[airflow] branch main updated (338a633fc9 -> 56175e4afa)

2023-02-02 Thread jedcunningham
This is an automated email from the ASF dual-hosted git repository. jedcunningham pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git from 338a633fc9 Initialize and validate conf early and intentionally (#29257) add 56175e4afa Helm deployment

[GitHub] [airflow] jedcunningham merged pull request #29214: Helm deployment fails when postgresql.nameOverride is used

2023-02-02 Thread via GitHub
jedcunningham merged PR #29214: URL: https://github.com/apache/airflow/pull/29214 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [airflow] jedcunningham commented on issue #29273: Status of testing of Apache Airflow Helm Chart 1.8.0rc1

2023-02-02 Thread via GitHub
jedcunningham commented on issue #29273: URL: https://github.com/apache/airflow/issues/29273#issuecomment-1414065922 @JulesTriomphe, I fully intend to do releases more quickly than we have in the past. So "sometime soon" meaning in the next month, yes, in the next week or so, no. --

[GitHub] [airflow] uranusjr commented on pull request #29016: Fix leak sensitive field via V1EnvVar on exception

2023-02-02 Thread via GitHub
uranusjr commented on PR #29016: URL: https://github.com/apache/airflow/pull/29016#issuecomment-1414056451 We actually have `Protocol` available on 3.7 via `airflow.typing_compat`, but either way is fine as I mentioned above, so since you already made the change let’s just keep this. --

[airflow] branch main updated (91c0ce7666 -> 338a633fc9)

2023-02-02 Thread jedcunningham
This is an automated email from the ASF dual-hosted git repository. jedcunningham pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git from 91c0ce7666 Drop Connection.schema use in DbtCloudHook (#29166) add 338a633fc9 Initialize and validate conf

[GitHub] [airflow] jedcunningham merged pull request #29257: Initialize and validate conf early and intentionally

2023-02-02 Thread via GitHub
jedcunningham merged PR #29257: URL: https://github.com/apache/airflow/pull/29257 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [airflow] tsoud commented on a diff in pull request #29158: Update pipeline.rst - Fix query in `merge_data()` task

2023-02-02 Thread via GitHub
tsoud commented on code in PR #29158: URL: https://github.com/apache/airflow/pull/29158#discussion_r1092087567 ## docs/apache-airflow/tutorial/pipeline.rst: ## @@ -170,7 +170,7 @@ Here we select completely unique records from the retrieved data, then we check

[GitHub] [airflow] emilymitacc commented on issue #29199: TaskFlow AirflowSkipException causes downstream step to fail when multiple_outputs is true

2023-02-02 Thread via GitHub
emilymitacc commented on issue #29199: URL: https://github.com/apache/airflow/issues/29199#issuecomment-1414044622 I also have this problem, please your help to solve it -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

[GitHub] [airflow] CarrascoEr commented on issue #29199: TaskFlow AirflowSkipException causes downstream step to fail when multiple_outputs is true

2023-02-02 Thread via GitHub
CarrascoEr commented on issue #29199: URL: https://github.com/apache/airflow/issues/29199#issuecomment-1414041105 stuck with this too, hoping for this feature to be released soon -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

[airflow] branch main updated: Drop Connection.schema use in DbtCloudHook (#29166)

2023-02-02 Thread joshfell
This is an automated email from the ASF dual-hosted git repository. joshfell pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 91c0ce7666 Drop Connection.schema use in

[GitHub] [airflow] josh-fell merged pull request #29166: Drop Connection.schema use in DbtCloudHook

2023-02-02 Thread via GitHub
josh-fell merged PR #29166: URL: https://github.com/apache/airflow/pull/29166 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [airflow] alexandermalyga commented on a diff in pull request #29314: Add ttlSecondsAfterFinished to migrateDatabaseJob and createUserJob

2023-02-02 Thread via GitHub
alexandermalyga commented on code in PR #29314: URL: https://github.com/apache/airflow/pull/29314#discussion_r1094788115 ## chart/templates/jobs/create-user-job.yaml: ## @@ -48,6 +48,9 @@ metadata: {{- $annotations | toYaml | nindent 4 }} {{- end }} spec: + {{- if not

[GitHub] [airflow] ephraimbuddy commented on pull request #29309: Add `airflow db drop-archived` command

2023-02-02 Thread via GitHub
ephraimbuddy commented on PR #29309: URL: https://github.com/apache/airflow/pull/29309#issuecomment-1414029554 > (He repeated himself. :) Haha :) 臘‍♂️ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

[GitHub] [airflow] Taragolis commented on a diff in pull request #27616: Flush the session in `handle_failure()` prior to invoking callbacks

2023-02-02 Thread via GitHub
Taragolis commented on code in PR #27616: URL: https://github.com/apache/airflow/pull/27616#discussion_r1094709928 ## airflow/models/taskinstance.py: ## @@ -1808,13 +1808,13 @@ def handle_failure( except Exception: self.log.exception("Failed to

  1   2   >