[GitHub] [airflow] alexott commented on a diff in pull request #28950: Sensor for Databricks partition and table changes

2023-02-19 Thread via GitHub
alexott commented on code in PR #28950: URL: https://github.com/apache/airflow/pull/28950#discussion_r590565 ## airflow/providers/databricks/sensors/databricks_sql.py: ## @@ -0,0 +1,136 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one +# or more

[GitHub] [airflow] pateash commented on issue #28465: Airflow 2.2.4 Jenkins Connection - unable to set as the hook expects to be

2023-02-19 Thread via GitHub
pateash commented on issue #28465: URL: https://github.com/apache/airflow/issues/28465#issuecomment-1436491373 @eladkal , I would like to pick this up. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [airflow] harishkrao commented on a diff in pull request #28950: Sensor for Databricks partition and table changes

2023-02-19 Thread via GitHub
harishkrao commented on code in PR #28950: URL: https://github.com/apache/airflow/pull/28950#discussion_r577966 ## airflow/providers/databricks/sensors/databricks_sql.py: ## @@ -0,0 +1,136 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one +# or more

[GitHub] [airflow] eladkal commented on issue #28465: Airflow 2.2.4 Jenkins Connection - unable to set as the hook expects to be

2023-02-19 Thread via GitHub
eladkal commented on issue #28465: URL: https://github.com/apache/airflow/issues/28465#issuecomment-1436479493 **The needed fix(s):** 1. `connection.extra` accepts only json while Jenkins conn attempts to set boolean value. This needs to be changed. Probably to: `{"use_https": true}`

[GitHub] [airflow] Taragolis commented on a diff in pull request #29623: Implement file credentials provider for AWS hook AssumeRoleWithWebIdentity

2023-02-19 Thread via GitHub
Taragolis commented on code in PR #29623: URL: https://github.com/apache/airflow/pull/29623#discussion_r561157 ## airflow/providers/amazon/aws/hooks/base_aws.py: ## @@ -312,19 +312,35 @@ def _get_web_identity_credential_fetcher( base_session =

[GitHub] [airflow] takersk commented on issue #29561: CeleryKubernetesExecutor @task(queue="kubernetes")  not work

2023-02-19 Thread via GitHub
takersk commented on issue #29561: URL: https://github.com/apache/airflow/issues/29561#issuecomment-1436439675 @tirkarthi thank you I solved it. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [airflow] talnagar commented on pull request #29413: Add a new graph inside of the grid view

2023-02-19 Thread via GitHub
talnagar commented on PR #29413: URL: https://github.com/apache/airflow/pull/29413#issuecomment-1436416320 this feature looks amazing! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [airflow] talnagar commented on pull request #29042: Clarify graph view dynamic task labels

2023-02-19 Thread via GitHub
talnagar commented on PR #29042: URL: https://github.com/apache/airflow/pull/29042#issuecomment-1436414683 thanks @potiuk -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

[GitHub] [airflow] takersk commented on issue #29561: CeleryKubernetesExecutor @task(queue="kubernetes")  not work

2023-02-19 Thread via GitHub
takersk commented on issue #29561: URL: https://github.com/apache/airflow/issues/29561#issuecomment-1436408171 @tirkarthi There is no value returned by task, exec_trino_query_pod task will run the actual ==> return KubernetesPodOperator( name=task_id.replace("_", "-"),

[GitHub] [airflow] Bowrna commented on issue #29405: Add pagination to get_log in the rest API

2023-02-19 Thread via GitHub
Bowrna commented on issue #29405: URL: https://github.com/apache/airflow/issues/29405#issuecomment-1436393744 i will start working on this today. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [airflow] tirkarthi commented on issue #29561: CeleryKubernetesExecutor @task(queue="kubernetes")  not work

2023-02-19 Thread via GitHub
tirkarthi commented on issue #29561: URL: https://github.com/apache/airflow/issues/29561#issuecomment-1436385102 Values returned by tasks should be JSON serializable. Please see comments https://github.com/apache/airflow/issues/28492#issuecomment-1359564322 and

[GitHub] [airflow] takersk commented on issue #29561: CeleryKubernetesExecutor @task(queue="kubernetes")  not work

2023-02-19 Thread via GitHub
takersk commented on issue #29561: URL: https://github.com/apache/airflow/issues/29561#issuecomment-1436378898 @tirkarthi cc: @hussein-awala Thank you for answer. I tested it after changing task_id, but another error is occurring. Can I check it? dags code modiftyed ```

[GitHub] [airflow] uranusjr opened a new pull request, #29625: Aggressively cache entry points in process

2023-02-19 Thread via GitHub
uranusjr opened a new pull request, #29625: URL: https://github.com/apache/airflow/pull/29625 `importlib.metadata.distributions()` reads information from the actual installations, which is a lot of IO that we can avoid by caching. The benefit of this depends on how many packages you

[GitHub] [airflow] amoghrajesh opened a new pull request, #29624: Can't configure Kubernetes and Celery workers in Helm Chart

2023-02-19 Thread via GitHub
amoghrajesh opened a new pull request, #29624: URL: https://github.com/apache/airflow/pull/29624 The "workers" section in the values.yaml is only used to configure the celery executor and celerykubernetes executors. We should split the common section of the workers section into celery

[airflow] branch main updated: Add Amazon Redshift-data to S3<>RS Transfer Operators (#27947)

2023-02-19 Thread eladkal
This is an automated email from the ASF dual-hosted git repository. eladkal pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 0604033829 Add Amazon Redshift-data to S3<>RS

[GitHub] [airflow] eladkal merged pull request #27947: Add Amazon Redshift-data to S3<>RS Transfer Operators

2023-02-19 Thread via GitHub
eladkal merged PR #27947: URL: https://github.com/apache/airflow/pull/27947 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [airflow] tirkarthi commented on issue #29561: CeleryKubernetesExecutor @task(queue="kubernetes")  not work

2023-02-19 Thread via GitHub
tirkarthi commented on issue #29561: URL: https://github.com/apache/airflow/issues/29561#issuecomment-1436341424 `example_task_kubernetes` is the function name decorated by `@task` . You also seem to be passing the same value as task_id to `exec_trino_query_pod` where if you are trying to

[GitHub] [airflow] pgagnon opened a new pull request, #29623: Implement file credentials provider for AWS hook AssumeRoleWithWebIdentity

2023-02-19 Thread via GitHub
pgagnon opened a new pull request, #29623: URL: https://github.com/apache/airflow/pull/29623 This PR implements a new `web_identity_token_loader` for the AWS connection `AssumeRoleWithWebIdentity` authentication mechanism that fetches the access token from a location on the Airflow

[airflow] branch constraints-main updated: Updating constraints. Github run id:4219899182

2023-02-19 Thread github-bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch constraints-main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/constraints-main by this push: new 6fd769aa4d Updating

[GitHub] [airflow] amoghrajesh opened a new pull request, #29622: Fix adding annotations for dag persistence PVC

2023-02-19 Thread via GitHub
amoghrajesh opened a new pull request, #29622: URL: https://github.com/apache/airflow/pull/29622 As per the review comments here: https://github.com/apache/airflow/pull/29270#pullrequestreview-1304890651, due to this design, the upgrades might suffer. Fix them to be helm upgrade

[airflow] branch main updated: Avoid importing executor during conf validation (#29569)

2023-02-19 Thread uranusjr
This is an automated email from the ASF dual-hosted git repository. uranusjr pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 7dd19731f2 Avoid importing executor during conf

[GitHub] [airflow] uranusjr merged pull request #29569: Avoid importing executor during conf validation

2023-02-19 Thread via GitHub
uranusjr merged PR #29569: URL: https://github.com/apache/airflow/pull/29569 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [airflow] uranusjr commented on a diff in pull request #29569: Avoid importing executor during conf validation

2023-02-19 Thread via GitHub
uranusjr commented on code in PR #29569: URL: https://github.com/apache/airflow/pull/29569#discussion_r432584 ## tests/www/views/test_views_tasks.py: ## @@ -492,7 +496,7 @@ def test_code_from_db_all_example_dags(admin_client):

[GitHub] [airflow] amoghrajesh opened a new issue, #29621: Fix adding annotations for dag persistence PVC

2023-02-19 Thread via GitHub
amoghrajesh opened a new issue, #29621: URL: https://github.com/apache/airflow/issues/29621 ### Official Helm Chart version 1.8.0 (latest released) ### Apache Airflow version 2.5.0 ### Kubernetes Version v1.25.4 ### Helm Chart configuration

[GitHub] [airflow] boring-cyborg[bot] commented on issue #29621: Fix adding annotations for dag persistence PVC

2023-02-19 Thread via GitHub
boring-cyborg[bot] commented on issue #29621: URL: https://github.com/apache/airflow/issues/29621#issuecomment-1436259360 Thanks for opening your first issue here! Be sure to follow the issue template! -- This is an automated message from the Apache Git Service. To respond to the

[GitHub] [airflow] amoghrajesh commented on a diff in pull request #29270: Adding possibility for annotations in logs pvc

2023-02-19 Thread via GitHub
amoghrajesh commented on code in PR #29270: URL: https://github.com/apache/airflow/pull/29270#discussion_r417094 ## chart/templates/logs-persistent-volume-claim.yaml: ## @@ -29,6 +29,10 @@ metadata: {{- with .Values.labels }} {{- toYaml . | nindent 4 }} {{-

[airflow] branch main updated: Fix bitnami/postgresql default username and password (#29478)

2023-02-19 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 2d13d04064 Fix bitnami/postgresql default username

[GitHub] [airflow] potiuk merged pull request #29478: Fix bitnami/postgresql default username and password

2023-02-19 Thread via GitHub
potiuk merged PR #29478: URL: https://github.com/apache/airflow/pull/29478 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [airflow] potiuk commented on pull request #29357: Template extra volumes in helm chart

2023-02-19 Thread via GitHub
potiuk commented on PR #29357: URL: https://github.com/apache/airflow/pull/29357#issuecomment-1436171637 I think this one needs documentation explaining usage of this feature. Otherwise it won't be discoverable. -- This is an automated message from the Apache Git Service. To respond to

[GitHub] [airflow] potiuk closed issue #29458: Identify logging statements from Airflow versus top level code

2023-02-19 Thread via GitHub
potiuk closed issue #29458: Identify logging statements from Airflow versus top level code URL: https://github.com/apache/airflow/issues/29458 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [airflow] potiuk commented on issue #29458: Identify logging statements from Airflow versus top level code

2023-02-19 Thread via GitHub
potiuk commented on issue #29458: URL: https://github.com/apache/airflow/issues/29458#issuecomment-1436167051 I think it should be done via convention. In your DAGs you can add specific logger "filtered.out" for example and configue it in the way you want in LOGGING_CONFIG:

[GitHub] [airflow] michaelmicheal commented on pull request #29441: datasets, next_run_datasets, remove unnecessary timestamp filter

2023-02-19 Thread via GitHub
michaelmicheal commented on PR #29441: URL: https://github.com/apache/airflow/pull/29441#issuecomment-1436166837 @potiuk is this good to merge? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [airflow] potiuk commented on issue #25060: Airflow scheduler crashes due to 'duplicate key value violates unique constraint "task_instance_pkey"'

2023-02-19 Thread via GitHub
potiuk commented on issue #25060: URL: https://github.com/apache/airflow/issues/25060#issuecomment-1436163207 Detailed logs and circumstance would be needed (see https://github.com/apache/airflow/issues/25200 for example how detailed information helped in fixing similar issue). -- This

[GitHub] [airflow] wonhyeongseo commented on pull request #27841: fix: add pre-commit to ban urlparse

2023-02-19 Thread via GitHub
wonhyeongseo commented on PR #27841: URL: https://github.com/apache/airflow/pull/27841#issuecomment-1436162381 @potiuk Thank you so much for helping! Hope you have a great day  -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [airflow] goodseog commented on issue #29393: S3TaskHandler continuously returns "*** Falling back to local log" even if log_pos is provided when log not in s3

2023-02-19 Thread via GitHub
goodseog commented on issue #29393: URL: https://github.com/apache/airflow/issues/29393#issuecomment-1436162264 Same here :) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

[GitHub] [airflow] potiuk closed issue #27715: Add pre-commit rule to validate using `urlsplit` rather than `urlparse`

2023-02-19 Thread via GitHub
potiuk closed issue #27715: Add pre-commit rule to validate using `urlsplit` rather than `urlparse` URL: https://github.com/apache/airflow/issues/27715 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[airflow] branch main updated: fix: add pre-commit to ban urlparse (#27841)

2023-02-19 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new a99254ffd3 fix: add pre-commit to ban urlparse

[GitHub] [airflow] potiuk merged pull request #27841: fix: add pre-commit to ban urlparse

2023-02-19 Thread via GitHub
potiuk merged PR #27841: URL: https://github.com/apache/airflow/pull/27841 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [airflow] potiuk commented on issue #12771: Instrument Airflow with Opentracing/Opentelemetry

2023-02-19 Thread via GitHub
potiuk commented on issue #12771: URL: https://github.com/apache/airflow/issues/12771#issuecomment-1436161279 FYI. Suppport and POC on how to do the OpenTelemetry integration is already aproved

[airflow] branch main updated: Add ttlSecondsAfterFinished to migrateDatabaseJob and createUserJob (#29314)

2023-02-19 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new cd01650192 Add ttlSecondsAfterFinished to

[GitHub] [airflow] potiuk merged pull request #29314: Add ttlSecondsAfterFinished to migrateDatabaseJob and createUserJob

2023-02-19 Thread via GitHub
potiuk merged PR #29314: URL: https://github.com/apache/airflow/pull/29314 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[airflow] branch main updated: Add deferrable mode to DataprocInstantiateWorkflowTemplateOperator (#28618)

2023-02-19 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 1677d80e65 Add deferrable mode to

[GitHub] [airflow] potiuk merged pull request #28618: Add deferrable mode to DataprocInstantiateWorkflowTemplateOperator

2023-02-19 Thread via GitHub
potiuk merged PR #28618: URL: https://github.com/apache/airflow/pull/28618 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [airflow] potiuk commented on issue #29258: ComputeEngineSSHHook on parallel runs in Composer gives banner Error reading SSH protocol banner

2023-02-19 Thread via GitHub
potiuk commented on issue #29258: URL: https://github.com/apache/airflow/issues/29258#issuecomment-1436158237 > @Taragolis Can we have somebody to assist on the banner timeout errors? Its coming inspite of maintaining banner timeout parameters in Compute Engine SSH Operator Also if

[GitHub] [airflow] potiuk commented on pull request #29409: Fix Rest API update user output

2023-02-19 Thread via GitHub
potiuk commented on PR #29409: URL: https://github.com/apache/airflow/pull/29409#issuecomment-1436157344 @ephraimbuddy ? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

[GitHub] [airflow] takersk commented on issue #29561: CeleryKubernetesExecutor @task(queue="kubernetes")  not work

2023-02-19 Thread via GitHub
takersk commented on issue #29561: URL: https://github.com/apache/airflow/issues/29561#issuecomment-1436153289 @hussein-awala yes!! ``` import os from datetime import datetime, timedelta from airflow import DAG from common.enum.enum import DagEnum,

[airflow] branch main updated: Include Rancher in Helm chart install instructions (#28416)

2023-02-19 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new c5d548b2d9 Include Rancher in Helm chart install

[GitHub] [airflow] boring-cyborg[bot] commented on pull request #28416: Include Rancher in Helm chart install instructions

2023-02-19 Thread via GitHub
boring-cyborg[bot] commented on PR #28416: URL: https://github.com/apache/airflow/pull/28416#issuecomment-1436153155 Awesome work, congrats on your first merged pull request! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [airflow] potiuk merged pull request #28416: Include Rancher in Helm chart install instructions

2023-02-19 Thread via GitHub
potiuk merged PR #28416: URL: https://github.com/apache/airflow/pull/28416 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [airflow] potiuk commented on pull request #28626: Introduce logging integration for Apache WebHDFS

2023-02-19 Thread via GitHub
potiuk commented on PR #28626: URL: https://github.com/apache/airflow/pull/28626#issuecomment-1436151044 > Can we leave it outside the scope of this PR? Likely yes. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

[GitHub] [airflow] potiuk commented on pull request #29042: Clarify graph view dynamic task labels

2023-02-19 Thread via GitHub
potiuk commented on PR #29042: URL: https://github.com/apache/airflow/pull/29042#issuecomment-1436150282 Sure. You can build your own airflow package and install it. Look at the INSTALL and https://github.com/apache/airflow/blob/main/INSTALL#L114 . You can also use breeze CLI

[GitHub] [airflow] potiuk commented on pull request #27264: Attempt to add Python 3.11 support

2023-02-19 Thread via GitHub
potiuk commented on PR #27264: URL: https://github.com/apache/airflow/pull/27264#issuecomment-1436149095 > @potiuk any updated on this? No. Not yet. It's not the highest priority - but if someone would like to pick it up from here in the meantime, that would be awesome. -- This is

[GitHub] [airflow] potiuk closed issue #29365: Airflow Workers Trying to Create Pods in Default Namespace

2023-02-19 Thread via GitHub
potiuk closed issue #29365: Airflow Workers Trying to Create Pods in Default Namespace URL: https://github.com/apache/airflow/issues/29365 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[airflow] branch main updated: Missing timezone in next_dagrun in parameterized docs (#29362)

2023-02-19 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new e9e3f2c8c7 Missing timezone in next_dagrun in

[GitHub] [airflow] boring-cyborg[bot] commented on pull request #29362: Missing timezone in next_dag_run in parameterized docs

2023-02-19 Thread via GitHub
boring-cyborg[bot] commented on PR #29362: URL: https://github.com/apache/airflow/pull/29362#issuecomment-1436147137 Awesome work, congrats on your first merged pull request! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [airflow] potiuk merged pull request #29362: Missing timezone in next_dag_run in parameterized docs

2023-02-19 Thread via GitHub
potiuk merged PR #29362: URL: https://github.com/apache/airflow/pull/29362 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [airflow] potiuk merged pull request #29146: Add ability to access context in functions decorated by task.sensor

2023-02-19 Thread via GitHub
potiuk merged PR #29146: URL: https://github.com/apache/airflow/pull/29146 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[airflow] branch main updated (0a4184e34c -> 2d3cc504db)

2023-02-19 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git from 0a4184e34c Add note about workaround to potential error with argparse to hdfs (#29614) add 2d3cc504db Remove

[GitHub] [airflow] boring-cyborg[bot] commented on pull request #29146: Add ability to access context in functions decorated by task.sensor

2023-02-19 Thread via GitHub
boring-cyborg[bot] commented on PR #29146: URL: https://github.com/apache/airflow/pull/29146#issuecomment-1436145128 Awesome work, congrats on your first merged pull request! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [airflow] potiuk closed issue #29137: Add ability to access context in functions decorated by task.sensor

2023-02-19 Thread via GitHub
potiuk closed issue #29137: Add ability to access context in functions decorated by task.sensor URL: https://github.com/apache/airflow/issues/29137 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [airflow] potiuk closed issue #29339: Failing to execute successive Dynamic Task Group Mapping

2023-02-19 Thread via GitHub
potiuk closed issue #29339: Failing to execute successive Dynamic Task Group Mapping URL: https://github.com/apache/airflow/issues/29339 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [airflow] potiuk closed issue #28836: Airflow Scheduler Hangs After Failed K8 API Call

2023-02-19 Thread via GitHub
potiuk closed issue #28836: Airflow Scheduler Hangs After Failed K8 API Call URL: https://github.com/apache/airflow/issues/28836 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

[GitHub] [airflow] github-actions[bot] commented on pull request #28265: ability to set referenceId in ECSRunTaskOperator

2023-02-19 Thread via GitHub
github-actions[bot] commented on PR #28265: URL: https://github.com/apache/airflow/pull/28265#issuecomment-1436142743 This pull request has been automatically marked as stale because it has not had recent activity. It will be closed in 5 days if no further activity occurs. Thank you for

[GitHub] [airflow] potiuk commented on pull request #26269: Added Feature - Glue job continuously log printing in Airflow task logs

2023-02-19 Thread via GitHub
potiuk commented on PR #26269: URL: https://github.com/apache/airflow/pull/26269#issuecomment-1436141544 > Hi is there any update on this PR, thanks! Not really - as you see it's closed. But if you want to pick it up where the original author left it - feel free to continue in a new

[GitHub] [airflow] hussein-awala commented on issue #29555: Dag Fails while creating a Dynamic Tasks using airflow variables.

2023-02-19 Thread via GitHub
hussein-awala commented on issue #29555: URL: https://github.com/apache/airflow/issues/29555#issuecomment-1436137871 Can you provide some code for the mapped task group? It's not clear to me why would you map different TIs based on the batches list instead of mapping the same TIs with

[GitHub] [airflow] potiuk commented on issue #29275: Allow user DAGs to emit DagWarning (or similar)

2023-02-19 Thread via GitHub
potiuk commented on issue #29275: URL: https://github.com/apache/airflow/issues/29275#issuecomment-1436133430 Yes it would be interesting feature to add. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

[GitHub] [airflow] potiuk commented on issue #25297: on_failure_callback is not called when task is terminated externally

2023-02-19 Thread via GitHub
potiuk commented on issue #25297: URL: https://github.com/apache/airflow/issues/25297#issuecomment-1436130977 I think if If you can provide reproducible case with logs @seub from the last Airflow version, that would be good new issue. I guess the case is different than explained originally

[GitHub] [airflow] hussein-awala commented on a diff in pull request #29270: Adding possibility for annotations in logs pvc

2023-02-19 Thread via GitHub
hussein-awala commented on code in PR #29270: URL: https://github.com/apache/airflow/pull/29270#discussion_r335700 ## chart/templates/logs-persistent-volume-claim.yaml: ## @@ -29,6 +29,10 @@ metadata: {{- with .Values.labels }} {{- toYaml . | nindent 4 }} {{-

[GitHub] [airflow] hussein-awala commented on issue #29557: Execution Timeout is not working properly on airflow 2.5.0

2023-02-19 Thread via GitHub
hussein-awala commented on issue #29557: URL: https://github.com/apache/airflow/issues/29557#issuecomment-1436106024 Airflow raises `AirflowTaskTimeout` exception when the task timed out, and your code can catch this exception and handle it if needed. Since you have `[2023-02-15,

[GitHub] [airflow] hussein-awala commented on issue #29582: Cannot trigger the same dag_id within the same second

2023-02-19 Thread via GitHub
hussein-awala commented on issue #29582: URL: https://github.com/apache/airflow/issues/29582#issuecomment-1436093181 > Cannot trigger the same dag_id within the same second You cannot trigger the dag multiple time with the same execution date because there are two unique constraint

[airflow] branch main updated: Add note about workaround to potential error with argparse to hdfs (#29614)

2023-02-19 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 0a4184e34c Add note about workaround to potential

[GitHub] [airflow] potiuk closed issue #22689: HDFS provider causes TypeError: __init__() got an unexpected keyword argument 'encoding'

2023-02-19 Thread via GitHub
potiuk closed issue #22689: HDFS provider causes TypeError: __init__() got an unexpected keyword argument 'encoding' URL: https://github.com/apache/airflow/issues/22689 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

[GitHub] [airflow] potiuk merged pull request #29614: Add note about workaround to potential error with argparse to hdfs

2023-02-19 Thread via GitHub
potiuk merged PR #29614: URL: https://github.com/apache/airflow/pull/29614 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [airflow] hussein-awala commented on issue #29561: CeleryKubernetesExecutor @task(queue="kubernetes")  not work

2023-02-19 Thread via GitHub
hussein-awala commented on issue #29561: URL: https://github.com/apache/airflow/issues/29561#issuecomment-1436086556 Can you add your dag code? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [airflow] Taragolis closed pull request #29616: Refactor docker-compose quick start test

2023-02-19 Thread via GitHub
Taragolis closed pull request #29616: Refactor docker-compose quick start test URL: https://github.com/apache/airflow/pull/29616 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

[airflow] branch main updated: convert moment with timezone to UTC instead of raising an exception (#29606)

2023-02-19 Thread taragolis
This is an automated email from the ASF dual-hosted git repository. taragolis pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 79c07e3fc5 convert moment with timezone to UTC

[GitHub] [airflow] Taragolis closed issue #29576: DateTimeSensorAsync breaks if target_time is timezone-aware

2023-02-19 Thread via GitHub
Taragolis closed issue #29576: DateTimeSensorAsync breaks if target_time is timezone-aware URL: https://github.com/apache/airflow/issues/29576 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [airflow] Taragolis merged pull request #29606: convert moment with timezone to UTC instead of raising an exception

2023-02-19 Thread via GitHub
Taragolis merged PR #29606: URL: https://github.com/apache/airflow/pull/29606 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [airflow] Taragolis opened a new pull request, #29616: Refactor docker-compose quick start test

2023-02-19 Thread via GitHub
Taragolis opened a new pull request, #29616: URL: https://github.com/apache/airflow/pull/29616 There is quite a few tests failed with the same error during the PR CI pipeline ```console > assert dag_state == "success" E AssertionError:

[GitHub] [airflow] dimberman commented on a diff in pull request #29580: Allow to specify which connection, variable or config are being looked up in the backend using *_lookup_pattern parameters

2023-02-19 Thread via GitHub
dimberman commented on code in PR #29580: URL: https://github.com/apache/airflow/pull/29580#discussion_r314542 ## airflow/providers/amazon/aws/secrets/secrets_manager.py: ## @@ -84,12 +88,24 @@ class SecretsManagerBackend(BaseSecretsBackend, LoggingMixin): :param

[GitHub] [airflow] dimberman commented on a diff in pull request #29606: convert moment with timezone to UTC instead of raising an exception

2023-02-19 Thread via GitHub
dimberman commented on code in PR #29606: URL: https://github.com/apache/airflow/pull/29606#discussion_r314236 ## airflow/triggers/temporal.py: ## @@ -41,10 +41,8 @@ def __init__(self, moment: datetime.datetime): # Make sure it's in UTC elif moment.tzinfo

[GitHub] [airflow] dimberman commented on pull request #29609: Decouple production executor warning in dags UI

2023-02-19 Thread via GitHub
dimberman commented on PR #29609: URL: https://github.com/apache/airflow/pull/29609#issuecomment-1436081501 @o-nikolas looks like there are a few failing tests. Once these are fixed would be glad to merge. -- This is an automated message from the Apache Git Service. To respond to the

[GitHub] [airflow] potiuk commented on pull request #29614: Add note about workaround to potential error with argparse to hdfs

2023-02-19 Thread via GitHub
potiuk commented on PR #29614: URL: https://github.com/apache/airflow/pull/29614#issuecomment-1436067561 It's a niche case but yes. We could detect it -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[airflow] branch main updated (a21c17bc07 -> fd000684d0)

2023-02-19 Thread taragolis
This is an automated email from the ASF dual-hosted git repository. taragolis pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git from a21c17bc07 Specific use-case: adding packages via requirements.txt in compose (#29598) add fd000684d0 SSH

[GitHub] [airflow] Taragolis closed issue #29282: Ssh connection extra parameter conn_timeout doesn't work with ssh operator

2023-02-19 Thread via GitHub
Taragolis closed issue #29282: Ssh connection extra parameter conn_timeout doesn't work with ssh operator URL: https://github.com/apache/airflow/issues/29282 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

[GitHub] [airflow] Taragolis merged pull request #29347: SSH Provider: Add cmd_timeout to ssh connection extra

2023-02-19 Thread via GitHub
Taragolis merged PR #29347: URL: https://github.com/apache/airflow/pull/29347 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [airflow] potiuk closed issue #29613: Importing CreateCustomContainerTrainingJobOperator takes too much time

2023-02-19 Thread via GitHub
potiuk closed issue #29613: Importing CreateCustomContainerTrainingJobOperator takes too much time URL: https://github.com/apache/airflow/issues/29613 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [airflow] potiuk commented on pull request #28968: Required Param requires None to be explicitly passed as default

2023-02-19 Thread via GitHub
potiuk commented on PR #28968: URL: https://github.com/apache/airflow/pull/28968#issuecomment-1436057533 > @potiuk @aamster following up on my last comment I do not know by heart - I would have to go to details, and I think it's a good exercise to do if you want to make sure the fix

[GitHub] [airflow] Taragolis commented on issue #29613: Importing CreateCustomContainerTrainingJobOperator takes too much time

2023-02-19 Thread via GitHub
Taragolis commented on issue #29613: URL: https://github.com/apache/airflow/issues/29613#issuecomment-1436057506 Check it locally, most of the time spend during this import: `from google.cloud.aiplatform.models import Model`

[GitHub] [airflow] potiuk commented on pull request #27841: fix: add pre-commit to ban urlparse

2023-02-19 Thread via GitHub
potiuk commented on PR #27841: URL: https://github.com/apache/airflow/pull/27841#issuecomment-1436056893 You need rebase and resolve conflicts @wonhyeongseo -> this was likely temporary issue. -- This is an automated message from the Apache Git Service. To respond to the message, please

[GitHub] [airflow] potiuk commented on pull request #29347: SSH Provider: Add cmd_timeout to ssh connection extra

2023-02-19 Thread via GitHub
potiuk commented on PR #29347: URL: https://github.com/apache/airflow/pull/29347#issuecomment-1436056370 @Taragolis - are you ok ? This one LGTM. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[airflow] branch main updated (83cdd5031b -> a21c17bc07)

2023-02-19 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git from 83cdd5031b Handling error on cluster policy itself (#29056) add a21c17bc07 Specific use-case: adding packages via

[GitHub] [airflow] potiuk merged pull request #29598: Specific use-case: adding packages via requirements.txt in compose

2023-02-19 Thread via GitHub
potiuk merged PR #29598: URL: https://github.com/apache/airflow/pull/29598 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [airflow] potiuk commented on issue #22689: HDFS provider causes TypeError: __init__() got an unexpected keyword argument 'encoding'

2023-02-19 Thread via GitHub
potiuk commented on issue #22689: URL: https://github.com/apache/airflow/issues/22689#issuecomment-1436055249 This is what #29614 does. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [airflow] potiuk opened a new pull request, #29614: Add note about workaround to potential error with argparse to hdfs

2023-02-19 Thread via GitHub
potiuk opened a new pull request, #29614: URL: https://github.com/apache/airflow/pull/29614 Instead of removing or reimplementing hdfs provider - in the event of a possibility to break argparse, we simply add a note in the provider documentation to remove argprase if such problem happens.

[GitHub] [airflow] Taragolis commented on pull request #29511: Add Landing Times entry to UI docs

2023-02-19 Thread via GitHub
Taragolis commented on PR #29511: URL: https://github.com/apache/airflow/pull/29511#issuecomment-1436048654 @john-jac could you fix Static Checks, the only way in your case it is use `pre-commit`, see [Static code

[GitHub] [airflow] Aakcht commented on pull request #29347: SSH Provider: Add cmd_timeout to ssh connection extra

2023-02-19 Thread via GitHub
Aakcht commented on PR #29347: URL: https://github.com/apache/airflow/pull/29347#issuecomment-1436048563 Looks like the unsuccessful check is not related to this PR. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

[GitHub] [airflow] Aakcht commented on a diff in pull request #29347: SSH Provider: Add cmd_timeout to ssh connection extra

2023-02-19 Thread via GitHub
Aakcht commented on code in PR #29347: URL: https://github.com/apache/airflow/pull/29347#discussion_r290081 ## airflow/providers/ssh/hooks/ssh.py: ## @@ -173,6 +182,14 @@ def __init__( if "conn_timeout" in extra_options and self.conn_timeout is None:

[GitHub] [airflow] RachitSharma2001 commented on pull request #29406: Add Fail Fast feature for DAGs

2023-02-19 Thread via GitHub
RachitSharma2001 commented on PR #29406: URL: https://github.com/apache/airflow/pull/29406#issuecomment-1436047140 Hi everybody, it looks like all the tests pass now. I was wondering if I could get some feedback on the approach I used for adding this feature. Does the approach look good,

[GitHub] [airflow] potiuk commented on pull request #29012: Fix airflow-webserver cannot create resource pods for k8s exceutor deployed using helm chart

2023-02-19 Thread via GitHub
potiuk commented on PR #29012: URL: https://github.com/apache/airflow/pull/29012#issuecomment-1436046117 So @snjypl are you going to work on it as explained by @jedcunningham ? Or maybe you would like to create theissue to fix the manual behaviour if you are not sure how to fix it? --

  1   2   >