[GitHub] [airflow] alexott commented on pull request #28950: Sensor for Databricks partition and table changes

2023-02-19 Thread via GitHub
alexott commented on PR #28950: URL: https://github.com/apache/airflow/pull/28950#issuecomment-1435939111 @eladkal yes, it will solve #21381 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the s

[GitHub] [airflow] Taragolis commented on issue #28940: Required `Param` requires `None` to be explicitly passed as default

2023-02-19 Thread via GitHub
Taragolis commented on issue #28940: URL: https://github.com/apache/airflow/issues/28940#issuecomment-1435939632 > Found out that you need to explicitly pass None as the default, otherwise it will crash with above. For me it make sense. If no default value provided than parameter is a

[GitHub] [airflow] Taragolis commented on pull request #28923: Add samba hook copyfile functionality

2023-02-19 Thread via GitHub
Taragolis commented on PR #28923: URL: https://github.com/apache/airflow/pull/28923#issuecomment-1435940206 > Unfortunately no, found another solution you fan actually get the samba client args from the hook and run the function No problem at all. -- This is an automated message fr

[GitHub] [airflow] Taragolis closed pull request #28923: Add samba hook copyfile functionality

2023-02-19 Thread via GitHub
Taragolis closed pull request #28923: Add samba hook copyfile functionality URL: https://github.com/apache/airflow/pull/28923 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

[GitHub] [airflow] alexott commented on a diff in pull request #28950: Sensor for Databricks partition and table changes

2023-02-19 Thread via GitHub
alexott commented on code in PR #28950: URL: https://github.com/apache/airflow/pull/28950#discussion_r202296 ## airflow/providers/databricks/sensors/databricks_sql.py: ## @@ -0,0 +1,136 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributo

[GitHub] [airflow] Taragolis commented on a diff in pull request #29347: SSH Provider: Add cmd_timeout to ssh connection extra

2023-02-19 Thread via GitHub
Taragolis commented on code in PR #29347: URL: https://github.com/apache/airflow/pull/29347#discussion_r203656 ## airflow/providers/ssh/hooks/ssh.py: ## @@ -173,6 +182,14 @@ def __init__( if "conn_timeout" in extra_options and self.conn_timeout is None:

[GitHub] [airflow] confusedpublic commented on pull request #28545: Fix Provider Amazon Test Connection when using custom endpoint url

2023-02-19 Thread via GitHub
confusedpublic commented on PR #28545: URL: https://github.com/apache/airflow/pull/28545#issuecomment-1435943597 Sorry, I've been busy. I can look at it again this week. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

[GitHub] [airflow] Taragolis commented on a diff in pull request #29606: convert moment with timezone to UTC instead of raising an exception

2023-02-19 Thread via GitHub
Taragolis commented on code in PR #29606: URL: https://github.com/apache/airflow/pull/29606#discussion_r206098 ## airflow/triggers/temporal.py: ## @@ -41,10 +41,8 @@ def __init__(self, moment: datetime.datetime): # Make sure it's in UTC elif moment.tzinfo i

[GitHub] [airflow] Taragolis commented on pull request #28545: Fix Provider Amazon Test Connection when using custom endpoint url

2023-02-19 Thread via GitHub
Taragolis commented on PR #28545: URL: https://github.com/apache/airflow/pull/28545#issuecomment-1435948533 > Sorry, I've been busy. No problem, it's just a clarification -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [airflow] Taragolis commented on pull request #29056: Handling error on cluster policy itself

2023-02-19 Thread via GitHub
Taragolis commented on PR #29056: URL: https://github.com/apache/airflow/pull/29056#issuecomment-1435950001 Seems like tests should be fixed, because right now this changes broke expected exception for TimeTable. And also need to fix [Static code checks](https://github.com/apache/airflow

[GitHub] [airflow] Aakcht commented on a diff in pull request #29347: SSH Provider: Add cmd_timeout to ssh connection extra

2023-02-19 Thread via GitHub
Aakcht commented on code in PR #29347: URL: https://github.com/apache/airflow/pull/29347#discussion_r214874 ## airflow/providers/ssh/hooks/ssh.py: ## @@ -173,6 +182,14 @@ def __init__( if "conn_timeout" in extra_options and self.conn_timeout is None:

[GitHub] [airflow] boring-cyborg[bot] commented on issue #29611: SQLToGoogleSheetsOperator not executing

2023-02-19 Thread via GitHub
boring-cyborg[bot] commented on issue #29611: URL: https://github.com/apache/airflow/issues/29611#issuecomment-1435962957 Thanks for opening your first issue here! Be sure to follow the issue template! -- This is an automated message from the Apache Git Service. To respond to the mess

[GitHub] [airflow] tegardp opened a new issue, #29611: SQLToGoogleSheetsOperator not executing

2023-02-19 Thread via GitHub
tegardp opened a new issue, #29611: URL: https://github.com/apache/airflow/issues/29611 ### Apache Airflow version 2.5.1 ### What happened ``` [2023-02-19 11:16:22,032] {dagrun.py:658} INFO - DagRun Finished: dag_id=xxx, execution_date=2022-09-23T00:00:00+00:00,

[GitHub] [airflow] IKholopov commented on a diff in pull request #29518: [DRAFT] Google Providers - Fix _MethodDefault deepcopy failure

2023-02-19 Thread via GitHub
IKholopov commented on code in PR #29518: URL: https://github.com/apache/airflow/pull/29518#discussion_r223973 ## airflow/providers/google/common/operators/base_google.py: ## @@ -0,0 +1,34 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one +# or more contrib

[GitHub] [airflow] tegardp closed issue #29611: SQLToGoogleSheetsOperator not executing properly

2023-02-19 Thread via GitHub
tegardp closed issue #29611: SQLToGoogleSheetsOperator not executing properly URL: https://github.com/apache/airflow/issues/29611 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

[GitHub] [airflow] tegardp commented on issue #29611: SQLToGoogleSheetsOperator not executing properly

2023-02-19 Thread via GitHub
tegardp commented on issue #29611: URL: https://github.com/apache/airflow/issues/29611#issuecomment-1435972877 test execution date was earlier than start_date -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL abo

[GitHub] [airflow] okayhooni commented on pull request #29056: Handling error on cluster policy itself

2023-02-19 Thread via GitHub
okayhooni commented on PR #29056: URL: https://github.com/apache/airflow/pull/29056#issuecomment-1435978575 > Seems like tests should be fixed, because right now this changes broke expected exception for TimeTable. And also need to fix [Static code checks](https://github.com/apache/airflow/

[GitHub] [airflow] potiuk commented on issue #28880: Can't configure Kubernetes and Celery workers in Helm Chart

2023-02-19 Thread via GitHub
potiuk commented on issue #28880: URL: https://github.com/apache/airflow/issues/28880#issuecomment-1435980218 Feel free -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To u

[GitHub] [airflow] potiuk commented on pull request #28799: introduce dag processor job fix #27140

2023-02-19 Thread via GitHub
potiuk commented on PR #28799: URL: https://github.com/apache/airflow/pull/28799#issuecomment-1435984665 Sorry for the delay:: 1. I think it would be good to maka a PR with this change. 2. Yeah. rebase and add it. @mhenc - also can yoy please let us know what you think about

[GitHub] [airflow] potiuk commented on issue #26429: ExternalTaskSensor: An option to use the data_interval variables rather than execution_date

2023-02-19 Thread via GitHub
potiuk commented on issue #26429: URL: https://github.com/apache/airflow/issues/26429#issuecomment-1435985080 Opening a PR is a good idea. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the spe

[GitHub] [airflow] potiuk commented on pull request #28256: Include full path to Python files under zip path while clearing import errors.

2023-02-19 Thread via GitHub
potiuk commented on PR #28256: URL: https://github.com/apache/airflow/pull/28256#issuecomment-1435985318 @ephraimbuddy - WDYT? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific commen

[GitHub] [airflow] justabaka commented on issue #29365: Airflow Workers Trying to Create Pods in Default Namespace

2023-02-19 Thread via GitHub
justabaka commented on issue #29365: URL: https://github.com/apache/airflow/issues/29365#issuecomment-1435988327 I have a similar issue but in my case the namespace name is a correct one. No signs of the proper RBAC for the scheduler SA though: `pods is forbidden: User \"system:servic

[GitHub] [airflow] Taragolis commented on a diff in pull request #29347: SSH Provider: Add cmd_timeout to ssh connection extra

2023-02-19 Thread via GitHub
Taragolis commented on code in PR #29347: URL: https://github.com/apache/airflow/pull/29347#discussion_r243238 ## airflow/providers/ssh/hooks/ssh.py: ## @@ -173,6 +182,14 @@ def __init__( if "conn_timeout" in extra_options and self.conn_timeout is None:

[GitHub] [airflow] potiuk commented on pull request #28125: Set limits for XCOM pod

2023-02-19 Thread via GitHub
potiuk commented on PR #28125: URL: https://github.com/apache/airflow/pull/28125#issuecomment-1436006357 Now. There are some confflicts to resolve - It's been some time it was out (and there is some backlog of issues/PRs so this takes longer than usual) - can you please rebase and fix the c

[GitHub] [airflow] potiuk merged pull request #28930: Make json and yaml available in templates

2023-02-19 Thread via GitHub
potiuk merged PR #28930: URL: https://github.com/apache/airflow/pull/28930 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.a

[GitHub] [airflow] potiuk closed issue #27079: Option to deserialize JSON from last log line in BashOperator and DockerOperator before sending to XCom

2023-02-19 Thread via GitHub
potiuk closed issue #27079: Option to deserialize JSON from last log line in BashOperator and DockerOperator before sending to XCom URL: https://github.com/apache/airflow/issues/27079 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitH

[airflow] branch main updated: Make json and yaml available in templates (#28930)

2023-02-19 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new ffdc696942 Make json and yaml available in template

[GitHub] [airflow] Taragolis commented on a diff in pull request #29016: Fix leak sensitive field via V1EnvVar on exception

2023-02-19 Thread via GitHub
Taragolis commented on code in PR #29016: URL: https://github.com/apache/airflow/pull/29016#discussion_r255261 ## kubernetes_tests/test_kubernetes_pod_operator.py: ## @@ -1283,3 +1283,39 @@ class MyK8SPodOperator(KubernetesPodOperator): == "apple-sauce"

[GitHub] [airflow] Taragolis commented on pull request #28846: Updated app to support configuring the caching hash method for FIPS

2023-02-19 Thread via GitHub
Taragolis commented on PR #28846: URL: https://github.com/apache/airflow/pull/28846#issuecomment-1436011429 @uranusjr @jedcunningham are you satisfied with the changes? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use t

[GitHub] [airflow] potiuk closed issue #29167: Airflow webserver emits malformed HTTP Response Status Line when endpoint/resource is not found

2023-02-19 Thread via GitHub
potiuk closed issue #29167: Airflow webserver emits malformed HTTP Response Status Line when endpoint/resource is not found URL: https://github.com/apache/airflow/issues/29167 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and u

[GitHub] [airflow] potiuk commented on pull request #29156: Add to provider.FTP possibility to use file-like objects.

2023-02-19 Thread via GitHub
potiuk commented on PR #29156: URL: https://github.com/apache/airflow/pull/29156#issuecomment-1436014426 I agree with @Taragolis - that passing file-like objects in FileTransferOperator is not going to work. Each operator when executing will run in its own process/interpreter and there is s

[GitHub] [airflow] potiuk closed pull request #29156: Add to provider.FTP possibility to use file-like objects.

2023-02-19 Thread via GitHub
potiuk closed pull request #29156: Add to provider.FTP possibility to use file-like objects. URL: https://github.com/apache/airflow/pull/29156 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the spe

[GitHub] [airflow] potiuk closed issue #29153: FTP(S)FileTransmitOperator class should allow to use file-like buffer.

2023-02-19 Thread via GitHub
potiuk closed issue #29153: FTP(S)FileTransmitOperator class should allow to use file-like buffer. URL: https://github.com/apache/airflow/issues/29153 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [airflow] potiuk commented on issue #29153: FTP(S)FileTransmitOperator class should allow to use file-like buffer.

2023-02-19 Thread via GitHub
potiuk commented on issue #29153: URL: https://github.com/apache/airflow/issues/29153#issuecomment-1436014777 As described in #29156 - this is not going to work - this is not how airflow works, and there are already other solution (building custom transfer operators or using TaskFlow and Ho

[GitHub] [airflow] potiuk closed issue #29178: Add `output` format to missing cli commands

2023-02-19 Thread via GitHub
potiuk closed issue #29178: Add `output` format to missing cli commands URL: https://github.com/apache/airflow/issues/29178 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To u

[GitHub] [airflow] potiuk merged pull request #29224: add output format arg for `cli.dags.trigger`

2023-02-19 Thread via GitHub
potiuk merged PR #29224: URL: https://github.com/apache/airflow/pull/29224 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.a

[airflow] branch main updated: add output format arg for `cli.dags.trigger` (#29224)

2023-02-19 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 60fc407911 add output format arg for `cli.dags.trig

[GitHub] [airflow] hussein-awala commented on pull request #29000: Skip KubernetesPodOperator task when it returns a provided exit code

2023-02-19 Thread via GitHub
hussein-awala commented on PR #29000: URL: https://github.com/apache/airflow/pull/29000#issuecomment-1436023616 This needs some tests for deferred mode, I convert it to draft -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

[GitHub] [airflow] hussein-awala commented on a diff in pull request #29606: convert moment with timezone to UTC instead of raising an exception

2023-02-19 Thread via GitHub
hussein-awala commented on code in PR #29606: URL: https://github.com/apache/airflow/pull/29606#discussion_r270818 ## airflow/triggers/temporal.py: ## @@ -41,10 +41,8 @@ def __init__(self, moment: datetime.datetime): # Make sure it's in UTC elif moment.tzin

[GitHub] [airflow] Taragolis merged pull request #29056: Handling error on cluster policy itself

2023-02-19 Thread via GitHub
Taragolis merged PR #29056: URL: https://github.com/apache/airflow/pull/29056 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflo

[airflow] branch main updated (60fc407911 -> 83cdd5031b)

2023-02-19 Thread taragolis
This is an automated email from the ASF dual-hosted git repository. taragolis pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git from 60fc407911 add output format arg for `cli.dags.trigger` (#29224) add 83cdd5031b Handling error on cluster polic

[GitHub] [airflow] Taragolis commented on a diff in pull request #29606: convert moment with timezone to UTC instead of raising an exception

2023-02-19 Thread via GitHub
Taragolis commented on code in PR #29606: URL: https://github.com/apache/airflow/pull/29606#discussion_r272142 ## airflow/triggers/temporal.py: ## @@ -41,10 +41,8 @@ def __init__(self, moment: datetime.datetime): # Make sure it's in UTC elif moment.tzinfo i

[GitHub] [airflow] Taragolis commented on pull request #29394: Add a new param for bg operators to support additional actions when resource exists

2023-02-19 Thread via GitHub
Taragolis commented on PR #29394: URL: https://github.com/apache/airflow/pull/29394#issuecomment-1436030870 cc: @mfjackson if you have a time could you also have a look. Is it what you expected for solve #29301 ? -- This is an automated message from the Apache Git Service. To respond to t

[GitHub] [airflow] hussein-awala commented on issue #28940: Required `Param` requires `None` to be explicitly passed as default

2023-02-19 Thread via GitHub
hussein-awala commented on issue #28940: URL: https://github.com/apache/airflow/issues/28940#issuecomment-1436032391 I agree with @Taragolis, we should check if the required parameters (with no default value) are provided or not, and fail the dag run if they are not provided. I'm working

[GitHub] [airflow] shahar1 opened a new issue, #29613: Importing CreateCustomContainerTrainingJobOperator takes too much time

2023-02-19 Thread via GitHub
shahar1 opened a new issue, #29613: URL: https://github.com/apache/airflow/issues/29613 ### Apache Airflow version Other Airflow 2 version (please specify below) ### What happened When importing CreateCustomContainerTrainingJobOperator it seems to take 3-4 seconds just t

[GitHub] [airflow] boring-cyborg[bot] commented on issue #29613: Importing CreateCustomContainerTrainingJobOperator takes too much time

2023-02-19 Thread via GitHub
boring-cyborg[bot] commented on issue #29613: URL: https://github.com/apache/airflow/issues/29613#issuecomment-1436038316 Thanks for opening your first issue here! Be sure to follow the issue template! -- This is an automated message from the Apache Git Service. To respond to the mess

[airflow] branch constraints-main updated: Updating constraints. Github run id:4216874752

2023-02-19 Thread github-bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch constraints-main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/constraints-main by this push: new bc6dee3637 Updating con

[GitHub] [airflow] hussein-awala commented on issue #28940: Required `Param` requires `None` to be explicitly passed as default

2023-02-19 Thread via GitHub
hussein-awala commented on issue #28940: URL: https://github.com/apache/airflow/issues/28940#issuecomment-1436040011 It seems like currently we don't support required params, to fix this, I think we need to differentiate between parsing the params during the dag parsing/creation and parsing

[GitHub] [airflow] potiuk commented on pull request #29598: Specific use-case: adding packages via requirements.txt in compose

2023-02-19 Thread via GitHub
potiuk commented on PR #29598: URL: https://github.com/apache/airflow/pull/29598#issuecomment-1436044348 All comments reflected and docs building fixed. Merging -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL a

[GitHub] [airflow] PodbiAtGS commented on issue #29393: S3TaskHandler continuously returns "*** Falling back to local log" even if log_pos is provided when log not in s3

2023-02-19 Thread via GitHub
PodbiAtGS commented on issue #29393: URL: https://github.com/apache/airflow/issues/29393#issuecomment-1436044382 Same here. Any ideas? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [airflow] potiuk commented on issue #28010: Airflow does not pass through Celery's support for Redis Sentinel over SSL.

2023-02-19 Thread via GitHub
potiuk commented on issue #28010: URL: https://github.com/apache/airflow/issues/28010#issuecomment-1436044858 I hope then someone using sentinel can further debug it and solve. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub a

[GitHub] [airflow] potiuk commented on pull request #29171: Added Snowflake provider to the Docker image

2023-02-19 Thread via GitHub
potiuk commented on PR #29171: URL: https://github.com/apache/airflow/pull/29171#issuecomment-1436045506 Do you still want to follow-up o that one @francescomucio ? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the U

[GitHub] [airflow] potiuk commented on pull request #29012: Fix airflow-webserver cannot create resource pods for k8s exceutor deployed using helm chart

2023-02-19 Thread via GitHub
potiuk commented on PR #29012: URL: https://github.com/apache/airflow/pull/29012#issuecomment-1436046117 So @snjypl are you going to work on it as explained by @jedcunningham ? Or maybe you would like to create theissue to fix the manual behaviour if you are not sure how to fix it? -- Th

[GitHub] [airflow] RachitSharma2001 commented on pull request #29406: Add Fail Fast feature for DAGs

2023-02-19 Thread via GitHub
RachitSharma2001 commented on PR #29406: URL: https://github.com/apache/airflow/pull/29406#issuecomment-1436047140 Hi everybody, it looks like all the tests pass now. I was wondering if I could get some feedback on the approach I used for adding this feature. Does the approach look good, or

[GitHub] [airflow] Aakcht commented on a diff in pull request #29347: SSH Provider: Add cmd_timeout to ssh connection extra

2023-02-19 Thread via GitHub
Aakcht commented on code in PR #29347: URL: https://github.com/apache/airflow/pull/29347#discussion_r290081 ## airflow/providers/ssh/hooks/ssh.py: ## @@ -173,6 +182,14 @@ def __init__( if "conn_timeout" in extra_options and self.conn_timeout is None:

[GitHub] [airflow] Aakcht commented on pull request #29347: SSH Provider: Add cmd_timeout to ssh connection extra

2023-02-19 Thread via GitHub
Aakcht commented on PR #29347: URL: https://github.com/apache/airflow/pull/29347#issuecomment-1436048563 Looks like the unsuccessful check is not related to this PR. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

[GitHub] [airflow] Taragolis commented on pull request #29511: Add Landing Times entry to UI docs

2023-02-19 Thread via GitHub
Taragolis commented on PR #29511: URL: https://github.com/apache/airflow/pull/29511#issuecomment-1436048654 @john-jac could you fix Static Checks, the only way in your case it is use `pre-commit`, see [Static code checks](https://github.com/apache/airflow/blob/main/STATIC_CODE_CHECKS.rst#id

[GitHub] [airflow] potiuk opened a new pull request, #29614: Add note about workaround to potential error with argparse to hdfs

2023-02-19 Thread via GitHub
potiuk opened a new pull request, #29614: URL: https://github.com/apache/airflow/pull/29614 Instead of removing or reimplementing hdfs provider - in the event of a possibility to break argparse, we simply add a note in the provider documentation to remove argprase if such problem happens.

[GitHub] [airflow] potiuk commented on issue #22689: HDFS provider causes TypeError: __init__() got an unexpected keyword argument 'encoding'

2023-02-19 Thread via GitHub
potiuk commented on issue #22689: URL: https://github.com/apache/airflow/issues/22689#issuecomment-1436055249 This is what #29614 does. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specifi

[GitHub] [airflow] potiuk merged pull request #29598: Specific use-case: adding packages via requirements.txt in compose

2023-02-19 Thread via GitHub
potiuk merged PR #29598: URL: https://github.com/apache/airflow/pull/29598 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.a

[airflow] branch main updated (83cdd5031b -> a21c17bc07)

2023-02-19 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git from 83cdd5031b Handling error on cluster policy itself (#29056) add a21c17bc07 Specific use-case: adding packages via

[GitHub] [airflow] potiuk commented on pull request #29347: SSH Provider: Add cmd_timeout to ssh connection extra

2023-02-19 Thread via GitHub
potiuk commented on PR #29347: URL: https://github.com/apache/airflow/pull/29347#issuecomment-1436056370 @Taragolis - are you ok ? This one LGTM. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [airflow] potiuk commented on pull request #27841: fix: add pre-commit to ban urlparse

2023-02-19 Thread via GitHub
potiuk commented on PR #27841: URL: https://github.com/apache/airflow/pull/27841#issuecomment-1436056893 You need rebase and resolve conflicts @wonhyeongseo -> this was likely temporary issue. -- This is an automated message from the Apache Git Service. To respond to the message, please l

[GitHub] [airflow] Taragolis commented on issue #29613: Importing CreateCustomContainerTrainingJobOperator takes too much time

2023-02-19 Thread via GitHub
Taragolis commented on issue #29613: URL: https://github.com/apache/airflow/issues/29613#issuecomment-1436057506 Check it locally, most of the time spend during this import: `from google.cloud.aiplatform.models import Model` ![image](https://user-images.githubusercontent.com/39

[GitHub] [airflow] potiuk commented on pull request #28968: Required Param requires None to be explicitly passed as default

2023-02-19 Thread via GitHub
potiuk commented on PR #28968: URL: https://github.com/apache/airflow/pull/28968#issuecomment-1436057533 > @potiuk @aamster following up on my last comment I do not know by heart - I would have to go to details, and I think it's a good exercise to do if you want to make sure the fix w

[GitHub] [airflow] potiuk closed issue #29613: Importing CreateCustomContainerTrainingJobOperator takes too much time

2023-02-19 Thread via GitHub
potiuk closed issue #29613: Importing CreateCustomContainerTrainingJobOperator takes too much time URL: https://github.com/apache/airflow/issues/29613 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [airflow] Taragolis merged pull request #29347: SSH Provider: Add cmd_timeout to ssh connection extra

2023-02-19 Thread via GitHub
Taragolis merged PR #29347: URL: https://github.com/apache/airflow/pull/29347 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflo

[GitHub] [airflow] Taragolis closed issue #29282: Ssh connection extra parameter conn_timeout doesn't work with ssh operator

2023-02-19 Thread via GitHub
Taragolis closed issue #29282: Ssh connection extra parameter conn_timeout doesn't work with ssh operator URL: https://github.com/apache/airflow/issues/29282 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above t

[airflow] branch main updated (a21c17bc07 -> fd000684d0)

2023-02-19 Thread taragolis
This is an automated email from the ASF dual-hosted git repository. taragolis pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git from a21c17bc07 Specific use-case: adding packages via requirements.txt in compose (#29598) add fd000684d0 SSH Prov

[GitHub] [airflow] potiuk commented on pull request #29614: Add note about workaround to potential error with argparse to hdfs

2023-02-19 Thread via GitHub
potiuk commented on PR #29614: URL: https://github.com/apache/airflow/pull/29614#issuecomment-1436067561 It's a niche case but yes. We could detect it -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [airflow] dimberman commented on pull request #29609: Decouple production executor warning in dags UI

2023-02-19 Thread via GitHub
dimberman commented on PR #29609: URL: https://github.com/apache/airflow/pull/29609#issuecomment-1436081501 @o-nikolas looks like there are a few failing tests. Once these are fixed would be glad to merge. -- This is an automated message from the Apache Git Service. To respond to the mess

[GitHub] [airflow] dimberman commented on a diff in pull request #29606: convert moment with timezone to UTC instead of raising an exception

2023-02-19 Thread via GitHub
dimberman commented on code in PR #29606: URL: https://github.com/apache/airflow/pull/29606#discussion_r314236 ## airflow/triggers/temporal.py: ## @@ -41,10 +41,8 @@ def __init__(self, moment: datetime.datetime): # Make sure it's in UTC elif moment.tzinfo i

[GitHub] [airflow] dimberman commented on a diff in pull request #29580: Allow to specify which connection, variable or config are being looked up in the backend using *_lookup_pattern parameters

2023-02-19 Thread via GitHub
dimberman commented on code in PR #29580: URL: https://github.com/apache/airflow/pull/29580#discussion_r314542 ## airflow/providers/amazon/aws/secrets/secrets_manager.py: ## @@ -84,12 +88,24 @@ class SecretsManagerBackend(BaseSecretsBackend, LoggingMixin): :param conne

[GitHub] [airflow] Taragolis opened a new pull request, #29616: Refactor docker-compose quick start test

2023-02-19 Thread via GitHub
Taragolis opened a new pull request, #29616: URL: https://github.com/apache/airflow/pull/29616 There is quite a few tests failed with the same error during the PR CI pipeline ```console > assert dag_state == "success" E AssertionError: ass

[GitHub] [airflow] Taragolis merged pull request #29606: convert moment with timezone to UTC instead of raising an exception

2023-02-19 Thread via GitHub
Taragolis merged PR #29606: URL: https://github.com/apache/airflow/pull/29606 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflo

[GitHub] [airflow] Taragolis closed issue #29576: DateTimeSensorAsync breaks if target_time is timezone-aware

2023-02-19 Thread via GitHub
Taragolis closed issue #29576: DateTimeSensorAsync breaks if target_time is timezone-aware URL: https://github.com/apache/airflow/issues/29576 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the spe

[airflow] branch main updated: convert moment with timezone to UTC instead of raising an exception (#29606)

2023-02-19 Thread taragolis
This is an automated email from the ASF dual-hosted git repository. taragolis pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 79c07e3fc5 convert moment with timezone to UTC i

[GitHub] [airflow] Taragolis closed pull request #29616: Refactor docker-compose quick start test

2023-02-19 Thread via GitHub
Taragolis closed pull request #29616: Refactor docker-compose quick start test URL: https://github.com/apache/airflow/pull/29616 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

[GitHub] [airflow] hussein-awala commented on issue #29561: CeleryKubernetesExecutor @task(queue="kubernetes")  not work

2023-02-19 Thread via GitHub
hussein-awala commented on issue #29561: URL: https://github.com/apache/airflow/issues/29561#issuecomment-1436086556 Can you add your dag code? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [airflow] potiuk merged pull request #29614: Add note about workaround to potential error with argparse to hdfs

2023-02-19 Thread via GitHub
potiuk merged PR #29614: URL: https://github.com/apache/airflow/pull/29614 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.a

[GitHub] [airflow] potiuk closed issue #22689: HDFS provider causes TypeError: __init__() got an unexpected keyword argument 'encoding'

2023-02-19 Thread via GitHub
potiuk closed issue #22689: HDFS provider causes TypeError: __init__() got an unexpected keyword argument 'encoding' URL: https://github.com/apache/airflow/issues/22689 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

[airflow] branch main updated: Add note about workaround to potential error with argparse to hdfs (#29614)

2023-02-19 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 0a4184e34c Add note about workaround to potential e

[GitHub] [airflow] hussein-awala commented on issue #29582: Cannot trigger the same dag_id within the same second

2023-02-19 Thread via GitHub
hussein-awala commented on issue #29582: URL: https://github.com/apache/airflow/issues/29582#issuecomment-1436093181 > Cannot trigger the same dag_id within the same second You cannot trigger the dag multiple time with the same execution date because there are two unique constraint on

[GitHub] [airflow] hussein-awala commented on issue #29557: Execution Timeout is not working properly on airflow 2.5.0

2023-02-19 Thread via GitHub
hussein-awala commented on issue #29557: URL: https://github.com/apache/airflow/issues/29557#issuecomment-1436106024 Airflow raises `AirflowTaskTimeout` exception when the task timed out, and your code can catch this exception and handle it if needed. Since you have `[2023-02-15, 20:5

[GitHub] [airflow] hussein-awala commented on a diff in pull request #29270: Adding possibility for annotations in logs pvc

2023-02-19 Thread via GitHub
hussein-awala commented on code in PR #29270: URL: https://github.com/apache/airflow/pull/29270#discussion_r335700 ## chart/templates/logs-persistent-volume-claim.yaml: ## @@ -29,6 +29,10 @@ metadata: {{- with .Values.labels }} {{- toYaml . | nindent 4 }} {{-

[GitHub] [airflow] potiuk commented on issue #25297: on_failure_callback is not called when task is terminated externally

2023-02-19 Thread via GitHub
potiuk commented on issue #25297: URL: https://github.com/apache/airflow/issues/25297#issuecomment-1436130977 I think if If you can provide reproducible case with logs @seub from the last Airflow version, that would be good new issue. I guess the case is different than explained originally

[GitHub] [airflow] potiuk commented on issue #29275: Allow user DAGs to emit DagWarning (or similar)

2023-02-19 Thread via GitHub
potiuk commented on issue #29275: URL: https://github.com/apache/airflow/issues/29275#issuecomment-1436133430 Yes it would be interesting feature to add. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [airflow] hussein-awala commented on issue #29555: Dag Fails while creating a Dynamic Tasks using airflow variables.

2023-02-19 Thread via GitHub
hussein-awala commented on issue #29555: URL: https://github.com/apache/airflow/issues/29555#issuecomment-1436137871 Can you provide some code for the mapped task group? It's not clear to me why would you map different TIs based on the batches list instead of mapping the same TIs with diffe

[GitHub] [airflow] potiuk commented on pull request #26269: Added Feature - Glue job continuously log printing in Airflow task logs

2023-02-19 Thread via GitHub
potiuk commented on PR #26269: URL: https://github.com/apache/airflow/pull/26269#issuecomment-1436141544 > Hi is there any update on this PR, thanks! Not really - as you see it's closed. But if you want to pick it up where the original author left it - feel free to continue in a new P

[GitHub] [airflow] github-actions[bot] commented on pull request #28265: ability to set referenceId in ECSRunTaskOperator

2023-02-19 Thread via GitHub
github-actions[bot] commented on PR #28265: URL: https://github.com/apache/airflow/pull/28265#issuecomment-1436142743 This pull request has been automatically marked as stale because it has not had recent activity. It will be closed in 5 days if no further activity occurs. Thank you for you

[GitHub] [airflow] potiuk closed issue #28836: Airflow Scheduler Hangs After Failed K8 API Call

2023-02-19 Thread via GitHub
potiuk closed issue #28836: Airflow Scheduler Hangs After Failed K8 API Call URL: https://github.com/apache/airflow/issues/28836 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

[GitHub] [airflow] potiuk closed issue #29339: Failing to execute successive Dynamic Task Group Mapping

2023-02-19 Thread via GitHub
potiuk closed issue #29339: Failing to execute successive Dynamic Task Group Mapping URL: https://github.com/apache/airflow/issues/29339 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [airflow] potiuk closed issue #29137: Add ability to access context in functions decorated by task.sensor

2023-02-19 Thread via GitHub
potiuk closed issue #29137: Add ability to access context in functions decorated by task.sensor URL: https://github.com/apache/airflow/issues/29137 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to th

[GitHub] [airflow] boring-cyborg[bot] commented on pull request #29146: Add ability to access context in functions decorated by task.sensor

2023-02-19 Thread via GitHub
boring-cyborg[bot] commented on PR #29146: URL: https://github.com/apache/airflow/pull/29146#issuecomment-1436145128 Awesome work, congrats on your first merged pull request! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

[airflow] branch main updated (0a4184e34c -> 2d3cc504db)

2023-02-19 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git from 0a4184e34c Add note about workaround to potential error with argparse to hdfs (#29614) add 2d3cc504db Remove poke

[GitHub] [airflow] potiuk merged pull request #29146: Add ability to access context in functions decorated by task.sensor

2023-02-19 Thread via GitHub
potiuk merged PR #29146: URL: https://github.com/apache/airflow/pull/29146 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.a

[GitHub] [airflow] potiuk merged pull request #29362: Missing timezone in next_dag_run in parameterized docs

2023-02-19 Thread via GitHub
potiuk merged PR #29362: URL: https://github.com/apache/airflow/pull/29362 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.a

[GitHub] [airflow] boring-cyborg[bot] commented on pull request #29362: Missing timezone in next_dag_run in parameterized docs

2023-02-19 Thread via GitHub
boring-cyborg[bot] commented on PR #29362: URL: https://github.com/apache/airflow/pull/29362#issuecomment-1436147137 Awesome work, congrats on your first merged pull request! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

[airflow] branch main updated: Missing timezone in next_dagrun in parameterized docs (#29362)

2023-02-19 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new e9e3f2c8c7 Missing timezone in next_dagrun in param

[GitHub] [airflow] potiuk closed issue #29365: Airflow Workers Trying to Create Pods in Default Namespace

2023-02-19 Thread via GitHub
potiuk closed issue #29365: Airflow Workers Trying to Create Pods in Default Namespace URL: https://github.com/apache/airflow/issues/29365 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specifi

  1   2   >