Re: [PR] [FEAT] adds repair run functionality for databricks [airflow]

2024-01-09 Thread via GitHub
gaurav7261 commented on PR #36601: URL: https://github.com/apache/airflow/pull/36601#issuecomment-1882593356 Hi @dirrao , since repair run will work with `DatabricksRunNowOperator` only, test case was failing because same handle method was getting called for `DatabricksSubmitRunOperator` as

Re: [PR] Deprecate AutoMLTrainModelOperator for Vision and Video [airflow]

2024-01-09 Thread via GitHub
VladaZakharova commented on code in PR #36473: URL: https://github.com/apache/airflow/pull/36473#discussion_r1445769655 ## tests/system/providers/google/cloud/automl/example_automl_video_intelligence_tracking.py: ## @@ -22,49 +22,49 @@ import os from datetime import datetime

Re: [I] SFTPSensor.newer_than not working with jinja logical ds/ts expression [airflow]

2024-01-09 Thread via GitHub
fpopic commented on issue #36629: URL: https://github.com/apache/airflow/issues/36629#issuecomment-1882609788 @RNHTTR @vatsrahul1001 Sorry for closing, but was too early. My pattern was testing for non-existing file and it worked, but once the file exists on SFTP server, I am getting

Re: [I] SFTPSensor.newer_than not working with jinja logical ds/ts expression [airflow]

2024-01-09 Thread via GitHub
fpopic commented on issue #36629: URL: https://github.com/apache/airflow/issues/36629#issuecomment-1882612654 and seems that not all pendulum method are imported. Substract doesn't work. Add works. ```py newer_than='{{ dag_run.logical_date.substract(days=1, hours=-2) }}', ```

Re: [I] SFTPSensor.newer_than not working with jinja logical ds/ts expression [airflow]

2024-01-09 Thread via GitHub
fpopic commented on issue #36629: URL: https://github.com/apache/airflow/issues/36629#issuecomment-1882637394 @AdamPaslawski @hussein-awala might you be able to help with ```py File "/opt/python3.8/lib/python3.8/site-packages/airflow/utils/timezone.py", line 39, in is_localized

[PR] Fix on docstring for apply_wildcard parameter in S3ListOperator. #36593 [airflow]

2024-01-09 Thread via GitHub
sasidharan-rathinam opened a new pull request, #36679: URL: https://github.com/apache/airflow/pull/36679 Changed the order of docstring for fixing this issue (https://github.com/apache/airflow/issues/36593) --- **^ Add meaningful description above** R

[PR] check transform job status status before deferring SageMakerTransformOperator [airflow]

2024-01-09 Thread via GitHub
Lee-W opened a new pull request, #36680: URL: https://github.com/apache/airflow/pull/36680 While running a sagemaker transform job in deferrable mode, the condition might already be met before we defer the task into the trigger. This PR intends to check the job status before deferring the t

[PR] Update test_file_not_new_enough in test_sftp.py with full datetime [airflow]

2024-01-09 Thread via GitHub
fpopic opened a new pull request, #36681: URL: https://github.com/apache/airflow/pull/36681 --- **^ Add meaningful description above** Read the **[Pull Request Guidelines](https://github.com/apache/airflow/blob/main/CONTRIBUTING.rst#pull-request-guidelines)*

(airflow) branch switch-to-hatch updated (c5d89910ce -> 2e74eced66)

2024-01-09 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a change to branch switch-to-hatch in repository https://gitbox.apache.org/repos/asf/airflow.git omit c5d89910ce Standardize airflow build process and switch to Hatchling build backend add 8e8c080050 Prov

[PR] Fix spelling of `airflow` [airflow]

2024-01-09 Thread via GitHub
jbampton opened a new pull request, #36682: URL: https://github.com/apache/airflow/pull/36682 --- **^ Add meaningful description above** Read the **[Pull Request Guidelines](https://github.com/apache/airflow/blob/main/CONTRIBUTING.rst#pull-request-guidelines

Re: [I] Broken installation with `--editable` flag [airflow]

2024-01-09 Thread via GitHub
potiuk commented on issue #30764: URL: https://github.com/apache/airflow/issues/30764#issuecomment-1882731503 A huge - but one that fixes it properly fix for that one is coming in https://github.com/apache/airflow/pull/36537 - moving airflow into the modern packaging tools world ! -- Thi

Re: [PR] Fix spelling of `airflow` [airflow]

2024-01-09 Thread via GitHub
potiuk commented on PR #36682: URL: https://github.com/apache/airflow/pull/36682#issuecomment-1882762339 But tests need to be fixed. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific c

[I] Webserver: signal 15 [airflow]

2024-01-09 Thread via GitHub
alucryd opened a new issue, #36683: URL: https://github.com/apache/airflow/issues/36683 ### Official Helm Chart version 1.11.0 (latest released) ### Apache Airflow version 2.7.1 ### Kubernetes Version 1.27.3-gke.100 ### Helm Chart configuration

Re: [I] Webserver: signal 15 [airflow]

2024-01-09 Thread via GitHub
boring-cyborg[bot] commented on issue #36683: URL: https://github.com/apache/airflow/issues/36683#issuecomment-1882763826 Thanks for opening your first issue here! Be sure to follow the issue template! If you are willing to raise PR to address this issue please do so, no need to wait for ap

Re: [I] Dag blocked when winter time change happen with specific cron [airflow]

2024-01-09 Thread via GitHub
V0lantis closed issue #35558: Dag blocked when winter time change happen with specific cron URL: https://github.com/apache/airflow/issues/35558 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the sp

Re: [I] Dag blocked when winter time change happen with specific cron [airflow]

2024-01-09 Thread via GitHub
V0lantis commented on issue #35558: URL: https://github.com/apache/airflow/issues/35558#issuecomment-1882785636 I think this is fixed by #35887 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

Re: [I] apache-airflow-providers-cncf-kubernetes requires latest kubernetes version [airflow]

2024-01-09 Thread via GitHub
raphaelauv commented on issue #36678: URL: https://github.com/apache/airflow/issues/36678#issuecomment-1882794935 ![image](https://github.com/apache/airflow/assets/10202690/78ea4bc5-b2cc-445e-b889-f27b22d62754) -- This is an automated message from the Apache Git Service. To respon

Re: [PR] Mark daskexecutor provider as removed [airflow]

2024-01-09 Thread via GitHub
MaiHoangViet1809 commented on PR #35965: URL: https://github.com/apache/airflow/pull/35965#issuecomment-1882798070 I know it is late to comment this but we're using DaskExecutor here in production and it quite good, config easier than celery as it not require broker backend like redis. --

[PR] bump: kubernetes client version [airflow]

2024-01-09 Thread via GitHub
raphaelauv opened a new pull request, #36684: URL: https://github.com/apache/airflow/pull/36684 close: https://github.com/apache/airflow/issues/36678 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[PR] check sagemaker training job status before deferring SageMakerTrainingOperator [airflow]

2024-01-09 Thread via GitHub
Lee-W opened a new pull request, #36685: URL: https://github.com/apache/airflow/pull/36685 While running a sagemaker training job in deferrable mode, the condition might already be met before we defer the task into the trigger. This PR intends to check the job status before deferring the ta

Re: [PR] check transform job status before deferring SageMakerTransformOperator [airflow]

2024-01-09 Thread via GitHub
Lee-W commented on PR #36680: URL: https://github.com/apache/airflow/pull/36680#issuecomment-1882853941 The static check failure should be fixed in https://github.com/apache/airflow/pull/36677 -- This is an automated message from the Apache Git Service. To respond to the message, please l

Re: [PR] check ProcessingJobStatus status before deferring SageMakerProcessingOperator [airflow]

2024-01-09 Thread via GitHub
Lee-W commented on PR #36658: URL: https://github.com/apache/airflow/pull/36658#issuecomment-1882854054 The static check failure should be fixed in https://github.com/apache/airflow/pull/36677 -- This is an automated message from the Apache Git Service. To respond to the message, please l

Re: [I] API Endpoints for Dataset Creation and Updating [airflow]

2024-01-09 Thread via GitHub
boring-cyborg[bot] commented on issue #36686: URL: https://github.com/apache/airflow/issues/36686#issuecomment-1882858416 Thanks for opening your first issue here! Be sure to follow the issue template! If you are willing to raise PR to address this issue please do so, no need to wait for ap

Re: [PR] bump: kubernetes client version [airflow]

2024-01-09 Thread via GitHub
eladkal commented on code in PR #36684: URL: https://github.com/apache/airflow/pull/36684#discussion_r1445949713 ## airflow/providers/cncf/kubernetes/provider.yaml: ## @@ -88,13 +88,13 @@ dependencies: # limiting minimum airflow version supported in cncf.kubernetes provider,

Re: [I] Support External Dataset Updates [airflow]

2024-01-09 Thread via GitHub
edumuellerFSL commented on issue #29162: URL: https://github.com/apache/airflow/issues/29162#issuecomment-1882872689 Hi, we use Airflow in our team and this is something we also feel is currently missing in airflow, especially to facilitate cross-instance dataset awareness and for custom in

Re: [PR] Mark daskexecutor provider as removed [airflow]

2024-01-09 Thread via GitHub
potiuk commented on PR #35965: URL: https://github.com/apache/airflow/pull/35965#issuecomment-1882890410 > I know it is late to comment this but we're using DaskExecutor here in production and it quite good, config easier than celery as it not require broker backend like redis. And

Re: [I] API Endpoints for Dataset Creation and Updating [airflow]

2024-01-09 Thread via GitHub
naltimari commented on issue #36686: URL: https://github.com/apache/airflow/issues/36686#issuecomment-1882892435 I also believe this would be helpful and aligns with the data-aware scheduling feature, as well as the dataset listener feature. -- This is an automated message from the Apache

Re: [PR] feat: Stream Transfer Enhancement for Airflow SFTP to GCS Operator [airflow]

2024-01-09 Thread via GitHub
he-gong-sardine commented on code in PR #36545: URL: https://github.com/apache/airflow/pull/36545#discussion_r1445988080 ## airflow/providers/google/cloud/transfers/sftp_to_gcs.py: ## @@ -85,13 +106,18 @@ def __init__( source_path: str, destination_bucket: str,

Re: [PR] feat: Stream Transfer Enhancement for Airflow SFTP to GCS Operator [airflow]

2024-01-09 Thread via GitHub
he-gong-sardine commented on code in PR #36545: URL: https://github.com/apache/airflow/pull/36545#discussion_r1445989162 ## airflow/providers/google/cloud/transfers/sftp_to_gcs.py: ## @@ -168,6 +204,68 @@ def _copy_single_object( self.log.info("Executing delete of %

[I] Airflow does not mask secrets in Database URI's when secret has a special character. [airflow]

2024-01-09 Thread via GitHub
Rurushu13 opened a new issue, #36688: URL: https://github.com/apache/airflow/issues/36688 ### Apache Airflow version Other Airflow 2 version (please specify below) ### If "Other Airflow 2 version" selected, which one? 2.5.1 ### What happened? While we were t

Re: [PR] OpenLineage: Add dag_id when generating run_id for task instance. [airflow]

2024-01-09 Thread via GitHub
mobuchowski merged PR #36659: URL: https://github.com/apache/airflow/pull/36659 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airf

(airflow) branch main updated: feat: Add dag_id when generating OpenLineage run_id for task instance. (#36659)

2024-01-09 Thread mobuchowski
This is an automated email from the ASF dual-hosted git repository. mobuchowski pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 95a83102e8 feat: Add dag_id when generating Op

Re: [PR] Add support of Pendulum 3 [airflow]

2024-01-09 Thread via GitHub
Taragolis commented on PR #36281: URL: https://github.com/apache/airflow/pull/36281#issuecomment-1882966528 I will rebase and add release notes today if I feel better -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use th

Re: [PR] feat: Stream Transfer Enhancement for Airflow SFTP to GCS Operator [airflow]

2024-01-09 Thread via GitHub
elaye-canopy commented on code in PR #36545: URL: https://github.com/apache/airflow/pull/36545#discussion_r1446051467 ## airflow/providers/google/cloud/transfers/sftp_to_gcs.py: ## @@ -168,6 +204,68 @@ def _copy_single_object( self.log.info("Executing delete of %s",

Re: [PR] Allow custom columns in cli dags list [airflow]

2024-01-09 Thread via GitHub
jtmiclat commented on PR #35250: URL: https://github.com/apache/airflow/pull/35250#issuecomment-1883051952 Hi @sidrahsayyad I haven't worked on it -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

Re: [I] SFTPSensor.newer_than not working with jinja logical ds/ts expression [airflow]

2024-01-09 Thread via GitHub
fpopic commented on issue #36629: URL: https://github.com/apache/airflow/issues/36629#issuecomment-1883120505 I think I recreated here the same problem in unit test https://github.com/apache/airflow/actions/runs/7459580855/job/20296049813#step:5:781 -- This is an automated message

Re: [I] Airflow unable to reserialize deleted DAG file [airflow]

2024-01-09 Thread via GitHub
ThR3742 commented on issue #21329: URL: https://github.com/apache/airflow/issues/21329#issuecomment-1883191678 I am in 2.2.2 and have the same issue ; is there a way to fix this ? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

(airflow) branch switch-to-hatch updated (2e74eced66 -> 86f68a409a)

2024-01-09 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a change to branch switch-to-hatch in repository https://gitbox.apache.org/repos/asf/airflow.git omit 2e74eced66 Standardize airflow build process and switch to Hatchling build backend add 86f68a409a Stan

Re: [I] `HttpOperator` no longer accepts `https`-prefixed connection URIs from environment variables [airflow]

2024-01-09 Thread via GitHub
Taragolis commented on issue #36602: URL: https://github.com/apache/airflow/issues/36602#issuecomment-1883234684 >and [the documentation for the connection type](https://airflow.apache.org/docs/apache-airflow-providers-http/4.8.0/connections/http.html) says that https is a valid schema whic

Re: [PR] Standardize airflow build process and switch to Hatchling build backend [airflow]

2024-01-09 Thread via GitHub
potiuk commented on PR #36537: URL: https://github.com/apache/airflow/pull/36537#issuecomment-1883270253 > I was "naively" testing and miserably failed. Used the current branch, created a new venv, then called pip install -e ., followed by pip install -e .[ssh,docker]. But then calling airf

(airflow) branch pendulum-3 updated (e623348429 -> 669d9e1e7b)

2024-01-09 Thread taragolis
This is an automated email from the ASF dual-hosted git repository. taragolis pushed a change to branch pendulum-3 in repository https://gitbox.apache.org/repos/asf/airflow.git omit e623348429 Update airflow/serialization/serialized_objects.py omit 72b806371d Add backcompat to pendulum 2

Re: [I] Airflow does not mask secrets in Database URI's when secret has a special character. [airflow]

2024-01-09 Thread via GitHub
Rurushu13 commented on issue #36688: URL: https://github.com/apache/airflow/issues/36688#issuecomment-1883340047 > @Rurushu13 > Good finding. If possible, can you share the sample DAG? @dirrao There it is `from datetime import timedelta import pendulum import os

(airflow) branch pendulum-3 updated (669d9e1e7b -> b08df5802e)

2024-01-09 Thread taragolis
This is an automated email from the ASF dual-hosted git repository. taragolis pushed a change to branch pendulum-3 in repository https://gitbox.apache.org/repos/asf/airflow.git from 669d9e1e7b Add newsfragments add b08df5802e Fix static checks annd redundand $ in workflow No new revisi

Re: [PR] [FEAT] adds repair run functionality for databricks [airflow]

2024-01-09 Thread via GitHub
gaurav7261 commented on PR #36601: URL: https://github.com/apache/airflow/pull/36601#issuecomment-1883398575 Hi @dirrao can you please rereview -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

Re: [PR] check transform job status before deferring SageMakerTransformOperator [airflow]

2024-01-09 Thread via GitHub
dirrao commented on code in PR #36680: URL: https://github.com/apache/airflow/pull/36680#discussion_r1446340539 ## airflow/providers/amazon/aws/operators/sagemaker.py: ## @@ -700,8 +700,24 @@ def execute(self, context: Context) -> dict: raise AirflowException(f"Sage

[I] Details tab blank for Mapped Tasks [airflow]

2024-01-09 Thread via GitHub
tanmaykansara opened a new issue, #36690: URL: https://github.com/apache/airflow/issues/36690 ### Apache Airflow version 2.8.0 ### If "Other Airflow 2 version" selected, which one? _No response_ ### What happened? The details tab in previous versions had the

[PR] Change in dag details from `concurrency` to `max active tasks` [airflow]

2024-01-09 Thread via GitHub
romsharon98 opened a new pull request, #36691: URL: https://github.com/apache/airflow/pull/36691 Since concurrency is deprecated, change in dag details page from concurrency to max active tasks https://github.com/apache/airflow/assets/33751805/3b0e5243-40e7-4ad5-9c45-b38408af446d";>

(airflow) branch switch-to-hatch updated (86f68a409a -> 6b7f09cccf)

2024-01-09 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a change to branch switch-to-hatch in repository https://gitbox.apache.org/repos/asf/airflow.git discard 86f68a409a Standardize airflow build process and switch to Hatchling build backend add 6b7f09cccf Stan

Re: [PR] Fix on docstring for apply_wildcard parameter in S3ListOperator [airflow]

2024-01-09 Thread via GitHub
vincbeck merged PR #36679: URL: https://github.com/apache/airflow/pull/36679 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow

Re: [PR] Fix on docstring for apply_wildcard parameter in S3ListOperator [airflow]

2024-01-09 Thread via GitHub
boring-cyborg[bot] commented on PR #36679: URL: https://github.com/apache/airflow/pull/36679#issuecomment-1883555681 Awesome work, congrats on your first merged pull request! You are invited to check our [Issue Tracker](https://github.com/apache/airflow/issues) for additional contributions.

(airflow) branch main updated: Fix docstring for apply_wildcard parameter in `S3ListOperator`. Changed the order of docstring for fix (#36679)

2024-01-09 Thread vincbeck
This is an automated email from the ASF dual-hosted git repository. vincbeck pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 9cb2052810 Fix docstring for apply_wildcard param

(airflow) branch switch-to-hatch updated (6b7f09cccf -> 2a2c204e85)

2024-01-09 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a change to branch switch-to-hatch in repository https://gitbox.apache.org/repos/asf/airflow.git omit 6b7f09cccf Standardize airflow build process and switch to Hatchling build backend add 2a2c204e85 Stan

[PR] Secret masker ignores passwords with special chars [airflow]

2024-01-09 Thread via GitHub
aritra24 opened a new pull request, #36692: URL: https://github.com/apache/airflow/pull/36692 Connection uri's get connection uses quote to change the password and certain other fields to escape special chars due to this, when the connection object is passed through the masker this changed

[PR] Fix Callback exception when a removed task is the last one in the task instance list [airflow]

2024-01-09 Thread via GitHub
joaopamaral opened a new pull request, #36693: URL: https://github.com/apache/airflow/pull/36693 --- **^ Add meaningful description above** Read the **[Pull Request Guidelines](https://github.com/apache/airflow/blob/main/CONTRIBUTING.rst#pull-request-guideli

Re: [PR] Secret masker ignores passwords with special chars [airflow]

2024-01-09 Thread via GitHub
aritra24 commented on PR #36692: URL: https://github.com/apache/airflow/pull/36692#issuecomment-1883616070 Wasn't entirely sure how to add testing for this since it needs to pass through the masker? 🤔 If anyone has ideas let me know. -- This is an automated message from the Apache Git Ser

(airflow) branch switch-to-hatch updated (2a2c204e85 -> d005d04925)

2024-01-09 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a change to branch switch-to-hatch in repository https://gitbox.apache.org/repos/asf/airflow.git omit 2a2c204e85 Standardize airflow build process and switch to Hatchling build backend add d005d04925 Stan

[I] Bulk Variable Updates to allow transactional rollbacks [airflow]

2024-01-09 Thread via GitHub
NathanielRose opened a new issue, #36694: URL: https://github.com/apache/airflow/issues/36694 ### Description Currently we have to iterate through variables in a json file to be able to upload new values. We should have a native way to simply hand off the JSON file to set all variabl

Re: [I] Bulk Variable Updates to allow transactional rollbacks [airflow]

2024-01-09 Thread via GitHub
boring-cyborg[bot] commented on issue #36694: URL: https://github.com/apache/airflow/issues/36694#issuecomment-1883633314 Thanks for opening your first issue here! Be sure to follow the issue template! If you are willing to raise PR to address this issue please do so, no need to wait for ap

[PR] Get rid of pyarrow-hotfix fora CVE-2023-47248 [airflow]

2024-01-09 Thread via GitHub
potiuk opened a new pull request, #36695: URL: https://github.com/apache/airflow/pull/36695 The #35650 introduced a hotfix for Pyarrow CVE-2023-47248. So far we have been blocked from removing it by Apache Beam that limited Airflow from bumping pyarrow to a version that was not vulnerable.

[I] Helm chart fail on Openshift OKD 4.13 [airflow]

2024-01-09 Thread via GitHub
maczg opened a new issue, #36696: URL: https://github.com/apache/airflow/issues/36696 ### Official Helm Chart version 1.11.0 (latest released) ### Apache Airflow version 2.7.1 ### Kubernetes Version Openshift OKD 4.13 Kubernetes v1.26 ### Helm Chart c

Re: [I] Helm chart fail on Openshift OKD 4.13 [airflow]

2024-01-09 Thread via GitHub
boring-cyborg[bot] commented on issue #36696: URL: https://github.com/apache/airflow/issues/36696#issuecomment-1883644648 Thanks for opening your first issue here! Be sure to follow the issue template! If you are willing to raise PR to address this issue please do so, no need to wait for ap

[PR] Get rid of pyarrow-hotfix for CVE-2023-47248 [airflow]

2024-01-09 Thread via GitHub
potiuk opened a new pull request, #36697: URL: https://github.com/apache/airflow/pull/36697 The #35650 introduced a hotfix for Pyarrow CVE-2023-47248. So far we have been blocked from removing it by Apache Beam that limited Airflow from bumping pyarrow to a version that was not vulnerable.

Re: [PR] Refresh credentials in `AwsEcsExecutor` [airflow]

2024-01-09 Thread via GitHub
o-nikolas merged PR #36179: URL: https://github.com/apache/airflow/pull/36179 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflo

(airflow) branch main updated (9cb2052810 -> cb1182ea4f)

2024-01-09 Thread onikolas
This is an automated email from the ASF dual-hosted git repository. onikolas pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git from 9cb2052810 Fix docstring for apply_wildcard parameter in `S3ListOperator`. Changed the order of docstring for fix (#

Re: [I] Details tab blank for Mapped Tasks [airflow]

2024-01-09 Thread via GitHub
aritra24 commented on issue #36690: URL: https://github.com/apache/airflow/issues/36690#issuecomment-1883657936 @tanmaykansara could you share any specific steps to repro? I do not see this behaviour on using the [sample](https://airflow.apache.org/docs/apache-airflow/stable/_modules/airflo

Re: [PR] Get rid of pyarrow-hotfix fora CVE-2023-47248 [airflow]

2024-01-09 Thread via GitHub
potiuk closed pull request #36695: Get rid of pyarrow-hotfix fora CVE-2023-47248 URL: https://github.com/apache/airflow/pull/36695 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment

Re: [PR] Get rid of pyarrow-hotfix fora CVE-2023-47248 [airflow]

2024-01-09 Thread via GitHub
potiuk commented on PR #36695: URL: https://github.com/apache/airflow/pull/36695#issuecomment-1883677585 Opened it twice by mistake #36697 is the right fix -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

[PR] Set min pandas dependency to 1.2.5 for all providers and airflow [airflow]

2024-01-09 Thread via GitHub
potiuk opened a new pull request, #36698: URL: https://github.com/apache/airflow/pull/36698 We had some REALLY old minimum version of Pandas set for all our pandas dependency - Pandas 0.17.1 has been released in 2015 (!) Looking at the dependency tree - most of our dependencies had

Re: [PR] Get rid of pyarrow-hotfix for CVE-2023-47248 [airflow]

2024-01-09 Thread via GitHub
hussein-awala commented on code in PR #36697: URL: https://github.com/apache/airflow/pull/36697#discussion_r1446533287 ## generated/provider_dependencies.json: ## @@ -56,7 +56,8 @@ "apache.beam": { "deps": [ "apache-airflow>=2.6.0", - "apache-beam>=2.47.0" +

Re: [PR] Get rid of pyarrow-hotfix for CVE-2023-47248 [airflow]

2024-01-09 Thread via GitHub
potiuk commented on code in PR #36697: URL: https://github.com/apache/airflow/pull/36697#discussion_r1446537433 ## generated/provider_dependencies.json: ## @@ -56,7 +56,8 @@ "apache.beam": { "deps": [ "apache-airflow>=2.6.0", - "apache-beam>=2.47.0" + "a

Re: [PR] Get rid of pyarrow-hotfix for CVE-2023-47248 [airflow]

2024-01-09 Thread via GitHub
potiuk commented on code in PR #36697: URL: https://github.com/apache/airflow/pull/36697#discussion_r1446537433 ## generated/provider_dependencies.json: ## @@ -56,7 +56,8 @@ "apache.beam": { "deps": [ "apache-airflow>=2.6.0", - "apache-beam>=2.47.0" + "a

Re: [PR] Get rid of pyarrow-hotfix for CVE-2023-47248 [airflow]

2024-01-09 Thread via GitHub
potiuk commented on code in PR #36697: URL: https://github.com/apache/airflow/pull/36697#discussion_r1446537433 ## generated/provider_dependencies.json: ## @@ -56,7 +56,8 @@ "apache.beam": { "deps": [ "apache-airflow>=2.6.0", - "apache-beam>=2.47.0" + "a

Re: [I] Details tab blank for Mapped Tasks [airflow]

2024-01-09 Thread via GitHub
tanmaykansara commented on issue #36690: URL: https://github.com/apache/airflow/issues/36690#issuecomment-1883699012 HI, here is the code snippet that I can share: ` def get_work_items_task(): logger.info(f"secret_name:{secret_name}") logger.info(f"bucket={b

Re: [PR] Get rid of pyarrow-hotfix for CVE-2023-47248 [airflow]

2024-01-09 Thread via GitHub
hussein-awala commented on code in PR #36697: URL: https://github.com/apache/airflow/pull/36697#discussion_r1446542990 ## generated/provider_dependencies.json: ## @@ -56,7 +56,8 @@ "apache.beam": { "deps": [ "apache-airflow>=2.6.0", - "apache-beam>=2.47.0" +

Re: [I] Details tab blank for Mapped Tasks [airflow]

2024-01-09 Thread via GitHub
eladkal closed issue #36690: Details tab blank for Mapped Tasks URL: https://github.com/apache/airflow/issues/36690 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscri

Re: [PR] Fix security manager inheritance in fab provider [airflow]

2024-01-09 Thread via GitHub
nicolamarangoni commented on PR #36538: URL: https://github.com/apache/airflow/pull/36538#issuecomment-1883718381 Hello, does this mean that we need to wait for 2.8.1 to test the setup with SSO over OAUTH and Azure? -- This is an automated message from the Apache Git Service. To respond t

Re: [PR] Get rid of pyarrow-hotfix for CVE-2023-47248 [airflow]

2024-01-09 Thread via GitHub
potiuk commented on code in PR #36697: URL: https://github.com/apache/airflow/pull/36697#discussion_r1446564939 ## generated/provider_dependencies.json: ## @@ -56,7 +56,8 @@ "apache.beam": { "deps": [ "apache-airflow>=2.6.0", - "apache-beam>=2.47.0" + "a

Re: [PR] Fix security manager inheritance in fab provider [airflow]

2024-01-09 Thread via GitHub
vincbeck commented on PR #36538: URL: https://github.com/apache/airflow/pull/36538#issuecomment-1883729836 > Hello, does this mean that we need to wait for 2.8.1 to test the setup with SSO over OAUTH and Azure? If you make your custom security manager inherit from `FabAirflowSecurity

(airflow) branch constraints-main updated: Updating constraints. Github run id:7465891363

2024-01-09 Thread github-bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch constraints-main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/constraints-main by this push: new 323150c26e Updating con

Re: [PR] Get rid of pyarrow-hotfix for CVE-2023-47248 [airflow]

2024-01-09 Thread via GitHub
potiuk merged PR #36697: URL: https://github.com/apache/airflow/pull/36697 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.a

[PR] Bump follow-redirects from 1.15.3 to 1.15.4 in /airflow/www [airflow]

2024-01-09 Thread via GitHub
dependabot[bot] opened a new pull request, #36700: URL: https://github.com/apache/airflow/pull/36700 Bumps [follow-redirects](https://github.com/follow-redirects/follow-redirects) from 1.15.3 to 1.15.4. Commits https://github.com/follow-redirects/follow-redirects/commit/658582

(airflow) branch dependabot/npm_and_yarn/airflow/www/follow-redirects-1.15.4 created (now 53b48a9954)

2024-01-09 Thread github-bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to branch dependabot/npm_and_yarn/airflow/www/follow-redirects-1.15.4 in repository https://gitbox.apache.org/repos/asf/airflow.git at 53b48a9954 Bump follow-redirects from 1.15.3 to 1.15.4 in /

(airflow) branch main updated: Get rid of pyarrow-hotfix for CVE-2023-47248 (#36697)

2024-01-09 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new d105c7115f Get rid of pyarrow-hotfix for CVE-2023-4

Re: [PR] Configure HttpHook's auth_type from Connection [airflow]

2024-01-09 Thread via GitHub
Joffreybvn commented on PR #35591: URL: https://github.com/apache/airflow/pull/35591#issuecomment-1883765966 Alright, aside from a last issue with the non-DB test, thi PR is ready for review. I updated the first post with current status. -- This is an automated message from the Apache

Re: [PR] add use_regex argument for allowing S3KeySensor to check s3 keys with regular expression [airflow]

2024-01-09 Thread via GitHub
vincbeck commented on code in PR #36578: URL: https://github.com/apache/airflow/pull/36578#discussion_r1446602251 ## tests/system/providers/amazon/aws/example_s3.py: ## @@ -200,7 +202,18 @@ def check_fn(files: list) -> bool: check_fn=check_fn, deferrable=True,

Re: [PR] Update mask-sensitive-values.rst [airflow]

2024-01-09 Thread via GitHub
hussein-awala merged PR #36699: URL: https://github.com/apache/airflow/pull/36699 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@ai

(airflow) branch main updated: Update mask-sensitive-values.rst (#36699)

2024-01-09 Thread husseinawala
This is an automated email from the ASF dual-hosted git repository. husseinawala pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 4c6be5476c Update mask-sensitive-values.rst (

Re: [I] Webserver: signal 15 [airflow]

2024-01-09 Thread via GitHub
Taragolis closed issue #36683: Webserver: signal 15 URL: https://github.com/apache/airflow/issues/36683 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [PR] Fix Callback exception when a removed task is the last one in the task instance list [airflow]

2024-01-09 Thread via GitHub
hussein-awala commented on code in PR #36693: URL: https://github.com/apache/airflow/pull/36693#discussion_r1446624698 ## airflow/models/dag.py: ## @@ -1466,6 +1466,9 @@ def fetch_callback( # context for the callback. if dag.partial: ti

Re: [PR] Change in dag details from `concurrency` to `max active tasks` [airflow]

2024-01-09 Thread via GitHub
hussein-awala merged PR #36691: URL: https://github.com/apache/airflow/pull/36691 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@ai

(airflow) branch main updated: rename concurrency label to max active tasks (#36691)

2024-01-09 Thread husseinawala
This is an automated email from the ASF dual-hosted git repository. husseinawala pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 9cbfed474c rename concurrency label to max ac

[I] default_webserver_config.py isn't compatible with the latest changes [airflow]

2024-01-09 Thread via GitHub
zachliu opened a new issue, #36702: URL: https://github.com/apache/airflow/issues/36702 ### Apache Airflow version 2.8.0 ### If "Other Airflow 2 version" selected, which one? _No response_ ### What happened? [default_webserver_config.py ](https://github.

Re: [PR] Bump follow-redirects from 1.15.3 to 1.15.4 in /airflow/www [airflow]

2024-01-09 Thread via GitHub
Taragolis merged PR #36700: URL: https://github.com/apache/airflow/pull/36700 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflo

(airflow) branch main updated: Bump follow-redirects from 1.15.3 to 1.15.4 in /airflow/www (#36700)

2024-01-09 Thread taragolis
This is an automated email from the ASF dual-hosted git repository. taragolis pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 0d7bad20d4 Bump follow-redirects from 1.15.3 to

(airflow) branch dependabot/npm_and_yarn/airflow/www/follow-redirects-1.15.4 deleted (was 53b48a9954)

2024-01-09 Thread github-bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to branch dependabot/npm_and_yarn/airflow/www/follow-redirects-1.15.4 in repository https://gitbox.apache.org/repos/asf/airflow.git was 53b48a9954 Bump follow-redirects from 1.15.3 to 1.15.4 in /

Re: [PR] Set min pandas dependency to 1.2.5 for all providers and airflow [airflow]

2024-01-09 Thread via GitHub
potiuk merged PR #36698: URL: https://github.com/apache/airflow/pull/36698 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.a

(airflow) branch main updated: Set min pandas dependency to 1.2.5 for all providers and airflow (#36698)

2024-01-09 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new ecb2c9f24d Set min pandas dependency to 1.2.5 for a

(airflow) branch switch-to-hatch updated (d005d04925 -> a1d5ed4875)

2024-01-09 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a change to branch switch-to-hatch in repository https://gitbox.apache.org/repos/asf/airflow.git omit d005d04925 Standardize airflow build process and switch to Hatchling build backend add 95a83102e8 feat

Re: [PR] Standardize airflow build process and switch to Hatchling build backend [airflow]

2024-01-09 Thread via GitHub
potiuk commented on PR #36537: URL: https://github.com/apache/airflow/pull/36537#issuecomment-1883873987 Also @jscheffl -> I reviewed and cleaned up envs - now `devel` is a bit smaller and non-conflicting + I bumped min-version for pandas/pyarrow in main (and rebased) and added min-version

Re: [PR] Standardize airflow build process and switch to Hatchling build backend [airflow]

2024-01-09 Thread via GitHub
potiuk commented on PR #36537: URL: https://github.com/apache/airflow/pull/36537#issuecomment-1883881029 When you earlier do `hatch python install all`, then doing `hatch shell` (and creating the default env with `devel` dependency takes now ~ 30 seconds. Not bad. -- This is an automate

  1   2   >