Re: [PR] Add kubernetes_resources to differentiate from Celery worker resources [airflow]

2024-08-20 Thread via GitHub
SKisContent commented on PR #41628: URL: https://github.com/apache/airflow/pull/41628#issuecomment-2301208182 @nevcohen I may not be understanding your question. The proposed change allows for a separate set of resource specifications for the Kubernetes worker pod. In my case, my values.yam

Re: [I] Helm chart can't remove .securityContexts for Openshift compatibility [airflow]

2024-08-20 Thread via GitHub
boring-cyborg[bot] commented on issue #41630: URL: https://github.com/apache/airflow/issues/41630#issuecomment-2301194334 Thanks for opening your first issue here! Be sure to follow the issue template! If you are willing to raise PR to address this issue please do so, no need to wait for ap

[I] Helm chart can't remove .securityContexts for Openshift compatibility [airflow]

2024-08-20 Thread via GitHub
philthynz opened a new issue, #41630: URL: https://github.com/apache/airflow/issues/41630 ### Official Helm Chart version 1.15.0 (latest released) ### Apache Airflow version 2.9.3 ### Kubernetes Version v1.26.15+4818370 ### Helm Chart configuration

Re: [PR] Add kubernetes_resources to differentiate from Celery worker resources [airflow]

2024-08-20 Thread via GitHub
nevcohen commented on PR #41628: URL: https://github.com/apache/airflow/pull/41628#issuecomment-2301191858 Now that there is a hybrid executor, we might want an entirely separate set of values ​​for the `pod-template`? Or it will just be duplication of values? -- This is an automated mess

Re: [I] Airflow SFTPToGCSOperator sftp file exist check [airflow]

2024-08-20 Thread via GitHub
kandharvishnu commented on issue #41472: URL: https://github.com/apache/airflow/issues/41472#issuecomment-2301174472 > > > Is there an issue with using an `SFTPSensor` for checking file existence, prior to running `SFTPToGCSOperator`? > > > > > > The SFTP sensor will wait until th

Re: [PR] Separate FAB migration from Core Airflow migration [airflow]

2024-08-20 Thread via GitHub
ephraimbuddy commented on code in PR #41437: URL: https://github.com/apache/airflow/pull/41437#discussion_r1724406933 ## airflow/providers/fab/migrations/env.py: ## @@ -0,0 +1,125 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agr

Re: [PR] Separate FAB migration from Core Airflow migration [airflow]

2024-08-20 Thread via GitHub
jedcunningham commented on code in PR #41437: URL: https://github.com/apache/airflow/pull/41437#discussion_r1724399931 ## airflow/providers/fab/migrations/env.py: ## @@ -0,0 +1,125 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license ag

Re: [PR] Separate FAB migration from Core Airflow migration [airflow]

2024-08-20 Thread via GitHub
ephraimbuddy commented on PR #41437: URL: https://github.com/apache/airflow/pull/41437#issuecomment-2301069429 > > I would appreciate some help on this CI issue cc @potiuk @jedcunningham > > Which one :) ? Honestly, it's a lot. Something is fundamentally wrong, but I can't find

Re: [PR] Switch cloudant provider from cloudant library to ibmcloudant library [airflow]

2024-08-20 Thread via GitHub
eladkal commented on code in PR #41555: URL: https://github.com/apache/airflow/pull/41555#discussion_r1724392868 ## airflow/providers/cloudant/provider.yaml: ## @@ -46,7 +47,17 @@ versions: dependencies: - apache-airflow>=2.8.0 - - cloudant>=2.13.0 + - ibmcloudant>=0.9.1

[PR] Add kubernetes_resources to differentiate from Celery worker resources [airflow]

2024-08-20 Thread via GitHub
SKisContent opened a new pull request, #41628: URL: https://github.com/apache/airflow/pull/41628 This adds a kubernetes_resources setting to the pod template so that those using the CeleryKubernetesExecutor can specify different resource settings for the Celery workers and t

Re: [I] Deprecation of operators should be prominently highlighted instead of the one line remark in the documentation [airflow]

2024-08-20 Thread via GitHub
geraj1010 commented on issue #41532: URL: https://github.com/apache/airflow/issues/41532#issuecomment-2300677916 @kaxil I'd be happy to take this on, if the issue is still open. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

Re: [I] Airflow SFTPToGCSOperator sftp file exist check [airflow]

2024-08-20 Thread via GitHub
geraj1010 commented on issue #41472: URL: https://github.com/apache/airflow/issues/41472#issuecomment-2300595119 > > Is there an issue with using an `SFTPSensor` for checking file existence, prior to running `SFTPToGCSOperator`? > > The SFTP sensor will wait until the file is detected

Re: [I] Airflow SFTPToGCSOperator sftp file exist check [airflow]

2024-08-20 Thread via GitHub
geraj1010 commented on issue #41472: URL: https://github.com/apache/airflow/issues/41472#issuecomment-2300591945 I see. In that case, I don't think it makes sense to have the task succeed if a file is not found. A 'success' would not have much meaning anymore besides that the task simply ra

Re: [PR] remove the removed --use-migration-files argument of "airflow db reset" command in run_generate_migration.sh [airflow]

2024-08-20 Thread via GitHub
Lee-W merged PR #41621: URL: https://github.com/apache/airflow/pull/41621 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.ap

(airflow) branch main updated: remove the removed --use-migration-files argument of "airflow db reset" command in run_generate_migration.sh (#41621)

2024-08-20 Thread weilee
This is an automated email from the ASF dual-hosted git repository. weilee pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 73cc7538ad remove the removed --use-migration-files

Re: [PR] Added logging device and logging device options [airflow]

2024-08-20 Thread via GitHub
geraj1010 commented on code in PR #41416: URL: https://github.com/apache/airflow/pull/41416#discussion_r1724297423 ## .pre-commit-config.yaml: ## @@ -779,7 +779,7 @@ repos: files: ^\.pre-commit-config\.yaml$|^.*_installed_providers\.txt$ pass_filenames: false

Re: [PR] Added logging device and logging device options [airflow]

2024-08-20 Thread via GitHub
geraj1010 commented on code in PR #41416: URL: https://github.com/apache/airflow/pull/41416#discussion_r1724297423 ## .pre-commit-config.yaml: ## @@ -779,7 +779,7 @@ repos: files: ^\.pre-commit-config\.yaml$|^.*_installed_providers\.txt$ pass_filenames: false

Re: [PR] Support the unpack operator in signature [airflow]

2024-08-20 Thread via GitHub
uranusjr commented on code in PR #41316: URL: https://github.com/apache/airflow/pull/41316#discussion_r1724281456 ## tests/utils/test_operator_helpers.py: ## @@ -224,3 +224,8 @@ def func(ds_nodash): kwargs_callable(*args, **kwargs) assert "ds_nodash" in str(exc_i

Re: [PR] Support the unpack operator in signature [airflow]

2024-08-20 Thread via GitHub
uranusjr commented on code in PR #41316: URL: https://github.com/apache/airflow/pull/41316#discussion_r1724281456 ## tests/utils/test_operator_helpers.py: ## @@ -224,3 +224,8 @@ def func(ds_nodash): kwargs_callable(*args, **kwargs) assert "ds_nodash" in str(exc_i

Re: [PR] Fix argument unpacking conflict with keyword argument [airflow]

2024-08-20 Thread via GitHub
uranusjr commented on PR #41415: URL: https://github.com/apache/airflow/pull/41415#issuecomment-2300368865 Closing in favour to the other one. This misses `VAR_KEYWORD`. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

Re: [PR] Fix argument unpacking conflict with keyword argument [airflow]

2024-08-20 Thread via GitHub
uranusjr closed pull request #41415: Fix argument unpacking conflict with keyword argument URL: https://github.com/apache/airflow/pull/41415 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the speci

Re: [PR] Unify DAG schedule args and change default to None [airflow]

2024-08-20 Thread via GitHub
uranusjr commented on code in PR #41453: URL: https://github.com/apache/airflow/pull/41453#discussion_r1724278749 ## newsfragments/41453.significant.rst: ## @@ -0,0 +1,11 @@ +Removed legacy scheduling arguments on DAG + +The ``schedule_interval`` and ``timetable`` arguments are

Re: [PR] Unify DAG schedule args and change default to None [airflow]

2024-08-20 Thread via GitHub
mertozb commented on code in PR #41453: URL: https://github.com/apache/airflow/pull/41453#discussion_r1724141565 ## newsfragments/41453.significant.rst: ## @@ -0,0 +1,11 @@ +Removed legacy scheduling arguments on DAG + +The ``schedule_interval`` and ``timetable`` arguments are r

Re: [PR] ci: extracts step from several workflows into action [airflow]

2024-08-20 Thread via GitHub
github-actions[bot] closed pull request #40280: ci: extracts step from several workflows into action URL: https://github.com/apache/airflow/pull/40280 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

Re: [PR] Add instruction: where to put image pull secret for private registry [airflow]

2024-08-20 Thread via GitHub
github-actions[bot] closed pull request #39687: Add instruction: where to put image pull secret for private registry URL: https://github.com/apache/airflow/pull/39687 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the UR

Re: [PR] Updated kuberentes pod operator to find pods only in running state [airflow]

2024-08-20 Thread via GitHub
github-actions[bot] closed pull request #39702: Updated kuberentes pod operator to find pods only in running state URL: https://github.com/apache/airflow/pull/39702 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

Re: [PR] fix: delete pod when base container completes [airflow]

2024-08-20 Thread via GitHub
github-actions[bot] closed pull request #39694: fix: delete pod when base container completes URL: https://github.com/apache/airflow/pull/39694 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the sp

Re: [PR] handle gitsync v4 env var change [airflow]

2024-08-20 Thread via GitHub
github-actions[bot] closed pull request #40333: handle gitsync v4 env var change URL: https://github.com/apache/airflow/pull/40333 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment

Re: [PR] Switch cloudant provider from cloudant library to ibmcloudant library [airflow]

2024-08-20 Thread via GitHub
potiuk commented on code in PR #41555: URL: https://github.com/apache/airflow/pull/41555#discussion_r1724080680 ## airflow/providers/cloudant/cloudant_fake.py: ## Review Comment: We should just leave a note to drop it after the issue is solved ( PR merged) explaining why

Re: [PR] Switch cloudant provider from cloudant library to ibmcloudant library [airflow]

2024-08-20 Thread via GitHub
potiuk commented on code in PR #41555: URL: https://github.com/apache/airflow/pull/41555#discussion_r1724080680 ## airflow/providers/cloudant/cloudant_fake.py: ## Review Comment: We should just leave a note / comment to drop it after the issue is solved ( PR merged) expl

(airflow) branch constraints-main updated: Updating constraints. Github run id:10477963110

2024-08-20 Thread github-bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch constraints-main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/constraints-main by this push: new 18cb55f2b8 Updating con

Re: [PR] Switch cloudant provider from cloudant library to ibmcloudant library [airflow]

2024-08-20 Thread via GitHub
potiuk commented on code in PR #41555: URL: https://github.com/apache/airflow/pull/41555#discussion_r1724076522 ## airflow/providers/cloudant/provider.yaml: ## @@ -25,7 +25,7 @@ state: ready source-date-epoch: 1723969866 # note that those versions are maintained by release man

Re: [PR] Switch cloudant provider from cloudant library to ibmcloudant library [airflow]

2024-08-20 Thread via GitHub
topherinternational commented on code in PR #41555: URL: https://github.com/apache/airflow/pull/41555#discussion_r1724076743 ## airflow/providers/cloudant/cloudant_fake.py: ## Review Comment: @eladkal @potiuk so this is a big hack...static tests were failing bc we run stat

Re: [PR] Switch cloudant provider from cloudant library to ibmcloudant library [airflow]

2024-08-20 Thread via GitHub
topherinternational commented on code in PR #41555: URL: https://github.com/apache/airflow/pull/41555#discussion_r1724073931 ## airflow/providers/cloudant/provider.yaml: ## @@ -25,7 +25,7 @@ state: ready source-date-epoch: 1723969866 # note that those versions are maintained b

Re: [PR] Switch cloudant provider from cloudant library to ibmcloudant library [airflow]

2024-08-20 Thread via GitHub
potiuk commented on code in PR #41555: URL: https://github.com/apache/airflow/pull/41555#discussion_r1724061965 ## docs/spelling_wordlist.txt: ## @@ -250,8 +250,8 @@ classpaths cleartext cli clientId -Cloudant cloudant +CloudantV Review Comment: Add `` around. (`Cloundan

Re: [PR] Switch cloudant provider from cloudant library to ibmcloudant library [airflow]

2024-08-20 Thread via GitHub
topherinternational commented on code in PR #41555: URL: https://github.com/apache/airflow/pull/41555#discussion_r1724058435 ## docs/spelling_wordlist.txt: ## @@ -250,8 +250,8 @@ classpaths cleartext cli clientId -Cloudant cloudant +CloudantV Review Comment: When I tried

Re: [PR] Handle executor events in `dag.test()` [airflow]

2024-08-20 Thread via GitHub
o-nikolas commented on code in PR #41625: URL: https://github.com/apache/airflow/pull/41625#discussion_r1724052994 ## airflow/jobs/scheduler_job_runner.py: ## @@ -739,9 +739,18 @@ def _process_task_event_logs(log_records: deque[Log], session: Session): session.bulk_sav

Re: [PR] Add callback to process Azure Service Bus message contents [airflow]

2024-08-20 Thread via GitHub
perry2of5 commented on PR #41601: URL: https://github.com/apache/airflow/pull/41601#issuecomment-2299839998 This addresses https://github.com/apache/airflow/discussions/26446 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

(airflow) branch main updated: Enable running Pull Requests against v2-10-stable branch (#41624)

2024-08-20 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new e306e7f7bc Enable running Pull Requests against v2-

Re: [PR] Enable running Pull Requests against v2-10-stable branch [airflow]

2024-08-20 Thread via GitHub
potiuk merged PR #41624: URL: https://github.com/apache/airflow/pull/41624 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.a

Re: [PR] Enable running Pull Requests against v2-10-stable branch [airflow]

2024-08-20 Thread via GitHub
potiuk commented on code in PR #41624: URL: https://github.com/apache/airflow/pull/41624#discussion_r1724015353 ## .github/workflows/ci.yml: ## @@ -23,7 +23,7 @@ on: # yamllint disable-line rule:truthy push: branches: ['v[0-9]+-[0-9]+-test'] pull_request: -branch

Re: [PR] Enable running Pull Requests against v2-10-stable branch [airflow]

2024-08-20 Thread via GitHub
hussein-awala commented on code in PR #41624: URL: https://github.com/apache/airflow/pull/41624#discussion_r1724009396 ## .github/workflows/ci.yml: ## @@ -23,7 +23,7 @@ on: # yamllint disable-line rule:truthy push: branches: ['v[0-9]+-[0-9]+-test'] pull_request: -

Re: [PR] Adding config max_active_tasks_include_deferred [airflow]

2024-08-20 Thread via GitHub
gopidesupavan commented on PR #41560: URL: https://github.com/apache/airflow/pull/41560#issuecomment-2299778718 I am not sure exact version, so have updated to 3.0.0 version, but please let me know if this version needs to be changed. -- This is an automated message from the Apache Git Se

Re: [PR] Implement `on_killed` support for tasks during DAG run timeout [airflow]

2024-08-20 Thread via GitHub
MRLab12 commented on PR #41627: URL: https://github.com/apache/airflow/pull/41627#issuecomment-2299777200 I'm having some issues with the test I added. Looking at other tests in `test_scheduler_job` I see that `session.refresh(dr)` is used before asserting the dag run state. In my test it c

[PR] Implement `on_killed` support for tasks during DAG run timeout [airflow]

2024-08-20 Thread via GitHub
MRLab12 opened a new pull request, #41627: URL: https://github.com/apache/airflow/pull/41627 --- This PR addresses issue #41036 by adding support for the `on_killed` callback on tasks that are still running when a DAG run reaches its timeout. Key cha

(airflow) branch main updated (50e2ecb2b8 -> ef5da26166)

2024-08-20 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git from 50e2ecb2b8 Wait sensor async (#41557) add ef5da26166 Add missing flag (#41464) No new revisions were added by thi

Re: [PR] Add missing field to KubernetesHook [airflow]

2024-08-20 Thread via GitHub
potiuk merged PR #41464: URL: https://github.com/apache/airflow/pull/41464 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.a

Re: [PR] Make latest botocore tests green [airflow]

2024-08-20 Thread via GitHub
potiuk closed pull request #41626: Make latest botocore tests green URL: https://github.com/apache/airflow/pull/41626 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubsc

Re: [PR] Fix: Keep compatibility with old FAB versions [airflow]

2024-08-20 Thread via GitHub
potiuk commented on PR #41549: URL: https://github.com/apache/airflow/pull/41549#issuecomment-2299717276 Possibly - but we have not seen this kind of need before - we assumed we backport everything from main and we do not have "2.10 only" fixes - but yes, this one looks like a legitimate c

Re: [PR] Adding config max_active_tasks_include_deferred [airflow]

2024-08-20 Thread via GitHub
gopidesupavan commented on PR #41560: URL: https://github.com/apache/airflow/pull/41560#issuecomment-2299708676 Hi , can you please help me , which airflow_version should i set in the migration file? -- This is an automated message from the Apache Git Service. To respond to the message,

[PR] Make latest botocore tests green [airflow]

2024-08-20 Thread via GitHub
potiuk opened a new pull request, #41626: URL: https://github.com/apache/airflow/pull/41626 The latest botocore tests are conflicting with a few requirements and until apache-beam upcoming version is released we need to do some manual exclusions. Those exclusions should make latest botocore

Re: [PR] Separate FAB migration from Core Airflow migration [airflow]

2024-08-20 Thread via GitHub
potiuk commented on PR #41437: URL: https://github.com/apache/airflow/pull/41437#issuecomment-2299702693 > I would appreciate some help on this CI issue cc @potiuk @jedcunningham Which one :) ? -- This is an automated message from the Apache Git Service. To respond to the message,

Re: [PR] Fix: Keep compatibility with old FAB versions [airflow]

2024-08-20 Thread via GitHub
vincbeck commented on PR #41549: URL: https://github.com/apache/airflow/pull/41549#issuecomment-2299683546 Good call, actually (and I should have thought about it before merging it), do we need it in main at all? Should it be only in 2.10 branch? Because as Kaxil mentioned: > Older F

Re: [PR] Fix: Keep compatibility with old FAB versions [airflow]

2024-08-20 Thread via GitHub
potiuk commented on PR #41549: URL: https://github.com/apache/airflow/pull/41549#issuecomment-2299677966 (we means merging committer shoudl decide if they want to do it themselves or ask the author to do so). -- This is an automated message from the Apache Git Service. To respond to the m

Re: [PR] Fix: Keep compatibility with old FAB versions [airflow]

2024-08-20 Thread via GitHub
potiuk commented on PR #41549: URL: https://github.com/apache/airflow/pull/41549#issuecomment-2299675922 According to the new rules- we should also back-port it to v2-10-test by PR -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitH

Re: [PR] log handler deprecated filename_template argument removal [airflow]

2024-08-20 Thread via GitHub
potiuk commented on PR #41552: URL: https://github.com/apache/airflow/pull/41552#issuecomment-2299671625 Just reverting will not be enough, but I think if the base Handler will accept extra arguments (and ignores them) we should be home -- This is an automated message from the Apache Git

[PR] Handle executor events in `dag.test()` [airflow]

2024-08-20 Thread via GitHub
vincbeck opened a new pull request, #41625: URL: https://github.com/apache/airflow/pull/41625 If the task does not reach the task runner environment, if the task fails, no update is made in the DB. The executor is sending "events" that are handled by the scheduler. In `dag.test()`, there is

(airflow) branch main updated: Wait sensor async (#41557)

2024-08-20 Thread vincbeck
This is an automated email from the ASF dual-hosted git repository. vincbeck pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 50e2ecb2b8 Wait sensor async (#41557) 50e2ecb2b8

Re: [PR] Wait sensor async [airflow]

2024-08-20 Thread via GitHub
vincbeck merged PR #41557: URL: https://github.com/apache/airflow/pull/41557 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow

(airflow) branch main updated: Fix: Keep compatibility with old FAB versions (#41549)

2024-08-20 Thread vincbeck
This is an automated email from the ASF dual-hosted git repository. vincbeck pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new d7d944e381 Fix: Keep compatibility with old FAB v

Re: [I] Access control error after upgrading from version 2.9.3 to 2.10.0 [airflow]

2024-08-20 Thread via GitHub
vincbeck closed issue #41540: Access control error after upgrading from version 2.9.3 to 2.10.0 URL: https://github.com/apache/airflow/issues/41540 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to th

Re: [PR] Fix: Keep compatibility with old FAB versions [airflow]

2024-08-20 Thread via GitHub
vincbeck merged PR #41549: URL: https://github.com/apache/airflow/pull/41549 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow

Re: [PR] Add fixes by breeze/precommit-lint static checks (#41604) [airflow]

2024-08-20 Thread via GitHub
vincbeck merged PR #41618: URL: https://github.com/apache/airflow/pull/41618 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow

(airflow) branch main updated: Add fixes by breeze/precommit-lint static checks (#41604) (#41618)

2024-08-20 Thread vincbeck
This is an automated email from the ASF dual-hosted git repository. vincbeck pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new c78a004210 Add fixes by breeze/precommit-lint sta

(airflow) branch v2-10-stable updated: Make PROD image building works in non-main PRs (#41480) (#41484) (#41623)

2024-08-20 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a commit to branch v2-10-stable in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/v2-10-stable by this push: new 535f27b137 Make PROD image building

Re: [PR] Apply GitHub workflow changes to stable branch [airflow]

2024-08-20 Thread via GitHub
potiuk merged PR #41623: URL: https://github.com/apache/airflow/pull/41623 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.a

Re: [PR] Enable running Pull Requests against v2-10-stable branch [airflow]

2024-08-20 Thread via GitHub
potiuk commented on PR #41624: URL: https://github.com/apache/airflow/pull/41624#issuecomment-2299565442 We have not been doing it before because we were directly cherry-picking and only "push" events happened for v2-10-test, not PRs - by enabling branch protection, and only allowing PRs to

Re: [PR] Enable running Pull Requests against v2-10-stable branch [airflow]

2024-08-20 Thread via GitHub
potiuk commented on PR #41624: URL: https://github.com/apache/airflow/pull/41624#issuecomment-2299562796 cc: @eladkal @utkarsharma2 -> you will also have to merge and backport-that one to #41610 in order to make the CI PR workflow to run -- This is an automated message from the Apache Gi

[PR] Enable running Pull Requests against v2-10-stable branch [airflow]

2024-08-20 Thread via GitHub
potiuk opened a new pull request, #41624: URL: https://github.com/apache/airflow/pull/41624 --- **^ Add meaningful description above** Read the **[Pull Request Guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#pul

Re: [PR] Wait sensor async [airflow]

2024-08-20 Thread via GitHub
RNHTTR commented on code in PR #41557: URL: https://github.com/apache/airflow/pull/41557#discussion_r1723825971 ## airflow/sensors/time_delta.py: ## @@ -89,3 +92,37 @@ def execute(self, context: Context) -> bool | NoReturn: def execute_complete(self, context: Context, event

Re: [PR] Sync v2-10-stable with v2-10-test to release python client v2.10.0 [airflow]

2024-08-20 Thread via GitHub
potiuk commented on PR #41610: URL: https://github.com/apache/airflow/pull/41610#issuecomment-2299550529 BTW, This one will not work still - we also need to enable PRs for v2-10-stable which are not enabled. -- This is an automated message from the Apache Git Service. To respond to t

Re: [PR] Apply GitHub workflow changes to stable branch [airflow]

2024-08-20 Thread via GitHub
potiuk commented on PR #41623: URL: https://github.com/apache/airflow/pull/41623#issuecomment-2299547105 @eladkal @utkarsharma2 -> please merge it regardless if it fails (it should fail with PROD images too) and then you will be able to rerun the failing "sync PR: #41610 -- This is an

Re: [PR] Switch cloudant provider from cloudant library to ibmcloudant library [airflow]

2024-08-20 Thread via GitHub
topherinternational commented on code in PR #41555: URL: https://github.com/apache/airflow/pull/41555#discussion_r1723817914 ## airflow/providers/cloudant/provider.yaml: ## @@ -46,7 +47,17 @@ versions: dependencies: - apache-airflow>=2.8.0 - - cloudant>=2.13.0 + - ibmclo

Re: [PR] Apply GitHub workflow changes to stable branch [airflow]

2024-08-20 Thread via GitHub
potiuk commented on code in PR #41623: URL: https://github.com/apache/airflow/pull/41623#discussion_r1723814929 ## .github/workflows/build-images.yml: ## @@ -241,14 +241,14 @@ jobs: pull-request-target: "true" is-committer-build: ${{ needs.build-info.outputs.is-com

[PR] Apply GitHub workflow changes to stable branch [airflow]

2024-08-20 Thread via GitHub
potiuk opened a new pull request, #41623: URL: https://github.com/apache/airflow/pull/41623 --- **^ Add meaningful description above** Read the **[Pull Request Guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#pul

Re: [PR] Wait sensor async [airflow]

2024-08-20 Thread via GitHub
collinmcnulty commented on code in PR #41557: URL: https://github.com/apache/airflow/pull/41557#discussion_r1723782013 ## tests/sensors/test_time_delta.py: ## @@ -71,3 +71,17 @@ def test_timedelta_sensor(self, defer_mock, should_defer): defer_mock.assert_called_once

Re: [PR] Switch cloudant provider from cloudant library to ibmcloudant library [airflow]

2024-08-20 Thread via GitHub
eladkal commented on code in PR #41555: URL: https://github.com/apache/airflow/pull/41555#discussion_r1723781663 ## airflow/providers/cloudant/provider.yaml: ## @@ -46,7 +47,17 @@ versions: dependencies: - apache-airflow>=2.8.0 - - cloudant>=2.13.0 + - ibmcloudant>=0.9.1

Re: [PR] feat(providers/openai): support batch api in hook/operator/trigger [airflow]

2024-08-20 Thread via GitHub
josix commented on code in PR #41554: URL: https://github.com/apache/airflow/pull/41554#discussion_r1723161345 ## airflow/providers/openai/hooks/openai.py: ## @@ -43,9 +46,26 @@ ChatCompletionUserMessageParam, ) +from airflow.exceptions import AirflowException f

Re: [PR] feat(providers/openai): support batch api in hook/operator/trigger [airflow]

2024-08-20 Thread via GitHub
josix commented on code in PR #41554: URL: https://github.com/apache/airflow/pull/41554#discussion_r1723720593 ## tests/providers/openai/operators/test_openai.py: ## @@ -43,8 +65,53 @@ def test_execute_with_input_text(): @pytest.mark.parametrize("invalid_input", ["", None, 123]

Re: [PR] feat(providers/openai): support batch api in hook/operator/trigger [airflow]

2024-08-20 Thread via GitHub
vincbeck commented on code in PR #41554: URL: https://github.com/apache/airflow/pull/41554#discussion_r1723718574 ## tests/providers/openai/operators/test_openai.py: ## @@ -43,8 +65,53 @@ def test_execute_with_input_text(): @pytest.mark.parametrize("invalid_input", ["", None, 1

Re: [PR] feat(providers/openai): support batch api in hook/operator/trigger [airflow]

2024-08-20 Thread via GitHub
josix commented on code in PR #41554: URL: https://github.com/apache/airflow/pull/41554#discussion_r1723715879 ## tests/providers/openai/operators/test_openai.py: ## @@ -43,8 +65,53 @@ def test_execute_with_input_text(): @pytest.mark.parametrize("invalid_input", ["", None, 123]

Re: [PR] feat(providers/openai): support batch api in hook/operator/trigger [airflow]

2024-08-20 Thread via GitHub
josix commented on code in PR #41554: URL: https://github.com/apache/airflow/pull/41554#discussion_r1723715459 ## tests/providers/openai/operators/test_openai.py: ## @@ -43,8 +65,53 @@ def test_execute_with_input_text(): @pytest.mark.parametrize("invalid_input", ["", None, 123]

Re: [PR] log handler deprecated filename_template argument removal [airflow]

2024-08-20 Thread via GitHub
dirrao commented on PR #41552: URL: https://github.com/apache/airflow/pull/41552#issuecomment-2299399556 @eladkal / @potiuk Let me know if you want me to revert it for providers alone. -- This is an automated message from the Apache Git Service. To respond to the message, please log on

Re: [PR] feat(providers/openai): support batch api in hook/operator/trigger [airflow]

2024-08-20 Thread via GitHub
vincbeck commented on code in PR #41554: URL: https://github.com/apache/airflow/pull/41554#discussion_r1723688798 ## tests/providers/openai/operators/test_openai.py: ## @@ -48,3 +51,55 @@ def test_execute_with_invalid_input(invalid_input): context = Context() with pyte

Re: [PR] Sync v2-10-stable with v2-10-test to release python client v2.10.0 [airflow]

2024-08-20 Thread via GitHub
potiuk commented on PR #41610: URL: https://github.com/apache/airflow/pull/41610#issuecomment-2299338550 > This can be done by submitting separate PRs for each branch, allowing us to continue the release from the 2-10-stable without any unexpected changes and not being concerned about what'

(airflow) branch main updated: Make kerberos an optional and devel dependency for impala and fab (#41616)

2024-08-20 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new d6df0786cf Make kerberos an optional and devel depe

Re: [PR] Make kerberos an optional and devel dependency for impala and fab [airflow]

2024-08-20 Thread via GitHub
potiuk merged PR #41616: URL: https://github.com/apache/airflow/pull/41616 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.a

Re: [PR] [FEAT] databricks repair run with reason match and appropriate new settings [airflow]

2024-08-20 Thread via GitHub
gaurav7261 commented on code in PR #41412: URL: https://github.com/apache/airflow/pull/41412#discussion_r1723654239 ## airflow/providers/databricks/operators/databricks.py: ## @@ -872,7 +896,23 @@ def execute_complete(self, context: Context, event: dict[str, Any] | None = None

Re: [I] BashOperator is killed before it can execute [airflow]

2024-08-20 Thread via GitHub
crutis commented on issue #41597: URL: https://github.com/apache/airflow/issues/41597#issuecomment-2299317507 The entire worker log (same as above) is just this: ``` *** Found local files: *** * /opt/airflow_logs/joblogs/dag_id=hello_world_dag/run_id=manual__2024-08-19T16:43:03.06

Re: [PR] feat(providers/openai): support batch api in hook/operator/trigger [airflow]

2024-08-20 Thread via GitHub
josix commented on code in PR #41554: URL: https://github.com/apache/airflow/pull/41554#discussion_r1723257205 ## airflow/providers/openai/triggers/openai.py: ## @@ -0,0 +1,116 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreem

Re: [I] Status of testing Providers that were prepared on August 19, 2024 [airflow]

2024-08-20 Thread via GitHub
perry2of5 commented on issue #41577: URL: https://github.com/apache/airflow/issues/41577#issuecomment-2299297335 41142 seems to fail with multiple_outputs=True which I'd never tested before. Will debug tonight. On Tuesday, August 20, 2024 at 06:19:08 AM PDT, Błażej Tecław ***@**

[PR] fix(run_generate_migration): remove the removed --use-migration-files argument of "airflow db reset" command [airflow]

2024-08-20 Thread via GitHub
Lee-W opened a new pull request, #41621: URL: https://github.com/apache/airflow/pull/41621 … --- **^ Add meaningful description above** Read the **[Pull Request Guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_request

Re: [PR] feat(providers/openai): support batch api in hook/operator/trigger [airflow]

2024-08-20 Thread via GitHub
josix commented on code in PR #41554: URL: https://github.com/apache/airflow/pull/41554#discussion_r1723614414 ## tests/providers/openai/operators/test_openai.py: ## @@ -48,3 +51,55 @@ def test_execute_with_invalid_input(invalid_input): context = Context() with pytest.

[I] Webserver Graph View Partial Load [airflow]

2024-08-20 Thread via GitHub
collinmcnulty opened a new issue, #41620: URL: https://github.com/apache/airflow/issues/41620 ### Description The Graph view currently load all tasks in the DAG and then allows you to scroll around the graph. This is very inefficient for large DAGs because the user can only actually

Re: [PR] fix(rest-api): Add order_by query param to TI listing APIs (#41283) [airflow]

2024-08-20 Thread via GitHub
omkar-foss commented on PR #41307: URL: https://github.com/apache/airflow/pull/41307#issuecomment-2299246662 Fixed all linting issues and added all your suggestions from comments above. Please review when you get a chance, thanks. -- This is an automated message from the Apache Git Servic

Re: [PR] [redis]adding additional values for existing persistence claim [airflow]

2024-08-20 Thread via GitHub
boring-cyborg[bot] commented on PR #41619: URL: https://github.com/apache/airflow/pull/41619#issuecomment-2299214428 Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our Contributors'

[PR] [redis]adding additional values for existing persistence claim [airflow]

2024-08-20 Thread via GitHub
kongdewen opened a new pull request, #41619: URL: https://github.com/apache/airflow/pull/41619 For redis persistence section, adding a value `existingClaim`, similar to dags and logs, if existingClaim value exisits, using existing claim name instead of using volumeClaimTemplates. *

[PR] Add fixes by breeze/precommit-lint static checks (#41604) [airflow]

2024-08-20 Thread via GitHub
omkar-foss opened a new pull request, #41618: URL: https://github.com/apache/airflow/pull/41618 related: #41604. This adds the fixes by breeze/precommit-lint static checks. --- **^ Add meaningful description above** Read the **[Pull Request Guideli

Re: [PR] Enable EksPodOperator deferrable [airflow]

2024-08-20 Thread via GitHub
o-nikolas commented on PR #41380: URL: https://github.com/apache/airflow/pull/41380#issuecomment-2299197837 I was under the impression this was already fixed by #41178 and the deferrable logic was inherited through the k8s pod operator itself. Were you testing with an older version of the p

Re: [PR] Switch cloudant provider from cloudant library to ibmcloudant library [airflow]

2024-08-20 Thread via GitHub
topherinternational commented on code in PR #41555: URL: https://github.com/apache/airflow/pull/41555#discussion_r1723445214 ## airflow/providers/cloudant/provider.yaml: ## @@ -46,7 +47,17 @@ versions: dependencies: - apache-airflow>=2.8.0 - - cloudant>=2.13.0 + - ibmclo

  1   2   3   >