[PR] Gh 35072 docs move cron basics to authoring and sheduling [airflow]

2024-01-27 Thread via GitHub
ketozhang opened a new pull request, #37049: URL: https://github.com/apache/airflow/pull/37049 closes: #35072 --- Move cron table in `dag-run.rst` to dedicated page in `authoring-and-scheduling/cron.rst`. Refactor references in `dags.rst` -- This is

(airflow) branch main updated: Upgrade yarn stack in pre-commits (#37048)

2024-01-27 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 3b3494c16d Upgrade yarn stack in pre-commits (#3704

Re: [PR] Upgrade yarn stack in pre-commits [airflow]

2024-01-27 Thread via GitHub
potiuk merged PR #37048: URL: https://github.com/apache/airflow/pull/37048 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.a

Re: [PR] Upgrade yarn stack in pre-commits [airflow]

2024-01-27 Thread via GitHub
potiuk commented on PR #37048: URL: https://github.com/apache/airflow/pull/37048#issuecomment-1913487893 Static checks passed. Merging. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specifi

Re: [I] API Bug: patching a task instance state to skipped crashes. [airflow]

2024-01-27 Thread via GitHub
eladkal commented on issue #37034: URL: https://github.com/apache/airflow/issues/37034#issuecomment-1913485932 > Hi, Can I work on this? Assigned -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] Fix `log_query` to format SQL statement correctly in `AthenaOperator` [airflow]

2024-01-27 Thread via GitHub
eladkal commented on PR #36962: URL: https://github.com/apache/airflow/pull/36962#issuecomment-1913485512 > Could you suggest a test case you'd like to see? I think 1 unit test for `query_params_to_string` that gets a mock of `params` and verify the expected result of the function is

[PR] Upgrade yarn stack in pre-commits [airflow]

2024-01-27 Thread via GitHub
potiuk opened a new pull request, #37048: URL: https://github.com/apache/airflow/pull/37048 The openapi-typescripts 6.7.4 generates slightly different typescript output and fails pre-commit. Using that opportunity to upgrade the environment for typescript/www compilation. Upgrading yarn to

Re: [PR] Fix KPO docs argument precedence [airflow]

2024-01-27 Thread via GitHub
boring-cyborg[bot] commented on PR #37047: URL: https://github.com/apache/airflow/pull/37047#issuecomment-1913480028 Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our Contributors'

[PR] Fix KPO docs argument precedence [airflow]

2024-01-27 Thread via GitHub
ketozhang opened a new pull request, #37047: URL: https://github.com/apache/airflow/pull/37047 closes: #35969 --- Fixes the Argument precedence section of KPO docs. Removed incorrect mention of KPO creates pods based off the Airflow connection. It should

Re: [I] [Docker-compose] apache-airflow-providers-daskexecutor FileNotFoundError(2, 'No such file or directory') [airflow]

2024-01-27 Thread via GitHub
lutuantai95 commented on issue #37024: URL: https://github.com/apache/airflow/issues/37024#issuecomment-1913437558 I reviewed chat messages from dask community for issues of DaskExecutor. So maybe I will research and switch to using some remote offical executor that you recommended. Than

Re: [PR] Add encryption methods to serde for primitives [airflow]

2024-01-27 Thread via GitHub
github-actions[bot] closed pull request #35867: Add encryption methods to serde for primitives URL: https://github.com/apache/airflow/pull/35867 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the s

Re: [PR] Add support for catchup="ignore_first" to disable catchup for the first DagRun [airflow]

2024-01-27 Thread via GitHub
github-actions[bot] closed pull request #35392: Add support for catchup="ignore_first" to disable catchup for the first DagRun URL: https://github.com/apache/airflow/pull/35392 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

Re: [PR] Allow changing the unfinished ti state [airflow]

2024-01-27 Thread via GitHub
github-actions[bot] commented on PR #36078: URL: https://github.com/apache/airflow/pull/36078#issuecomment-1913380129 This pull request has been automatically marked as stale because it has not had recent activity. It will be closed in 5 days if no further activity occurs. Thank you for you

Re: [PR] Adding Yeedu Provider to the airflow. [airflow]

2024-01-27 Thread via GitHub
github-actions[bot] closed pull request #36126: Adding Yeedu Provider to the airflow. URL: https://github.com/apache/airflow/pull/36126 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific c

Re: [PR] fix: downstream on list [airflow]

2024-01-27 Thread via GitHub
github-actions[bot] closed pull request #36107: fix: downstream on list URL: https://github.com/apache/airflow/pull/36107 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To uns

Re: [PR] Add bigquery partition sensor for sequential partition id [airflow]

2024-01-27 Thread via GitHub
github-actions[bot] commented on PR #36201: URL: https://github.com/apache/airflow/pull/36201#issuecomment-1913380108 This pull request has been automatically marked as stale because it has not had recent activity. It will be closed in 5 days if no further activity occurs. Thank you for you

Re: [I] Serialization error for op_kwargs and op_args when none are passed [airflow]

2024-01-27 Thread via GitHub
github-actions[bot] commented on issue #36738: URL: https://github.com/apache/airflow/issues/36738#issuecomment-1913380097 This issue has been automatically marked as stale because it has been open for 14 days with no response from the author. It will be closed in next 7 days if no further

Re: [I] API Bug: patching a task instance state to skipped crashes. [airflow]

2024-01-27 Thread via GitHub
killua1zoldyck commented on issue #37034: URL: https://github.com/apache/airflow/issues/37034#issuecomment-1913363927 Hi, Can I work on this? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the s

Re: [PR] Add "airflow users reset-password" command [airflow]

2024-01-27 Thread via GitHub
jscheffl commented on code in PR #37044: URL: https://github.com/apache/airflow/pull/37044#discussion_r1468667674 ## airflow/providers/fab/auth_manager/security_manager/override.py: ## @@ -470,7 +470,9 @@ def reset_password(self, userid, password): user = self.get_user_

Re: [PR] Python3.12 [airflow]

2024-01-27 Thread via GitHub
potiuk commented on PR #36755: URL: https://github.com/apache/airflow/pull/36755#issuecomment-1913341944 The good news ( I am following the universal_pathlib PR https://github.com/fsspec/universal_pathlib/pull/152) the author made it green, and I tested installing the universal_pathlib libr

Re: [I] Status of testing Providers that were prepared on January 26, 2024 [airflow]

2024-01-27 Thread via GitHub
potiuk commented on issue #36948: URL: https://github.com/apache/airflow/issues/36948#issuecomment-1913334433 Thank you everyone. Providers are released! I invite everyone to help improve providers for the next release, a list of open issues can be found [here](https://github.com/apa

Re: [I] Status of testing Providers that were prepared on January 26, 2024 [airflow]

2024-01-27 Thread via GitHub
potiuk closed issue #36948: Status of testing Providers that were prepared on January 26, 2024 URL: https://github.com/apache/airflow/issues/36948 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

(airflow) branch main updated: Update providers metadata 2024-01-27 (#37046)

2024-01-27 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new d1e5831cca Update providers metadata 2024-01-27 (#3

Re: [PR] Update providers metadata 2024-01-27 [airflow]

2024-01-27 Thread via GitHub
potiuk merged PR #37046: URL: https://github.com/apache/airflow/pull/37046 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.a

Re: [I] UI: DAG Conf Parameters Form - Param Type of Boolean, Null converted to String for JSON config [airflow]

2024-01-27 Thread via GitHub
jscheffl commented on issue #36369: URL: https://github.com/apache/airflow/issues/36369#issuecomment-1913326274 > @jscheffl sorry for the delay. That is a great question, at a high level, the goal is to create an optional parameter so that code which the DAG calls can define a default. With

Re: [PR] The task is stuck in a queued state forever in case of pod launch errors [airflow]

2024-01-27 Thread via GitHub
hussein-awala commented on code in PR #36882: URL: https://github.com/apache/airflow/pull/36882#discussion_r1468587468 ## airflow/providers/cncf/kubernetes/executors/kubernetes_executor.py: ## @@ -434,9 +434,9 @@ def sync(self) -> None: )

[PR] Update providers metadata 2024-01-27 [airflow]

2024-01-27 Thread via GitHub
potiuk opened a new pull request, #37046: URL: https://github.com/apache/airflow/pull/37046 --- **^ Add meaningful description above** Read the **[Pull Request Guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#pul

Re: [PR] Add documentation for packages - 2024-01-26 [airflow-site]

2024-01-27 Thread via GitHub
potiuk merged PR #944: URL: https://github.com/apache/airflow-site/pull/944 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.

Re: [I] Dag grouping [airflow]

2024-01-27 Thread via GitHub
mariotaddeucci commented on issue #37042: URL: https://github.com/apache/airflow/issues/37042#issuecomment-1913293229 Just for curiosity, did you try to create a pool for this scenario? I have an issue based when i consume an API and hit the rate limit, pools are good to manage this. A

svn commit: r66892 - /dev/airflow/providers/ /release/airflow/providers/

2024-01-27 Thread potiuk
Author: potiuk Date: Sat Jan 27 18:47:45 2024 New Revision: 66892 Log: Release Airflow Providers on 2024-01-27 Added: release/airflow/providers/apache_airflow_providers_airbyte-3.6.0-py3-none-any.whl - copied unchanged from r66891, dev/airflow/providers/apache_airflow_providers_airbyt

Re: [I] Celery worker enters a catatonic state after redis restart [airflow]

2024-01-27 Thread via GitHub
potiuk commented on issue #26542: URL: https://github.com/apache/airflow/issues/26542#issuecomment-1913286153 > We could set those in the Helm Chart too ? Yeah. Good idea. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitH

Re: [PR] Add ability to automatically set DAG to off after X times it failed sequentially [airflow]

2024-01-27 Thread via GitHub
pateash commented on code in PR #36935: URL: https://github.com/apache/airflow/pull/36935#discussion_r1468545330 ## airflow/models/dagrun.py: ## @@ -498,6 +498,37 @@ def fetch_task_instances( tis = tis.where(TI.task_id.in_(task_ids)) return session.scalars(

Re: [PR] Move CI documentation to inside Breeze docs [airflow]

2024-01-27 Thread via GitHub
potiuk merged PR #37039: URL: https://github.com/apache/airflow/pull/37039 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.a

(airflow) branch python3.12 updated (c05a8e1c15 -> 539a733f13)

2024-01-27 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a change to branch python3.12 in repository https://gitbox.apache.org/repos/asf/airflow.git omit c05a8e1c15 fixup! python 3.12 support add 539a733f13 fixup! python 3.12 support This update added new revis

Re: [PR] Python3.12 [airflow]

2024-01-27 Thread via GitHub
potiuk commented on PR #36755: URL: https://github.com/apache/airflow/pull/36755#issuecomment-1913272732 Seems like we have a green PR in the universal_pathlib `Python 3.12` branch - added installing it from the Github Branch to see if it fixes the problem :) -- This is an automated mes

(airflow) branch python3.12 updated (84763a9a09 -> c05a8e1c15)

2024-01-27 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a change to branch python3.12 in repository https://gitbox.apache.org/repos/asf/airflow.git discard 84763a9a09 python 3.12 support add e9a4bcaa16 Improve ExternalTaskSensor Async Implementation (#36916)

Re: [PR] Add "airflow users reset-password" command [airflow]

2024-01-27 Thread via GitHub
dirrao commented on code in PR #37044: URL: https://github.com/apache/airflow/pull/37044#discussion_r1468516075 ## airflow/providers/fab/auth_manager/security_manager/override.py: ## @@ -470,7 +470,9 @@ def reset_password(self, userid, password): user = self.get_user_by

Re: [PR] Add "airflow users reset-password" command [airflow]

2024-01-27 Thread via GitHub
dirrao commented on code in PR #37044: URL: https://github.com/apache/airflow/pull/37044#discussion_r1468516075 ## airflow/providers/fab/auth_manager/security_manager/override.py: ## @@ -470,7 +470,9 @@ def reset_password(self, userid, password): user = self.get_user_by

(airflow) branch main updated: add description to queue_when (#36997)

2024-01-27 Thread taragolis
This is an automated email from the ASF dual-hosted git repository. taragolis pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 1bb8126ddc add description to queue_when (#36997

Re: [PR] add description to queue_when [airflow]

2024-01-27 Thread via GitHub
Taragolis merged PR #36997: URL: https://github.com/apache/airflow/pull/36997 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflo

Re: [PR] add description to queue_when [airflow]

2024-01-27 Thread via GitHub
boring-cyborg[bot] commented on PR #36997: URL: https://github.com/apache/airflow/pull/36997#issuecomment-1913188331 Awesome work, congrats on your first merged pull request! You are invited to check our [Issue Tracker](https://github.com/apache/airflow/issues) for additional contributions.

[PR] Add "airflow users reset-password" command [airflow]

2024-01-27 Thread via GitHub
aritra24 opened a new pull request, #37044: URL: https://github.com/apache/airflow/pull/37044 closes: #37009 Added support for reset-password command which can either randomly generate a password, or take a password from user. --- **^ Add me

Re: [PR] Draft: kubernetes version bump [airflow]

2024-01-27 Thread via GitHub
dirrao commented on PR #37040: URL: https://github.com/apache/airflow/pull/37040#issuecomment-1913149355 > `#52 25.79 The conflict is caused by: [7496](https://github.com/apache/airflow/actions/runs/7676485430/job/20924456098?pr=37040#step:7:7567) #52 25.79 apache-airflow-providers-cncf-kub

[PR] Feature make hive client connection simpler [airflow]

2024-01-27 Thread via GitHub
romsharon98 opened a new pull request, #37043: URL: https://github.com/apache/airflow/pull/37043 related: #36221 this pr solve: - complication of running hive client with proxy user. - make hive client connection more user friendly. --- **^ Add

Re: [I] WORKER_AUTOSCALE should not allow max workers < min workers [airflow]

2024-01-27 Thread via GitHub
Bowrna commented on issue #35332: URL: https://github.com/apache/airflow/issues/35332#issuecomment-1913129539 yes it works the same way for both config. yeah, let me update the docs -- This is an automated message from the Apache Git Service. To respond to the message, please log on to Git

(airflow) branch constraints-main updated: Updating constraints. Github run id:7677986131

2024-01-27 Thread github-bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch constraints-main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/constraints-main by this push: new 24a303eaf8 Updating con

Re: [PR] Draft: kubernetes version bump [airflow]

2024-01-27 Thread via GitHub
hussein-awala commented on PR #37040: URL: https://github.com/apache/airflow/pull/37040#issuecomment-1913107394 Close and reopen to run all the tests -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

Re: [PR] Draft: kubernetes version bump [airflow]

2024-01-27 Thread via GitHub
hussein-awala closed pull request #37040: Draft: kubernetes version bump URL: https://github.com/apache/airflow/pull/37040 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To un

(airflow) branch main updated: Allow for helm plugin installation to be happening more than once (#37038)

2024-01-27 Thread husseinawala
This is an automated email from the ASF dual-hosted git repository. husseinawala pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new eb86df7d68 Allow for helm plugin installation

Re: [PR] Allow for helm plugin installation to be happening more than once [airflow]

2024-01-27 Thread via GitHub
hussein-awala merged PR #37038: URL: https://github.com/apache/airflow/pull/37038 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@ai

Re: [I] Dag grouping [airflow]

2024-01-27 Thread via GitHub
boring-cyborg[bot] commented on issue #37042: URL: https://github.com/apache/airflow/issues/37042#issuecomment-1913097414 Thanks for opening your first issue here! Be sure to follow the issue template! If you are willing to raise PR to address this issue please do so, no need to wait for ap

[I] Dag grouping [airflow]

2024-01-27 Thread via GitHub
MatthewStrickland opened a new issue, #37042: URL: https://github.com/apache/airflow/issues/37042 ### Description Support grouping dags, where a dag can be part of many groups, with a configuration similar to max_active_runs across the whole group. (I'm sure this must have bee

Re: [PR] Draft: kubernetes version bump [airflow]

2024-01-27 Thread via GitHub
dirrao commented on PR #37040: URL: https://github.com/apache/airflow/pull/37040#issuecomment-1913084786 @potiuk Prod image build failing due to user constraints. How to update the user constraints file and test? -- This is an automated message from the Apache Git Service. To respond