[PR] Develop [airflow]

2024-02-05 Thread via GitHub
raprobert23 opened a new pull request, #37174: URL: https://github.com/apache/airflow/pull/37174 --- **^ Add meaningful description above** Read the **[Pull Request Guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rs

Re: [PR] Develop [airflow]

2024-02-05 Thread via GitHub
raprobert23 closed pull request #37174: Develop URL: https://github.com/apache/airflow/pull/37174 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commit

[PR] Re-add old licences-UI file to gitignore [airflow]

2024-02-05 Thread via GitHub
potiuk opened a new pull request, #37175: URL: https://github.com/apache/airflow/pull/37175 It's been moved recently, so if someone built airflow before, this file might still be around, we should just ignore it. --- **^ Add meaningful description above*

Re: [I] Datasets: Allow time based force dependency when dataset is not ready. [airflow]

2024-02-05 Thread via GitHub
dakshin-k commented on issue #30974: URL: https://github.com/apache/airflow/issues/30974#issuecomment-1926517418 @cmarteepants Yes that would solve the problem at hand, however it's not ideal as re-computing `C` multiple times could cause an issue as there's no guarantee that operation is i

Re: [PR] Update providers metadata 2024-02-04 [airflow]

2024-02-05 Thread via GitHub
potiuk commented on code in PR #37163: URL: https://github.com/apache/airflow/pull/37163#discussion_r1477864819 ## licenses/LICENSES-ui.txt: ## Review Comment: This is an old file that used to be there and .gitignored, but it's remaining there in your code from past airflo

Re: [PR] Re-add old licences-UI file to gitignore [airflow]

2024-02-05 Thread via GitHub
potiuk merged PR #37175: URL: https://github.com/apache/airflow/pull/37175 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.a

(airflow) branch main updated: Re-add old licences-UI file to gitignore (#37175)

2024-02-05 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 8664ef258e Re-add old licences-UI file to gitignore

Re: [I] Running tasks marked as skipped on DagRun timeout [airflow]

2024-02-05 Thread via GitHub
raphaelauv commented on issue #30264: URL: https://github.com/apache/airflow/issues/30264#issuecomment-1926604905 I agree with @pankajkoti ``` Failed / Shutdown sounds more reasonable than the Skipped state. ``` I think we should re-open this issue -- This is an automated

Re: [I] DatabricksSQLOperator struggles with parsing the data [airflow]

2024-02-05 Thread via GitHub
amoralca16 commented on issue #36839: URL: https://github.com/apache/airflow/issues/36839#issuecomment-1926613645 Using pandas-2.1.4 solved the issue for me (Airflow 2.8.1) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and u

Re: [I] DatabricksSQLOperator struggles with parsing the data [airflow]

2024-02-05 Thread via GitHub
amoralca16 commented on issue #36838: URL: https://github.com/apache/airflow/issues/36838#issuecomment-1926615969 Using pandas-2.1.4 solved the issue for me (Airflow 2.8.1) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and u

Re: [PR] Add new run type "triggered" to distinguish manual DAG runs and triggered DAG runs [airflow]

2024-02-05 Thread via GitHub
VladaZakharova commented on PR #37087: URL: https://github.com/apache/airflow/pull/37087#issuecomment-1926640135 Hi @uranusjr @eladkal ! Can we merge these changes? Thanks :) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

Re: [PR] Fix location requirement in DataflowTemplatedJobStartOperator [airflow]

2024-02-05 Thread via GitHub
VladaZakharova commented on PR #37069: URL: https://github.com/apache/airflow/pull/37069#issuecomment-1926661951 Hi @dirrao @potiuk ! Can you please check the changes here? Thank you! -- This is an automated message from the Apache Git Service. To respond to the message, please log on t

[PR] add endpoint to delete DatasetDagRunQueue [airflow]

2024-02-05 Thread via GitHub
Lee-W opened a new pull request, #37176: URL: https://github.com/apache/airflow/pull/37176 --- **^ Add meaningful description above** Read the **[Pull Request Guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#pull

[PR] Add airflow version substitution into Docker Compose Howto [airflow]

2024-02-05 Thread via GitHub
Taragolis opened a new pull request, #37177: URL: https://github.com/apache/airflow/pull/37177 related: https://github.com/apache/airflow/pull/36714 I'm not sure why `|version|` substitution not works correctly in particular this blocks out of the box, so I add HTML in

Re: [PR] Fix Airflow serialization for namedtuple [airflow]

2024-02-05 Thread via GitHub
bolkedebruin commented on code in PR #37168: URL: https://github.com/apache/airflow/pull/37168#discussion_r1478045494 ## airflow/serialization/serde.py: ## @@ -147,6 +147,13 @@ def serialize(o: object, depth: int = 0) -> U | None: if is_serialized: return e

Re: [I] Improve classes and modules docstring [airflow]

2024-02-05 Thread via GitHub
NewtonChutney commented on issue #32535: URL: https://github.com/apache/airflow/issues/32535#issuecomment-1926795608 > The purpose of this task is to enhance the docstrings, focusing particularly on utilizing the Sphinx cross-referencing syntax to improve the documentation. I'm sadly

(airflow) branch main updated: Fix location requirement in DataflowTemplatedJobStartOperator (#37069)

2024-02-05 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 1fe8cddde8 Fix location requirement in DataflowTem

Re: [PR] Fix location requirement in DataflowTemplatedJobStartOperator [airflow]

2024-02-05 Thread via GitHub
potiuk merged PR #37069: URL: https://github.com/apache/airflow/pull/37069 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.a

Re: [PR] Fix location requirement in DataflowTemplatedJobStartOperator [airflow]

2024-02-05 Thread via GitHub
boring-cyborg[bot] commented on PR #37069: URL: https://github.com/apache/airflow/pull/37069#issuecomment-1926897524 Awesome work, congrats on your first merged pull request! You are invited to check our [Issue Tracker](https://github.com/apache/airflow/issues) for additional contributions.

[PR] [Doc]: cncf provider fix spark operator doc link [airflow]

2024-02-05 Thread via GitHub
Aakcht opened a new pull request, #37179: URL: https://github.com/apache/airflow/pull/37179 Fix a minor documentation link issue that is visible at https://airflow.apache.org/docs/apache-airflow-providers-cncf-kubernetes/stable/operators.html#sparkkubernetesoperator .

Re: [PR] [Doc]: cncf provider fix spark operator doc link [airflow]

2024-02-05 Thread via GitHub
potiuk merged PR #37179: URL: https://github.com/apache/airflow/pull/37179 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.a

(airflow) branch main updated (1fe8cddde8 -> b7731280fc)

2024-02-05 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git from 1fe8cddde8 Fix location requirement in DataflowTemplatedJobStartOperator (#37069) add b7731280fc [Doc]: cncf prov

Re: [PR] Added ability to enable/disable scheduler and webserver [airflow]

2024-02-05 Thread via GitHub
potiuk merged PR #36991: URL: https://github.com/apache/airflow/pull/36991 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.a

Re: [PR] Added ability to enable/disable scheduler and webserver [airflow]

2024-02-05 Thread via GitHub
boring-cyborg[bot] commented on PR #36991: URL: https://github.com/apache/airflow/pull/36991#issuecomment-1926965199 Awesome work, congrats on your first merged pull request! You are invited to check our [Issue Tracker](https://github.com/apache/airflow/issues) for additional contributions.

(airflow) branch main updated: Added ability to enable/disable scheduler and webserver (#36991)

2024-02-05 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 681d2ba012 Added ability to enable/disable schedule

[I] Run the same trigger in 2+ triggerer in the same time. [airflow]

2024-02-05 Thread via GitHub
MaksYermak opened a new issue, #37180: URL: https://github.com/apache/airflow/issues/37180 ### Apache Airflow version Other Airflow 2 version (please specify below) ### If "Other Airflow 2 version" selected, which one? 2.6.3 ### What happened? When trying to

Re: [PR] Implement task duration page in react. [airflow]

2024-02-05 Thread via GitHub
tirkarthi commented on PR #35863: URL: https://github.com/apache/airflow/pull/35863#issuecomment-1927081448 @bbovenzi Is there anything I can help here to have this reviewed and merged? There is one pending item left to redirect links from old task duration page but I would like your though

(airflow) branch constraints-main updated: Updating constraints. Github run id:7784981276

2024-02-05 Thread github-bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch constraints-main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/constraints-main by this push: new 9699220906 Updating con

(airflow) branch main updated: Add airflow version substitution into Docker Compose Howto (#37177)

2024-02-05 Thread taragolis
This is an automated email from the ASF dual-hosted git repository. taragolis pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 8fb4c71857 Add airflow version substitution into

Re: [PR] Add airflow version substitution into Docker Compose Howto [airflow]

2024-02-05 Thread via GitHub
Taragolis merged PR #37177: URL: https://github.com/apache/airflow/pull/37177 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflo

Re: [PR] Partially enable D401 rule [airflow]

2024-02-05 Thread via GitHub
potiuk commented on code in PR #37167: URL: https://github.com/apache/airflow/pull/37167#discussion_r1478332654 ## pyproject.toml: ## @@ -1273,7 +1273,10 @@ extend-select = [ "D2", "D3", "D400", -# "D401", # Not enabled by ruff, but we don't want it Review Co

Re: [PR] Partially enable D401 rule [airflow]

2024-02-05 Thread via GitHub
Taragolis commented on code in PR #37167: URL: https://github.com/apache/airflow/pull/37167#discussion_r1478344631 ## pyproject.toml: ## @@ -1273,7 +1273,10 @@ extend-select = [ "D2", "D3", "D400", -# "D401", # Not enabled by ruff, but we don't want it Review

Re: [I] All DAGs triggered through UI fail after upgrading from 2.6.1 to 2.8.1 when using LocalExecutor [airflow]

2024-02-05 Thread via GitHub
luederm commented on issue #37149: URL: https://github.com/apache/airflow/issues/37149#issuecomment-1927135274 Hello @jscheffl, thanks for looking into this. > From your descriptions I assume you can re-produce the error also with a fresh/clean setup? Yep, I tested with a fresh

Re: [PR] Affixing exit summary to publish-docs command [airflow]

2024-02-05 Thread via GitHub
amoghrajesh commented on PR #37172: URL: https://github.com/apache/airflow/pull/37172#issuecomment-1927152914 > Actually one thing should be changed. Rather than ysing print() you should use `get_console().print()` (and colors - `[success]` for published and `[warning]` for skipped + plus u

Re: [PR] Doc: fix version to 2.8.0 [airflow]

2024-02-05 Thread via GitHub
Taragolis closed pull request #36714: Doc: fix version to 2.8.0 URL: https://github.com/apache/airflow/pull/36714 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe

Re: [PR] Doc: fix version to 2.8.0 [airflow]

2024-02-05 Thread via GitHub
Taragolis commented on PR #36714: URL: https://github.com/apache/airflow/pull/36714#issuecomment-1927155425 Closed in favor of #37177 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

Re: [PR] Affixing exit summary to publish-docs command [airflow]

2024-02-05 Thread via GitHub
amoghrajesh commented on PR #37172: URL: https://github.com/apache/airflow/pull/37172#issuecomment-1927168578 Good feedback @potiuk this is how it looks now: (I am quite impressed actually) ``` (new-env) ➜ airflow git:(summaryPublishDocs) ✗ breeze release-management publish-doc

Re: [PR] Affixing exit summary to publish-docs command [airflow]

2024-02-05 Thread via GitHub
amoghrajesh commented on PR #37172: URL: https://github.com/apache/airflow/pull/37172#issuecomment-1927169952 Woops my bad, it has coloring. This is a screenshot ![image](https://github.com/apache/airflow/assets/35884252/cae45b3f-bc01-485b-855a-dd82cc3ad110) -- This is an automated

Re: [PR] add endpoint to delete DatasetDagRunQueue [airflow]

2024-02-05 Thread via GitHub
Lee-W commented on PR #37176: URL: https://github.com/apache/airflow/pull/37176#issuecomment-1927174666 I came up a few questions during implementation 1. should we use `uri` instead of `dataset_id` in this endpoint? but we'll have to use Dataset to find its id and then query DDRQ.

Re: [PR] Affixing exit summary to publish-docs command [airflow]

2024-02-05 Thread via GitHub
potiuk commented on code in PR #37172: URL: https://github.com/apache/airflow/pull/37172#discussion_r1478392055 ## dev/breeze/src/airflow_breeze/commands/release_management_commands.py: ## @@ -1230,14 +1229,23 @@ def run_publish_docs_in_parallel( )

Re: [PR] Affixing exit summary to publish-docs command [airflow]

2024-02-05 Thread via GitHub
potiuk commented on code in PR #37172: URL: https://github.com/apache/airflow/pull/37172#discussion_r1478392441 ## dev/breeze/src/airflow_breeze/commands/release_management_commands.py: ## @@ -1307,10 +1315,24 @@ def publish_docs( override_versioned=override_version

Re: [PR] Affixing exit summary to publish-docs command [airflow]

2024-02-05 Thread via GitHub
potiuk commented on code in PR #37172: URL: https://github.com/apache/airflow/pull/37172#discussion_r1478392828 ## dev/breeze/src/airflow_breeze/commands/release_management_commands.py: ## @@ -1307,10 +1315,24 @@ def publish_docs( override_versioned=override_version

Re: [PR] Affixing exit summary to publish-docs command [airflow]

2024-02-05 Thread via GitHub
potiuk commented on code in PR #37172: URL: https://github.com/apache/airflow/pull/37172#discussion_r1478391758 ## dev/breeze/src/airflow_breeze/commands/release_management_commands.py: ## @@ -1230,14 +1229,23 @@ def run_publish_docs_in_parallel( )

Re: [PR] Implement task duration page in react. [airflow]

2024-02-05 Thread via GitHub
bbovenzi commented on PR #35863: URL: https://github.com/apache/airflow/pull/35863#issuecomment-1927208576 > @bbovenzi Is there anything I can help here to have this reviewed and merged? There is one pending item left to redirect links from old task duration page but I would like your thoug

(airflow) branch constraints-main updated: Updating constraints. Github run id:7785777791

2024-02-05 Thread github-bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch constraints-main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/constraints-main by this push: new a28c9725e2 Updating con

Re: [I] Quickstart: not having `virtualenv` installed results in broken DAGs [airflow-site]

2024-02-05 Thread via GitHub
potiuk commented on issue #945: URL: https://github.com/apache/airflow-site/issues/945#issuecomment-1927254240 The problem is that those examples are there, and they will fail to import resulting in red nasty errors as you see when you enable exampes. That's a bad first-encounter with Airfl

Re: [PR] Fix assignment of template field in `__init__` in `custom_job` [airflow]

2024-02-05 Thread via GitHub
potiuk commented on code in PR #36789: URL: https://github.com/apache/airflow/pull/36789#discussion_r1478436450 ## airflow/providers/google/cloud/operators/vertex_ai/custom_job.py: ## @@ -445,9 +445,19 @@ def __init__( self, *, command: Sequence[str] =

Re: [PR] Added ability to enable/disable scheduler and webserver [airflow]

2024-02-05 Thread via GitHub
jedcunningham commented on PR #36991: URL: https://github.com/apache/airflow/pull/36991#issuecomment-1927260888 Thanks @kajanajustin! Congrats on your first commit 🎉 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

Re: [PR] Fix assignment of template field in `__init__` in `KubernetesPodOperator` [airflow]

2024-02-05 Thread via GitHub
potiuk commented on code in PR #37010: URL: https://github.com/apache/airflow/pull/37010#discussion_r1478438159 ## airflow/providers/cncf/kubernetes/operators/pod.py: ## @@ -309,16 +309,19 @@ def __init__( self.labels = labels or {} self.startup_timeout_seconds

Re: [PR] Fix assignment of template field in `__init__` in `KubernetesPodOperator` [airflow]

2024-02-05 Thread via GitHub
potiuk merged PR #37010: URL: https://github.com/apache/airflow/pull/37010 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.a

(airflow) branch main updated: Fix assignment of template field in `__init__` in `KubernetesPodOperator` (#37010)

2024-02-05 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 7932958488 Fix assignment of template field in `__i

Re: [I] Cli export / import roles including permissions Part 2 (refinement) [airflow]

2024-02-05 Thread via GitHub
rawwar commented on issue #36580: URL: https://github.com/apache/airflow/issues/36580#issuecomment-1927285438 This issue seems to be fixed. Can this be closed? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL ab

Re: [I] Datasets: Allow time based force dependency when dataset is not ready. [airflow]

2024-02-05 Thread via GitHub
cmarteepants commented on issue #30974: URL: https://github.com/apache/airflow/issues/30974#issuecomment-1927291755 Gotcha, makes sense. Paraphrasing: DAG `C`: DAG that contains Task that outputs to dataset `C` The crux of the problem is that DAG `C` cannot leverage data

Re: [PR] Update providers metadata 2024-02-04 [airflow]

2024-02-05 Thread via GitHub
hussein-awala merged PR #37163: URL: https://github.com/apache/airflow/pull/37163 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@ai

(airflow) branch main updated: Update providers metadata 2024-02-04 (#37163)

2024-02-05 Thread husseinawala
This is an automated email from the ASF dual-hosted git repository. husseinawala pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new d1aea8ec22 Update providers metadata 2024-02-

Re: [I] `PythonVirtualenvOperator` crashes if any `python_callable` function is defined in the same source as DAG [airflow]

2024-02-05 Thread via GitHub
rawwar commented on issue #35529: URL: https://github.com/apache/airflow/issues/35529#issuecomment-1927358607 > 2. Make the original DAG available by copying it to the temporary directory and rename it to the modified module name(which is `unusual_prefix_*` . This temporary directory is cre

(airflow) branch python3.12 updated (568fb7f81e -> 030f4a2d5a)

2024-02-05 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a change to branch python3.12 in repository https://gitbox.apache.org/repos/asf/airflow.git omit 568fb7f81e python 3.12 support add 11564a0297 Fixed the hardcoded default namespace value for GCP Data Fusi

Re: [PR] Add Teradata Provider [airflow]

2024-02-05 Thread via GitHub
SatishChGit commented on code in PR #36953: URL: https://github.com/apache/airflow/pull/36953#discussion_r1478588646 ## airflow/providers/teradata/example_dags/example_teradata_to_teradata_transfer_operator.py: ## @@ -0,0 +1,158 @@ +# Review Comment: example dags are removed

(airflow) branch constraints-main updated: Updating constraints. Github run id:7787317119

2024-02-05 Thread github-bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch constraints-main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/constraints-main by this push: new 4e0f9d1886 Updating con

Re: [PR] Add Teradata Provider [airflow]

2024-02-05 Thread via GitHub
SatishChGit commented on code in PR #36953: URL: https://github.com/apache/airflow/pull/36953#discussion_r1477019071 ## airflow/providers/teradata/operators/teradata.py: ## @@ -0,0 +1,67 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor l

Re: [PR] Add Teradata Provider [airflow]

2024-02-05 Thread via GitHub
SatishChGit commented on code in PR #36953: URL: https://github.com/apache/airflow/pull/36953#discussion_r1478592864 ## pyproject.toml: ## @@ -533,6 +533,7 @@ alibaba = [ "oss2>=2.14.0", ] amazon = [ + "PyAthena>=3.0.10", Review Comment: No. Now it should be in sync wi

Re: [PR] Add Teradata Provider [airflow]

2024-02-05 Thread via GitHub
SatishChGit commented on code in PR #36953: URL: https://github.com/apache/airflow/pull/36953#discussion_r1478594282 ## tests/providers/teradata/hooks/test_teradata.py: ## @@ -0,0 +1,272 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor l

Re: [PR] Add Teradata Provider [airflow]

2024-02-05 Thread via GitHub
SatishChGit commented on code in PR #36953: URL: https://github.com/apache/airflow/pull/36953#discussion_r1478598188 ## airflow/providers/teradata/hooks/teradata.py: ## @@ -0,0 +1,212 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor lice

Re: [I] All DAGs triggered through UI fail after upgrading from 2.6.1 to 2.8.1 when using LocalExecutor [airflow]

2024-02-05 Thread via GitHub
potiuk commented on issue #37149: URL: https://github.com/apache/airflow/issues/37149#issuecomment-1927500318 I tried to reproduce it with venv and could not. I guess the problem is with mixing of conda packages and `pip` packages - looks like you have conda python abi installed (which is t

Re: [PR] Add Teradata Provider [airflow]

2024-02-05 Thread via GitHub
SatishChGit commented on code in PR #36953: URL: https://github.com/apache/airflow/pull/36953#discussion_r1478597184 ## tests/providers/teradata/operators/test_teradata.py: ## @@ -0,0 +1,122 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one +# or more contribut

Re: [PR] Add Teradata Provider [airflow]

2024-02-05 Thread via GitHub
SatishChGit commented on code in PR #36953: URL: https://github.com/apache/airflow/pull/36953#discussion_r1478600448 ## airflow/providers/teradata/hooks/teradata.py: ## @@ -0,0 +1,212 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor lice

Re: [PR] Add Teradata Provider [airflow]

2024-02-05 Thread via GitHub
SatishChGit commented on code in PR #36953: URL: https://github.com/apache/airflow/pull/36953#discussion_r1478601238 ## docs/apache-airflow-providers-teradata/redirects.txt: ## Review Comment: Removed. Placed this as like other providers aws, sqlite. -- This is an auto

Re: [PR] Add Teradata Provider [airflow]

2024-02-05 Thread via GitHub
SatishChGit commented on code in PR #36953: URL: https://github.com/apache/airflow/pull/36953#discussion_r1478602118 ## airflow/providers/teradata/operators/teradata.py: ## @@ -0,0 +1,71 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor l

Re: [PR] Partially enable D401 rule [airflow]

2024-02-05 Thread via GitHub
ferruzzi commented on code in PR #37167: URL: https://github.com/apache/airflow/pull/37167#discussion_r1478608195 ## pyproject.toml: ## @@ -1273,7 +1273,10 @@ extend-select = [ "D2", "D3", "D400", -# "D401", # Not enabled by ruff, but we don't want it Review

Re: [PR] Partially enable D401 rule [airflow]

2024-02-05 Thread via GitHub
ferruzzi commented on PR #37167: URL: https://github.com/apache/airflow/pull/37167#issuecomment-1927519637 I started making these fixes before the holidays then never got around to finishing them. They go very fast. -- This is an automated message from the Apache Git Service. To respond

Re: [PR] Type Check for retries [airflow]

2024-02-05 Thread via GitHub
rawwar commented on code in PR #34914: URL: https://github.com/apache/airflow/pull/34914#discussion_r1478638440 ## airflow/models/baseoperator.py: ## @@ -124,7 +124,7 @@ def parse_retries(retries: Any) -> int | None: -if retries is None or isinstance(retries, int): +

Re: [PR] Add Teradata Provider [airflow]

2024-02-05 Thread via GitHub
SatishChGit commented on PR #36953: URL: https://github.com/apache/airflow/pull/36953#issuecomment-1927583430 > Solid work here! A few changes needed notwithstanding the example DAG question. Thank you. Addressed all review comments and commented on each ask. -- This is an automate

Re: [PR] Add Teradata Provider [airflow]

2024-02-05 Thread via GitHub
SatishChGit commented on PR #36953: URL: https://github.com/apache/airflow/pull/36953#issuecomment-1927589326 > Partial review for now Thank you for review. Addressed review comments. Please re-review it. -- This is an automated message from the Apache Git Service. To respond to the

Re: [PR] Type Check for retries [airflow]

2024-02-05 Thread via GitHub
rawwar commented on PR #34914: URL: https://github.com/apache/airflow/pull/34914#issuecomment-1927663905 @kandharvishnu , thanks for this PR. I added tests [here ](https://github.com/apache/airflow/pull/37183) if you want to take a look. -- This is an automated message from the Apache

[I] Issue with AWS CloudFormation CreateStack Operator Parameter Validation [airflow]

2024-02-05 Thread via GitHub
adarshsaraogi opened a new issue, #37184: URL: https://github.com/apache/airflow/issues/37184 ### Apache Airflow version main (development) ### If "Other Airflow 2 version" selected, which one? 2.7.2 ### What happened? I hope this message finds you well. I w

Re: [I] Issue with AWS CloudFormation CreateStack Operator Parameter Validation [airflow]

2024-02-05 Thread via GitHub
boring-cyborg[bot] commented on issue #37184: URL: https://github.com/apache/airflow/issues/37184#issuecomment-1927678521 Thanks for opening your first issue here! Be sure to follow the issue template! If you are willing to raise PR to address this issue please do so, no need to wait for ap

Re: [PR] Add Teradata Provider [airflow]

2024-02-05 Thread via GitHub
Taragolis commented on code in PR #36953: URL: https://github.com/apache/airflow/pull/36953#discussion_r1478705027 ## docs/apache-airflow-providers-teradata/redirects.txt: ## Review Comment: I guess for other providers in the past connections stored into the index.rst, so

Re: [I] Quickstart: not having `virtualenv` installed results in broken DAGs [airflow-site]

2024-02-05 Thread via GitHub
xionams commented on issue #945: URL: https://github.com/apache/airflow-site/issues/945#issuecomment-1927740965 > The problem is that those examples are there, and they will fail to import resulting in red nasty errors as you see when you enable exampes. That's a bad first-encounter with Ai

Re: [PR] Add Teradata Provider [airflow]

2024-02-05 Thread via GitHub
Taragolis commented on code in PR #36953: URL: https://github.com/apache/airflow/pull/36953#discussion_r1478710236 ## tests/providers/teradata/operators/test_teradata.py: ## @@ -0,0 +1,122 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor

Re: [I] Kubernetes executor pod reconcilation is slow due to recurssive deep copy [airflow]

2024-02-05 Thread via GitHub
jedcunningham commented on issue #37131: URL: https://github.com/apache/airflow/issues/37131#issuecomment-1927763734 I think not deepcopying may cause issues for KPO, see [build_pod_request_obj](https://github.com/apache/airflow/blob/d1aea8ec222cb358847a41bae81e62756b358dc5/airflow/providers

Re: [PR] Partially enable D401 rule [airflow]

2024-02-05 Thread via GitHub
Taragolis commented on code in PR #37167: URL: https://github.com/apache/airflow/pull/37167#discussion_r1478719723 ## pyproject.toml: ## @@ -1273,7 +1273,10 @@ extend-select = [ "D2", "D3", "D400", -# "D401", # Not enabled by ruff, but we don't want it Review

Re: [PR] ECS Executor - Add backoff on failed task retry [airflow]

2024-02-05 Thread via GitHub
ferruzzi merged PR #37109: URL: https://github.com/apache/airflow/pull/37109 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow

(airflow) branch main updated: ECS Executor - Add backoff on failed task retry (#37109)

2024-02-05 Thread ferruzzi
This is an automated email from the ASF dual-hosted git repository. ferruzzi pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 41ebf28103 ECS Executor - Add backoff on failed t

Re: [I] Issue with AWS CloudFormation CreateStack Operator Parameter Validation [airflow]

2024-02-05 Thread via GitHub
Taragolis commented on issue #37184: URL: https://github.com/apache/airflow/issues/37184#issuecomment-1927833502 [Dashboard](https://aws-mwaa.github.io/#/open-source/system-tests/dashboard.html) provided by Amazon (AWS) Team shows that system tests with CloudFormation operators works fine.

Re: [I] Issue with AWS CloudFormation CreateStack Operator Parameter Validation [airflow]

2024-02-05 Thread via GitHub
Taragolis closed issue #37184: Issue with AWS CloudFormation CreateStack Operator Parameter Validation URL: https://github.com/apache/airflow/issues/37184 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to g

(airflow) branch python3.12 updated (030f4a2d5a -> 4240736d15)

2024-02-05 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a change to branch python3.12 in repository https://gitbox.apache.org/repos/asf/airflow.git omit 030f4a2d5a python 3.12 support add 41ebf28103 ECS Executor - Add backoff on failed task retry (#37109)

(airflow) branch python3.12 updated (4240736d15 -> 01b744bc11)

2024-02-05 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a change to branch python3.12 in repository https://gitbox.apache.org/repos/asf/airflow.git discard 4240736d15 python 3.12 support add 01b744bc11 python 3.12 support This update added new revisions after und

Re: [I] Quickstart: not having `virtualenv` installed results in broken DAGs [airflow-site]

2024-02-05 Thread via GitHub
potiuk commented on issue #945: URL: https://github.com/apache/airflow-site/issues/945#issuecomment-1927861613 assigned. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

(airflow) branch constraints-main updated: Updating constraints. Github run id:7789348798

2024-02-05 Thread github-bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch constraints-main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/constraints-main by this push: new fb2b20d323 Updating con

Re: [I] All DAGs triggered through UI fail after upgrading from 2.6.1 to 2.8.1 when using LocalExecutor [airflow]

2024-02-05 Thread via GitHub
luederm commented on issue #37149: URL: https://github.com/apache/airflow/issues/37149#issuecomment-1927995275 I ran the task manually as you suggested. I didn't see much in the log output besides for: `INFO - Running on host C` It also looks like it failed in the UI. I

Re: [PR] Partially enable D401 rule [airflow]

2024-02-05 Thread via GitHub
ferruzzi commented on code in PR #37167: URL: https://github.com/apache/airflow/pull/37167#discussion_r1478850976 ## pyproject.toml: ## @@ -1273,7 +1273,10 @@ extend-select = [ "D2", "D3", "D400", -# "D401", # Not enabled by ruff, but we don't want it Review

Re: [I] Enable More PyDocStyle Checks [airflow]

2024-02-05 Thread via GitHub
ferruzzi commented on issue #10742: URL: https://github.com/apache/airflow/issues/10742#issuecomment-1928003944 No, holidays came and distracted me, I never finished the changes. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

Re: [I] `PythonVirtualenvOperator` crashes if any `python_callable` function is defined in the same source as DAG [airflow]

2024-02-05 Thread via GitHub
potiuk commented on issue #35529: URL: https://github.com/apache/airflow/issues/35529#issuecomment-1928014249 Maybe there is an easier solution: you already have the source of the callable: ``` {{ python_callable_source }} ``` and name ``` {{ python_callable }}(

Re: [I] All DAGs triggered through UI fail after upgrading from 2.6.1 to 2.8.1 when using LocalExecutor [airflow]

2024-02-05 Thread via GitHub
potiuk commented on issue #37149: URL: https://github.com/apache/airflow/issues/37149#issuecomment-1928043023 > One notable thing is that when using venv, google-re2 installed through pip without error. Of course. This will also work in conda as soon as conda maintainers will stop g

Re: [PR] Added ability to enable/disable scheduler and webserver [airflow]

2024-02-05 Thread via GitHub
kajanajustin commented on PR #36991: URL: https://github.com/apache/airflow/pull/36991#issuecomment-1928072608 @jedcunningham thank you for all your help! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above t

Re: [PR] Celery provider: Add task_acks_late setting [airflow]

2024-02-05 Thread via GitHub
potiuk commented on code in PR #37066: URL: https://github.com/apache/airflow/pull/37066#discussion_r1478921486 ## airflow/providers/celery/provider.yaml: ## @@ -276,6 +276,20 @@ config: type: float example: ~ default: "1.0" + task_acks_late: +

Re: [PR] AIP-58: Add object storage backend for xcom [airflow]

2024-02-05 Thread via GitHub
jedcunningham commented on PR #37058: URL: https://github.com/apache/airflow/pull/37058#issuecomment-1928121474 (@bolkedebruin, moving this back to 2.9 as we won't ship this in a core patch release) -- This is an automated message from the Apache Git Service. To respond to the message, pl

Re: [I] dag_processing.processes should be a gauge, not a counter [airflow]

2024-02-05 Thread via GitHub
potiuk commented on issue #36188: URL: https://github.com/apache/airflow/issues/36188#issuecomment-1928121937 We need a new metric, I think and deprecate the old one. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

Re: [I] Button on UI to pause all DAGs [airflow]

2024-02-05 Thread via GitHub
potiuk commented on issue #36994: URL: https://github.com/apache/airflow/issues/36994#issuecomment-1928124659 I am fine with that. @eladkal ? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the s

Re: [PR] Celery provider: Add task_acks_late setting [airflow]

2024-02-05 Thread via GitHub
ninsbl commented on code in PR #37066: URL: https://github.com/apache/airflow/pull/37066#discussion_r1478928304 ## airflow/providers/celery/provider.yaml: ## @@ -276,6 +276,20 @@ config: type: float example: ~ default: "1.0" + task_acks_late: +

  1   2   >