Re: [I] "Celery command failed", but "Marking task as SUCCESS" [airflow]

2024-01-07 Thread via GitHub
EinavDanielDX commented on issue #35610: URL: https://github.com/apache/airflow/issues/35610#issuecomment-1879997134 I have update on the matter. More background regarding the deployment: We are using airflow with Celery Executor on Kubernetes, in addition, as we don't need the workers

Re: [I] Search by Owner in task instances view [airflow]

2024-01-07 Thread via GitHub
eladkal closed issue #16638: Search by Owner in task instances view URL: https://github.com/apache/airflow/issues/16638 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsub

Re: [I] Multiple pgbouncer instances using the helm chart [airflow]

2024-01-07 Thread via GitHub
eladkal closed issue #24954: Multiple pgbouncer instances using the helm chart URL: https://github.com/apache/airflow/issues/24954 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment

Re: [I] Multiple pgbouncer instances using the helm chart [airflow]

2024-01-07 Thread via GitHub
eladkal commented on issue #24954: URL: https://github.com/apache/airflow/issues/24954#issuecomment-1879998734 https://github.com/apache/airflow/pull/27439 made it configurable -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

Re: [I] Launch Pod/job in separate namespace and support in helm chart [airflow]

2024-01-07 Thread via GitHub
eladkal commented on issue #25222: URL: https://github.com/apache/airflow/issues/25222#issuecomment-187012 Should be solved by https://github.com/apache/airflow/pull/35639 @mikebell90 can you help with review? -- This is an automated message from the Apache Git Service. To respond t

Re: [PR] Run mypy checks for full packages in CI [airflow]

2024-01-07 Thread via GitHub
potiuk commented on PR #36638: URL: https://github.com/apache/airflow/pull/36638#issuecomment-188485 Update @jscheffl. I spent a little time and implemented the final version that I foresaw initiallly for that. That one is quite a bit more complex, because it implements all the s

Re: [PR] Run mypy checks for full packages in CI [airflow]

2024-01-07 Thread via GitHub
jscheffl commented on code in PR #36638: URL: https://github.com/apache/airflow/pull/36638#discussion_r1443966349 ## .github/workflows/ci.yml: ## @@ -550,6 +552,52 @@ jobs: DEFAULT_BRANCH: ${{ needs.build-info.outputs.default-branch }} RUFF_FORMAT: "github"

Re: [PR] Run mypy checks for full packages in CI [airflow]

2024-01-07 Thread via GitHub
jscheffl commented on PR #36638: URL: https://github.com/apache/airflow/pull/36638#issuecomment-1880007727 Convinced. I'd do a final review and approve once the CI turns green :-D -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

Re: [PR] Run mypy checks for full packages in CI [airflow]

2024-01-07 Thread via GitHub
potiuk commented on code in PR #36638: URL: https://github.com/apache/airflow/pull/36638#discussion_r1443972876 ## .github/workflows/ci.yml: ## @@ -550,6 +552,52 @@ jobs: DEFAULT_BRANCH: ${{ needs.build-info.outputs.default-branch }} RUFF_FORMAT: "github"

Re: [PR] Refactor the Design of AzureSynapsePipeline class [airflow]

2024-01-07 Thread via GitHub
pankajastro commented on code in PR #36622: URL: https://github.com/apache/airflow/pull/36622#discussion_r1443972162 ## airflow/providers/microsoft/azure/hooks/synapse.py: ## @@ -56,8 +56,64 @@ class AzureSynapseSparkBatchRunStatus: TERMINAL_STATUSES = {SUCCESS, DEAD, KILLE

Re: [PR] Run mypy checks for full packages in CI [airflow]

2024-01-07 Thread via GitHub
potiuk commented on PR #36638: URL: https://github.com/apache/airflow/pull/36638#issuecomment-1880012160 Yeah. I had to add and expand a bit the tests and the code because I forgot about one more case (but the seletive check unit tests reminded me about it). We ALSO want to run `airfl

Re: [PR] Run mypy checks for full packages in CI [airflow]

2024-01-07 Thread via GitHub
potiuk commented on PR #36638: URL: https://github.com/apache/airflow/pull/36638#issuecomment-1880013497 BTW. Nice side effect of this one will be that our CI jobs will run even faster. Static checks was quite a long job (and mypy was a significant part of it). By separating out the

Re: [PR] feat: Stream Transfer Enhancement for Airflow SFTP to GCS Operator [airflow]

2024-01-07 Thread via GitHub
elaye-canopy commented on code in PR #36545: URL: https://github.com/apache/airflow/pull/36545#discussion_r1443975480 ## airflow/providers/google/cloud/transfers/sftp_to_gcs.py: ## @@ -168,6 +204,68 @@ def _copy_single_object( self.log.info("Executing delete of %s",

Re: [PR] Run mypy checks for full packages in CI [airflow]

2024-01-07 Thread via GitHub
jscheffl commented on code in PR #36638: URL: https://github.com/apache/airflow/pull/36638#discussion_r1443975448 ## .github/workflows/ci.yml: ## @@ -550,6 +551,53 @@ jobs: DEFAULT_BRANCH: ${{ needs.build-info.outputs.default-branch }} RUFF_FORMAT: "github"

Re: [PR] Run mypy checks for full packages in CI [airflow]

2024-01-07 Thread via GitHub
jscheffl commented on code in PR #36638: URL: https://github.com/apache/airflow/pull/36638#discussion_r1443975448 ## .github/workflows/ci.yml: ## @@ -550,6 +551,53 @@ jobs: DEFAULT_BRANCH: ${{ needs.build-info.outputs.default-branch }} RUFF_FORMAT: "github"

Re: [PR] Run mypy checks for full packages in CI [airflow]

2024-01-07 Thread via GitHub
jscheffl commented on PR #36638: URL: https://github.com/apache/airflow/pull/36638#issuecomment-1880015507 The "mypy (airflow)" check took now very long (9min) - is there some `--exclude` missing such that it does not check provider subfolder redundant to the parallel provider test? -- T

Re: [I] Enable Streaming from SFTP to GCS in `SFTPToGCSOperator` [airflow]

2024-01-07 Thread via GitHub
elaye-canopy commented on issue #34995: URL: https://github.com/apache/airflow/issues/34995#issuecomment-1880016235 @he-gong-sardine Great to see this is worked on, we independently solved it on our side and I put a comment in the code about that. Your solution does seem to work b

Re: [I] Enable Streaming from SFTP to GCS in `SFTPToGCSOperator` [airflow]

2024-01-07 Thread via GitHub
elaye-canopy commented on issue #34995: URL: https://github.com/apache/airflow/issues/34995#issuecomment-1880016337 Also, again, not directly related but logically related - perhaps similar logic should be made for the converse operator as well (GCS to SFTP) -- This is an automated messag

Re: [PR] Run mypy checks for full packages in CI [airflow]

2024-01-07 Thread via GitHub
potiuk commented on PR #36638: URL: https://github.com/apache/airflow/pull/36638#issuecomment-1880016862 > The "mypy (airflow)" check took now very long (9min) - is there some `--exclude` missing such that it does not check provider subfolder redundant to the parallel provider test?

Re: [PR] Run mypy checks for full packages in CI [airflow]

2024-01-07 Thread via GitHub
potiuk commented on code in PR #36638: URL: https://github.com/apache/airflow/pull/36638#discussion_r1443977169 ## .github/workflows/ci.yml: ## @@ -550,6 +551,53 @@ jobs: DEFAULT_BRANCH: ${{ needs.build-info.outputs.default-branch }} RUFF_FORMAT: "github"

Re: [PR] Run mypy checks for full packages in CI [airflow]

2024-01-07 Thread via GitHub
potiuk commented on PR #36638: URL: https://github.com/apache/airflow/pull/36638#issuecomment-1880018607 BTW. We also might want to add some extra instructions on how to run the tests if `Mypy` test fails - because it's not obvious that you should run ``` MYPY_PACKAGES="airflow" p

Re: [PR] Prepare docs 1st wave of Providers January 2024 [airflow]

2024-01-07 Thread via GitHub
eladkal merged PR #36640: URL: https://github.com/apache/airflow/pull/36640 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.

(airflow) branch main updated: Prepare docs 1st wave of Providers January 2024 (#36640)

2024-01-07 Thread eladkal
This is an automated email from the ASF dual-hosted git repository. eladkal pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 19ebcac239 Prepare docs 1st wave of Providers Janu

Re: [PR] Run mypy checks for full packages in CI [airflow]

2024-01-07 Thread via GitHub
potiuk commented on code in PR #36638: URL: https://github.com/apache/airflow/pull/36638#discussion_r1443982164 ## .github/workflows/ci.yml: ## @@ -550,6 +551,53 @@ jobs: DEFAULT_BRANCH: ${{ needs.build-info.outputs.default-branch }} RUFF_FORMAT: "github"

Re: [PR] Run mypy checks for full packages in CI [airflow]

2024-01-07 Thread via GitHub
potiuk commented on PR #36638: URL: https://github.com/apache/airflow/pull/36638#issuecomment-1880024443 Added the instructions. And also added one more thing that was wrong - when we update constraints we need to also make a dependency on the `mypy` job - otherwise we will update constrain

svn commit: r66455 - /dev/airflow/providers/

2024-01-07 Thread eladkal
Author: eladkal Date: Sun Jan 7 11:02:06 2024 New Revision: 66455 Log: Add artifacts for Airflow Providers 2024-01-07 Added: dev/airflow/providers/apache_airflow_providers_amazon-8.16.0-py3-none-any.whl (with props) dev/airflow/providers/apache_airflow_providers_amazon-8.16.0-py3-no

Re: [PR] Run mypy checks for full packages in CI [airflow]

2024-01-07 Thread via GitHub
potiuk commented on PR #36638: URL: https://github.com/apache/airflow/pull/36638#issuecomment-1880028121 And added `STATIC_CHECKS` documentaiton explaining all that. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

Re: [PR] enhance druid connection parameters [airflow]

2024-01-07 Thread via GitHub
blcksrx commented on PR #35244: URL: https://github.com/apache/airflow/pull/35244#issuecomment-1880031917 Yeah, I do! Actually i asked a question but didn’t got any answers  On Sun, Jan 7, 2024 at 07:47, Elad Kalif ***@***.***> wrote: @blcksrx are you planning completing this PR?

Re: [PR] Run mypy checks for full packages in CI [airflow]

2024-01-07 Thread via GitHub
jscheffl commented on code in PR #36638: URL: https://github.com/apache/airflow/pull/36638#discussion_r1443987182 ## scripts/ci/pre_commit/pre_commit_mypy.py: ## @@ -51,19 +55,26 @@ "MOUNT_SOURCES": "selected", }, ) +ci_environment = os.environ.get("CI") if res.r

Re: [PR] Run mypy checks for full packages in CI [airflow]

2024-01-07 Thread via GitHub
potiuk commented on PR #36638: URL: https://github.com/apache/airflow/pull/36638#issuecomment-1880032729 Final touches: Added `--color always` (in CI this is a pseudo-terminal and pre-commit does not detect color terminal support automatically) and `--verbose` - we will see a little more ou

Re: [PR] Run mypy checks for full packages in CI [airflow]

2024-01-07 Thread via GitHub
potiuk commented on code in PR #36638: URL: https://github.com/apache/airflow/pull/36638#discussion_r1443988423 ## scripts/ci/pre_commit/pre_commit_mypy.py: ## @@ -51,19 +55,26 @@ "MOUNT_SOURCES": "selected", }, ) +ci_environment = os.environ.get("CI") if res.ret

Re: [PR] Run mypy checks for full packages in CI [airflow]

2024-01-07 Thread via GitHub
potiuk commented on code in PR #36638: URL: https://github.com/apache/airflow/pull/36638#discussion_r1443988533 ## scripts/ci/pre_commit/pre_commit_mypy.py: ## @@ -51,19 +55,26 @@ "MOUNT_SOURCES": "selected", }, ) +ci_environment = os.environ.get("CI") if res.ret

Re: [PR] enhance druid connection parameters [airflow]

2024-01-07 Thread via GitHub
eladkal commented on PR #35244: URL: https://github.com/apache/airflow/pull/35244#issuecomment-1880033476 > Actually i asked a question but didn’t got any answers Which question? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

Re: [PR] enhance druid connection parameters [airflow]

2024-01-07 Thread via GitHub
blcksrx commented on PR #35244: URL: https://github.com/apache/airflow/pull/35244#issuecomment-1880034326 About warning or decision about deprecation on the Major release On Sun, Jan 7, 2024 at 12:39, Elad Kalif ***@***.***> wrote: Actually i asked a question but didn’t got any

Re: [PR] Run mypy checks for full packages in CI [airflow]

2024-01-07 Thread via GitHub
potiuk commented on PR #36638: URL: https://github.com/apache/airflow/pull/36638#issuecomment-1880034630 A little too much verbosity :) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specifi

Re: [PR] enhance druid connection parameters [airflow]

2024-01-07 Thread via GitHub
eladkal commented on PR #35244: URL: https://github.com/apache/airflow/pull/35244#issuecomment-1880035274 > About warning or decision about deprecation on the Major release I answered about it in https://github.com/apache/airflow/pull/35244#discussion_r1404152273 Is there a followu

Re: [PR] Run mypy checks for full packages in CI [airflow]

2024-01-07 Thread via GitHub
potiuk commented on PR #36638: URL: https://github.com/apache/airflow/pull/36638#issuecomment-1880037772 Right... That's good enough: https://github.com/apache/airflow/assets/595491/cc3cdeaa-ccad-45dc-8d97-413eaa66889b";> -- This is an automated message from the Apache Gi

(airflow-site) branch add-documentation-2024-01-07 created (now 52580d73d0)

2024-01-07 Thread eladkal
This is an automated email from the ASF dual-hosted git repository. eladkal pushed a change to branch add-documentation-2024-01-07 in repository https://gitbox.apache.org/repos/asf/airflow-site.git at 52580d73d0 Add documentation for packages - 2024-01-07 This branch includes the followin

[PR] Add documentation for packages - 2024-01-07 [airflow-site]

2024-01-07 Thread via GitHub
eladkal opened a new pull request, #931: URL: https://github.com/apache/airflow-site/pull/931 (no comment) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e

Re: [PR] [API] can return the specified field when get dag/dagRun [airflow]

2024-01-07 Thread via GitHub
HarryWu-CHN commented on PR #36641: URL: https://github.com/apache/airflow/pull/36641#issuecomment-1880040419 > Very nice! Indeed that nicely solves the most pressing need of partial retrieval of the DAG / DAGRun. > > Can you please add some tests covering this ? @potiuk Hi! Si

[I] Status of testing Providers that were prepared on January 07, 2024 [airflow]

2024-01-07 Thread via GitHub
eladkal opened a new issue, #36644: URL: https://github.com/apache/airflow/issues/36644 ### Body I have a kind request for all the contributors to the latest provider packages release. Could you please help us to test the RC versions of the providers? The guidelines on how t

[PR] Add Airflow logo to the README [airflow]

2024-01-07 Thread via GitHub
kaxil opened a new pull request, #36645: URL: https://github.com/apache/airflow/pull/36645 Similar to Apache Superset, adding this repo to README so that Airflow repo shows up nicely with thumbnails on various Topics pages like https://github.com/topics/airflow

[PR] Upgrade to latest versions of `pip` and `pipx` in CI runners [airflow]

2024-01-07 Thread via GitHub
potiuk opened a new pull request, #36646: URL: https://github.com/apache/airflow/pull/36646 The CI runners did not have latest version of `pip` and `pipx`. This change updates the installation scripts to fix `pip` to the same version as in the CI image and down-binds pipx to 1.4.1 which is

Re: [PR] Upgrade to latest versions of `pip` and `pipx` in CI runners [airflow]

2024-01-07 Thread via GitHub
potiuk commented on code in PR #36646: URL: https://github.com/apache/airflow/pull/36646#discussion_r1443996825 ## dev/breeze/pyproject.toml: ## @@ -54,6 +54,7 @@ dependencies = [ "jinja2>=3.1.0", "jsonschema>=4.19.1", "packaging>=23.2", +"pipx>=1.4.1", Revie

Re: [PR] Standardize airflow build process and switch to Hatchling build backend [airflow]

2024-01-07 Thread via GitHub
potiuk commented on code in PR #36537: URL: https://github.com/apache/airflow/pull/36537#discussion_r144436 ## INSTALL: ## @@ -1,118 +1,324 @@ # INSTALL / BUILD instructions for Apache Airflow -This is a generic installation method that requires a number of dependencies

Re: [PR] Upgrade to latest versions of `pip` and `pipx` in CI runners [airflow]

2024-01-07 Thread via GitHub
potiuk merged PR #36646: URL: https://github.com/apache/airflow/pull/36646 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.a

(airflow) branch main updated: Upgrade to latest versions of `pip` and `pipx` in CI runners (#36646)

2024-01-07 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 75bc05ce1f Upgrade to latest versions of `pip` and

Re: [PR] Run mypy checks for full packages in CI [airflow]

2024-01-07 Thread via GitHub
potiuk merged PR #36638: URL: https://github.com/apache/airflow/pull/36638 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.a

(airflow) branch main updated: Run mypy checks for full packages in CI (#36638)

2024-01-07 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new f7b663d9af Run mypy checks for full packages in CI

(airflow) branch main updated: Add Airflow logo to the README (#36645)

2024-01-07 Thread husseinawala
This is an automated email from the ASF dual-hosted git repository. husseinawala pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 9a20aa8f0f Add Airflow logo to the README (#3

Re: [PR] Add Airflow logo to the README [airflow]

2024-01-07 Thread via GitHub
hussein-awala merged PR #36645: URL: https://github.com/apache/airflow/pull/36645 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@ai

Re: [PR] Add Airflow logo to the README [airflow]

2024-01-07 Thread via GitHub
hussein-awala commented on PR #36645: URL: https://github.com/apache/airflow/pull/36645#issuecomment-1880058362 It didn't work; I will check the README files in the other repositories -- This is an automated message from the Apache Git Service. To respond to the message, please log on to G

[PR] Make `graphviz` dependency optional [airflow]

2024-01-07 Thread via GitHub
potiuk opened a new pull request, #36647: URL: https://github.com/apache/airflow/pull/36647 The `graphviz` dependency has been problematic as Airflow required dependency - especially for ARM-based installations. Graphviz packages require binary graphviz libraries - which is already a limita

Re: [PR] feat: Stream Transfer Enhancement for Airflow SFTP to GCS Operator [airflow]

2024-01-07 Thread via GitHub
elaye-canopy commented on code in PR #36545: URL: https://github.com/apache/airflow/pull/36545#discussion_r1443975333 ## airflow/providers/google/cloud/transfers/sftp_to_gcs.py: ## @@ -168,6 +204,68 @@ def _copy_single_object( self.log.info("Executing delete of %s",

Re: [PR] Standardize airflow build process and switch to Hatchling build backend [airflow]

2024-01-07 Thread via GitHub
potiuk commented on code in PR #36537: URL: https://github.com/apache/airflow/pull/36537#discussion_r1444006201 ## airflow/utils/dot_renderer.py: ## @@ -19,9 +19,14 @@ """Renderer DAG (tasks and dependencies) to the graphviz object.""" from __future__ import annotations +imp

(airflow) branch switch-to-hatch updated (5cba0a1607 -> e18efed381)

2024-01-07 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a change to branch switch-to-hatch in repository https://gitbox.apache.org/repos/asf/airflow.git discard 5cba0a1607 Standardize airflow build process and switch to Hatchling build backend add 4469baad4e Add

Re: [PR] feat: Stream Transfer Enhancement for Airflow SFTP to GCS Operator [airflow]

2024-01-07 Thread via GitHub
elaye-canopy commented on code in PR #36545: URL: https://github.com/apache/airflow/pull/36545#discussion_r1443975333 ## airflow/providers/google/cloud/transfers/sftp_to_gcs.py: ## @@ -168,6 +204,68 @@ def _copy_single_object( self.log.info("Executing delete of %s",

Re: [PR] Add Airflow logo to the README [airflow]

2024-01-07 Thread via GitHub
jscheffl commented on PR #36645: URL: https://github.com/apache/airflow/pull/36645#issuecomment-1880062299 The logo is not served from README :-( You need to be repo admin, go to repo settings and there add a "Social preview" image. -- This is an automated message from the Apache Git S

Re: [PR] Add Airflow logo to the README [airflow]

2024-01-07 Thread via GitHub
jscheffl commented on PR #36645: URL: https://github.com/apache/airflow/pull/36645#issuecomment-1880062687 I assume we _could_ set it manually but a follow-up weekend activity was to check if there is an option in `.asf.yaml` or maybe a feature needs to be added there to define it programat

Re: [I] [WIP] SFTPSensor.newer_than not working with jinja logical date expression [airflow]

2024-01-07 Thread via GitHub
RNHTTR commented on issue #36629: URL: https://github.com/apache/airflow/issues/36629#issuecomment-1880070155 For what it's worth, [the `SFTPSensor` does support templating `newer_than`](https://github.com/apache/airflow/blob/providers-sftp/4.4.0/airflow/providers/sftp/sensors/sftp.py#L47),

Re: [PR] Add Airflow logo to the README [airflow]

2024-01-07 Thread via GitHub
kaxil commented on PR #36645: URL: https://github.com/apache/airflow/pull/36645#issuecomment-1880071661 > The logo is not served from README :-( You need to be repo admin, go to repo settings and there add a "Social preview" image. Are you sure about that? Most of the repos I am seein

(airflow) branch switch-to-hatch updated (e18efed381 -> 7c8df822a0)

2024-01-07 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a change to branch switch-to-hatch in repository https://gitbox.apache.org/repos/asf/airflow.git omit e18efed381 Standardize airflow build process and switch to Hatchling build backend add 7c8df822a0 Stan

Re: [PR] Standardize airflow build process and switch to Hatchling build backend [airflow]

2024-01-07 Thread via GitHub
potiuk commented on code in PR #36537: URL: https://github.com/apache/airflow/pull/36537#discussion_r1444014436 ## hatch_build.py: ## Review Comment: Moved `hatch_build.py` to `dev` folder. -- This is an automated message from the Apache Git Service. To respond to the m

(airflow) branch constraints-main updated: Updating constraints. Github run id:7438490206

2024-01-07 Thread github-bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch constraints-main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/constraints-main by this push: new f55337839c Updating con

Re: [PR] Add Airflow logo to the README [airflow]

2024-01-07 Thread via GitHub
kaxil commented on PR #36645: URL: https://github.com/apache/airflow/pull/36645#issuecomment-1880073879 You are probably right, I experimented with another repo -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL a

Re: [PR] Add parameter confirm to GCSToSFTPOperator [airflow]

2024-01-07 Thread via GitHub
Nimo139 commented on PR #35704: URL: https://github.com/apache/airflow/pull/35704#issuecomment-1880074271 Sorry for the late reply, I didn't have time to set up the test environment during the holidays. I have now adjusted the tests for the parameter so that they pass. I have added a very b

Re: [PR] Add Airflow logo to the README [airflow]

2024-01-07 Thread via GitHub
kaxil commented on PR #36645: URL: https://github.com/apache/airflow/pull/36645#issuecomment-1880074340 Yup found the Superset thing - [here](https://issues.apache.org/jira/browse/INFRA-20803?jql=text%20~%20%22social%20preview%22), we could request the same. And we should make it sel

Re: [PR] Add Airflow logo to the README [airflow]

2024-01-07 Thread via GitHub
kaxil commented on PR #36645: URL: https://github.com/apache/airflow/pull/36645#issuecomment-1880075252 Created this ticket - https://issues.apache.org/jira/browse/INFRA-25351 for now -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

Re: [PR] [API] can return the specified field when get dag/dagRun [airflow]

2024-01-07 Thread via GitHub
potiuk commented on PR #36641: URL: https://github.com/apache/airflow/pull/36641#issuecomment-1880078290 > > Very nice! Indeed that nicely solves the most pressing need of partial retrieval of the DAG / DAGRun. > > Can you please add some tests covering this ? > > @potiuk Hi! Since

Re: [PR] Add Airflow logo to the README [airflow]

2024-01-07 Thread via GitHub
potiuk commented on PR #36645: URL: https://github.com/apache/airflow/pull/36645#issuecomment-1880079809 Yeah.. I'v already added a feature there - it's Python based so it's not a rocket science https://github.com/apache/infrastructure-p6/pull/1740. - we just need to check if there is an AP

Re: [PR] Add Airflow logo to the README [airflow]

2024-01-07 Thread via GitHub
jscheffl commented on PR #36645: URL: https://github.com/apache/airflow/pull/36645#issuecomment-1880082614 > You are probably right, I experimented with another repo @kaxil I was not fast enough with response... Yes I also tried with a temporary repo which was empty yesterday... from

Re: [PR] Add Airflow logo to the README [airflow]

2024-01-07 Thread via GitHub
jscheffl commented on PR #36645: URL: https://github.com/apache/airflow/pull/36645#issuecomment-1880083915 > Created this ticket - https://issues.apache.org/jira/browse/INFRA-25351 for now Oh, they already added it - that was FAST. Problem solved for now :-D -- This is an automated

[PR] Replace build errors output from spellcheck-only output [airflow]

2024-01-07 Thread via GitHub
potiuk opened a new pull request, #36648: URL: https://github.com/apache/airflow/pull/36648 After #36591 has been implemented, in cas of `--spellcheck-only` checks for parallel packages, it might be that some build errors remain in build errors output (depends on some race conditions while

Re: [I] Names for expanded tasks [airflow]

2024-01-07 Thread via GitHub
RNHTTR commented on issue #23020: URL: https://github.com/apache/airflow/issues/23020#issuecomment-1880089536 I'm interested in taking this on (or at least moving it forward). Would it be sufficient to start by: 1. Adding something like `named_mapped_index` to BaseOperator and as a c

Re: [I] Add dark mode to Airflow UI [airflow]

2024-01-07 Thread via GitHub
Andrew-Wichmann commented on issue #11334: URL: https://github.com/apache/airflow/issues/11334#issuecomment-1880091543 I hear ya'. I'm not demanding anything. I really like the software. Thank you for your time and effort -- This is an automated message from the Apache Git Service. To res

[PR] Upgrade to latest ruff and remove ISC001 warning from output [airflow]

2024-01-07 Thread via GitHub
potiuk opened a new pull request, #36649: URL: https://github.com/apache/airflow/pull/36649 This PR upgrades to latest ruff, and removes the ISC001 warning that warns us against potential conflict between ruff and ruff-formatter when two strings in one line get concatenated. This war

Re: [PR] Make `graphviz` dependency optional [airflow]

2024-01-07 Thread via GitHub
potiuk merged PR #36647: URL: https://github.com/apache/airflow/pull/36647 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.a

(airflow) branch main updated: Make `graphviz` dependency optional (#36647)

2024-01-07 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 89f1737afb Make `graphviz` dependency optional (#36

Re: [PR] Remove build errors output from spellcheck-only output [airflow]

2024-01-07 Thread via GitHub
potiuk merged PR #36648: URL: https://github.com/apache/airflow/pull/36648 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.a

(airflow) branch main updated: Remove build errors output from spellcheck-only output (#36648)

2024-01-07 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 579c4b5c59 Remove build errors output from spellche

Re: [PR] Upgrade to latest ruff and remove ISC001 warning from output [airflow]

2024-01-07 Thread via GitHub
potiuk commented on PR #36649: URL: https://github.com/apache/airflow/pull/36649#issuecomment-1880099482 These warnings get removed: https://github.com/apache/airflow/assets/595491/08a30262-8612-46ba-9615-000cb9ea778f";> -- This is an automated message from the Apache Git Servic

Re: [PR] Upgrade to latest ruff and remove ISC001 warning from output [airflow]

2024-01-07 Thread via GitHub
potiuk commented on code in PR #36649: URL: https://github.com/apache/airflow/pull/36649#discussion_r1444031321 ## pyproject.toml: ## @@ -65,7 +65,7 @@ extend-select = [ "ISC", # Checks for implicit literal string concatenation (auto-fixable) "B006", # Checks for uses

Re: [I] Names for expanded tasks [airflow]

2024-01-07 Thread via GitHub
kaxil commented on issue #23020: URL: https://github.com/apache/airflow/issues/23020#issuecomment-1880105381 > I'm interested in taking this on (or at least moving it forward). Would it be sufficient to start by: > > 1. Adding something like `named_mapped_index` to BaseOperator and as

[PR] [FEATURE] Added oauth support with snowflake sql api operator [airflow]

2024-01-07 Thread via GitHub
ighoshsubho opened a new pull request, #36650: URL: https://github.com/apache/airflow/pull/36650 ## Functionality added - - SnowflakeSqlApiOperator support for OAuth closes: #36621 --- **^ Add meaningful description above** Read the **[Pu

Re: [PR] [FEATURE] Added oauth support with snowflake sql api operator [airflow]

2024-01-07 Thread via GitHub
boring-cyborg[bot] commented on PR #36650: URL: https://github.com/apache/airflow/pull/36650#issuecomment-1880109101 Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our Contribution

Re: [PR] enhance druid connection parameters [airflow]

2024-01-07 Thread via GitHub
blcksrx commented on PR #35244: URL: https://github.com/apache/airflow/pull/35244#issuecomment-1880109251 I see, Im gonna fix it this week. On Sun, Jan 7, 2024 at 12:47, Elad Kalif ***@***.***> wrote: About warning or decision about deprecation on the Major release I answ

[PR] pre-commit(codespell): use all lowercase for `spelling_wordlist.txt` [airflow]

2024-01-07 Thread via GitHub
jbampton opened a new pull request, #36651: URL: https://github.com/apache/airflow/pull/36651 https://github.com/apache/airflow/blob/579c4b5c59e3855dd1285cbfe482f41c57b2b82e/.pre-commit-config.yaml#L281C6-L281C6 Convert the `spelling_wordlist.txt` to all lowercase. Thi

Re: [PR] Add Airflow logo to the README [airflow]

2024-01-07 Thread via GitHub
potiuk commented on PR #36645: URL: https://github.com/apache/airflow/pull/36645#issuecomment-1880118253 The image though is probably not perfect (refreshed preview in LinkedIn - Facebook is similar). https://github.com/apache/airflow/assets/595491/6d121782-2a67-48be-8803-4e457e58e911

(airflow) branch switch-to-hatch updated (7c8df822a0 -> 87c8d24423)

2024-01-07 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a change to branch switch-to-hatch in repository https://gitbox.apache.org/repos/asf/airflow.git discard 7c8df822a0 Standardize airflow build process and switch to Hatchling build backend add 9a20aa8f0f Add

Re: [PR] [FEATURE] Added oauth support with snowflake sql api operator [airflow]

2024-01-07 Thread via GitHub
potiuk closed pull request #36650: [FEATURE] Added oauth support with snowflake sql api operator URL: https://github.com/apache/airflow/pull/36650 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

Re: [PR] [FEATURE] Added oauth support with snowflake sql api operator [airflow]

2024-01-07 Thread via GitHub
potiuk commented on PR #36650: URL: https://github.com/apache/airflow/pull/36650#issuecomment-1880124442 You need to add tests, it makes little sense to run CI if there are no tests added for it. -- This is an automated message from the Apache Git Service. To respond to the message, pleas

Re: [PR] [FEATURE] Added oauth support with snowflake sql api operator [airflow]

2024-01-07 Thread via GitHub
potiuk commented on PR #36650: URL: https://github.com/apache/airflow/pull/36650#issuecomment-1880124501 Closed by mistake. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

Re: [PR] Add parameter confirm to GCSToSFTPOperator [airflow]

2024-01-07 Thread via GitHub
potiuk commented on PR #35704: URL: https://github.com/apache/airflow/pull/35704#issuecomment-1880125333 Should be good enough - now the only thing left is to rebase to latest `main` and fix failing static checks -- This is an automated message from the Apache Git Service. To respond to t

Re: [PR] Standardize airflow build process and switch to Hatchling build backend [airflow]

2024-01-07 Thread via GitHub
potiuk commented on code in PR #36537: URL: https://github.com/apache/airflow/pull/36537#discussion_r1444047774 ## dev/old_setup/MANIFEST.in: ## Review Comment: OK. Removed it for now. I have a copy in git and it is enough :) -- This is an automated message from the Ap

Re: [PR] [FEATURE] Added oauth support with snowflake sql api operator [airflow]

2024-01-07 Thread via GitHub
ighoshsubho commented on PR #36650: URL: https://github.com/apache/airflow/pull/36650#issuecomment-1880129786 > You need to add tests, it makes little sense to run CI if there are no tests added for it. sure will do it -- This is an automated message from the Apache Git Service. To

Re: [PR] Add parameter confirm to GCSToSFTPOperator [airflow]

2024-01-07 Thread via GitHub
eladkal commented on code in PR #35704: URL: https://github.com/apache/airflow/pull/35704#discussion_r1444052299 ## tests/providers/google/cloud/transfers/test_gcs_to_sftp.py: ## @@ -116,7 +116,55 @@ def test_execute_move_single_file( ) sftp_hook_mock.return_

[I] "Celery command failed", but "Marking task as SUCCESS" [airflow]

2024-01-07 Thread via GitHub
EinavDanielDX opened a new issue, #35610: URL: https://github.com/apache/airflow/issues/35610 ### Apache Airflow version 2.7.3 ### What happened I've executed a task, on flower and worker log I see this error: ``` airflow.exceptions.AirflowException: Celery command

Re: [I] "Celery command failed", but "Marking task as SUCCESS" [airflow]

2024-01-07 Thread via GitHub
potiuk commented on issue #35610: URL: https://github.com/apache/airflow/issues/35610#issuecomment-1880141693 That's an interesting lead. I will mark it for 2.8.1 and maybe we could guesss what could be the readon. Do you see any logs from the workers indicating that there might be an error

Re: [PR] Upgrade to latest ruff and remove ISC001 warning from output [airflow]

2024-01-07 Thread via GitHub
potiuk merged PR #36649: URL: https://github.com/apache/airflow/pull/36649 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.a

(airflow) branch main updated: Upgrade to latest ruff and remove ISC001 warning from output (#36649)

2024-01-07 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 11c46fd2ec Upgrade to latest ruff and remove ISC001

  1   2   >