Re: [I] Removed DAG Reporting import_errors to Datadog [airflow]

2023-10-25 Thread via GitHub
github-actions[bot] commented on issue #24286: URL: https://github.com/apache/airflow/issues/24286#issuecomment-1778637388 This issue has been closed because it has not received response from the issue author. -- This is an automated message from the Apache Git Service. To respond to the

Re: [I] Removed DAG Reporting import_errors to Datadog [airflow]

2023-10-25 Thread via GitHub
github-actions[bot] closed issue #24286: Removed DAG Reporting import_errors to Datadog URL: https://github.com/apache/airflow/issues/24286 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specif

[PR] Improve breeze command image generation and consistency check [airflow]

2023-10-25 Thread via GitHub
potiuk opened a new pull request, #35169: URL: https://github.com/apache/airflow/pull/35169 There is no more need to regenerate the images manually via `breeze setup regenerate-command-images`. Also check for consistency of command configuration vs command parameters in Breeze is now done i

Re: [PR] Improve breeze command image generation and consistency check [airflow]

2023-10-25 Thread via GitHub
potiuk commented on code in PR #35169: URL: https://github.com/apache/airflow/pull/35169#discussion_r1371250007 ## .pre-commit-config.yaml: ## @@ -900,7 +900,7 @@ repos: files: newsfragments/.*\.rst entry: ./scripts/ci/pre_commit/pre_commit_newsfragments.py

Re: [PR] Improve breeze command image generation and consistency check [airflow]

2023-10-25 Thread via GitHub
potiuk commented on PR #35169: URL: https://github.com/apache/airflow/pull/35169#issuecomment-1778643802 Should be easier to keep those images regenerated now for anyone who: * has breeze installed * has pre-commit installed * changes commands in Breeze Instead of finding

Re: [PR] Rewrite build_provider_documentation in python [airflow]

2023-10-25 Thread via GitHub
anudeep-pv commented on PR #35080: URL: https://github.com/apache/airflow/pull/35080#issuecomment-1778646403 > Actually.. This might be a bit sad news for you @anudeep-pv but we do not need this script any more. And rather than converting it to python we should change the instructions to us

[I] clear_task_instances support all parameters [airflow]

2023-10-25 Thread via GitHub
romanzdk opened a new issue, #35170: URL: https://github.com/apache/airflow/issues/35170 ### Description `clear_task_instances` function does not support all parameters same REST API endpoint does ### Use case/motivation I need to clear task instances with β€œdownstream” s

Re: [I] clear_task_instances support all parameters [airflow]

2023-10-25 Thread via GitHub
Taragolis commented on issue #35170: URL: https://github.com/apache/airflow/issues/35170#issuecomment-1778715991 > clear_task_instances function does not support all parameters same REST API endpoint does [airflow.models.taskinstance.clear_task_instances](https://airflow.apache.org/d

[I] ComputeEngineSSHHook unintentionally deletes instance metadata [airflow]

2023-10-25 Thread via GitHub
teslur opened a new issue, #35171: URL: https://github.com/apache/airflow/issues/35171 ### Apache Airflow version Other Airflow 2 version (please specify below) ### What happened When I run the workflow in [this code example](https://github.com/apache/airflow/blob/main/t

Re: [I] ComputeEngineSSHHook unintentionally deletes instance metadata [airflow]

2023-10-25 Thread via GitHub
boring-cyborg[bot] commented on issue #35171: URL: https://github.com/apache/airflow/issues/35171#issuecomment-1778732002 Thanks for opening your first issue here! Be sure to follow the issue template! If you are willing to raise PR to address this issue please do so, no need to wait for ap

Re: [PR] Added to the rendering of KubernetesOperator V1VolumeMount, sub_path [airflow]

2023-10-25 Thread via GitHub
mescanne commented on PR #35129: URL: https://github.com/apache/airflow/pull/35129#issuecomment-1778785267 @uranusjr Updated with pre-commit enforced Python syntax. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the U

Re: [PR] Extend papermill operator to support remote kernels [airflow]

2023-10-25 Thread via GitHub
Taragolis commented on code in PR #34840: URL: https://github.com/apache/airflow/pull/34840#discussion_r1371356084 ## airflow/providers/papermill/provider.yaml: ## @@ -44,6 +44,7 @@ dependencies: - apache-airflow>=2.5.0 - papermill[all]>=1.2.1 - scrapbook[all] + - ipyk

Re: [PR] Yandex dataproc deduce default service account [airflow]

2023-10-25 Thread via GitHub
Piatachock commented on PR #35059: URL: https://github.com/apache/airflow/pull/35059#issuecomment-1778805964 > Static checks are failing Sorry, forgot to mark this PR as WIP. I'm actually trying to build provider package locally to check it in our Managed Airflow installation end-t

Re: [PR] Add extra operator links for EMR Serverless [airflow]

2023-10-25 Thread via GitHub
Taragolis commented on code in PR #34225: URL: https://github.com/apache/airflow/pull/34225#discussion_r1371397419 ## airflow/providers/amazon/aws/links/emr.py: ## @@ -66,3 +82,98 @@ def get_log_uri( return None log_uri = S3Hook.parse_s3_url(cluster_info["LogUri"])

Re: [I] EcsRunTaskOperator [airflow]

2023-10-25 Thread via GitHub
rotemlevibigabid closed issue #34802: EcsRunTaskOperator URL: https://github.com/apache/airflow/issues/34802 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-m

Re: [PR] Permitting airflow kerberos to run in different modes [airflow]

2023-10-25 Thread via GitHub
potiuk commented on PR #35146: URL: https://github.com/apache/airflow/pull/35146#issuecomment-1778852631 looks great, but docs and tests need fixing. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

Re: [PR] Yandex dataproc deduce default service account [airflow]

2023-10-25 Thread via GitHub
Taragolis commented on PR #35059: URL: https://github.com/apache/airflow/pull/35059#issuecomment-1778863223 > it in our Managed Airflow BTW, you or some one from the Yandex also could add link to Yandex Managed Service for Apache Airflow into [Airflow Ecosystem](https://airflow.apac

Re: [PR] Add pagination to `SimpleHttpOperator` and make it more modular [airflow]

2023-10-25 Thread via GitHub
hussein-awala commented on PR #34669: URL: https://github.com/apache/airflow/pull/34669#issuecomment-1778874699 If we decide to implement the feature in the existing operator, I suggest renaming it to `HttpOperator` and creating a deprecated subclass `SimpleHttpOperator` (without any overri

Re: [PR] Improve breeze command image generation and consistency check [airflow]

2023-10-25 Thread via GitHub
potiuk merged PR #35169: URL: https://github.com/apache/airflow/pull/35169 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.a

[airflow] branch main updated: Improve breeze command image generation and consistency check (#35169)

2023-10-25 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 7e5c827472 Improve breeze command image generation

Re: [I] ComputeEngineSSHHook unintentionally deletes instance metadata [airflow]

2023-10-25 Thread via GitHub
potiuk commented on issue #35171: URL: https://github.com/apache/airflow/issues/35171#issuecomment-1778916343 cc: @VladaZakharova -> I think you should answer this one :) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

Re: [PR] Yandex dataproc deduce default service account [airflow]

2023-10-25 Thread via GitHub
Taragolis commented on PR #35059: URL: https://github.com/apache/airflow/pull/35059#issuecomment-1778920067 > I'm actually trying to build provider package locally to check it in our Managed Airflow installation end-to-end. Do you, by any chance, know where should I look for any info on t

Re: [PR] Permitting airflow kerberos to run in different modes [airflow]

2023-10-25 Thread via GitHub
amoghrajesh commented on PR #35146: URL: https://github.com/apache/airflow/pull/35146#issuecomment-1778934922 Thank you @potiuk Just pushed a fix for this -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL abov

Re: [PR] Make Dataprep system test self-sufficient [airflow]

2023-10-25 Thread via GitHub
moiseenkov commented on PR #34880: URL: https://github.com/apache/airflow/pull/34880#issuecomment-1778946760 @potiuk , hi, Could we merge it please? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to g

Re: [PR] Limit Airflow and breeze to < 3.12 [airflow]

2023-10-25 Thread via GitHub
potiuk merged PR #35123: URL: https://github.com/apache/airflow/pull/35123 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.a

[airflow] branch main updated: Limit Airflow and breeze to < 3.12 (#35123)

2023-10-25 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 020691f5cc Limit Airflow and breeze to < 3.12 (#351

Re: [PR] Make Dataprep system test self-sufficient [airflow]

2023-10-25 Thread via GitHub
potiuk commented on PR #34880: URL: https://github.com/apache/airflow/pull/34880#issuecomment-1778950304 > @potiuk , hi, Could we merge it please? Ass soon as @eladkal changes his review status. Until that I CAN't merge it. -- This is an automated message from the Apache Git Service

Re: [PR] AIP44 Fix DAG serialization [airflow]

2023-10-25 Thread via GitHub
potiuk commented on PR #34042: URL: https://github.com/apache/airflow/pull/34042#issuecomment-1778968734 @kaxil :pray: -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

Re: [PR] Add Open Search Provider [airflow]

2023-10-25 Thread via GitHub
pateash commented on PR #34705: URL: https://github.com/apache/airflow/pull/34705#issuecomment-1778986522 Hi @cjames23, Just a quick question regarding this one. as we know OpenSearch is just a fork of Elasticsearch which happened couple of years ago, Can we not use Elastic search

Re: [PR] openlineage, aws: Add OpenLineage support for AthenaOperator. [airflow]

2023-10-25 Thread via GitHub
mobuchowski commented on code in PR #35090: URL: https://github.com/apache/airflow/pull/35090#discussion_r1371557024 ## airflow/providers/amazon/aws/hooks/athena.py: ## @@ -121,7 +122,26 @@ def run_query( self.log.info("Query execution id: %s", query_execution_id)

[I] Node label/capability based scheduling (for CeleryExecutor) [airflow]

2023-10-25 Thread via GitHub
hterik opened a new issue, #35174: URL: https://github.com/apache/airflow/issues/35174 ### Description Schedule tasks based on labels/capabilities with and+or matching, instead of just queue names. ### Use case/motivation Imagine you have tasks that can only run on speci

Re: [PR] Add Cohere Provider [airflow]

2023-10-25 Thread via GitHub
utkarsharma2 commented on code in PR #34921: URL: https://github.com/apache/airflow/pull/34921#discussion_r1371602720 ## airflow/providers/cohere/hooks/cohere.py: ## @@ -0,0 +1,80 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license

Re: [I] Enhanced DAG Navigation with Search Bar on Dag View Page [airflow]

2023-10-25 Thread via GitHub
hterik commented on issue #33210: URL: https://github.com/apache/airflow/issues/33210#issuecomment-1779092233 Voting yes on the search bar, looks really useful and should be simple to implement. Voting no on the tabs. Difficult to implement and my browser already has tabs :) Somet

Re: [PR] Add Cohere Provider [airflow]

2023-10-25 Thread via GitHub
utkarsharma2 commented on code in PR #34921: URL: https://github.com/apache/airflow/pull/34921#discussion_r1371602720 ## airflow/providers/cohere/hooks/cohere.py: ## @@ -0,0 +1,80 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license

Re: [I] Add CSRF protection to "/logout" [airflow]

2023-10-25 Thread via GitHub
hterik commented on issue #33030: URL: https://github.com/apache/airflow/issues/33030#issuecomment-1779099322 Will this interfere with single-sign-*out*, when using Oauth2 or Saml login? If a user logs out from the IDP, the IDP somehow need to notify Airflow to clear the the session, tha

[airflow] branch constraints-main updated: Updating constraints. Github run id:6638965943

2023-10-25 Thread github-bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch constraints-main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/constraints-main by this push: new 7224f38902 Updating con

Re: [PR] Use timeout when inntializing _timeout_sec of TaskStateTrigger [airflow]

2023-10-25 Thread via GitHub
Xiroo commented on PR #34715: URL: https://github.com/apache/airflow/pull/34715#issuecomment-1779177551 @hussein-awala hello, I changed to use timeout parameter Can I get review for change? -- This is an automated message from the Apache Git Service. To respond to the message, plea

[PR] update README_RELEASE_PROVIDER_PACKAGES.md [airflow]

2023-10-25 Thread via GitHub
eladkal opened a new pull request, #35175: URL: https://github.com/apache/airflow/pull/35175 --- **^ Add meaningful description above** Read the **[Pull Request Guidelines](https://github.com/apache/airflow/blob/main/CONTRIBUTING.rst#pull-request-guidelines)

[PR] Add http to s3 operator [airflow]

2023-10-25 Thread via GitHub
simonprydden opened a new pull request, #35176: URL: https://github.com/apache/airflow/pull/35176 Adding HttpToS3Operator and unit tests. --- **^ Add meaningful description above** Read the **[Pull Request Guidelines](https://github.com/apache/airflow

Re: [PR] Permitting airflow kerberos to run in different modes [airflow]

2023-10-25 Thread via GitHub
amoghrajesh commented on PR #35146: URL: https://github.com/apache/airflow/pull/35146#issuecomment-1779214261 @potiuk @eladkal @jedcunningham I'd like to run the PR again by all of you. CI is green now Take a look when you have some time -- This is an automated message from the Apa

[I] Airflow Does Not Display ImportError for certain Roles [airflow]

2023-10-25 Thread via GitHub
ahipp13 opened a new issue, #35177: URL: https://github.com/apache/airflow/issues/35177 ### Apache Airflow version Other Airflow 2 version (please specify below) ### What happened So I am on Airflow 2.6.3, and we ran into a weird bug that I am wondering if it has been se

Re: [PR] Remove offset-based pagination from `list_jobs` function in `DatabricksHook` [airflow]

2023-10-25 Thread via GitHub
oleksiidav commented on PR #34926: URL: https://github.com/apache/airflow/pull/34926#issuecomment-1779249193 Hi @eladkal, could you ptal? This is ready to be merged -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the U

Re: [PR] update README_RELEASE_PROVIDER_PACKAGES.md [airflow]

2023-10-25 Thread via GitHub
eladkal merged PR #35175: URL: https://github.com/apache/airflow/pull/35175 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.

[airflow] branch main updated: update README_RELEASE_PROVIDER_PACKAGES.md (#35175)

2023-10-25 Thread eladkal
This is an automated email from the ASF dual-hosted git repository. eladkal pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 2a0cd3a8d8 update README_RELEASE_PROVIDER_PACKAGES

Re: [I] logging/remote_logging - option to disable creation of cloudwatch log groups [airflow]

2023-10-25 Thread via GitHub
nwsparks commented on issue #35164: URL: https://github.com/apache/airflow/issues/35164#issuecomment-1779293956 good suggestion, thanks @Taragolis -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

Re: [PR] Permitting airflow kerberos to run in different modes [airflow]

2023-10-25 Thread via GitHub
potiuk merged PR #35146: URL: https://github.com/apache/airflow/pull/35146 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.a

[airflow] branch main updated: Permitting airflow kerberos to run in different modes (#35146)

2023-10-25 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 36c5c111ec Permitting airflow kerberos to run in di

Re: [I] logging/remote_logging - option to disable creation of cloudwatch log groups [airflow]

2023-10-25 Thread via GitHub
nwsparks commented on issue #35164: URL: https://github.com/apache/airflow/issues/35164#issuecomment-1779320595 so it does seem watchtower was updated to not do this every time, however the code for it is wrapped in a try/catch that first does describe log groups, then swallows any errors a

Re: [PR] Fix task id copy button copying wrong id [airflow]

2023-10-25 Thread via GitHub
bbovenzi commented on code in PR #34904: URL: https://github.com/apache/airflow/pull/34904#discussion_r1371812335 ## airflow/www/static/js/components/Clipboard.tsx: ## @@ -43,9 +43,13 @@ export const ClipboardButton = forwardRef( }, ref ) => { -const { hasCopied

Re: [I] Track SparkSubmitHook Yarn Cluster application with Yarn CLI [airflow]

2023-10-25 Thread via GitHub
vchiapaikeo commented on issue #24171: URL: https://github.com/apache/airflow/issues/24171#issuecomment-1779390194 Ooo @tokoko , I just recently learned that Spark master exposes a rest API with `spark.master.rest.enabled true`. Can you check if that's the case on your yarn deployment? If p

Re: [PR] Add extra operator links for EMR Serverless [airflow]

2023-10-25 Thread via GitHub
dacort commented on code in PR #34225: URL: https://github.com/apache/airflow/pull/34225#discussion_r1371855339 ## airflow/providers/amazon/aws/links/emr.py: ## @@ -66,3 +82,98 @@ def get_log_uri( return None log_uri = S3Hook.parse_s3_url(cluster_info["LogUri"])

Re: [PR] Add extra operator links for EMR Serverless [airflow]

2023-10-25 Thread via GitHub
dacort commented on code in PR #34225: URL: https://github.com/apache/airflow/pull/34225#discussion_r1371857665 ## airflow/providers/amazon/aws/operators/emr.py: ## @@ -1173,13 +1184,35 @@ class EmrServerlessStartJobOperator(BaseOperator): "execution_role_arn",

Re: [PR] Add extra operator links for EMR Serverless [airflow]

2023-10-25 Thread via GitHub
dacort commented on code in PR #34225: URL: https://github.com/apache/airflow/pull/34225#discussion_r1371859865 ## airflow/providers/amazon/aws/operators/emr.py: ## @@ -1352,6 +1391,101 @@ def on_kill(self) -> None: check_interval_seconds=self.waiter_delay,

Re: [PR] Add extra operator links for EMR Serverless [airflow]

2023-10-25 Thread via GitHub
dacort commented on code in PR #34225: URL: https://github.com/apache/airflow/pull/34225#discussion_r1371878861 ## airflow/providers/amazon/aws/links/emr.py: ## @@ -66,3 +82,98 @@ def get_log_uri( return None log_uri = S3Hook.parse_s3_url(cluster_info["LogUri"])

Re: [PR] Add Cohere Provider [airflow]

2023-10-25 Thread via GitHub
utkarsharma2 commented on code in PR #34921: URL: https://github.com/apache/airflow/pull/34921#discussion_r1371886493 ## airflow/providers/cohere/operators/embedding.py: ## @@ -0,0 +1,82 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor lice

Re: [PR] Add Cohere Provider [airflow]

2023-10-25 Thread via GitHub
utkarsharma2 commented on code in PR #34921: URL: https://github.com/apache/airflow/pull/34921#discussion_r1371890752 ## airflow/providers/cohere/operators/embedding.py: ## @@ -0,0 +1,82 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor lice

[PR] Implement Dag SLAs [airflow]

2023-10-25 Thread via GitHub
syun64 opened a new pull request, #33532: URL: https://github.com/apache/airflow/pull/33532 closes #32816 ### Summary This PR introduces Dag SLAs as a part of **AIP-57 Refactor SLA Feature**. After many discussions, the community has voted to introduce new

Re: [PR] Add Cohere Provider [airflow]

2023-10-25 Thread via GitHub
utkarsharma2 commented on PR #34921: URL: https://github.com/apache/airflow/pull/34921#issuecomment-1779461088 > Do we need the .svg files to be added to this PR? Yes, we do need to add them, as the provider dependency changes, the breeze output command interface also changes and that

Re: [PR] Enable encryption in S3 download_files() hook. [airflow]

2023-10-25 Thread via GitHub
Taragolis merged PR #35037: URL: https://github.com/apache/airflow/pull/35037 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflo

[airflow] branch main updated (36c5c111ec -> 95980a9bc5)

2023-10-25 Thread taragolis
This is an automated email from the ASF dual-hosted git repository. taragolis pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git from 36c5c111ec Permitting airflow kerberos to run in different modes (#35146) add 95980a9bc5 Enable encryption in

[I] The logical date parameter for DAGRun is incorrect [airflow-client-python]

2023-10-25 Thread via GitHub
junqilv opened a new issue, #98: URL: https://github.com/apache/airflow-client-python/issues/98 `Exception when calling DAGRunAPI->post_dag_run: (400) Reason: BAD REQUEST HTTP response headers: HTTPHeaderDict({'Server': 'gunicorn', 'Date': 'Wed, 25 Oct 2023 15:29:05 GMT', 'Connection':

Re: [I] The logical date parameter for DAGRun is incorrect [airflow-client-python]

2023-10-25 Thread via GitHub
junqilv commented on issue #98: URL: https://github.com/apache/airflow-client-python/issues/98#issuecomment-1779554271 The sanitize_for_serialization(cls, obj) function outputs an incorrect time format -- This is an automated message from the Apache Git Service. To respond to the message

[PR] The sanitize_for_serialization(cls, obj) function outputs an incorret time format [airflow-client-python]

2023-10-25 Thread via GitHub
junqilv opened a new pull request, #99: URL: https://github.com/apache/airflow-client-python/pull/99 (no comment) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubsc

Re: [PR] Add distinct function to MongoHook in apache-airflow-providers-mongo [airflow]

2023-10-25 Thread via GitHub
Taragolis commented on code in PR #34466: URL: https://github.com/apache/airflow/pull/34466#discussion_r1372017572 ## airflow/providers/mongo/hooks/mongo.py: ## @@ -366,3 +366,22 @@ def delete_many( collection = self.get_collection(mongo_collection, mongo_db=mongo_db)

[I] Airflow Can't Terminate Tasks that are created by a DAG that uses DAG Decorator. [airflow]

2023-10-25 Thread via GitHub
ImmortalLotus opened a new issue, #35179: URL: https://github.com/apache/airflow/issues/35179 ### Apache Airflow version 2.7.2 ### What happened airflow throws this error when trying to terminate a task that comes from a DAG that is created using DAG decorator and then i

Re: [I] Airflow Can't Terminate Tasks that are created by a DAG that uses DAG Decorator. [airflow]

2023-10-25 Thread via GitHub
boring-cyborg[bot] commented on issue #35179: URL: https://github.com/apache/airflow/issues/35179#issuecomment-1779648317 Thanks for opening your first issue here! Be sure to follow the issue template! If you are willing to raise PR to address this issue please do so, no need to wait for ap

Re: [PR] Reset stats logger in daemon context to reinitialize in child process [airflow]

2023-10-25 Thread via GitHub
pavansharma36 closed pull request #34769: Reset stats logger in daemon context to reinitialize in child process URL: https://github.com/apache/airflow/pull/34769 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL abo

Re: [PR] Chart: configurable mountPath for DAGs volume [airflow]

2023-10-25 Thread via GitHub
jedcunningham merged PR #35083: URL: https://github.com/apache/airflow/pull/35083 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@ai

Re: [I] Airflow Can't Terminate Tasks that are created by a DAG that uses DAG Decorator. [airflow]

2023-10-25 Thread via GitHub
ImmortalLotus commented on issue #35179: URL: https://github.com/apache/airflow/issues/35179#issuecomment-1779653517 the main thing is that the DAG runs correctly, however the pod isn't deleted. -- This is an automated message from the Apache Git Service. To respond to the message, please

[airflow] branch main updated: Chart: configurable mountPath for DAGs volume (#35083)

2023-10-25 Thread jedcunningham
This is an automated email from the ASF dual-hosted git repository. jedcunningham pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new ac53a9aaab Chart: configurable mountPath for

Re: [PR] Reset stats logger in daemon context to reinitialize in child process [airflow]

2023-10-25 Thread via GitHub
pavansharma36 commented on PR #34769: URL: https://github.com/apache/airflow/pull/34769#issuecomment-1779653801 Will raise new PR -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comm

Re: [I] Chart: Allow mounting DAGs to a custom path in airflow containers. [airflow]

2023-10-25 Thread via GitHub
jedcunningham closed issue #34959: Chart: Allow mounting DAGs to a custom path in airflow containers. URL: https://github.com/apache/airflow/issues/34959 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[PR] reset stats instance in daemon context [airflow]

2023-10-25 Thread via GitHub
pavansharma36 opened a new pull request, #35181: URL: https://github.com/apache/airflow/pull/35181 related: #33897 --- Fix services crashing in daemon mode when statsd is enabled. -- This is an automated message from the Apache Git Service. To

Re: [PR] Add distinct function to MongoHook in apache-airflow-providers-mongo [airflow]

2023-10-25 Thread via GitHub
vincbeck merged PR #34466: URL: https://github.com/apache/airflow/pull/34466 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow

Re: [PR] Add distinct function to MongoHook in apache-airflow-providers-mongo [airflow]

2023-10-25 Thread via GitHub
boring-cyborg[bot] commented on PR #34466: URL: https://github.com/apache/airflow/pull/34466#issuecomment-1779680349 Awesome work, congrats on your first merged pull request! You are invited to check our [Issue Tracker](https://github.com/apache/airflow/issues) for additional contributions.

[airflow] branch main updated: Add distinct function to MongoHook in apache-airflow-providers-mongo (#34466)

2023-10-25 Thread vincbeck
This is an automated email from the ASF dual-hosted git repository. vincbeck pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 647e76716e Add distinct function to MongoHook in

[airflow] branch main updated: Update dev instructions to use breeze command directly with short package ids to build docs (#35080)

2023-10-25 Thread jedcunningham
This is an automated email from the ASF dual-hosted git repository. jedcunningham pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new f1f5560d97 Update dev instructions to use br

Re: [PR] Update dev instructions to use breeze command directly with short package ids to build docs [airflow]

2023-10-25 Thread via GitHub
jedcunningham merged PR #35080: URL: https://github.com/apache/airflow/pull/35080 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@ai

Re: [PR] Update dev instructions to use breeze command directly with short package ids to build docs [airflow]

2023-10-25 Thread via GitHub
jedcunningham commented on PR #35080: URL: https://github.com/apache/airflow/pull/35080#issuecomment-1779692157 Thanks @anudeep-pv! Congrats on your first commit! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

Re: [PR] Update dev instructions to use breeze command directly with short package ids to build docs [airflow]

2023-10-25 Thread via GitHub
boring-cyborg[bot] commented on PR #35080: URL: https://github.com/apache/airflow/pull/35080#issuecomment-1779690654 Awesome work, congrats on your first merged pull request! You are invited to check our [Issue Tracker](https://github.com/apache/airflow/issues) for additional contributions.

Re: [I] Airflow Can't Terminate Tasks that are created by a DAG that uses DAG Decorator. [airflow]

2023-10-25 Thread via GitHub
Taragolis closed issue #35179: Airflow Can't Terminate Tasks that are created by a DAG that uses DAG Decorator. URL: https://github.com/apache/airflow/issues/35179 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL a

Re: [PR] AWS ECS Executor [airflow]

2023-10-25 Thread via GitHub
eladkal merged PR #34381: URL: https://github.com/apache/airflow/pull/34381 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.

[airflow] branch main updated: Add AWS ECS Executor (#34381)

2023-10-25 Thread eladkal
This is an automated email from the ASF dual-hosted git repository. eladkal pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 5f4d2b5dee Add AWS ECS Executor (#34381) 5f4d2b5de

Re: [PR] AWS ECS Executor [airflow]

2023-10-25 Thread via GitHub
eladkal commented on PR #34381: URL: https://github.com/apache/airflow/pull/34381#issuecomment-1779744622 πŸŽ‰ πŸŽ‰ πŸŽ‰ πŸŽ‰ πŸŽ‰ πŸŽ‰ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To un

Re: [PR] AWS ECS Executor [airflow]

2023-10-25 Thread via GitHub
Taragolis commented on PR #34381: URL: https://github.com/apache/airflow/pull/34381#issuecomment-1779755027 Nice. Still try to find time to actually try it 🀣 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

Re: [PR] Remove offset-based pagination from `list_jobs` function in `DatabricksHook` [airflow]

2023-10-25 Thread via GitHub
eladkal commented on code in PR #34926: URL: https://github.com/apache/airflow/pull/34926#discussion_r1372113523 ## airflow/providers/databricks/CHANGELOG.rst: ## @@ -61,6 +61,13 @@ Misc 4.5.0 . +Breaking changes + + +The ``offset`` parameter has been dep

Re: [PR] Remove offset-based pagination from `list_jobs` function in `DatabricksHook` [airflow]

2023-10-25 Thread via GitHub
eladkal commented on PR #34926: URL: https://github.com/apache/airflow/pull/34926#issuecomment-1779763315 Please also add `5.0.0` entry to versions in `provider.yaml:` https://github.com/apache/airflow/blob/e9987d50598f70d84cbb2a5d964e21020e81c080/airflow/providers/databricks/provider

Re: [I] Airflow Does Not Display ImportError for certain Roles [airflow]

2023-10-25 Thread via GitHub
ahipp13 commented on issue #35177: URL: https://github.com/apache/airflow/issues/35177#issuecomment-1779788524 I found similar issue here: https://github.com/apache/airflow/issues/29897 going to close this issue. -- This is an automated message from the Apache Git Service. To respond to

Re: [I] Airflow Does Not Display ImportError for certain Roles [airflow]

2023-10-25 Thread via GitHub
ahipp13 closed issue #35177: Airflow Does Not Display ImportError for certain Roles URL: https://github.com/apache/airflow/issues/35177 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific c

Re: [I] Import error of new dags cannot be shown on web UI for common users [airflow]

2023-10-25 Thread via GitHub
ahipp13 commented on issue #29897: URL: https://github.com/apache/airflow/issues/29897#issuecomment-1779790643 This is still open in 2.6.3. Commenting here because I want to know if there are any updates on this! -- This is an automated message from the Apache Git Service. To respond to t

Re: [I] Import error of new dags cannot be shown on web UI for common users [airflow]

2023-10-25 Thread via GitHub
timlichen commented on issue #29897: URL: https://github.com/apache/airflow/issues/29897#issuecomment-1779820918 I was unable to replicate this error/behavior when I looked into this previously, it may no longer be an issue. -- This is an automated message from the Apache Git Service. To

Re: [I] Import error of new dags cannot be shown on web UI for common users [airflow]

2023-10-25 Thread via GitHub
ahipp13 commented on issue #29897: URL: https://github.com/apache/airflow/issues/29897#issuecomment-1779842177 @timlichen I am able to replicate this behavior with my setup. I have 2 roles: TEAM_ROLE: ![image](https://github.com/apache/airflow/assets/118911990/bb164e4c-0a62-49cf-a

Re: [PR] Add http to s3 operator [airflow]

2023-10-25 Thread via GitHub
Taragolis commented on code in PR #35176: URL: https://github.com/apache/airflow/pull/35176#discussion_r1372191952 ## docs/apache-airflow-providers-amazon/transfer/http_to_s3.rst: ## @@ -0,0 +1,53 @@ + .. Licensed to the Apache Software Foundation (ASF) under one +or more co

Re: [PR] Add http to s3 operator [airflow]

2023-10-25 Thread via GitHub
simonprydden commented on code in PR #35176: URL: https://github.com/apache/airflow/pull/35176#discussion_r1372198163 ## docs/apache-airflow-providers-amazon/transfer/http_to_s3.rst: ## @@ -0,0 +1,53 @@ + .. Licensed to the Apache Software Foundation (ASF) under one +or more

Re: [PR] Added `overrides` parameter to CloudRunExecuteJobOperator [airflow]

2023-10-25 Thread via GitHub
eladkal merged PR #34874: URL: https://github.com/apache/airflow/pull/34874 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.

Re: [PR] Added `overrides` parameter to CloudRunExecuteJobOperator [airflow]

2023-10-25 Thread via GitHub
boring-cyborg[bot] commented on PR #34874: URL: https://github.com/apache/airflow/pull/34874#issuecomment-1779906158 Awesome work, congrats on your first merged pull request! You are invited to check our [Issue Tracker](https://github.com/apache/airflow/issues) for additional contributions.

[airflow] branch main updated: Added `overrides` parameter to CloudRunExecuteJobOperator (#34874)

2023-10-25 Thread eladkal
This is an automated email from the ASF dual-hosted git repository. eladkal pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 0bb56315e6 Added `overrides` parameter to CloudRun

Re: [PR] AWS ECS Executor [airflow]

2023-10-25 Thread via GitHub
potiuk commented on PR #34381: URL: https://github.com/apache/airflow/pull/34381#issuecomment-1779927333 Nice! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscri

Re: [PR] The sanitize_for_serialization(cls, obj) function outputs an incorret time format [airflow-client-python]

2023-10-25 Thread via GitHub
potiuk commented on PR #99: URL: https://github.com/apache/airflow-client-python/pull/99#issuecomment-1779932375 Could you please explain (in commit message ideally as well) what was the problem it solves ? I don't see why it was wrong b) it would require to have test added. -- This i

Re: [PR] Add http to s3 operator [airflow]

2023-10-25 Thread via GitHub
Taragolis commented on code in PR #35176: URL: https://github.com/apache/airflow/pull/35176#discussion_r1372244799 ## docs/apache-airflow-providers-amazon/transfer/http_to_s3.rst: ## @@ -0,0 +1,53 @@ + .. Licensed to the Apache Software Foundation (ASF) under one +or more co

  1   2   >