[GitHub] [airflow] proofrock commented on issue #32332: Installation: pydantic.errors.PydanticUserError: A non-annotated attribute was detected: `dag_id = `.

2023-07-03 Thread via GitHub
proofrock commented on issue #32332: URL: https://github.com/apache/airflow/issues/32332#issuecomment-1619623751 > > This will help - [#32316 (comment)](https://github.com/apache/airflow/issues/32316#issuecomment-1616703896) > > Thanks! I will try tomorrow morning (CET) and report bac

[GitHub] [airflow] uranusjr commented on a diff in pull request #32122: control task group concurrency: use allow list for mapping index WIP

2023-07-03 Thread via GitHub
uranusjr commented on code in PR #32122: URL: https://github.com/apache/airflow/pull/32122#discussion_r1251549036 ## airflow/decorators/task_group.py: ## @@ -124,14 +124,14 @@ def partial(self, **kwargs: Any) -> _TaskGroupFactory[FParams, FReturn]: # TODO: fixme when m

[GitHub] [airflow] Adaverse commented on a diff in pull request #32323: refresh connection if an exception is caught in "AzureDataFactory"

2023-07-03 Thread via GitHub
Adaverse commented on code in PR #32323: URL: https://github.com/apache/airflow/pull/32323#discussion_r1251536943 ## airflow/providers/microsoft/azure/hooks/data_factory.py: ## @@ -828,7 +834,14 @@ def wait_for_pipeline_run_status( # Wait to check the status of the

[GitHub] [airflow] uranusjr commented on a diff in pull request #32309: Hide sensitive values from extra in connection edit form

2023-07-03 Thread via GitHub
uranusjr commented on code in PR #32309: URL: https://github.com/apache/airflow/pull/32309#discussion_r1251536749 ## airflow/www/views.py: ## @@ -4819,8 +4835,21 @@ def process_form(self, form, is_created): # value isn't an empty string.

[GitHub] [airflow] assaf-xm commented on pull request #21092: Delete old Spark Application in SparkKubernetesOperator

2023-07-03 Thread via GitHub
assaf-xm commented on PR #21092: URL: https://github.com/apache/airflow/pull/21092#issuecomment-1619567746 @theesen , @potiuk , were the changes in this PR reverted in https://github.com/apache/airflow/commit/9a4f6748521c9c3b66d96598036be08fd94ccf89 ? I can see that with the latest a

[GitHub] [airflow] Adaverse commented on a diff in pull request #32323: refresh connection if an exception is caught in "AzureDataFactory"

2023-07-03 Thread via GitHub
Adaverse commented on code in PR #32323: URL: https://github.com/apache/airflow/pull/32323#discussion_r1251529709 ## airflow/providers/microsoft/azure/hooks/data_factory.py: ## @@ -828,7 +834,14 @@ def wait_for_pipeline_run_status( # Wait to check the status of the

[GitHub] [airflow] uranusjr commented on a diff in pull request #32309: Hide sensitive values from extra in connection edit form

2023-07-03 Thread via GitHub
uranusjr commented on code in PR #32309: URL: https://github.com/apache/airflow/pull/32309#discussion_r1251532557 ## airflow/www/views.py: ## @@ -4819,8 +4835,21 @@ def process_form(self, form, is_created): # value isn't an empty string.

[GitHub] [airflow] uranusjr commented on a diff in pull request #32309: Hide sensitive values from extra in connection edit form

2023-07-03 Thread via GitHub
uranusjr commented on code in PR #32309: URL: https://github.com/apache/airflow/pull/32309#discussion_r1251531548 ## airflow/providers_manager.py: ## @@ -882,7 +883,12 @@ def _add_widgets(self, package_name: str, hook_class: type, widgets: dict[str, A # In case

[GitHub] [airflow] Adaverse commented on a diff in pull request #32323: refresh connection if an exception is caught in "AzureDataFactory"

2023-07-03 Thread via GitHub
Adaverse commented on code in PR #32323: URL: https://github.com/apache/airflow/pull/32323#discussion_r1251529709 ## airflow/providers/microsoft/azure/hooks/data_factory.py: ## @@ -828,7 +834,14 @@ def wait_for_pipeline_run_status( # Wait to check the status of the

[airflow] branch revert-32325-exclude-cross-providers-for-non-releasable-providers deleted (was 20565e7bc6)

2023-07-03 Thread eladkal
This is an automated email from the ASF dual-hosted git repository. eladkal pushed a change to branch revert-32325-exclude-cross-providers-for-non-releasable-providers in repository https://gitbox.apache.org/repos/asf/airflow.git was 20565e7bc6 Revert "Exckude not-releaseable providers fro

[airflow] branch main updated: Revert "Exckude not-releaseable providers from extras not only from releasing (#32325)" (#32338)

2023-07-03 Thread eladkal
This is an automated email from the ASF dual-hosted git repository. eladkal pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new d7c966441e Revert "Exckude not-releaseable provide

[GitHub] [airflow] eladkal merged pull request #32338: Revert "Exclude not-releaseable providers from extras not only from releasing"

2023-07-03 Thread via GitHub
eladkal merged PR #32338: URL: https://github.com/apache/airflow/pull/32338 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.

[GitHub] [airflow] ephraimbuddy commented on a diff in pull request #32313: Introduce Heartbeat Parameter to Allow Per-LocalTaskJob Configuration

2023-07-03 Thread via GitHub
ephraimbuddy commented on code in PR #32313: URL: https://github.com/apache/airflow/pull/32313#discussion_r1251526258 ## airflow/jobs/local_task_job_runner.py: ## @@ -158,7 +158,7 @@ def sigusr2_debug_handler(signum, frame): try: self.task_runner.start()

[GitHub] [airflow] uranusjr commented on a diff in pull request #31663: Logging from all containers in KubernetesOperatorPod

2023-07-03 Thread via GitHub
uranusjr commented on code in PR #31663: URL: https://github.com/apache/airflow/pull/31663#discussion_r1251524796 ## airflow/providers/cncf/kubernetes/operators/pod.py: ## @@ -342,6 +348,9 @@ def __init__( self.cluster_context = cluster_context self.reattach_on

[airflow] branch revert-32325-exclude-cross-providers-for-non-releasable-providers created (now 20565e7bc6)

2023-07-03 Thread eladkal
This is an automated email from the ASF dual-hosted git repository. eladkal pushed a change to branch revert-32325-exclude-cross-providers-for-non-releasable-providers in repository https://gitbox.apache.org/repos/asf/airflow.git at 20565e7bc6 Revert "Exckude not-releaseable providers fro

[airflow] 01/01: Revert "Exckude not-releaseable providers from extras not only from releasing (#32325)"

2023-07-03 Thread eladkal
This is an automated email from the ASF dual-hosted git repository. eladkal pushed a commit to branch revert-32325-exclude-cross-providers-for-non-releasable-providers in repository https://gitbox.apache.org/repos/asf/airflow.git commit 20565e7bc66707bfa9089835927ec44f03fbee77 Author: eladkal <4

[GitHub] [airflow] eladkal opened a new pull request, #32338: Revert "Exclude not-releaseable providers from extras not only from releasing"

2023-07-03 Thread via GitHub
eladkal opened a new pull request, #32338: URL: https://github.com/apache/airflow/pull/32338 Reverts apache/airflow#32325 Oops :) merged the wrong PR -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL ab

[airflow] branch main updated: Exckude not-releaseable providers from extras not only from releasing (#32325)

2023-07-03 Thread eladkal
This is an automated email from the ASF dual-hosted git repository. eladkal pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new bc212f80e8 Exckude not-releaseable providers from

[GitHub] [airflow] eladkal merged pull request #32325: Exclude not-releaseable providers from extras not only from releasing

2023-07-03 Thread via GitHub
eladkal merged PR #32325: URL: https://github.com/apache/airflow/pull/32325 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.

[GitHub] [airflow] uranusjr opened a new pull request, #32337: Support Pydantic 2

2023-07-03 Thread via GitHub
uranusjr opened a new pull request, #32337: URL: https://github.com/apache/airflow/pull/32337 Fix #32311 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-m

[GitHub] [airflow] amoghrajesh commented on pull request #31663: Logging from all containers in KubernetesOperatorPod

2023-07-03 Thread via GitHub
amoghrajesh commented on PR #31663: URL: https://github.com/apache/airflow/pull/31663#issuecomment-1619531084 @hussein-awala can you take a look at this PR again when you have some time? -- This is an automated message from the Apache Git Service. To respond to the message, please log on t

[GitHub] [airflow] ephraimbuddy closed issue #32324: DAG run fails when leaf node task is skipped.

2023-07-03 Thread via GitHub
ephraimbuddy closed issue #32324: DAG run fails when leaf node task is skipped. URL: https://github.com/apache/airflow/issues/32324 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific commen

[GitHub] [airflow] ephraimbuddy commented on issue #32324: DAG run fails when leaf node task is skipped.

2023-07-03 Thread via GitHub
ephraimbuddy commented on issue #32324: URL: https://github.com/apache/airflow/issues/32324#issuecomment-1619526461 Closing for now. Do let us know if there's a way to reproduce it -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitH

[GitHub] [airflow] Lee-W commented on a diff in pull request #32323: refresh connection if an exception is caught in "AzureDataFactory"

2023-07-03 Thread via GitHub
Lee-W commented on code in PR #32323: URL: https://github.com/apache/airflow/pull/32323#discussion_r1251486897 ## airflow/providers/microsoft/azure/hooks/data_factory.py: ## @@ -828,7 +833,14 @@ def wait_for_pipeline_run_status( # Wait to check the status of the pip

[GitHub] [airflow] uranusjr commented on a diff in pull request #32310: Update python.rst with multi-Scheduler LocalExecutor info

2023-07-03 Thread via GitHub
uranusjr commented on code in PR #32310: URL: https://github.com/apache/airflow/pull/32310#discussion_r1251466697 ## docs/apache-airflow/core-concepts/executor/local.rst: ## @@ -47,3 +47,13 @@ Arguably, :class:`~airflow.executors.sequential_executor.SequentialExecutor` cou par

[GitHub] [airflow] uranusjr commented on a diff in pull request #31904: Fix `operator_extra_links` property serialization in mapped tasks

2023-07-03 Thread via GitHub
uranusjr commented on code in PR #31904: URL: https://github.com/apache/airflow/pull/31904#discussion_r1251424562 ## airflow/serialization/serialized_objects.py: ## @@ -287,7 +287,7 @@ class BaseSerialization: _datetime_types = (datetime.datetime,) # Object types tha

[GitHub] [airflow] github-actions[bot] commented on pull request #29433: Add dataset update endpoint

2023-07-03 Thread via GitHub
github-actions[bot] commented on PR #29433: URL: https://github.com/apache/airflow/pull/29433#issuecomment-1619283233 This pull request has been automatically marked as stale because it has not had recent activity. It will be closed in 5 days if no further activity occurs. Thank you for you

[GitHub] [airflow] github-actions[bot] closed pull request #31219: support from_email sourcing from email connection

2023-07-03 Thread via GitHub
github-actions[bot] closed pull request #31219: support from_email sourcing from email connection URL: https://github.com/apache/airflow/pull/31219 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to th

[GitHub] [airflow] github-actions[bot] closed pull request #31275: KPO: Add Typing on Security Context

2023-07-03 Thread via GitHub
github-actions[bot] closed pull request #31275: KPO: Add Typing on Security Context URL: https://github.com/apache/airflow/pull/31275 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific com

[GitHub] [airflow] github-actions[bot] commented on pull request #31378: Move metadata declarations to pyproject.toml

2023-07-03 Thread via GitHub
github-actions[bot] commented on PR #31378: URL: https://github.com/apache/airflow/pull/31378#issuecomment-1619283156 This pull request has been automatically marked as stale because it has not had recent activity. It will be closed in 5 days if no further activity occurs. Thank you for you

[GitHub] [airflow] github-actions[bot] closed pull request #30166: Update usage of accept_none arg in SQLColumn/TableCheckOperators

2023-07-03 Thread via GitHub
github-actions[bot] closed pull request #30166: Update usage of accept_none arg in SQLColumn/TableCheckOperators URL: https://github.com/apache/airflow/pull/30166 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL ab

[GitHub] [airflow] dwreeves commented on pull request #31846: add more accurate typing for DbApiHook.run method

2023-07-03 Thread via GitHub
dwreeves commented on PR #31846: URL: https://github.com/apache/airflow/pull/31846#issuecomment-1619261426 Fixed merge conflicts and merged changes that have passing CI. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

[airflow] branch constraints-main updated: Updating constraints. Github run id:5448920084

2023-07-03 Thread github-bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch constraints-main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/constraints-main by this push: new 7accd0df39 Updating con

[airflow] branch main updated: Added `AzureBlobStorageToS3Operator` transfer operator (#32270)

2023-07-03 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 17132ef4c6 Added `AzureBlobStorageToS3Operator` tra

[GitHub] [airflow] potiuk merged pull request #32270: Added `AzureBlobStorageToS3Operator` transfer operator

2023-07-03 Thread via GitHub
potiuk merged PR #32270: URL: https://github.com/apache/airflow/pull/32270 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.a

[GitHub] [airflow] potiuk commented on issue #32335: Git Sync 4 support

2023-07-03 Thread via GitHub
potiuk commented on issue #32335: URL: https://github.com/apache/airflow/issues/32335#issuecomment-1619204978 FYI: Our chart variables - for one - are completely not working (for example GIT_SYNC_WAIT -> GITSYNC_PERIOD. -- This is an automated message from the Apache Git Service. To respo

[GitHub] [airflow] potiuk opened a new issue, #32335: Git Sync 4 support

2023-07-03 Thread via GitHub
potiuk opened a new issue, #32335: URL: https://github.com/apache/airflow/issues/32335 ### Body Git Sync 4 - about to be released introduces a lot of changes (including changed names for env variables).That heavily impacts our Helm Chart and we should fix those incompatiblities befor

[airflow] branch main updated: Render list items in rendered fields view (#32042)

2023-07-03 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 2b5d4e319b Render list items in rendered fields vie

[GitHub] [airflow] potiuk merged pull request #32042: Render list items in rendered fields view

2023-07-03 Thread via GitHub
potiuk merged PR #32042: URL: https://github.com/apache/airflow/pull/32042 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.a

[GitHub] [airflow] boring-cyborg[bot] commented on pull request #32042: Render list items in rendered fields view

2023-07-03 Thread via GitHub
boring-cyborg[bot] commented on PR #32042: URL: https://github.com/apache/airflow/pull/32042#issuecomment-1619184181 Awesome work, congrats on your first merged pull request! You are invited to check our [Issue Tracker](https://github.com/apache/airflow/issues) for additional contributions.

[GitHub] [airflow] pragnareddye commented on a diff in pull request #32310: Update python.rst with multi-Scheduler LocalExecutor info

2023-07-03 Thread via GitHub
pragnareddye commented on code in PR #32310: URL: https://github.com/apache/airflow/pull/32310#discussion_r1251298660 ## docs/apache-airflow/core-concepts/executor/local.rst: ## @@ -47,3 +47,13 @@ Arguably, :class:`~airflow.executors.sequential_executor.SequentialExecutor` cou

[GitHub] [airflow] potiuk opened a new pull request, #32334: Add proxy_user template check

2023-07-03 Thread via GitHub
potiuk opened a new pull request, #32334: URL: https://github.com/apache/airflow/pull/32334 --- **^ Add meaningful description above** Read the **[Pull Request Guidelines](https://github.com/apache/airflow/blob/main/CONTRIBUTING.rst#pull-request-guidelines)** for m

[GitHub] [airflow] hussein-awala commented on issue #31648: Getting Unauthorized error messages with GKEStartPodOperator if pod execution is over 1 hour

2023-07-03 Thread via GitHub
hussein-awala commented on issue #31648: URL: https://github.com/apache/airflow/issues/31648#issuecomment-1619149837 I created this draft PR #32333, I will not have time to test it before Wednesday, if someone can test it on a GKE it would be great. I will also test if it masks the passed k

[GitHub] [airflow] hussein-awala opened a new pull request, #32333: Use kubeconfig file to auth to GKE instead of oauth2 token

2023-07-03 Thread via GitHub
hussein-awala opened a new pull request, #32333: URL: https://github.com/apache/airflow/pull/32333 closes: #31648 Before the PR #29266, we used to create a temporary configuration file by running the command gcloud container clusters get-credentials. We would then use thi

[GitHub] [airflow] Joffreybvn commented on a diff in pull request #32304: Add kerberos dependency to Impala Provider

2023-07-03 Thread via GitHub
Joffreybvn commented on code in PR #32304: URL: https://github.com/apache/airflow/pull/32304#discussion_r1251248837 ## airflow/providers/apache/impala/provider.yaml: ## @@ -28,7 +28,7 @@ versions: - 1.0.0 dependencies: - - impyla>=0.18.0,<1.0 + - impyla[kerberos]>=0.18.0

[GitHub] [airflow] Joffreybvn commented on a diff in pull request #32304: Add kerberos dependency to Impala Provider

2023-07-03 Thread via GitHub
Joffreybvn commented on code in PR #32304: URL: https://github.com/apache/airflow/pull/32304#discussion_r1251248837 ## airflow/providers/apache/impala/provider.yaml: ## @@ -28,7 +28,7 @@ versions: - 1.0.0 dependencies: - - impyla>=0.18.0,<1.0 + - impyla[kerberos]>=0.18.0

[GitHub] [airflow] stikkireddy commented on a diff in pull request #32221: Add `DatabricksJobsCreateOperator`

2023-07-03 Thread via GitHub
stikkireddy commented on code in PR #32221: URL: https://github.com/apache/airflow/pull/32221#discussion_r1251247493 ## airflow/providers/databricks/hooks/databricks.py: ## @@ -135,9 +137,27 @@ def __init__( ) -> None: super().__init__(databricks_conn_id, timeout_s

[GitHub] [airflow] stikkireddy commented on a diff in pull request #32221: Add `DatabricksJobsCreateOperator`

2023-07-03 Thread via GitHub
stikkireddy commented on code in PR #32221: URL: https://github.com/apache/airflow/pull/32221#discussion_r1251241946 ## airflow/providers/databricks/operators/databricks.py: ## @@ -160,6 +162,142 @@ def get_link( return XCom.get_value(key=XCOM_RUN_PAGE_URL_KEY, ti_key=t

[GitHub] [airflow] Joffreybvn commented on a diff in pull request #32319: Make ODBCHook result JSON-serializable

2023-07-03 Thread via GitHub
Joffreybvn commented on code in PR #32319: URL: https://github.com/apache/airflow/pull/32319#discussion_r1251241162 ## airflow/providers/odbc/hooks/odbc.py: ## @@ -211,3 +211,10 @@ def get_sqlalchemy_connection( engine = self.get_sqlalchemy_engine(engine_kwargs=engine_k

[GitHub] [airflow] Adaverse commented on a diff in pull request #32328: Append region info to S3ToRedshitOperator if present

2023-07-03 Thread via GitHub
Adaverse commented on code in PR #32328: URL: https://github.com/apache/airflow/pull/32328#discussion_r1251223065 ## tests/providers/amazon/aws/transfers/test_s3_to_redshift.py: ## @@ -420,16 +477,8 @@ def test_using_redshift_data_api(self, mock_rs, mock_run, mock_session, mock

[GitHub] [airflow] Adaverse commented on a diff in pull request #32328: Append region info to S3ToRedshitOperator if present

2023-07-03 Thread via GitHub
Adaverse commented on code in PR #32328: URL: https://github.com/apache/airflow/pull/32328#discussion_r1251222624 ## tests/providers/amazon/aws/transfers/test_s3_to_redshift.py: ## @@ -443,8 +492,21 @@ def test_using_redshift_data_api(self, mock_rs, mock_run, mock_session, mock

[GitHub] [airflow] Adaverse commented on issue #32290: Try number is incorrect

2023-07-03 Thread via GitHub
Adaverse commented on issue #32290: URL: https://github.com/apache/airflow/issues/32290#issuecomment-1619045557 I looked into it, the endpoint `api/v1/dags/{dag_id}/dagRuns/{dag_run_id}/taskInstances/{task_id}` and it was giving the right result. In the code, it is seems to be attached to t

[GitHub] [airflow] jens-scheffler-bosch closed issue #32332: Installation: pydantic.errors.PydanticUserError: A non-annotated attribute was detected: `dag_id = `.

2023-07-03 Thread via GitHub
jens-scheffler-bosch closed issue #32332: Installation: pydantic.errors.PydanticUserError: A non-annotated attribute was detected: `dag_id = `. URL: https://github.com/apache/airflow/issues/32332 -- This is an automated message from the Apache Git Service. To respond to the message, please l

[GitHub] [airflow] jens-scheffler-bosch commented on issue #32332: Installation: pydantic.errors.PydanticUserError: A non-annotated attribute was detected: `dag_id = `.

2023-07-03 Thread via GitHub
jens-scheffler-bosch commented on issue #32332: URL: https://github.com/apache/airflow/issues/32332#issuecomment-1619002259 Duplicate of #32301 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [airflow] bbovenzi commented on pull request #32288: Make the DAG Run UI view more reliable by removing the column `config`

2023-07-03 Thread via GitHub
bbovenzi commented on PR #32288: URL: https://github.com/apache/airflow/pull/32288#issuecomment-1618961466 Do we want to leave out the config entirely? I think we'd just want to convert it to a json object or a string. Because a user would want to see the config a run was triggered with.

[GitHub] [airflow] bhagany commented on issue #32106: GCSToBigQueryOperator and BigQueryToGCSOperator do not respect their project_id arguments

2023-07-03 Thread via GitHub
bhagany commented on issue #32106: URL: https://github.com/apache/airflow/issues/32106#issuecomment-1618909358 I'm still on vacation, but popping in here to add my 2 cents regarding your PR @avinashpandeshwar -- I don't think these PR's are mutually compatible, with regard to how they treat

[GitHub] [airflow] phanikumv commented on a diff in pull request #32323: refresh connection if an exception is caught in "AzureDataFactory"

2023-07-03 Thread via GitHub
phanikumv commented on code in PR #32323: URL: https://github.com/apache/airflow/pull/32323#discussion_r1251127914 ## airflow/providers/microsoft/azure/triggers/data_factory.py: ## @@ -147,33 +162,47 @@ async def run(self) -> AsyncIterator[TriggerEvent]: resourc

[GitHub] [airflow] proofrock commented on issue #32332: Installation: pydantic.errors.PydanticUserError: A non-annotated attribute was detected: `dag_id = `.

2023-07-03 Thread via GitHub
proofrock commented on issue #32332: URL: https://github.com/apache/airflow/issues/32332#issuecomment-1618898184 > This will help - https://github.com/apache/airflow/issues/32316#issuecomment-1616703896 Thanks! I will try tomorrow morning (CET) and report back. -- This is an automa

[GitHub] [airflow] Adaverse commented on issue #32332: Installation: pydantic.errors.PydanticUserError: A non-annotated attribute was detected: `dag_id = `.

2023-07-03 Thread via GitHub
Adaverse commented on issue #32332: URL: https://github.com/apache/airflow/issues/32332#issuecomment-1618892068 This will help - https://github.com/apache/airflow/issues/32316#issuecomment-1616703896 -- This is an automated message from the Apache Git Service. To respond to the message, p

[GitHub] [airflow] proofrock commented on issue #32332: Installation: pydantic.errors.PydanticUserError: A non-annotated attribute was detected: `dag_id = `.

2023-07-03 Thread via GitHub
proofrock commented on issue #32332: URL: https://github.com/apache/airflow/issues/32332#issuecomment-1618885069 Hello and thanks for answering. How do I do that? I am not too familiar with python, all I can see is a pip command that installs a predefined dependency set (that includes that

[GitHub] [airflow] Adaverse commented on issue #32332: Installation: pydantic.errors.PydanticUserError: A non-annotated attribute was detected: `dag_id = `.

2023-07-03 Thread via GitHub
Adaverse commented on issue #32332: URL: https://github.com/apache/airflow/issues/32332#issuecomment-1618881242 I see your Pydantic version is 2.0.0, can u try with a lower version? For more info see this - #32311 -- This is an automated message from the Apache Git Service. To respond to

[GitHub] [airflow] richardbadman commented on a diff in pull request #32328: Append region info to S3ToRedshitOperator if present

2023-07-03 Thread via GitHub
richardbadman commented on code in PR #32328: URL: https://github.com/apache/airflow/pull/32328#discussion_r1251089059 ## tests/providers/amazon/aws/transfers/test_s3_to_redshift.py: ## @@ -324,6 +324,55 @@ def test_execute_role_arn(self, mock_run, mock_session, mock_connection

[GitHub] [airflow] avinashpandeshwar commented on issue #32106: GCSToBigQueryOperator and BigQueryToGCSOperator do not respect their project_id arguments

2023-07-03 Thread via GitHub
avinashpandeshwar commented on issue #32106: URL: https://github.com/apache/airflow/issues/32106#issuecomment-1618775957 @eladkal Yeah, your'e right. I will be merging these fixes into my PR. -- This is an automated message from the Apache Git Service. To respond to the message, please log

[GitHub] [airflow] avinashpandeshwar commented on pull request #32232: Fixing Issue - Provided project_id parameter not getting used to subm…

2023-07-03 Thread via GitHub
avinashpandeshwar commented on PR #32232: URL: https://github.com/apache/airflow/pull/32232#issuecomment-1618767738 @eladkal Sure, that makes sense. Working on merging all the fixes into this PR. -- This is an automated message from the Apache Git Service. To respond to the message, pleas

[GitHub] [airflow] eladkal commented on pull request #32232: Fixing Issue - Provided project_id parameter not getting used to subm…

2023-07-03 Thread via GitHub
eladkal commented on PR #32232: URL: https://github.com/apache/airflow/pull/32232#issuecomment-1618734790 Lets please fix all occurrences as mentioned in https://github.com/apache/airflow/issues/32106#issuecomment-1614573765 -- This is an automated message from the Apache Git Service. To

[airflow] branch constraints-main updated: Updating constraints. Github run id:5445283882

2023-07-03 Thread github-bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch constraints-main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/constraints-main by this push: new e506624e34 Updating con

[GitHub] [airflow] boring-cyborg[bot] commented on issue #32332: Installation: pydantic.errors.PydanticUserError: A non-annotated attribute was detected: `dag_id = `.

2023-07-03 Thread via GitHub
boring-cyborg[bot] commented on issue #32332: URL: https://github.com/apache/airflow/issues/32332#issuecomment-1618579236 Thanks for opening your first issue here! Be sure to follow the issue template! If you are willing to raise PR to address this issue please do so, no need to wait for ap

[GitHub] [airflow] proofrock opened a new issue, #32332: Installation: pydantic.errors.PydanticUserError: A non-annotated attribute was detected: `dag_id = `.

2023-07-03 Thread via GitHub
proofrock opened a new issue, #32332: URL: https://github.com/apache/airflow/issues/32332 ### Apache Airflow version 2.6.2 ### What happened I install Airflow with: ``` pip install apache-airflow['postgresql'] pip install psycopg2-binary # airflow.cfg: exe

[GitHub] [airflow] potiuk commented on pull request #32232: Fixing Issue - Provided project_id parameter not getting used to subm…

2023-07-03 Thread via GitHub
potiuk commented on PR #32232: URL: https://github.com/apache/airflow/pull/32232#issuecomment-1618557507 Static checks need fixing + rebase. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the sp

[GitHub] [airflow] potiuk commented on pull request #31596: Add Deferrable switch to SnowflakeSqlApiOperator

2023-07-03 Thread via GitHub
potiuk commented on PR #31596: URL: https://github.com/apache/airflow/pull/31596#issuecomment-1618549959 I think you should rebase/solve conflicts on that one jsut to be sure. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub an

[GitHub] [airflow] Francesko90 commented on a diff in pull request #31868: Runtime class name

2023-07-03 Thread via GitHub
Francesko90 commented on code in PR #31868: URL: https://github.com/apache/airflow/pull/31868#discussion_r1250985992 ## chart/values.yaml: ## @@ -2056,36 +2062,36 @@ config: # `run_duration` included for Airflow 1.10 backward compatibility; removed in 2.0. run_duratio

[GitHub] [airflow] luos-fc opened a new pull request, #32331: Only update crawler tags if present in config dict

2023-07-03 Thread via GitHub
luos-fc opened a new pull request, #32331: URL: https://github.com/apache/airflow/pull/32331 Closes: #32330 This PR modifies the GlueCrawlerHook to only update the crawlers tags if the `Tags` key is present in the `config` dict. This prevent situations where tags that are set by ext

[GitHub] [airflow] amoghrajesh commented on a diff in pull request #31868: Runtime class name

2023-07-03 Thread via GitHub
amoghrajesh commented on code in PR #31868: URL: https://github.com/apache/airflow/pull/31868#discussion_r1250962465 ## tests/charts/airflow_core/test_worker.py: ## @@ -373,6 +373,17 @@ def test_should_create_default_affinity(self): docs[0], ) +def te

[GitHub] [airflow] Adaverse commented on a diff in pull request #32328: Append region info to S3ToRedshitOperator if present

2023-07-03 Thread via GitHub
Adaverse commented on code in PR #32328: URL: https://github.com/apache/airflow/pull/32328#discussion_r1250937194 ## tests/providers/amazon/aws/transfers/test_s3_to_redshift.py: ## @@ -324,6 +324,55 @@ def test_execute_role_arn(self, mock_run, mock_session, mock_connection, moc

[GitHub] [airflow] fsellecchia commented on issue #30722: Ability to set automountServiceAccountToken in values.yaml

2023-07-03 Thread via GitHub
fsellecchia commented on issue #30722: URL: https://github.com/apache/airflow/issues/30722#issuecomment-1618350707 Hi @amoghrajesh, I'm ok with that. Unfortunately I'be been busy in the last months so I was not able to work on this. Here is the PR https://github.com/apache/airflow/pull/3075

[GitHub] [airflow] richardbadman commented on a diff in pull request #32328: Append region info to S3ToRedshitOperator if present

2023-07-03 Thread via GitHub
richardbadman commented on code in PR #32328: URL: https://github.com/apache/airflow/pull/32328#discussion_r1250930999 ## tests/providers/amazon/aws/transfers/test_s3_to_redshift.py: ## @@ -324,6 +324,55 @@ def test_execute_role_arn(self, mock_run, mock_session, mock_connection

[GitHub] [airflow] richardbadman commented on a diff in pull request #32328: Append region info to S3ToRedshitOperator if present

2023-07-03 Thread via GitHub
richardbadman commented on code in PR #32328: URL: https://github.com/apache/airflow/pull/32328#discussion_r1250930999 ## tests/providers/amazon/aws/transfers/test_s3_to_redshift.py: ## @@ -324,6 +324,55 @@ def test_execute_role_arn(self, mock_run, mock_session, mock_connection

[airflow] branch main updated: Fixing typo in LOCAL_VIRTUALENV.rst (#32327)

2023-07-03 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 42eb93b87b Fixing typo in LOCAL_VIRTUALENV.rst (#32

[GitHub] [airflow] potiuk merged pull request #32327: Fixing typo in LOCAL_VIRTUALENV.rst

2023-07-03 Thread via GitHub
potiuk merged PR #32327: URL: https://github.com/apache/airflow/pull/32327 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.a

[GitHub] [airflow] Adaverse commented on a diff in pull request #32328: Append region info to S3ToRedshitOperator if present

2023-07-03 Thread via GitHub
Adaverse commented on code in PR #32328: URL: https://github.com/apache/airflow/pull/32328#discussion_r1250925367 ## tests/providers/amazon/aws/transfers/test_s3_to_redshift.py: ## @@ -324,6 +324,55 @@ def test_execute_role_arn(self, mock_run, mock_session, mock_connection, moc

[GitHub] [airflow] potiuk commented on pull request #32325: Exclude not-releaseable providers from extras not only from releasing

2023-07-03 Thread via GitHub
potiuk commented on PR #32325: URL: https://github.com/apache/airflow/pull/32325#issuecomment-1618297758 Up to @eladkal if he wants to merge it :) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [airflow] potiuk commented on a diff in pull request #32309: Hide sensitive values from extra in connection edit form

2023-07-03 Thread via GitHub
potiuk commented on code in PR #32309: URL: https://github.com/apache/airflow/pull/32309#discussion_r1250913706 ## airflow/providers_manager.py: ## @@ -882,7 +884,11 @@ def _add_widgets(self, package_name: str, hook_class: type, widgets: dict[str, A # In case o

[GitHub] [airflow] luos-fc opened a new issue, #32330: AWS GlueCrawlerOperator deletes existing tags on run

2023-07-03 Thread via GitHub
luos-fc opened a new issue, #32330: URL: https://github.com/apache/airflow/issues/32330 ### Apache Airflow version 2.6.2 ### What happened We are currently on AWS Provider 6.0.0 and looking to upgrade to the latest version 8.2.0. However, there are some issues with the G

[airflow] branch main updated: invalid args fix (#32326)

2023-07-03 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 985a6cd2a8 invalid args fix (#32326) 985a6cd2a8 is

[GitHub] [airflow] potiuk merged pull request #32326: Invalid args fix in `example_dag` sensor example Azure provider

2023-07-03 Thread via GitHub
potiuk merged PR #32326: URL: https://github.com/apache/airflow/pull/32326 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.a

[GitHub] [airflow] richardbadman opened a new pull request, #32328: Append region info to S3ToRedshitOperator if present

2023-07-03 Thread via GitHub
richardbadman opened a new pull request, #32328: URL: https://github.com/apache/airflow/pull/32328 It's possible to copy from S3 into Redshift across different regions, however, currently you are unable to do so with the S3ToRedshiftOperator. This PR simply makes this possible, by checking

[GitHub] [airflow] zach-b commented on issue #32289: Deadlock due to zombie task clean-up executed twice

2023-07-03 Thread via GitHub
zach-b commented on issue #32289: URL: https://github.com/apache/airflow/issues/32289#issuecomment-1618209203 A workaround to avoid this until it is fixed is to set the zombie clean up interval to a value higher than the DAG file parsing interval. -- This is an automated message from the

[GitHub] [airflow] amoghrajesh opened a new pull request, #32327: Fixing typo in LOCAL_VIRTUALENV.rst

2023-07-03 Thread via GitHub
amoghrajesh opened a new pull request, #32327: URL: https://github.com/apache/airflow/pull/32327 --- **^ Add meaningful description above** Read the **[Pull Request Guidelines](https://github.com/apache/airflow/blob/main/CONTRIBUTING.rst#pull-request-guidelines)**

[airflow] branch main updated: Refactor Sqlalchemy queries to 2.0 style (Part 3) (#32177)

2023-07-03 Thread phanikumv
This is an automated email from the ASF dual-hosted git repository. phanikumv pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 1065687ec6 Refactor Sqlalchemy queries to 2.0 st

[GitHub] [airflow] phanikumv merged pull request #32177: Refactor Sqlalchemy queries to 2.0 style (Part 3)

2023-07-03 Thread via GitHub
phanikumv merged PR #32177: URL: https://github.com/apache/airflow/pull/32177 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflo

[GitHub] [airflow] Adaverse opened a new pull request, #32326: Invalid args fix

2023-07-03 Thread via GitHub
Adaverse opened a new pull request, #32326: URL: https://github.com/apache/airflow/pull/32326 Fixes erroneous change made in #32299 that was missed in the PR CI. --- **^ Add meaningful description above** Read the **[Pull Request Guidelines](https://github.co

[GitHub] [airflow] ephraimbuddy commented on issue #32324: DAG run fails when leaf node task is skipped.

2023-07-03 Thread via GitHub
ephraimbuddy commented on issue #32324: URL: https://github.com/apache/airflow/issues/32324#issuecomment-1617947349 @Cricktom, we can't reproduce your issue with the dag you provided. Let us know if you have any other reproduction steps -- This is an automated message from the Apache Git

[GitHub] [airflow] ephraimbuddy commented on issue #32324: DAG run fails when leaf node task is skipped.

2023-07-03 Thread via GitHub
ephraimbuddy commented on issue #32324: URL: https://github.com/apache/airflow/issues/32324#issuecomment-1617943619 I tried in main and the dag succeeded too. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL abo

[GitHub] [airflow] mis98zb commented on pull request #32122: control task group concurrency: use allow list for mapping index WIP

2023-07-03 Thread via GitHub
mis98zb commented on PR #32122: URL: https://github.com/apache/airflow/pull/32122#issuecomment-1617941992 > However, I'm trying to use `task_map` table to store the limitation, so as it can consist concurrency control for task as well as task group... Since the task mapping info is in

[GitHub] [airflow] ephraimbuddy commented on a diff in pull request #32177: Refactor Sqlalchemy queries to 2.0 style (Part 3)

2023-07-03 Thread via GitHub
ephraimbuddy commented on code in PR #32177: URL: https://github.com/apache/airflow/pull/32177#discussion_r1250694947 ## airflow/utils/db.py: ## @@ -1101,7 +1102,7 @@ def check_conn_type_null(session: Session) -> Iterable[str]: n_nulls = [] try: -n_nulls =

[GitHub] [airflow] ephraimbuddy commented on a diff in pull request #32177: Refactor Sqlalchemy queries to 2.0 style (Part 3)

2023-07-03 Thread via GitHub
ephraimbuddy commented on code in PR #32177: URL: https://github.com/apache/airflow/pull/32177#discussion_r1250694947 ## airflow/utils/db.py: ## @@ -1101,7 +1102,7 @@ def check_conn_type_null(session: Session) -> Iterable[str]: n_nulls = [] try: -n_nulls =

[GitHub] [airflow] Adaverse commented on issue #32324: DAG run fails when leaf node task is skipped.

2023-07-03 Thread via GitHub
Adaverse commented on issue #32324: URL: https://github.com/apache/airflow/issues/32324#issuecomment-1617896439 I tried with the same code above; the DAG succeeded for the same Airflow version. -- This is an automated message from the Apache Git Service. To respond to the message, please

[GitHub] [airflow] mobuchowski commented on pull request #32325: Exclude not-releaseable providers from extras not only from releasing

2023-07-03 Thread via GitHub
mobuchowski commented on PR #32325: URL: https://github.com/apache/airflow/pull/32325#issuecomment-1617885799 Looks good - the lack of openlineage in extras does not prevent it from eventually working once it's released. -- This is an automated message from the Apache Git Service. To resp

[airflow] branch main updated: Adds Sensor section in the Azure providers docs (#32299)

2023-07-03 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new c03d7b7c33 Adds Sensor section in the Azure provide

[GitHub] [airflow] potiuk merged pull request #32299: Adds Sensor section in the Azure providers docs

2023-07-03 Thread via GitHub
potiuk merged PR #32299: URL: https://github.com/apache/airflow/pull/32299 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.a

  1   2   >