Re: [PR] Adapt GCP CloudSQL trigger to run in private cloud [airflow]
shahar1 merged PR #66917: URL: https://github.com/apache/airflow/pull/66917 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [PR] Adapt GCP CloudSQL trigger to run in private cloud [airflow]
olegkachur-e commented on PR #66917: URL: https://github.com/apache/airflow/pull/66917#issuecomment-4458393176 @shahar1 Thank you for the detailed review and serialization catch, I believe I've addressed the all comments, please take a look. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [PR] Adapt GCP CloudSQL trigger to run in private cloud [airflow]
olegkachur-e commented on code in PR #66917: URL: https://github.com/apache/airflow/pull/66917#discussion_r3246998333 ## providers/google/tests/unit/google/cloud/triggers/test_cloud_sql.py: ## @@ -48,8 +48,18 @@ def trigger(): ) [email protected] +def sync_hook_mock(): +mock_obj = mock.MagicMock() Review Comment: Sure, changed. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [PR] Adapt GCP CloudSQL trigger to run in private cloud [airflow]
olegkachur-e commented on code in PR #66917: URL: https://github.com/apache/airflow/pull/66917#discussion_r3246997030 ## providers/google/src/airflow/providers/google/cloud/triggers/cloud_sql.py: ## @@ -41,13 +43,15 @@ def __init__( gcp_conn_id: str = "google_cloud_default", impersonation_chain: str | Sequence[str] | None = None, poke_interval: int = 20, +api_version: str = "v1beta4", ): super().__init__() self.gcp_conn_id = gcp_conn_id self.impersonation_chain = impersonation_chain self.operation_name = operation_name self.project_id = project_id self.poke_interval = poke_interval +self.api_version = api_version Review Comment: Thank you for the common solution, I've updated trigger and test. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [PR] Adapt GCP CloudSQL trigger to run in private cloud [airflow]
olegkachur-e commented on code in PR #66917: URL: https://github.com/apache/airflow/pull/66917#discussion_r3246920368 ## providers/google/src/airflow/providers/google/cloud/hooks/cloud_sql.py: ## @@ -432,13 +441,8 @@ def _wait_for_operation_to_complete( :param time_to_sleep: Time to sleep between active checks of the operation results. :return: None """ -service = self.get_conn() Review Comment: I'm not against the idea of fighting regression but it makes this method usage really weird in other places, e.g. in trigger or in the future, since every other method calls `self.get_conn()` internally. I'll add the test to ensure `get_conn` being cached, I think it will help with regression, WDYT? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [PR] Adapt GCP CloudSQL trigger to run in private cloud [airflow]
shahar1 commented on code in PR #66917:
URL: https://github.com/apache/airflow/pull/66917#discussion_r3243169032
##
providers/google/src/airflow/providers/google/cloud/triggers/cloud_sql.py:
##
@@ -41,13 +43,15 @@ def __init__(
gcp_conn_id: str = "google_cloud_default",
impersonation_chain: str | Sequence[str] | None = None,
poke_interval: int = 20,
+api_version: str = "v1beta4",
):
super().__init__()
self.gcp_conn_id = gcp_conn_id
self.impersonation_chain = impersonation_chain
self.operation_name = operation_name
self.project_id = project_id
self.poke_interval = poke_interval
+self.api_version = api_version
Review Comment:
Triggers must include all constructor arguments in their `serialize()`
return value.
Please fix it here, I'll work on a generic pre-commit to fix existing bugged
triggers and prevent that for the next time.
##
providers/google/src/airflow/providers/google/cloud/hooks/cloud_sql.py:
##
@@ -432,13 +441,8 @@ def _wait_for_operation_to_complete(
:param time_to_sleep: Time to sleep between active checks of the
operation results.
:return: None
"""
-service = self.get_conn()
Review Comment:
nit: `self.get_conn()` will be called in every iteration of the `while`
loop. It is now cached internally, but to prevent potential regressions I'd
keep the original and pass it as parameter where needed.
##
providers/google/tests/unit/google/cloud/triggers/test_cloud_sql.py:
##
@@ -148,3 +161,28 @@ async def
test_async_export_trigger_exception_should_execute_successfully(
generator = trigger.run()
actual = await generator.asend(None)
assert TriggerEvent({"status": "failed", "message": "Test exception"})
== actual
+
[email protected]
[email protected](HOOK_STR.format("CloudSQLAsyncHook.get_operation"))
+async def
test_async_export_trigger_executest_successfully_in_custom_universe(
Review Comment:
```suggestion
async def
test_async_export_trigger_executes_successfully_in_custom_universe(
```
##
providers/google/tests/unit/google/cloud/triggers/test_cloud_sql.py:
##
@@ -48,8 +48,18 @@ def trigger():
)
[email protected]
+def sync_hook_mock():
+mock_obj = mock.MagicMock()
Review Comment:
nit: can it be mocked with a spec definition?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
