sadpandajoe commented on code in PR #43695: URL: https://github.com/apache/superset/pull/43695#discussion_r4257335224
########## tests/unit_tests/db_engine_specs/test_arc.py: ########## @@ -0,0 +1,57 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. +import pytest + +from superset.constants import TimeGrain +from superset.db_engine_specs.arc import ArcEngineSpec + + +def test_arc_properties() -> None: + assert ArcEngineSpec.engine == "arc" + assert ArcEngineSpec.engine_name == "Arc" + assert ArcEngineSpec.default_driver == "arrow" + + +def test_arc_metadata() -> None: + metadata = ArcEngineSpec.metadata + assert ( + metadata["description"] + == "Arc is a data platform with multiple connection options." + ) + assert metadata["logo"] == "arc.png" Review Comment: `ArcEngineSpec.metadata` no longer defines a `logo` at this head (the logo entries were dropped), so `metadata["logo"]` raises `KeyError` and `test_arc_metadata` fails. The same assertion fails in `test_ocient.py:46`, `test_odps.py:38`, and `test_parseable.py:37`, whose specs also have no `logo`. Could these assertions be dropped (or the logos restored together with the image assets) so the new suites pass? ########## tests/unit_tests/db_engine_specs/test_teradata.py: ########## @@ -0,0 +1,59 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. +import pytest + +from superset.db_engine_specs.teradata import TeradataEngineSpec + + +def test_teradata_properties() -> None: + assert TeradataEngineSpec.engine == "teradatasql" + assert TeradataEngineSpec.engine_name == "Teradata" + assert TeradataEngineSpec.max_column_name_length == 30 + + +def test_teradata_metadata() -> None: + metadata = TeradataEngineSpec.metadata + desc = metadata["description"] + assert "Teradata is an enterprise data warehouse platform" in desc + assert metadata["logo"] == "teradata.png" + assert "teradatasqlalchemy" in metadata["pypi_packages"] + assert metadata["default_port"] == 1025 + + +def test_teradata_epoch_to_dttm() -> None: + assert ( + TeradataEngineSpec.epoch_to_dttm().format(col="ts") + == "CAST(((CAST(DATE '1970-01-01' + (ts / 86400) AS TIMESTAMP(0) AT 0)) AT 0) + ((ts MOD 86400) * INTERVAL '00:00:01' HOUR TO SECOND) AS TIMESTAMP(0))" Review Comment: This expected-SQL literal is 159 characters, over the repo's 88-column Ruff limit, so the lint/pre-commit check will flag it. Could it be split into adjacent string literals while keeping the full assertion? ########## superset/db_engine_specs/databricks.py: ########## @@ -629,9 +657,22 @@ class DatabricksNativeEngineSpec(DatabricksDynamicBaseEngineSpec): "databricks+connector://token:{access_token}@{host}:{port}/{database_name}" ) - # Note: Primary metadata is in DatabricksPythonConnectorEngineSpec which - # consolidates all Databricks connection methods. This spec exists for - # backwards compatibility with legacy databricks-dbapi connections. + metadata = { + "description": ("Databricks legacy connector using databricks-dbapi."), + "logo": "databricks.png", + "homepage_url": "https://www.databricks.com/";, + "categories": [ + DatabaseCategory.CLOUD_DATA_WAREHOUSES, + DatabaseCategory.ANALYTICAL_DATABASES, + DatabaseCategory.HOSTED_OPEN_SOURCE, + ], + "pypi_packages": ["databricks-dbapi[sqlalchemy]"], Review Comment: Registering `databricks.connector` against `databricks.sqlalchemy.DatabricksDialect` doesn't line up with how `DatabricksNativeEngineSpec` builds its connections: it still emits `/{database_name}` and applies the catalog through `connect_args` and `USE CATALOG`, while that dialect takes its catalog only from the `?catalog=` query parameter. A legacy `databricks+connector://` URI without `?catalog=` would then list tables with ``SHOW TABLES FROM `None`.<schema>``, and the new test passes the catalog in the URL so it never exercises that path. Could the Native spec translate catalog/schema into query parameters the way the Python Connector spec does, with a test built from the URI the Native builder actually produces? ########## superset/db_engine_specs/databricks.py: ########## @@ -955,6 +980,29 @@ class DatabricksODBCEngineSpec(DatabricksBaseEngineSpec): drivers = {"pyodbc": "ODBC driver for SQL endpoint"} default_driver = "pyodbc" + metadata = { + "description": ("Databricks SQL Endpoint connectivity via the pyodbc driver."), + "logo": "databricks.png", + "homepage_url": "https://www.databricks.com/";, + "categories": [ + DatabaseCategory.CLOUD_DATA_WAREHOUSES, + DatabaseCategory.ANALYTICAL_DATABASES, + DatabaseCategory.PROPRIETARY, + ], + "pypi_packages": ["pyodbc"], Review Comment: `databricks-dbapi[sqlalchemy]` requires `sqlalchemy>=1.3,<2.0` and doesn't install `pyodbc` (that comes from its separate `odbc` extra), so the documented install still can't resolve against Superset's SQLAlchemy 2.x requirement, and the ODBC driver this URI needs is no longer listed. Could this list packages that work with SQLAlchemy 2, or record the incompatibility under `known_incompatibilities` the way the Kusto spec does? ########## superset/db_engine_specs/databricks.py: ########## @@ -974,6 +1022,31 @@ class DatabricksHiveEngineSpec(HiveEngineSpec): drivers = {"pyhive": "Hive driver for Interactive Cluster"} default_driver = "pyhive" + metadata = { + "description": ( + "Databricks Interactive Cluster connectivity via the PyHive connector." + ), + "logo": "databricks.png", + "homepage_url": "https://www.databricks.com/";, + "categories": [ + DatabaseCategory.CLOUD_DATA_WAREHOUSES, + DatabaseCategory.ANALYTICAL_DATABASES, + DatabaseCategory.HOSTED_OPEN_SOURCE, + ], + "pypi_packages": ["pyhive"], Review Comment: Same remaining problem for the Hive spec: `databricks-dbapi[sqlalchemy]` pins SQLAlchemy below 2.0 and doesn't pull in the `hive` extra (`pyhive`/`thrift`), so the documented install either can't resolve against Superset or lacks the driver `databricks+pyhive` needs. Could the package list or a `known_incompatibilities` entry reflect that? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
