s1ny1998 opened a new pull request, #43000:
URL: https://github.com/apache/superset/pull/43000
### SUMMARY
The `clickhouse-connect` engine spec (`ClickHouseConnectEngineSpec`)
inherited
`supports_file_upload = False` from `ClickHouseEngineSpec`, so CSV / Excel /
columnar uploads were disabled for it — even though the `clickhouse-connect`
driver is perfectly capable of inserting data.
This PR:
1. Sets `supports_file_upload = True` on `ClickHouseConnectEngineSpec`.
2. Adds a `df_to_sql` override. ClickHouse requires every table to declare a
table engine, which the `CREATE TABLE` emitted by pandas' `to_sql` does
not —
so uploads would otherwise fail at table-creation time. The override
creates
the table itself with a `MergeTree` engine and then appends the rows.
The engine is configurable per database through the `extra` JSON, defaulting
to
`ORDER BY tuple()` so any upload works with no configuration:
```json
{"clickhouse_file_upload": {
"order_by": ["col1", "col2"],
"partition_by": "toYYYYMM(col1)",
"primary_key": "col1",
"settings": {"index_granularity": 8192}
}}
```
Columns named in `order_by` are created as non-nullable sort keys; all other
columns are nullable. `if_exists` (`fail` / `append` / `replace`) is honored.
### TESTING INSTRUCTIONS
Unit tests added in `tests/unit_tests/db_engine_specs/test_clickhouse.py`
cover:
the flag being off for the legacy spec and on for connect; table creation
with
the default `MergeTree ORDER BY tuple()`; `order_by` / `partition_by` sourced
from `extra` with sort-key columns made non-nullable; `if_exists='fail'`
raising
on an existing table; and `if_exists='replace'` dropping before recreating.
Manual: add a ClickHouse database using the `clickhousedb+connect` driver,
go to
**Settings → Upload a CSV to database**, and upload a CSV. A `MergeTree`
table is
created and populated.
### ADDITIONAL INFORMATION
- [ ] Has associated issue:
- [ ] Required feature flags:
- [ ] Changes UI
- [ ] Includes DB Migration (follow approval process in
[SIP-59](https://github.com/apache/superset/issues/13351))
- [x] Introduces new feature or API
- [ ] Removes existing feature or API
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
