kokhlo opened a new pull request, #44156:
URL: https://github.com/apache/superset/pull/44156
## SUMMARY
Fixes #44155: BigQuery's special partition IDs `__NULL__` and
`__UNPARTITIONED__` are now filtered out when selecting the latest partition,
preventing `PARSE_DATE('%Y%m%d', '__NULL__')` query failures.
## BEFORE/AFTER
**Before**: Tables with NULL values in the partition column caused Copy
SELECT statement and table preview to generate invalid SQL:
```sql
WHERE partition_date = PARSE_DATE('%Y%m%d', '__NULL__') -- fails
```
**After**: Special partition IDs are excluded from max(partition_id),
ensuring only numeric YYYYMMDD partitions are selected:
```sql
WHERE partition_date = PARSE_DATE('%Y%m%d', '20251231') -- works
```
If no usable partition exists, the WHERE clause is omitted entirely.
## TEST PLAN
- Added unit test `test_where_latest_partition_filters_null_partitions` that
mocks a BigQuery table with `__NULL__` partitions and verifies the generated
SQL filters them out.
- Existing BigQuery partition tests remain unchanged.
## ADDITIONAL CONTEXT
This affects:
- SQL Lab Copy SELECT statement (when latest_partition=True)
- Table preview/sample queries for time-partitioned BigQuery tables
The root cause: `get_max_partition_id()` performed `SELECT
max(partition_id)` without filtering. Since `__NULL__` and `__UNPARTITIONED__`
sort higher than numeric strings, they became the max and then failed inside
`PARSE_DATE`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
