LauraXia123 opened a new issue, #12093:
URL: https://github.com/apache/gravitino/issues/12093
## Describe the bug
When creating a Doris table with list partitions, the create response
correctly returns `partitioning.assignments` with the partition data. However,
when loading the same table, the `partitioning.assignments` field returns an
empty array `[]`.
The same issue also affects range partition assignments.
## To Reproduce
1. Create a Doris table with list partition via Gravitino REST API
2. Observe the create response - `partitioning.assignments` correctly
contains the partition data
3. Load the same table via GET API
4. Observe the load response - `partitioning.assignments` is empty `[]`
### Create response (correct)
```json
"partitioning": [
{
"strategy": "list",
"fieldNames": [["col"]],
"assignments": [
{
"type": "list",
"name": "p1",
"lists": [
[{"type": "literal", "dataType": "integer", "value": "1"}],
[{"type": "literal", "dataType": "integer", "value": "2"}],
[{"type": "literal", "dataType": "integer", "value": "3"}]
],
"properties": null
}
]
}
]
```
### Load response (assignments empty)
```json
"partitioning": [
{
"strategy": "list",
"fieldNames": [["col"]],
"assignments": []
}
]
```
## Expected behavior
Loading a Doris table with list/range partitions should return the same
`partitioning.assignments` data as when the table was created.
## Root cause
In `DorisUtils.extractPartitionInfoFromSql()`
(`catalogs/catalog-jdbc-doris/src/main/java/org/apache/gravitino/catalog/doris/utils/DorisUtils.java:99-125`),
the method only parses the `PARTITION BY LIST(col)` clause from `SHOW CREATE
TABLE` output to extract column names, but does **not** parse the individual
partition definitions (`PARTITION p1 VALUES IN (...)`).
As a result, it calls `Transforms.list(filedNames)` (line 114) which is the
overload **without** assignments, instead of `Transforms.list(filedNames,
assignments)`.
The same issue exists for range partitions at line 116:
`Transforms.range(new String[] {columns[0]})` is called without assignments.
### Relevant code
```java
// DorisUtils.java:111-117
if (LIST_PARTITION.equals(partitionType)) {
String[][] filedNames =
Arrays.stream(columns).map(s -> new String[]
{s}).toArray(String[][]::new);
return Optional.of(Transforms.list(filedNames)); // <-- missing
assignments
} else if (RANGE_PARTITION.equals(partitionType)) {
return Optional.of(Transforms.range(new String[] {columns[0]})); // <--
missing assignments
}
```
### Suggested fix
The `extractPartitionInfoFromSql` method (or `getTablePartitioning` in
`DorisTableOperations`) should also parse the partition definitions from the
DDL and construct the appropriate `ListPartition[]` / `RangePartition[]`
assignments to pass to `Transforms.list(filedNames, assignments)` /
`Transforms.range(fieldName, assignments)`.
Alternatively, the existing
`DorisTablePartitionOperations.fromDorisPartition()` method already has the
logic to parse partition data from `SHOW PARTITIONS` output. This could be
leveraged during table load to populate assignments.
## Additional context
- Gravitino version: current main branch
- Catalog: jdbc-doris
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
