kz930 opened a new issue, #7562:
URL: https://github.com/apache/texera/issues/7562
### What happened?
Dumbbell Plot fails when the compared column is empty in any row.
`DumbbellPlotOpDesc.scala:116` takes the column's distinct values and sorts
them:
```python
entityNames = list(table[comparedColumnName].unique())
entityNames = sorted(entityNames, reverse=True)
```
A blank cell reaches the operator as `None`, so `unique()` returns
`['alpha', None, 'gamma']` and `sorted` compares `None` against a string.
An empty value is ordinary input here. A blank CSV cell arrives as null,
since univocity returns null for an empty field and
`AttributeTypeUtils.parseField` passes it through by design, its first line
being `if (field == null) return null`.
The execution then pauses on that row, with the exception shown in the
operator's console.
Expected: a row whose compared column is empty either passes through, or is
skipped, or fails with a message naming the column.
### How to reproduce?
Upload a CSV with a blank cell in the compared column:
```
id,entity,category,measure
1,alpha,before,10
2,,before,20
3,gamma,after,30
4,alpha,after,15
```
Build `CSV File Scan` to `Dumbbell Plot` with Compared Column Name =
`entity`, Category Column Name = `category`, Measurement Column Name =
`measure`, Dumbbell Start Value = `before`, Dumbbell End Value = `after`. Run
it. The operator's console shows the TypeError.
### Version/Branch
1.3.0-incubating-SNAPSHOT (main)
### Relevant log output
```shell
└ ['alpha', None, 'gamma']
TypeError: '<' not supported between instances of 'NoneType' and 'str'
udf-v1:process_table:23
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
