gortiz opened a new issue, #12186:
URL: https://github.com/apache/pinot/issues/12186
Right now Pinot fails when a table is modified in such a way that two
columns are added and one of them depend on the other.
More specifically, assuming table T has columns A1, A2 and A3, it should be
possible to add columns B and C such as:
```js
//table config
"ingestionConfig": {
"transformConfigs": [
{
"columnName": "C",
"transformFunction": "B + 1"
}
]
}
```
But right now this fails with:
```
Caused by: java.lang.RuntimeException: Failed to create derived column: C
because argument: B does not exist in the segment
```
This error is thrown in
https://github.com/apache/pinot/blob/85b5779cff8d33412937ceb6d4dfc972b6c8bd97/pinot-segment-local/src/main/java/org/apache/pinot/segment/local/segment/index/loader/defaultcolumn/BaseDefaultColumnHandler.java#L374
when at least one of the columns used in the transform function is not
included in the columns already stored in the segment.
This is not a blocker because the same change can be done by first adding
column B, reload and then modify the table again to add column D and reload.
It is important to note that at the same time we add this feature we should
add checks to avoid cycle dependencies. Specifically, it should be forbidden to
add columns `E1, E2, ..., En` when `E1` depends on `En` and `Ek` depends on
`E(k-1)` for all k in [1, n].
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
