andygrove commented on PR #6455:
URL:
https://github.com/apache/datafusion-comet/pull/6455#issuecomment-5920455977
Addressed in 04492d8:
- Rounding: `mills_as_money`, added after @comphead's comment, returns
thousandths, so `changePrecision` rounds half up (-1.005 to -1.01, 1.004 to
1.00, 0.005 to 0.01). For round-and-overflow it now declares `DECIMAL(7, 2)`:
99999.999 fits the type's five integer digits until rounding makes it
100000.00, so both engines return null, and the same for -99999.999.
- Array and struct writers: `array(fn(i))` and `named_struct('m', fn(i))`
don't reach the kernel's array or struct writer. Comet runs `CreateArray` and
`CreateNamedStruct` natively and dispatches only the call, so its value goes
through the top-level writer. I checked the plan: `array` and `named_struct`
run natively and only `invoke` is dispatched. The new test instead uses DSv2
functions that return `ARRAY<DECIMAL(10, 2)>`, `ARRAY<DECIMAL(12, 2) NOT NULL>`
and `STRUCT<m: DECIMAL(10, 2), non_null_m: DECIMAL(12, 2) NOT NULL>`, holding
scale-0 values. That covers both writers with and without their null check, and
the nullable children overflow to null. The non-nullable children use
`DECIMAL(12, 2)`, which holds any `INT`, so they never overflow.
- Null check: added the comment to the decimal branch.
- Non-nullable overflow: new test. It asserts that Spark fails and that
Comet fails in the native projection with "declared as non-nullable but
contains null values".
- `StaticInvoke`: added `static_as_money`, which is `as_money` with a static
`invoke`, to the main test. The test now also asserts that both `invoke` and
`staticinvoke` ran through the dispatcher.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
