1fanwang opened a new pull request, #25961: URL: https://github.com/apache/datafusion/pull/25961
## Which issue does this PR close? - Closes https://github.com/apache/datafusion/issues/25946. ## Rationale for this change On a NOT NULL decimal column with a positive scale, col % 1 returns zero for every row: 1.50 % 1 returns 0.00 instead of 0.50, and WHERE col % 1 = 0 keeps rows it should drop. The same expression on a nullable column is correct. The simplifier rewrites a non-null A % 1 to 0 for every type except floats, which holds for integers but not for decimals that carry fractional digits. ## What changes are included in this PR? The rewrite now applies only when A cannot have a fractional part: integer types and decimals with a scale of 0 or less. Other decimals keep the modulo, and Arrow's remainder kernel computes it at run time. The rule's comment now says so. ## What is the testing strategy for this PR? math.slt now inserts 1.50 into the existing non-null decimal table and checks c1 % 1 with an integer and a decimal literal 1, plus the WHERE filter. The existing simplifier tests that fold an Int64 column % 1, including with a decimal literal 1, are unchanged. ### Testing Done The issue's queries in datafusion-cli, built and run the same way on main (4d167a167) and on this branch: ```shell cat > issue-25946.sql <<'EOF' CREATE TABLE t (d DECIMAL(10,2) NOT NULL, n DECIMAL(10,2)) AS VALUES (1.50, 1.50); SELECT d % 1 AS not_null_col, n % 1 AS nullable_col FROM t; SELECT count(*) AS cnt FROM t WHERE d % 1 = 0; SELECT d % CAST(1 AS DECIMAL(10,2)) AS dec_literal FROM t; EOF cargo build --profile ci -p datafusion-cli target/ci/datafusion-cli -f issue-25946.sql ``` On main the non-null column returns 0.00 and the filter keeps the row: ```text DataFusion CLI v55.1.0 0 row(s) fetched. Elapsed 0.016 seconds. +--------------+--------------+ | not_null_col | nullable_col | +--------------+--------------+ | 0.00 | 0.50 | +--------------+--------------+ 1 row(s) fetched. Elapsed 0.003 seconds. +-----+ | cnt | +-----+ | 1 | +-----+ 1 row(s) fetched. Elapsed 0.004 seconds. +-------------+ | dec_literal | +-------------+ | 0.00 | +-------------+ 1 row(s) fetched. Elapsed 0.001 seconds. ``` On this branch both columns return 0.50 and the filter drops the row: ```text DataFusion CLI v55.1.0 0 row(s) fetched. Elapsed 0.017 seconds. +--------------+--------------+ | not_null_col | nullable_col | +--------------+--------------+ | 0.50 | 0.50 | +--------------+--------------+ 1 row(s) fetched. Elapsed 0.004 seconds. +-----+ | cnt | +-----+ | 0 | +-----+ 1 row(s) fetched. Elapsed 0.009 seconds. +-------------+ | dec_literal | +-------------+ | 0.50 | +-------------+ 1 row(s) fetched. Elapsed 0.002 seconds. ``` ## Are there any user-facing changes? Yes. A non-null decimal column with a positive scale now returns its fractional remainder for % 1 instead of zero. No API changes. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
