andygrove opened a new pull request, #4277: URL: https://github.com/apache/datafusion-comet/pull/4277
## Which issue does this PR close? N/A - new expression support ## Rationale for this change Spark's `pmod` (positive modulo) expression currently falls back to Spark when running under Comet. Adding native support avoids the fallback overhead for this commonly used function. ## What changes are included in this PR? Implements `pmod` end-to-end through the Comet stack: - **Proto**: Added `MathExpr pmod = 71` to the `Expr` oneof in `expr.proto` - **Scala serde**: Added `CometPmod` object in `arithmetic.scala` using the `MathExpr` / `MathBase` pattern, rejecting TRY eval mode - **Scala registry**: Registered `classOf[Pmod] -> CometPmod` in `QueryPlanSerde.scala` - **Rust scalar function**: Implemented `spark_pmod` in `modulo_expr.rs` — computes `((a % b) + b) % b` to ensure positive results, with proper null/error handling for division by zero - **Rust expression builder**: Added `create_pmod_expr` with null-if-zero wrapping for non-ANSI mode and Decimal256 overflow handling (mirrors `create_modulo_expr`) - **Rust expression registry**: Added `PmodBuilder` and `ExpressionType::Pmod` variant - **Tests**: Comet SQL Tests for int, long, float, double, decimal, short, byte types with NULL handling and division-by-zero; ANSI mode error test - **Docs**: Marked `pmod` as supported in `spark_expressions_support.md` The `implement-comet-expression` skill was used to scaffold this implementation. ## How are these changes tested? - `pmod.sql`: Tests all supported numeric types (int, long, float, double, decimal, short, byte) with positive/negative dividends and divisors, NULL inputs, and division by zero in non-ANSI mode (returns NULL). Uses `ConfigMatrix: parquet.enable.dictionary=false,true`. - `pmod_ansi.sql`: Tests that division by zero raises an error in ANSI mode (`spark.sql.ansi.enabled=true`). ``` ./mvnw test -Pspark-3.5 -Pscala-2.12 -Dsuites="org.apache.comet.CometSqlFileTestSuite pmod" -Dtest=none ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
