andygrove opened a new issue, #3099:
URL: https://github.com/apache/datafusion-comet/issues/3099
## What is the problem the feature request solves?
> **Note:** This issue was generated with AI assistance. The specification
details have been extracted from Spark documentation and may need verification.
Comet does not currently support the Spark `make_interval` function, causing
queries using this function to fall back to Spark's JVM execution instead of
running natively on DataFusion.
The `MakeInterval` expression creates a calendar interval from separate
year, month, week, day, hour, minute, and second components. It supports
flexible argument lists allowing omission of trailing components, which default
to zero.
Supporting this expression would allow more Spark workloads to benefit from
Comet's native acceleration.
## Describe the potential solution
### Spark Specification
**Syntax:**
```sql
make_interval(years, months, weeks, days, hours, mins, secs)
make_interval(years, months, weeks, days, hours, mins)
make_interval(years, months, weeks, days, hours)
-- ... (supports up to 0 arguments with overloaded constructors)
```
**Arguments:**
| Argument | Type | Description |
|----------|------|-------------|
| years | IntegerType | Number of years in the interval |
| months | IntegerType | Number of months in the interval |
| weeks | IntegerType | Number of weeks in the interval |
| days | IntegerType | Number of days in the interval |
| hours | IntegerType | Number of hours in the interval |
| mins | IntegerType | Number of minutes in the interval |
| secs | DecimalType(MAX_LONG_DIGITS, 6) | Number of seconds with
microsecond precision |
**Return Type:** `CalendarIntervalType` - Returns a calendar interval object
containing the specified time components.
**Supported Data Types:**
- **Integer types**: years, months, weeks, days, hours, minutes
- **Decimal type**: seconds (with scale 6 for microsecond precision)
- All arguments support implicit casting to their required types
**Edge Cases:**
- **Null handling**: Returns null if any input argument is null (null
intolerant)
- **Overflow behavior**:
- When `failOnError=true` (ANSI mode): Throws
`QueryExecutionErrors.arithmeticOverflowError`
- When `failOnError=false`: Returns null on arithmetic overflow
- **Default values**: Missing trailing arguments default to `Literal(0)`
except seconds which defaults to `Decimal(0, MAX_LONG_DIGITS, 6)`
- **Nullable result**: When `failOnError=false`, result is always nullable;
when `true`, nullable only if any child is nullable
**Examples:**
```sql
-- Create interval with all components
SELECT make_interval(1, 2, 3, 4, 5, 6, 7.123456);
-- Result: 1 years 2 months 25 days 5 hours 6 minutes 7.123456 seconds
-- Create interval with partial components
SELECT make_interval(0, 1, 0, 1);
-- Result: 1 months 1 days
-- Handle overflow in non-ANSI mode
SELECT make_interval(999999999, 0, 0, 0, 0, 0, 0);
-- Result: NULL (on overflow)
```
```scala
// Example DataFrame API usage
import org.apache.spark.sql.functions._
df.select(expr("make_interval(1, 1, 0, 1, 0, 1, 40.000001)"))
// Using literal expressions
df.select(lit(1).alias("years"), lit(1).alias("months"))
.select(expr("make_interval(years, months, 0, 1, 0, 1, 40.000001)"))
```
### Implementation Approach
See the [Comet guide on adding new
expressions](https://datafusion.apache.org/comet/contributor-guide/adding_a_new_expression.html)
for detailed instructions.
1. **Scala Serde**: Add expression handler in
`spark/src/main/scala/org/apache/comet/serde/`
2. **Register**: Add to appropriate map in `QueryPlanSerde.scala`
3. **Protobuf**: Add message type in `native/proto/src/proto/expr.proto` if
needed
4. **Rust**: Implement in `native/spark-expr/src/` (check if DataFusion has
built-in support first)
## Additional context
**Difficulty:** Large
**Spark Expression Class:**
`org.apache.spark.sql.catalyst.expressions.MakeInterval`
**Related:**
- `IntervalUtils.makeInterval()` - Underlying utility method
- Calendar interval arithmetic expressions
- `extract()` function for decomposing intervals
- Date/time interval operations
---
*This issue was auto-generated from Spark reference documentation.*
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
