andygrove opened a new issue, #3085:
URL: https://github.com/apache/datafusion-comet/issues/3085
## What is the problem the feature request solves?
> **Note:** This issue was generated with AI assistance. The specification
details have been extracted from Spark documentation and may need verification.
Comet does not currently support the Spark `current_date` function, causing
queries using this function to fall back to Spark's JVM execution instead of
running natively on DataFusion.
The `CurrentDate` expression returns the current date in the session time
zone. It is a deterministic function that returns the same date value
throughout a single query execution, making it foldable for optimization
purposes.
Supporting this expression would allow more Spark workloads to benefit from
Comet's native acceleration.
## Describe the potential solution
### Spark Specification
**Syntax:**
```sql
CURRENT_DATE
CURRENT_DATE()
```
```scala
// DataFrame API
import org.apache.spark.sql.functions.current_date
df.select(current_date())
```
**Arguments:**
| Argument | Type | Description |
|----------|------|-------------|
| timeZoneId | Option[String] | Optional time zone identifier for date
calculation (internal parameter) |
**Return Type:** Returns `DateType` - a date value without time component.
**Supported Data Types:**
This expression takes no input data types as it is a leaf expression that
generates a date value based on the current system time.
**Edge Cases:**
- Never returns null values (nullable = false)
- Returns consistent date across all nodes in a distributed execution
- Date boundary depends on the configured time zone
- Behavior is deterministic within query execution but varies between queries
- Time zone changes affect the returned date value
**Examples:**
```sql
-- Get current date
SELECT CURRENT_DATE;
-- Use in WHERE clause
SELECT * FROM events WHERE event_date = CURRENT_DATE;
-- Compare with timestamp column
SELECT *, CURRENT_DATE AS today FROM logs;
```
```scala
// DataFrame API usage
import org.apache.spark.sql.functions._
// Add current date column
df.withColumn("current_date", current_date())
// Filter by current date
df.filter(col("date_column") === current_date())
// Select current date
spark.sql("SELECT CURRENT_DATE").show()
```
### Implementation Approach
See the [Comet guide on adding new
expressions](https://datafusion.apache.org/comet/contributor-guide/adding_a_new_expression.html)
for detailed instructions.
1. **Scala Serde**: Add expression handler in
`spark/src/main/scala/org/apache/comet/serde/`
2. **Register**: Add to appropriate map in `QueryPlanSerde.scala`
3. **Protobuf**: Add message type in `native/proto/src/proto/expr.proto` if
needed
4. **Rust**: Implement in `native/spark-expr/src/` (check if DataFusion has
built-in support first)
## Additional context
**Difficulty:** Medium
**Spark Expression Class:**
`org.apache.spark.sql.catalyst.expressions.CurrentDate`
**Related:**
- `CurrentTimestamp` - returns current timestamp with time component
- `Now` - alias for current timestamp
- `UnixTimestamp` - returns current time as Unix timestamp
- `DateFormatClass` - for formatting date values
---
*This issue was auto-generated from Spark reference documentation.*
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
