andygrove opened a new issue, #236:
URL: https://github.com/apache/datafusion-spark/issues/236

   Part of #175 and apache/datafusion-comet#7300.
   
   ### What the function does
   
   `timestamp_micros(microseconds)` returns the `TIMESTAMP` that is 
`microseconds` µs after `1970-01-01 00:00:00 UTC`.
   The argument must be an integral type (`TINYINT`, `SMALLINT`, `INT`, 
`BIGINT`); the expression uses `ExpectsInputTypes`
   with `IntegralType`, so `DECIMAL`, `FLOAT`, `DOUBLE` and `STRING` are 
analysis errors rather than being cast. The value
   is used as is (scale factor 1), so it cannot overflow.
   
   The result is always `TIMESTAMP` (LTZ), independent of 
`spark.sql.timestampType`; only its display depends on the
   session time zone. NULL input gives NULL; nullability follows the input. 
Since 3.1.0; the inverse is `unix_micros`
   (already in this crate).
   
   ### Examples
   
   Session time zone `America/Los_Angeles` (as in Spark's docs and golden 
files):
   
   ```sql
   SELECT timestamp_micros(1230219000123123);
   -- 2008-12-25 07:30:00.123123
   SELECT timestamp_micros(-1230219000123123);
   -- 1931-01-07 00:29:59.876877
   SELECT timestamp_micros(null);
   -- NULL
   ```
   
   ### Edge cases to test
   
   - `i64::MIN` and `i64::MAX` are valid results (no error, no NULL).
   - Negative values are floor-style: `-1230219000123123` is `...:59.876877`.
   - `TINYINT` / `SMALLINT` / `INT` / `BIGINT` inputs (widened to `i64`); 
`DECIMAL`, `DOUBLE`, `STRING` fail with
     `DATATYPE_MISMATCH.UNEXPECTED_INPUT_TYPE`.
   - Round trip with `unix_micros(timestamp_micros(x)) = x`.
   - The stored value must not depend on `spark.sql.session.timeZone`; display 
across a DST change does.
   - Literal vs column input; dictionary-encoded input.
   
   ### Differences across Spark versions
   
   None found (3.4 through 4.2 have the same evaluation). ANSI mode does not 
affect this function.
   
   ### Implementation notes
   
   - Spark: 
`sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/datetimeExpressions.scala`,
     `MicrosToTimestamp` (extends `IntegralToTimestampBase`, `upScaleFactor = 
1`).
   - DataFusion core `to_timestamp_micros` is close for integer input (it casts 
to `Timestamp(Microsecond, tz)`), but it
     also accepts floats (truncated through `Int64`), decimals and strings, 
which Spark rejects at analysis time. A thin
     wrapper with Spark's integral-only signature returning 
`Timestamp(Microsecond, Some(tz))` (zone from
     `datafusion.execution.time_zone`) is enough.
   - Share code with `timestamp_seconds` / `timestamp_millis` (separate issues) 
in `crates/spark/src/function/datetime/`,
     next to `unix.rs`.
   - Tests: `crates/sqllogictest/test_files/datetime/timestamp_micros.slt` 
exists but every query is commented out
     (ported placeholders). Uncomment and expand it. Document with 
`#[user_doc]`.
   
   ---
   _This issue was generated by an LLM (Claude) from the Apache Spark source 
code and tests. Examples come from Spark's `ExpressionDescription` examples, 
test suites and `.sql.out` golden files, but please verify them against Spark 
before relying on them._


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to