PHILO-HE commented on code in PR #10248:
URL:
https://github.com/apache/incubator-gluten/pull/10248#discussion_r2244660340
##########
gluten-flink/ut/src/test/java/org/apache/gluten/table/runtime/stream/custom/ScalarFunctionsTest.java:
##########
@@ -169,4 +172,28 @@ void testDecimal() {
query = "select b + e as x from tblDecimal where a > 0";
runAndCheck(query, Arrays.asList("+I[2.0]", "+I[5.0]", "+I[7.0]"));
}
+
+ @Test
+ void testDateFormat() {
+ List<Row> rows =
+ Arrays.asList(Row.of(1, "2024-12-31 12:12:12"), Row.of(2, "2025-02-28
12:12:12"));
+ createSimpleBoundedValuesTable("dateFormatTbl", "a int, b string", rows);
+ String query =
+ "select a, DATE_FORMAT(cast(b as Timestamp(3)), 'yyyy-MM-dd'),
DATE_FORMAT(cast(b as Timestamp(3)), 'yyyy-MM-dd HH:mm:ss') from dateFormatTbl";
+ runAndCheck(
+ query,
+ Arrays.asList(
+ "+I[1, 2024-12-31, 2024-12-31 12:12:12]", "+I[2, 2025-02-28,
2025-02-28 12:12:12]"));
+ Map<String, String> configs =
+ Map.of(
+ VeloxQueryConfig.ADJUST_TIMESTMP_TO_SESSION_TIMEZONE.key(),
+ "true",
+ TableConfigOptions.LOCAL_TIME_ZONE.key(),
+ "America/Los_Angeles");
Review Comment:
@KevinyhZou, if I understand correctly, we should ensure the result type is
Flink's `timestamp` type for `cast(xxx as timestamp)`. You can submit another
PR to fix the issue.
I investigated Flink just now. It seems Flink's `timestamp` type is
corresponding to Spark's `timestamp_ntz` type (no counterpart in Velox), and
Flinks' `timestamp_ltz` type is corresponding to Spark/Velox's `timestamp`
type. Please confirm this to avoid wrong type mapping.
In Flink, when casting datetime string to `timestamp_ltz` or outputting
string represented result for `timestamp_ltz`, timezone is respected. But
regarding `timestamp` type, timezone is not respected in such conversions.
Both `timestamp` and `timestamp_ltz` are supported for Flink's date_format
function. So if `timestamp` is used, the result is not influenced by session
timezone. But if `timestamp_ltz` is used, the result should be adjusted based
on session timezone.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
