turcsanyip commented on a change in pull request #4781:
URL: https://github.com/apache/nifi/pull/4781#discussion_r564092023
##########
File path:
nifi-commons/nifi-record/src/main/java/org/apache/nifi/serialization/record/ResultSetRecordSet.java
##########
@@ -26,12 +27,12 @@
import java.math.BigDecimal;
import java.math.BigInteger;
import java.sql.Array;
+import java.sql.Date;
Review comment:
It was intentional because it seemed to be a bug.
`ResultSet.getMetaData().getColumnClassName()` would never return
`java.util.Date` but the SQL date/time types.
However, now I see that this code section is related to QueryRecord and it
may use `java.util.Date`. I'll revert it back.
Thanks for the heads up that made me double check it in more detail.
##########
File path:
nifi-commons/nifi-record/src/main/java/org/apache/nifi/serialization/record/util/DataTypeUtils.java
##########
@@ -1085,6 +1087,28 @@ private static Object toEnum(Object value, EnumDataType
dataType, String fieldNa
throw new IllegalTypeConversionException("Cannot convert value [" +
value + "] of type " + value.getClass() + " to Date for field " + fieldName);
}
+ /**
+ * Converts a java.sql.Date object with 00:00:00 in local time zone
(typically coming from a java.sql.ResultSet)
+ * to UTC normalized form (storing epoch corresponding to UTC 00:00:00 on
the given day).
+ *
+ * @param date java.sql.Date with local time zone 00:00:00
+ * @return java.sql.Date with UTC 00:00:00
+ */
+ public static Date convertDateToUTC(Date date) {
+ return new
Date(date.toLocalDate().atStartOfDay(ZoneId.of("UTC")).toInstant().toEpochMilli());
Review comment:
`atStartOfDay()` has `ZoneId` parameter only.
In my understanding `ZoneOffset` is DST-agnostic, so the `ZoneId` is the
right choice in any case.
##########
File path:
nifi-nar-bundles/nifi-extension-utils/nifi-database-utils/src/test/java/org/apache/nifi/util/db/TestJdbcCommon.java
##########
@@ -679,11 +680,12 @@ public void
testConvertToAvroStreamForDateTimeAsLogicalType() throws SQLExceptio
testConvertToAvroStreamForDateTime(options,
(record, date) -> {
+ java.sql.Date expected =
DataTypeUtils.convertDateToUTC(date);
final int daysSinceEpoch = (int) record.get("date");
final long millisSinceEpoch =
TimeUnit.MILLISECONDS.convert(daysSinceEpoch, TimeUnit.DAYS);
- java.sql.Date actual =
java.sql.Date.valueOf(Instant.ofEpochMilli(millisSinceEpoch).atZone(ZoneOffset.UTC).toLocalDate());
- LOGGER.debug("comparing dates, expecting '{}', actual
'{}'", date, actual);
- assertEquals(date, actual);
+ java.sql.Date actual = new java.sql.Date(millisSinceEpoch);
+ LOGGER.debug("comparing dates, expecting '{}', actual
'{}'", expected, actual);
+ assertEquals(expected, actual);
Review comment:
These tests (historically) do not assert the raw value (int / long) from
the Avro record but reconstruct the Date/Time/Timestamp objects and compare
them with the original objects (which were the input of the test case).
I believe the reason was to avoid to calculate the days/millis/etc. from the
test input data which would be a similarly complicated method.
As the test inputs are literals, it is not needed to calculate the expected
value from code, but it can be determined once and hard coded as literals too.
What do you think?
I mean
```
int expectedDaysSinceEpoch = 17444
assertDate.accept(record, expectedDaysSinceEpoch);
```
##########
File path:
nifi-nar-bundles/nifi-extension-utils/nifi-database-utils/src/test/java/org/apache/nifi/util/db/TestJdbcCommon.java
##########
@@ -679,11 +680,12 @@ public void
testConvertToAvroStreamForDateTimeAsLogicalType() throws SQLExceptio
testConvertToAvroStreamForDateTime(options,
(record, date) -> {
+ java.sql.Date expected =
DataTypeUtils.convertDateToUTC(date);
final int daysSinceEpoch = (int) record.get("date");
final long millisSinceEpoch =
TimeUnit.MILLISECONDS.convert(daysSinceEpoch, TimeUnit.DAYS);
- java.sql.Date actual =
java.sql.Date.valueOf(Instant.ofEpochMilli(millisSinceEpoch).atZone(ZoneOffset.UTC).toLocalDate());
- LOGGER.debug("comparing dates, expecting '{}', actual
'{}'", date, actual);
- assertEquals(date, actual);
+ java.sql.Date actual = new java.sql.Date(millisSinceEpoch);
+ LOGGER.debug("comparing dates, expecting '{}', actual
'{}'", expected, actual);
+ assertEquals(expected, actual);
Review comment:
These tests (historically) do not assert the raw value (int / long) from
the Avro record but reconstruct the Date/Time/Timestamp objects and compare
them with the original objects (which were the inputs of the test cases).
I believe the reason was to avoid to calculate the days/millis/etc. from the
test input data which would be a similarly complicated method.
As the test inputs are literals, it is not needed to calculate the expected
value from code, but it can be determined once and hard coded as literals too.
What do you think?
I mean:
```
int expectedDaysSinceEpoch = 17444;
assertDate.accept(record, expectedDaysSinceEpoch);
```
##########
File path:
nifi-commons/nifi-record/src/main/java/org/apache/nifi/serialization/record/util/DataTypeUtils.java
##########
@@ -1085,6 +1087,28 @@ private static Object toEnum(Object value, EnumDataType
dataType, String fieldNa
throw new IllegalTypeConversionException("Cannot convert value [" +
value + "] of type " + value.getClass() + " to Date for field " + fieldName);
}
+ /**
+ * Converts a java.sql.Date object with 00:00:00 in local time zone
(typically coming from a java.sql.ResultSet)
+ * to UTC normalized form (storing epoch corresponding to UTC 00:00:00 on
the given day).
+ *
+ * @param date java.sql.Date with local time zone 00:00:00
+ * @return java.sql.Date with UTC 00:00:00
+ */
+ public static Date convertDateToUTC(Date date) {
+ return new
Date(date.toLocalDate().atStartOfDay(ZoneId.of("UTC")).toInstant().toEpochMilli());
+ }
+
+ /**
+ * Converts a java.sql.Date object with 00:00:00 in UTC
+ * to local time zone normalized form (storing epoch corresponding to
00:00:00 in local time zone on the given day).
+ *
+ * @param date java.sql.Date with UTC 00:00:00
+ * @return java.sql.Date with local time zone 00:00:00
+ */
+ public static Date convertDateToLocalTZ(Date date) {
+ return
Date.valueOf(Instant.ofEpochMilli(date.getTime()).atZone(ZoneId.of("UTC")).toLocalDate());
Review comment:
Indeed... what do you think about this?
```
public static Date convertDateToUTC(Date dateLocalTZ) {
ZonedDateTime zdtLocalTZ =
ZonedDateTime.ofInstant(Instant.ofEpochMilli(dateLocalTZ.getTime()),
ZoneId.systemDefault());
ZonedDateTime zdtUTC =
zdtLocalTZ.withZoneSameLocal(ZoneId.of("UTC"));
Date dateUTC = new Date(zdtUTC.toInstant().toEpochMilli());
return dateUTC;
}
```
No `LocaDate` and I believe `withZoneSameLocal()` in the middle of the
method really grasps what is going on here.
##########
File path:
nifi-commons/nifi-record/src/main/java/org/apache/nifi/serialization/record/util/DataTypeUtils.java
##########
@@ -1085,6 +1087,28 @@ private static Object toEnum(Object value, EnumDataType
dataType, String fieldNa
throw new IllegalTypeConversionException("Cannot convert value [" +
value + "] of type " + value.getClass() + " to Date for field " + fieldName);
}
+ /**
+ * Converts a java.sql.Date object with 00:00:00 in local time zone
(typically coming from a java.sql.ResultSet)
+ * to UTC normalized form (storing epoch corresponding to UTC 00:00:00 on
the given day).
+ *
+ * @param date java.sql.Date with local time zone 00:00:00
+ * @return java.sql.Date with UTC 00:00:00
+ */
+ public static Date convertDateToUTC(Date date) {
+ return new
Date(date.toLocalDate().atStartOfDay(ZoneId.of("UTC")).toInstant().toEpochMilli());
+ }
+
+ /**
+ * Converts a java.sql.Date object with 00:00:00 in UTC
+ * to local time zone normalized form (storing epoch corresponding to
00:00:00 in local time zone on the given day).
+ *
+ * @param date java.sql.Date with UTC 00:00:00
+ * @return java.sql.Date with local time zone 00:00:00
+ */
+ public static Date convertDateToLocalTZ(Date date) {
+ return
Date.valueOf(Instant.ofEpochMilli(date.getTime()).atZone(ZoneId.of("UTC")).toLocalDate());
Review comment:
Indeed... what do you think about this?
```
public static Date convertDateToUTC(Date dateLocalTZ) {
ZonedDateTime zdtLocalTZ =
ZonedDateTime.ofInstant(Instant.ofEpochMilli(dateLocalTZ.getTime()),
ZoneId.systemDefault());
ZonedDateTime zdtUTC =
zdtLocalTZ.withZoneSameLocal(ZoneId.of("UTC"));
Date dateUTC = new Date(zdtUTC.toInstant().toEpochMilli());
return dateUTC;
}
```
No `LocaDate` and I believe `withZoneSameLocal()` in the middle of the
method really grasps what is going on here.
The same can be applied to `convertDateToLocalTZ()` similarly.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
