zabetak commented on code in PR #4760: URL: https://github.com/apache/hive/pull/4760#discussion_r1343951379
########## standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/parser/ExpressionTree.java: ########## @@ -18,7 +18,7 @@ package org.apache.hadoop.hive.metastore.parser; import java.sql.Timestamp; -import java.util.Date; Review Comment: The import was introduced by HIVE-5679 and you are right it was done accidentally. ########## standalone-metastore/metastore-server/src/test/java/org/apache/hadoop/hive/metastore/utils/TestMetaStoreUtils.java: ########## @@ -0,0 +1,78 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * <p> + * http://www.apache.org/licenses/LICENSE-2.0 + * <p> + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.hadoop.hive.metastore.utils; + +import org.apache.hadoop.hive.metastore.annotation.MetastoreUnitTest; +import org.junit.Test; +import org.junit.experimental.categories.Category; + +import java.sql.Date; +import java.sql.Timestamp; +import java.time.ZoneId; +import java.util.TimeZone; + +import static org.junit.Assert.assertEquals; + +@Category(MetastoreUnitTest.class) +public class TestMetaStoreUtils { Review Comment: Can we parameterize the tests using more input literals (date/timestamps) and timezones to improve test coverage? You can draw some inspiration from: `org.apache.hadoop.hive.ql.io.parquet.serde.TestParquetTimestampsHive2Compatibility` I am mostly interested for the `string -> date/timestamp -> string` case. I had the impression that I already left that comment but it appears I forgot about it in the previous review round. ########## ql/src/java/org/apache/hadoop/hive/ql/parse/BaseSemanticAnalyzer.java: ########## @@ -1804,9 +1803,9 @@ private static String normalizeDateCol(Object colValue) throws SemanticException throw new SemanticException("Unexpected date type " + colValue.getClass()); } try { - return MetaStoreUtils.PARTITION_DATE_FORMAT.get().format( - MetaStoreUtils.PARTITION_DATE_FORMAT.get().parse(value.toString())); - } catch (ParseException e) { + return MetaStoreUtils.convertDateToString( + MetaStoreUtils.convertStringToDate(value.toString())); + } catch (Exception e) { throw new SemanticException(e); } } Review Comment: By dead code, I was referring to the whole `normalizeDateCol` method. It was not really about the changes in this PR. ########## ql/src/test/results/clientpositive/llap/date_timestamp_partition_filter.q.out: ########## @@ -0,0 +1,214 @@ +PREHOOK: query: CREATE EXTERNAL TABLE testpd(col1 string, col2 String) PARTITIONED BY(PartitionDate DATE) STORED AS ORC Review Comment: This question is still pending. ########## standalone-metastore/metastore-server/src/test/java/org/apache/hadoop/hive/metastore/utils/TestMetaStoreUtils.java: ########## @@ -0,0 +1,78 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * <p> + * http://www.apache.org/licenses/LICENSE-2.0 + * <p> + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.hadoop.hive.metastore.utils; + +import org.apache.hadoop.hive.metastore.annotation.MetastoreUnitTest; +import org.junit.Test; +import org.junit.experimental.categories.Category; + +import java.sql.Date; +import java.sql.Timestamp; +import java.time.ZoneId; +import java.util.TimeZone; + +import static org.junit.Assert.assertEquals; + +@Category(MetastoreUnitTest.class) +public class TestMetaStoreUtils { + + @Test + public void testConvertDateToString() { + TimeZone defaultTZ = TimeZone.getDefault(); + try { + TimeZone.setDefault(TimeZone.getTimeZone(ZoneId.of("Asia/Hong_Kong"))); + String date = MetaStoreUtils.convertDateToString(Date.valueOf("2023-01-01")); + assertEquals("2023-01-01", date); + } finally { + TimeZone.setDefault(defaultTZ); + } + } + + @Test + public void testcConvertTimestampToString() { + TimeZone defaultTZ = TimeZone.getDefault(); + try { + String date = MetaStoreUtils.convertTimestampToString(Timestamp.valueOf("2023-01-01 10:20:30")); + assertEquals("2023-01-01 10:20:30", date); + } finally { + TimeZone.setDefault(defaultTZ); + } Review Comment: Since all tests are switching timezone it would be nice to do some refactoring to avoid the boilerplate. Using `@Before`, `@After` annotated methods and/or `@Parameterized` tests could aid in this. ########## standalone-metastore/metastore-server/src/test/java/org/apache/hadoop/hive/metastore/utils/TestMetaStoreUtils.java: ########## @@ -0,0 +1,78 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * <p> + * http://www.apache.org/licenses/LICENSE-2.0 + * <p> + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.hadoop.hive.metastore.utils; + +import org.apache.hadoop.hive.metastore.annotation.MetastoreUnitTest; +import org.junit.Test; +import org.junit.experimental.categories.Category; + +import java.sql.Date; +import java.sql.Timestamp; +import java.time.ZoneId; +import java.util.TimeZone; + +import static org.junit.Assert.assertEquals; + +@Category(MetastoreUnitTest.class) +public class TestMetaStoreUtils { Review Comment: > The issue was coming when SimpleDateFormat and java.sql.Date is used. Let me rephrase my question cause it was not completely clear. Can someone see the problem after switching the formatter behind `MetaStoreUtils.convertDateToString` using the unit tests here? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For additional commands, e-mail: gitbox-h...@hive.apache.org
