danny0405 commented on PR #11577:
URL: https://github.com/apache/hudi/pull/11577#issuecomment-2212995870

   > @danny0405 I checked the code and realised that the reason why the tests 
were failing because the `DATE` column is defined with an INT type in 
`org.apache.hudi.common.testutils.HoodieTestDataGenerator#TRIP_HIVE_COLUMN_TYPES`.
   > 
   > 
https://github.com/apache/hudi/blob/1f6520b1b26ff02127e1cfdece32b2a68f0c5d51/hudi-common/src/test/java/org/apache/hudi/common/testutils/HoodieTestDataGenerator.java#L157-L158
   > 
   > In our productionised tests environment, the HIVE_TYPES for dates are 
passed by `DATE` instead of `INT` types.
   > 
   > On top of that, if values are indeed passed as INT, we should be seeing 
the same error as what was described in #11576.
   > 
   > The return types for are as such:
   > 
   > # Hive2
   > 
org.apache.hadoop.hive.serde2.objectinspector.primitive.WritableDateObjectInspector#getPrimitiveJavaObject
   > 
   > 
![image](https://private-user-images.githubusercontent.com/6312314/346418622-6b847fa0-3399-4d08-86f6-8fc798be9af4.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MjA0MTI2ODQsIm5iZiI6MTcyMDQxMjM4NCwicGF0aCI6Ii82MzEyMzE0LzM0NjQxODYyMi02Yjg0N2ZhMC0zMzk5LTRkMDgtODZmNi04ZmM3OThiZTlhZjQucG5nP1gtQW16LUFsZ29yaXRobT1BV1M0LUhNQUMtU0hBMjU2JlgtQW16LUNyZWRlbnRpYWw9QUtJQVZDT0RZTFNBNTNQUUs0WkElMkYyMDI0MDcwOCUyRnVzLWVhc3QtMSUyRnMzJTJGYXdzNF9yZXF1ZXN0JlgtQW16LURhdGU9MjAyNDA3MDhUMDQxOTQ0WiZYLUFtei1FeHBpcmVzPTMwMCZYLUFtei1TaWduYXR1cmU9YWMzMDFkZjQwMjE2ZjBmNDkzN2RlMThlMjYzNjdkYTMzYzdmOWQ0YWZjYjhkNTU1MzQzZDY3MDZiZDc0MzU0ZiZYLUFtei1TaWduZWRIZWFkZXJzPWhvc3QmYWN0b3JfaWQ9MCZrZXlfaWQ9MCZyZXBvX2lkPTAifQ.uDmOWOXhpqZcFKG5ypPNuTS3BA5EcB6PUk_AZojMg5Q)
   > 
   > # Hive3
   > 
org.apache.hadoop.hive.serde2.objectinspector.primitive.WritableDateObjectInspector#getPrimitiveJavaObject
   > 
   > 
![image](https://private-user-images.githubusercontent.com/6312314/346419022-57bf629e-61a1-4430-b246-afa1e7173eb0.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MjA0MTI2ODQsIm5iZiI6MTcyMDQxMjM4NCwicGF0aCI6Ii82MzEyMzE0LzM0NjQxOTAyMi01N2JmNjI5ZS02MWExLTQ0MzAtYjI0Ni1hZmExZTcxNzNlYjAucG5nP1gtQW16LUFsZ29yaXRobT1BV1M0LUhNQUMtU0hBMjU2JlgtQW16LUNyZWRlbnRpYWw9QUtJQVZDT0RZTFNBNTNQUUs0WkElMkYyMDI0MDcwOCUyRnVzLWVhc3QtMSUyRnMzJTJGYXdzNF9yZXF1ZXN0JlgtQW16LURhdGU9MjAyNDA3MDhUMDQxOTQ0WiZYLUFtei1FeHBpcmVzPTMwMCZYLUFtei1TaWduYXR1cmU9MDY0NDE0MWE4YzYyNjJhZjdmNzYxODkxZWI5ZjJmYWZiNzU0ZDI2NDgzOWQ1OGU4OGE0MzhmZDFlNWRhZTQwMSZYLUFtei1TaWduZWRIZWFkZXJzPWhvc3QmYWN0b3JfaWQ9MCZrZXlfaWQ9MCZyZXBvX2lkPTAifQ.kLPPoHs9LTX9fdxyhSAImqfzh4sadH5JTOWmxBZjVa4)
   
   Great findings, maybe we can just keep the pattern match for `INT` in case 
there are some rare cases.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

Reply via email to