Re: [I] [SUPPORT] can't retrieve original partition column value when exacting date with CustomKeyGenerator [hudi]

2024-04-30 Thread via GitHub
ad1happy2go commented on issue #11002: URL: https://github.com/apache/hudi/issues/11002#issuecomment-2087971292 @liangchen-datanerd That's the good suggestion. Created a tracking JIRA too - https://issues.apache.org/jira/browse/HUDI-7698 We can think of introducing the reader side

Re: [I] [SUPPORT] can't retrieve original partition column value when exacting date with CustomKeyGenerator [hudi]

2024-04-17 Thread via GitHub
liangchen-datanerd commented on issue #11002: URL: https://github.com/apache/hudi/issues/11002#issuecomment-2063009955 @ad1happy2go Based on the IoT scenario on which I've been working, the event time would be adopted as the partition column. At the same time, we would query data based

Re: [I] [SUPPORT] can't retrieve original partition column value when exacting date with CustomKeyGenerator [hudi]

2024-04-16 Thread via GitHub
ad1happy2go commented on issue #11002: URL: https://github.com/apache/hudi/issues/11002#issuecomment-2059291917 @liangchen-datanerd Thanks. Got it. So currently this is not implemented. Currently it is transforming the partition column after reading from parquet. Will check if we can

Re: [I] [SUPPORT] can't retrieve original partition column value when exacting date with CustomKeyGenerator [hudi]

2024-04-14 Thread via GitHub
liangchen-datanerd commented on issue #11002: URL: https://github.com/apache/hudi/issues/11002#issuecomment-2054533892 @ad1happy2go Thanks for the reply. Hudi did transform the partition column timestamp value to the dataformat value based on the

Re: [I] [SUPPORT] can't retrieve original partition column value when exacting date with CustomKeyGenerator [hudi]

2024-04-14 Thread via GitHub
ad1happy2go commented on issue #11002: URL: https://github.com/apache/hudi/issues/11002#issuecomment-2054100970 @liangchen-datanerd As you are using '"hoodie.keygen.timebased.output.dateformat":"-MM-dd"' , it is expected for hudi to output the partition column value in date format only

[I] [SUPPORT] can't retrieve original partition column value when exacting date with CustomKeyGenerator [hudi]

2024-04-11 Thread via GitHub
liangchen-datanerd opened a new issue, #11002: URL: https://github.com/apache/hudi/issues/11002 **problem** the requirement was to extract date value as partition from event_time column. According to the hudi offical doc the ingestion config for hoodie would be like this ```