nsivabalan commented on issue #9751:
URL: https://github.com/apache/hudi/issues/9751#issuecomment-2044036786
hey @njalan @BruceKellan : any follow ups on this.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL a
ad1happy2go commented on issue #9751:
URL: https://github.com/apache/hudi/issues/9751#issuecomment-1919303669
@ad1happy2go I did internal benchmarks with different versions of hudi here.
With metadata enabled between various version, I didn't saw significant
increase in S3 calls.
@nj
BruceKellan commented on issue #9751:
URL: https://github.com/apache/hudi/issues/9751#issuecomment-1884566335
Any updates?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
T
ad1happy2go commented on issue #9751:
URL: https://github.com/apache/hudi/issues/9751#issuecomment-1776827378
@njalan Didn't got much time yet to look into this yet. I will prioritize
this one this week. Thanks. Will update.
--
This is an automated message from the Apache Git Service.
To
njalan commented on issue #9751:
URL: https://github.com/apache/hudi/issues/9751#issuecomment-1773018752
@ad1happy2go May I know any updates from you? If can't reduce object list ,
can we cache these metadatas on driver?
--
This is an automated message from the Apache Git Service.
To res
njalan commented on issue #9751:
URL: https://github.com/apache/hudi/issues/9751#issuecomment-1761879043
@ad1happy2go Thanks a lot for your help. Just let me know if you want any
other information from me.
--
This is an automated message from the Apache Git Service.
To respond to the m
ad1happy2go commented on issue #9751:
URL: https://github.com/apache/hudi/issues/9751#issuecomment-1761827180
Thanks a lot for your effort here. @njalan . Really appreciate it. Looks
like in your case metadata table got more list calls.
I will work on this. Thanks.
--
This is an automa
njalan commented on issue #9751:
URL: https://github.com/apache/hudi/issues/9751#issuecomment-1759783515
@ad1happy2go Below are the list count for one spark streaming micro batch:
bleow are top list opreations(**first line is list count**) for table with
hudi 0.13.1 and metadata enabled:
ad1happy2go commented on issue #9751:
URL: https://github.com/apache/hudi/issues/9751#issuecomment-1744541419
@njalan Do you also see similar behaviour for the tables which got written
with later versions of hudi (0.13) only and not 0.9.
--
This is an automated message from the Apache Git
