Re: [HELP] unstable tests in the travis CI

2021-06-26 Thread Raymond Xu
I did some backlog grooming; putting flaky tests by class in subtasks there in https://issues.apache.org/jira/browse/HUDI-1248 If you're working on any of those, please set the assignee. Let's parallelize the efforts :) Also, the Azure CI builds for master/release versions can be found here https

Website redesign

2021-06-26 Thread Raymond Xu
Hi all, We've completed a re-design of Hudi's website (hudi.apache.org) , in the goal of making the navigation more organized and information more discoverable. The design document can be found here (thanks to designer Joanna) https://www.figma.com/file/tipod1JZRw7anZRWBI6sZh/Hudi.Apache?node-id=

Re: Could Hudi Data lake support low latency, high throughput random reads?

2021-06-26 Thread Jialun Liu
Hey Vinoth, Thanks for your reply! I am actually looking into a different direction atm. Basically write the transformed data into a OLTP database, e.g. DynamoDB, any data need to support low latency high throughput read would be exported periodically. Not sure if this is the right pattern, appr

Re: Could Hudi Data lake support low latency, high throughput random reads?

2021-06-26 Thread Vinoth Chandar
Yes. Thats a working approach. One thing I would like to suggest is the use of Hudi’s incremental queries to update DynamoDB as opposed to full exporting periodically, depending on how much of your target dynamoDB table changes between loads, it can save you cost and time. On Sat, Jun 26, 2021 at