nsivabalan commented on issue #6226: URL: https://github.com/apache/hudi/issues/6226#issuecomment-1210091812
you have to use the same partition key for any lock. I mean, any writes happening to a single hudi table will be acquiring one dynamo db lock, even if they are writing to two diff partitions. Only during conflict solution stages, hudi detects that two writes are for two diff partitions and will let both writes succeed. if there was a overlap, hudi will fail one of them. So, locking is required irrespective of whether writers write to same or diff partitions. btw, our metadata table writes for a given data table happens within the same lock. So, may be thats playing a part. but you should not see so much latency in general. Did you enable access logs for dynamo DB. whats the latency you see w/ dynamoc DB. I will let @zhedoubushishi investigate this. just throwing in some ideas. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
