jihoonson commented on issue #6336: update insert pending segments logic to synchronous URL: https://github.com/apache/incubator-druid/pull/6336#issuecomment-450603775 @QiuMM thank you for reporting! I think `TaskLockbox` potentially has this problem for all methods because they are coordinated by a single `giant` lock. I guess this becomes significant in large clusters when `allocatePendingSegment` is executed inside of `doInCriticalSection` because it's a heavy method. I think this PR is still useful because we can avoid the slow pending segment allocation problem caused by the race in `allocatePendingSegment`, but the solution may need to be improved. How about improving the lock granularity for taskLockbox? There's no need to use a single lock, but it can use more granular lock per dataSource per interval.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
