jihoonson commented on issue #6336: update insert pending segments logic to synchronous URL: https://github.com/apache/incubator-druid/pull/6336#issuecomment-450693437 I see. Since the lock contention can happen between tasks if they're writing into the interval of the same dataSource, 600 running tasks at the same time can cause this issue. If this is the case, before this PR, I think such many tasks could cause frequent pending segment allocation failures due to the allowed race in `allocatePendingSegments`. Those failed actions will be retried, but make segment allocation slow. Were you able to see this kind of problem before upgrade? Maybe we need to use separate http thread pools for API and UI. Also, maybe https://github.com/apache/incubator-druid/pull/6348 and https://github.com/apache/incubator-druid/pull/6356 would help though you said your pendingSegment table is small.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
