mccormickt12 commented on PR #5322: URL: https://github.com/apache/hadoop/pull/5322#issuecomment-1430524690
> The change generally looks good to me. > > One concern is that when `refetchLocation` is called inside `hedgedFetchBlockByteRange` we could remove a node from the ignoredList that is already part of the futures array. This would lead to multiple reads to the same node. > > What I'm thinking of is > > 1. Node A is added to futures array > 2. getFirstToComplete throws an InterruptedException when doing hedgedService.take() > 3. We call retchLocation, which remove node A from ignored list. > 4. The while loop re-adds Node A to the futures list. > > I don't know if this actually can happen. Even if it is technically possible, it may not be an issue. Thoughts? yes @simbadzina I agree this was an issue. I've resolved this now. As you pointed out the future is removed in `getFirstToComplete`, so now we let it spin in the while loop, each time a future will be removed, and then once futures is empty and refetch is needed we will clear the ignore list -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
