shunping opened a new pull request, #36226: URL: https://github.com/apache/beam/pull/36226
A deadlock can occur in the `TestDynamicSplit/BlockOnSplit` test case due to a race condition between two go routines, the `processPlan` and `splitPlan` goroutines: - `processPlan` - This is the routine where `ProcessElement`, `GetRestriction` and `TryClaim` are called. https://github.com/apache/beam/blob/65dfd30684d38d27e86dc77b7d056119ee0447a9/sdks/go/pkg/beam/core/runtime/exec/dynsplit_test.go#L442 - `splitPlan` - This is the routine where `TrySplit` is called. The splitBlockingDriver uses channels for synchronization. https://github.com/apache/beam/blob/65dfd30684d38d27e86dc77b7d056119ee0447a9/sdks/go/pkg/beam/core/runtime/exec/dynsplit_test.go#L178-L182 However, a specific execution order can lead to a deadlock: - Driver executes and starts `processPlan` and `splitPlan`. - Driver waits for a signal on the `sdf.proc` channel from `ProcessElement` in `processPlan` routine - `TrySplit` in `splitPlan` routine executes first, acquires the `rt.mu` lock and then blocks while waiting on the `rt.blocksplit` channel. https://github.com/apache/beam/blob/65dfd30684d38d27e86dc77b7d056119ee0447a9/sdks/go/pkg/beam/core/runtime/exec/dynsplit_test.go#L399-L401 - `ProcessElement` in `processPlan` routine executes `GetRestriction` where it tries to acquire `rt.mu` but it is already-held. https://github.com/apache/beam/blob/65dfd30684d38d27e86dc77b7d056119ee0447a9/sdks/go/pkg/beam/core/runtime/exec/dynsplit_test.go#L422 - Unfortunately, `ProcessElement` can only send the signal to the `sdf.proc` channel after `GetRestriction` is finished. https://github.com/apache/beam/blob/65dfd30684d38d27e86dc77b7d056119ee0447a9/sdks/go/pkg/beam/core/runtime/exec/dynsplit_test.go#L443-L444 This creates a circular dependency and a deadlock. This change resolves the issue by moving the signal on `rt.blockSplit` to before the mutex is locked, preventing the deadlock. fixes #36224 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
