LuciferYang opened a new issue, #13099:
URL: https://github.com/apache/gluten/issues/13099
`ShuffleManagerRegistry.register` guards against registering
`GlutenShuffleManager` itself, because `GlutenShuffleManager` routes every
shuffle to the registered managers, so registering it (or a subclass) would
route into itself and recurse.
The guard condition is inverted:
```scala
require(
!clazz.isAssignableFrom(classOf[GlutenShuffleManager]),
"It's not allowed to register GlutenShuffleManager recursively")
```
`!clazz.isAssignableFrom(classOf[GlutenShuffleManager])` only rejects
`GlutenShuffleManager` and its supertypes. A subclass of `GlutenShuffleManager`
passes the check (for a subclass `GSub`,
`GSub.isAssignableFrom(GlutenShuffleManager)` is `false`, so the negation is
`true`). Registering such a subclass while `spark.shuffle.manager` is the
experimental `GlutenShuffleManager` makes the router build and route into
itself on the first routed shuffle (`registerShuffle` → `getOrBuild` →
instantiate → route → `getOrBuild` → …) until it overflows the stack.
The guard has been inverted since the registry was introduced in #8084, so
it has never actually blocked a subclass. Reachable only under the experimental
`GlutenShuffleManager` plus a user-defined subclass, so the practical impact is
small, but the guard does not do what it claims.
The correct condition also rejects subtypes:
`!classOf[GlutenShuffleManager].isAssignableFrom(clazz)`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
