voonhous opened a new issue, #19893:
URL: https://github.com/apache/hudi/issues/19893
Found while reviewing #19892 (part of #19524).
`Registry.REGISTRY_MAP` (hudi-io, `org.apache.hudi.common.metrics.Registry`)
is a JVM-wide static that is never cleared.
`SparkHoodieBackedTableMetadataWriter.initRegistry()` interns the metadata
registry under the table-name-less key `""::HoodieMetadata`. When metrics are
on and executor metrics are off it stores a `LocalRegistry`; when executor
metrics are on with a non-INMEMORY reporter it calls
`Registry.getRegistryOfClass(..., DistributedRegistry.class)`. On a class
mismatch `getRegistryOfClass` only logs "already exists with a different class"
and returns the existing `LocalRegistry`, which the caller then hard-casts.
Reproduces at one surefire fork just by ordering two existing tests in one
JVM:
```
mvn test -am -pl hudi-spark-datasource/hudi-spark -Dspark3.5 -Dscala-2.12
-DskipUTs=true \
-Dhudi.surefire.forkCount=1 -Dsurefire.runOrder=reversealphabetical \
-Dtest='TestMetricsReporter,TestMetadataTableWithSparkSQL' ...
# ClassCastException: LocalRegistry cannot be cast to DistributedRegistry
```
The alphabetical order passes, and each class passes alone. Any change to
how classes are grouped into JVMs (fork count, new tests, renamed classes) can
flip it.
Proposed fix: key the metadata registry per table (or per base path), and
have `getRegistryOfClass` replace or throw on a class mismatch rather than
return the wrong type. A test-side reset of `REGISTRY_MAP` between classes
would hide the symptom but not the production bug.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
