danny0405 commented on issue #20685:
URL: https://github.com/apache/hudi/issues/20685#issuecomment-6969462862

   I traced the existing callers and the RFC-118 stack to understand why this 
cache would need concurrent access despite its documented per-thread scope and 
`@NotThreadSafe` annotation.
   
   There is a concrete ownership mismatch in the existing Avro scanner path:
   
   - 
[`AvroRecordContext.getFieldAccessorInstance()`](https://github.com/apache/hudi/blob/f740680aa40862bf008f0f9c6ada11cb8d35e37c/hudi-common/src/main/java/org/apache/hudi/common/avro/AvroRecordContext.java#L57-L60)
 returns a static singleton, which inherits its own `LocalHoodieSchemaCache` 
from `RecordContext`.
   - 
[`HoodieMergedLogRecordScanner.processNextRecord()`](https://github.com/apache/hudi/blob/f740680aa40862bf008f0f9c6ada11cb8d35e37c/hudi-common/src/main/java/org/apache/hudi/common/table/log/HoodieMergedLogRecordScanner.java#L253-L266)
 passes that singleton to `BufferedRecords.fromHoodieRecord()`, which calls 
`encodeSchema()`.
   - Consequently, separate scanners executing concurrently in the same 
JVM/classloader can mutate the same cache. This establishes a possible race, 
but I have not reproduced a failing scanner workload.
   
   For RFC-118 specifically, the new 
[`TestBlobAndEmbeddingStreamsConcurrently`](https://github.com/Davis-Zhang-Onehouse/hudi-oss/blob/0cc05eb384833eddd8d07349caba1b432822f6da/hudi-utilities/src/test/java/org/apache/hudi/utilities/unstructured/embedding/streamer/TestBlobAndEmbeddingStreamsConcurrently.java)
 runs two Streamer jobs against the same events table using a shared Spark 
context. That could expose existing shared-state bugs, but it does not itself 
demonstrate concurrent schema-cache access. Searching the cumulative patches 
through #20733 found no new `RecordContext`, `encodeSchema`, or 
`HoodieMergedLogRecordScanner` references outside the cache change and its 
tests. #20686 also describes this fix as independent of the rest.
   
   The Spark singleton is not sufficient evidence either: the production 
callers I traced use field access or native delete-row construction, while 
[`SparkFileFormatInternalRowReaderContext`](https://github.com/apache/hudi/blob/f740680aa40862bf008f0f9c6ada11cb8d35e37c/hudi-client/hudi-spark-client/src/main/scala/org/apache/hudi/SparkFileFormatInternalRowReaderContext.scala#L86)
 creates a fresh record context with `apply(tableConfig)`.
   
   Could we fix the scanner's ownership by creating one `AvroRecordContext` per 
scanner and reusing it across records? Construction is lightweight, this 
preserves the local cache contract, and it avoids retaining schemas for the 
singleton's lifetime. If RFC-118 requires shared ownership instead, could you 
provide the specific call path and a reproducer that fails without this change? 
The cache-level concurrency test demonstrates what happens when the existing 
contract is violated, but does not establish why callers need that sharing.
   
   Also, `HoodieSchemaCache` / `HoodieAvroSchemaCache` are not direct 
replacements: they intern schema objects rather than maintain the bidirectional 
integer-ID mapping needed by buffered records.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to