xylaaaaa opened a new pull request, #70243: URL: https://github.com/apache/doris/pull/70243
### Background The BE service registers a file cache factory in `ExecEnv` at startup. BE writer unit tests also construct S3 writers and packed-file managers, but run without starting the service. ### What problem does this PR solve? Issue Number: None Related PR: #68797 (enabled file caching by default), #69605 (initial S3 fixture fix), #69670 (overlapping S3 client teardown) PR Type: Test Problem Summary: S3, Hive report-writer and packed-file-manager unit tests can crash when run alone with file caching enabled. A downloader or cloud-tablet test can mask the crash by leaving caching disabled. Give the writer fixtures a populated cache for the duration of the test and restore the factory and its enable flag afterward. ### Details - **Failure:** Isolated writers dereference a missing cache factory, either while constructing an S3 writer or while uploading a packed file. In a shuffled run, the S3 buffer-boundary test also removed the suite's mock callbacks, which made later tests send requests to S3. - **Cause:** The writer fixtures relied on process-wide state set by other tests. The boundary test used its own in-memory client but also cleared all registered callbacks and left a changed buffer size behind. - **Stability evidence:** The six affected suites pass together in three shuffled runs (seeds 7301, 7302 and 7303). The new scope tests exercise a missing factory, an empty factory, reuse of a populated factory, and restoration with either initial value of the enable flag. ### Testing - Reproduced three ASAN segmentation faults with the fixtures from `315c2e0033`. `S3FileWriterTest.multi_part_io_error` and `VHivePartitionWriterReportLifecycleTest.PeriodicReportDefersMetadataAndFinalReportTransfersUploadIdentity` fail during writer construction. `PackedFileManagerTest.ProcessUploadingFilesSetsFailedWhenMetaUpdateFails` fails during the cache write. All three pass in separate processes with this change. The Azure Hive case and three additional packed-upload cases also pass in separate processes (7/7). - Ran all six suites locally with the BE unit-test entry point under ASAN. The default run passed 68/68. Setting `GTEST_SHUFFLE=1 GTEST_REPEAT=3 GTEST_RANDOM_SEED=7301` passed 68/68 in each iteration and the process exited with status 0. ```sh ./run-be-ut.sh --run --filter='ScopedFileCacheTest.*:S3FileWriterTest.*:VHivePartitionWriterReportLifecycleTest.*:PackedFileManagerTest.*:FileCacheBlockDownloaderTest.*:CloudTabletSyncMetaTest.*' ``` Clang-format 16, clang-tidy on the changed lines, and the BE build-hygiene checks passed. ### Notes for Reviewers The scope reuses an existing populated factory. Otherwise it creates a 16 MiB memory cache. The S3 and packed-file fixtures drain pending uploads before releasing the cache. The S3 fixture also releases its filesystem before AWS SDK shutdown. ### Release note None ### Check List (For Author) - Test: BE unit tests under ASAN. - Behavior changed: No - Does this need documentation: No - Pre-submission status: Ready ### AI Disclosure - Tool: OpenAI Codex (GPT-6) - Generated: Implementation, unit tests, and PR description. - Human review: Pending author review. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
