The GitHub Actions job "cargo in /. for object_store - Update #891942938" on horaedb.git has failed. Run started by GitHub user dependabot[bot] (triggered by dependabot[bot]).
Head commit for run: aa438cae8c552a18f6701d6237bcbcc982bed25a / Draco <[email protected]> feat: use multithreading to optimize WAL replay (#1572) ## Rationale 1. Currently, the WAL replayer uses coroutines to replay the WAL logs of multiple tables in parallel. However, coroutines utilize at most one CPU. By switching to a multithreaded approach, we can fully leverage multiple CPUs. 2. We observed that during the replay phase, decoding the WAL log is a CPU-intensive operation, so parallelize it. ## Detailed Changes 1. Modify both `TableBasedReplay` and `RegionBasedReplay` to use the `spawn task` approach for parallelism, with a maximum of 20 tasks running concurrently. 2. Preload next segment in WAL based on local storage. 4. In `BatchLogIteratorAdapter::simulated_async_next`, we first retrieve all the payloads in a batch and then decode them in parallel. ## Test Plan Manual testing. Report URL: https://github.com/apache/horaedb/actions/runs/11066581867 With regards, GitHub Actions via GitBox --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
