This is an automated email from the ASF dual-hosted git repository.
nicholasjiang pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/celeborn.git
The following commit(s) were added to refs/heads/main by this push:
new 3c9fe9897 [MINOR] Fix doc about PushMergedData split
3c9fe9897 is described below
commit 3c9fe9897cc251c0e8d644bf20b698be65beb1f0
Author: sychen <[email protected]>
AuthorDate: Fri Aug 22 20:50:05 2025 +0800
[MINOR] Fix doc about PushMergedData split
### What changes were proposed in this pull request?
### Why are the changes needed?
[CELEBORN-1721][CIP-12] Support HARD_SPLIT in PushMergedData
https://issues.apache.org/jira/browse/CELEBORN-1721
<img width="775" height="149" alt="image"
src="https://github.com/user-attachments/assets/deb7a741-5d72-403c-8405-77f837c25f59";
/>
### Does this PR introduce _any_ user-facing change?
### How was this patch tested?
<img width="675" height="108" alt="image"
src="https://github.com/user-attachments/assets/b33bead1-6f26-42d7-8ef3-7fd6df3b334e";
/>
Closes #3442 from cxzl25/doc_PushMergedData_split.
Authored-by: sychen <[email protected]>
Signed-off-by: SteNicholas <[email protected]>
---
docs/developers/storage.md | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/docs/developers/storage.md b/docs/developers/storage.md
index 851e53d46..90d81abe5 100644
--- a/docs/developers/storage.md
+++ b/docs/developers/storage.md
@@ -82,7 +82,7 @@ Upon receiving `CommitFiles`, `Worker` will flush all
buffered data for `Partiti
the RPC and close files, then responds the succeeded and failed
`PartitionLocation` lists.
## Trigger Split
-Upon receiving `PushData` (note: currently receiving `PushMergedData` does not
trigger Split, it's future work),
+Upon receiving `PushData` or `PushMergedData`,
`Worker` will check whether disk usage exceeds disk reservation (defaults to
5GiB). If so, `Worker` will respond
Split to `ShuffleClient`.
@@ -130,4 +130,4 @@ threshold (defaults to 256KiB), `Worker` flushes it to some
storage layer. The p
flush the data into DFS/OSS, as ⑤ indicates.
- If the `Cache Region` exceeds the threshold, choose the largest
`PartitionLocation` and flush it to local disk, as ⑥
indicates.
-- Optionally, if local disk does not have enough memory, choose a
`PartitionLocation` split and evict to HDFS/OSS.
\ No newline at end of file
+- Optionally, if local disk does not have enough memory, choose a
`PartitionLocation` split and evict to HDFS/OSS.
