>From Ritik Raj <[email protected]>: Attention is currently required from: Ian Maxon, Murtadha Hubail.
Ritik Raj has posted comments on this change by Ritik Raj. ( https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/21372?usp=email ) Change subject: [ASTERIXDB-3788][STO] flush .dic LAF pages in ascending page-id order ...................................................................... Patch Set 3: Code-Review+1 (6 comments) Patchset: PS3: > was claude not used at all for this? the comments are wordy and remind me of > opus's usual writing st […] Yes, I used claude, and Opus did write those wordy comments. File asterixdb/asterix-app/src/test/java/org/apache/asterix/test/cloud_storage/CloudStorageMergeTest.java: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/21372/comment/7c9bcacc_dafe6e21?usp=email : PS3, Line 56: MB-72604 > ```suggestion […] Done File hyracks-fullstack/hyracks/hyracks-storage-common/src/main/java/org/apache/hyracks/storage/common/compression/file/LAFWriter.java: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/21372/comment/e59eabed_231c3d50?usp=email : PS3, Line 52: MB-72604 > refer to the apache issue instead? Done https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/21372/comment/693ab680_be249862?usp=email : PS3, Line 153: MB-72604 > same here Done https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/21372/comment/41d72b35_36518596?usp=email : PS3, Line 186: flushReadyPages(); > is there any realistic way that say, a page early in the file could remain > underfull and lead to som […] When page P is underfull, subsequently filled pages in the range (P, maxPageId] remain held in memory until P fills or endWriting drains them. In a columnar flush, the lowest-ID page (page-zero) of each leaf is written last; because completion is not globally ascending, nextPageToFlush can stall at a low page. In the worst case, a component's entire .dic set of LAF pages is held until endWriting. This quirk never surfaced on random-access local files, but append-only cloud copies strictly require sequential order. Fortunately, the memory overhead stays bounded, cheap, and per-component (not cumulative): one 128KB LAF page indexes 8,192 data pages (ENTRY_LENGTH=16), meaning even a massive 1M-data-page component holds only ~123 LAF pages (~15MB), which are safely released at endWriting. Correctness is fully preserved regardless, as endWriting ultimately forces an ascending flush. https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/21372/comment/b28268ae_03cfc7fb?usp=email : PS3, Line 215: The local file is : * random-access and indifferent to order; only the cloud copy needs this guarantee. > i don't follow this comment. […] Yes, the pages are flushed sequentially here. >The local file is random-access and indifferent to order; only the cloud copy >needs this guarantee. This line just states that the fact that this wasn't an issue if it were written on local disk, because we can write in disk in random order so a page 10 can be persisted to disk before page 4, but in the CloudWriter we tend to write sequentially, so flushing page 10 before page 4 will cause issue while flushing page 4 because the cloudWrite pointer have moved forward to page10's offset. The function `flushReadyPages` makes the flush sequential for both disk and cloud write. -- To view, visit https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/21372?usp=email To unsubscribe, or for help writing mail filters, visit https://asterix-gerrit.ics.uci.edu/settings?usp=email Gerrit-MessageType: comment Gerrit-Project: asterixdb Gerrit-Branch: lumina Gerrit-Change-Id: I2a8a83cd1b57d3ceb1d9be381edd6589a0e138b9 Gerrit-Change-Number: 21372 Gerrit-PatchSet: 3 Gerrit-Owner: Ritik Raj <[email protected]> Gerrit-Reviewer: Anon. E. Moose #1000171 Gerrit-Reviewer: Jenkins <[email protected]> Gerrit-Reviewer: Murtadha Hubail <[email protected]> Gerrit-Reviewer: Ritik Raj <[email protected]> Gerrit-CC: Ian Maxon <[email protected]> Gerrit-Attention: Murtadha Hubail <[email protected]> Gerrit-Attention: Ian Maxon <[email protected]> Gerrit-Comment-Date: Fri, 03 Jul 2026 05:51:23 +0000 Gerrit-HasComments: Yes Gerrit-Has-Labels: Yes Comment-In-Reply-To: Ian Maxon <[email protected]>
