github-actions[bot] commented on code in PR #66856:
URL: https://github.com/apache/doris/pull/66856#discussion_r3800388634
##########
be/src/storage/index/snii/snii_blob_staging_directory.cpp:
##########
@@ -184,27 +202,21 @@ std::vector<snii::writer::BlobFileSource>
SniiBlobStagingDirectory::blob_sources
// std::map iterates in name order, which is the order the filesystem
harvest
// produced by sorting list(). Two builds of one index therefore lay their
// sub-files out identically in the container.
- for (const auto& [name, buffer] : _files) {
+ for (const auto& [name, file] : _files) {
sources.push_back(snii::writer::BlobFileSource {
.name = name,
- .length = buffer->size(),
- .read_fn = [buffer](uint64_t offset, size_t len, uint8_t* out)
-> Status {
- if (offset > buffer->size() || len > buffer->size() -
offset) {
- return Status::Error<ErrorCode::INTERNAL_ERROR>(
- "SNII staging read [{}, +{}) is outside the
staged {} bytes",
- offset, len, buffer->size());
- }
- std::memcpy(out, buffer->data() + offset, len);
- return Status::OK();
+ .length = file->bytes_written(),
+ .read_fn = [file](uint64_t offset, size_t len, uint8_t* out)
-> Status {
Review Comment:
[P2] Release staged files whenever compound sealing fails
`read_fn` now owns the temp file, but once these callbacks are registered,
`SniiCompoundWriter::finish()` can become terminal before releasing them: it
sets `finished_` before `ensure_bootstrap()`, and a bootstrap append, staged
read, or blob-output append error returns before `release_blob_sources()`. The
`begin_close()` defer clears only the directory map, so the compound still
holds this callback, and `InvertedIndexFileCollection::begin_close()` returns
without clearing its writer map. An ANN-sized scratch file and fd can therefore
remain linked after the failed close—especially harmful for ENOSPC—until the
enclosing writer is eventually destroyed. Release/reset all pending blob
sources on every terminal seal failure, and add a held-writer test for both
initial-container and mid-blob failures.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
