csun5285 opened a new pull request, #68198:
URL: https://github.com/apache/doris/pull/68198
`StorageReadOptions::tablet_schema` carried a whole `TabletSchema` down to
every segment iterator, but it served two unrelated roles:
- tablet-level facts the segment already knows (keys type, index storage
format, whether a column is a key), and
- the compaction *output* schema, which genuinely differs from the segment's
own schema and is what variant compaction needs to lay out its subcolumns.
Conflating them made the schema look like a general read input, so callers
that only ever read on the query path had to produce one, and code that needed
the compaction output schema looked like it worked on any read.
This splits the two and deletes the field:
1. `StorageReadOptions::tablet_schema` is gone. Tablet-level facts are read
from `Segment::_tablet_schema`, and the two facts a read cannot recover from
the segment -- whether the tablet defines a sequence mapping, and whether it
materializes variant subcolumns as extracted columns -- move onto `ReadSchema`
as `tablet_has_sequence_map()` and `tablet_has_extracted_variant_columns()`.
2. `TabletSchema::_path_set_info_map` moves out into a new
`VariantCompactionPaths` (`be/src/storage/variant_compaction_paths.h`). It was
only ever built by compaction and only ever read by a compaction reader and
writer, yet it lived on the schema, where it was never persisted and never
survived a `RowsetMeta::set_tablet_schema` round trip. Living on the schema
made `TabletSchema::inverted_indexs()` look like it needed it everywhere; it
did not. The layout now travels explicitly through `RowsetWriterContext` /
`TabletReaderParams` / `StorageReadOptions` as a `shared_ptr<const ...>`,
shared read-only by the one compaction that built it.
3. `ReadSchema`'s three public init methods collapse into a single
`init_from_tablet_schema(schema, merge_by_sequence_mapping,
map_row_binlog_columns)`, and `RowsetReaderContext::tablet_schema` is deleted.
The two flags are decided by `BlockReader` / `VerticalBlockReader`, which is
where the reader type, direct mode and MoW-ness are known.
4. `Segment::seek_and_read_by_rowid` takes the `TabletColumn` it reads
instead of a whole `TabletSchema`, which is all it used. Its two callers
(`PointQueryExecutor`, `RowIdFetcher`) resolve the column once and pass it in.
5. `OrderedDataCompactionTest` and `SegmentsKeyBoundsTruncationTest` now
restore `ordered_data_compaction_min_segment_size`. Both leaked it, which sent
every later suite's compaction down the ordered link-file path instead of a
real merge -- this is what made `IndexStorageVariant*` fail depending on suite
order.
No behavior change intended on the query path.
Tests: `be/test/storage/variant_compaction_paths_test.cpp` is new and ports
the coverage that was deleted with `_path_set_info_map`; `read_schema_test.cpp`
gains the sequence-map and extracted-variant cases; `segcompaction_test.cpp`
gains `VariantRowsetIsNeverSegmentCompacted`, pinning the existing rule that a
rowset with a variant column is never segment-compacted.
### What problem does this PR solve?
Issue Number: close #xxx
Related PR: #xxx
Problem Summary:
### Release note
None
### Check List (For Author)
- Test <!-- At least one of them must be included. -->
- [ ] Regression test
- [ ] Unit Test
- [ ] Manual test (add detailed scripts or steps below)
- [ ] No need to test or manual test. Explain why:
- [ ] This is a refactor/code format and no logic has been changed.
- [ ] Previous test can cover this change.
- [ ] No code files have been changed.
- [ ] Other reason <!-- Add your reason? -->
- Behavior changed:
- [ ] No.
- [ ] Yes. <!-- Explain the behavior change -->
- Does this need documentation?
- [ ] No.
- [ ] Yes. <!-- Add document PR link here. eg:
https://github.com/apache/doris-website/pull/1214 -->
### Check List (For Reviewer who merge this PR)
- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label <!-- Add branch pick label that this PR should
merge into -->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
