FrankChen021 opened a new pull request, #20772:
URL: https://github.com/apache/druid/pull/20772
### Description
Druid can now write segments in the V10 format. However, the SQL system
tables offer no way to tell which format a published segment uses: the only
place the version shows up is the task log (`DataSegment{binaryVersion=10,
...}` at handoff). This PR exposes `DataSegment.binaryVersion` as a new
`binary_version` column in `sys.segments`, and shows it in the web console.
#### `sys.segments.binary_version`
Adds a `binary_version` (`BIGINT`) column to `sys.segments`. It holds the
binary format version of the segment file, such as `9` or `10`.
- The column is appended last, so existing column positions are unchanged.
- It is filled in both row builders: published segments (from the
Coordinator) and available-but-unpublished segments (from the Broker's segment
metadata cache).
- It is `null` when the version is unknown:
- **Realtime segments.** A realtime task announces its sink with
`DataSegment.builder(SegmentId)`, which defaults `binaryVersion` to `0`. The
real version is set only when the segment is pushed to deep storage.
- **Segments that still carry the `0` placeholder after handoff.** The
Broker's metadata cache keeps the first announced `DataSegment` for an ID, so
until the next Coordinator poll, a handed-off segment can still show the
realtime placeholder.
- **Tombstones**, which never set `binaryVersion`.
Because realtime segments always carry `0`, the check is value-based
(`null` or `0` becomes `null`); no separate `is_realtime` check is needed.
Example:
```sql
SELECT datasource, binary_version, COUNT(*) AS segments, SUM("size") AS bytes
FROM sys.segments
WHERE is_published = 1 AND is_overshadowed = 0
GROUP BY 1, 2
```
#### Web console
Adds a **Binary version** column to the Segments view. It is hidden by
default, including for users who already have a saved column selection, and it
is available in all capability modes:
- with SQL, it reads `binary_version`;
- without SQL, it reads `binaryVersion` from the Coordinator
`metadata/segments` API, using the same `null` rules.
Unknown values show as `-`.
#### Release note
`sys.segments` has a new `binary_version` column that shows each segment's
binary format version (for example `9` or `10`), so you can check with SQL
which segments use the V10 format. The web console Segments view has a matching
**Binary version** column, hidden by default.
<hr>
##### Key changed/added classes in this PR
* `SystemSchema`
* `SystemSchemaTest`
* `segments-view.tsx`
<hr>
This PR has:
- [x] been self-reviewed.
- [x] added documentation for new or modified features or behaviors.
- [x] a release note entry in the PR description.
- [x] added comments explaining the "why" and the intent of the code
wherever would not be obvious for an unfamiliar reader.
- [x] added unit tests or modified existing tests to cover new code paths,
ensuring the threshold for [code
coverage](https://github.com/apache/druid/blob/master/dev/code-review/code-coverage.md)
is met.
- [x] been tested in a test Druid cluster.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
