gianm commented on code in PR #17943:
URL: https://github.com/apache/druid/pull/17943#discussion_r2060756738
##########
processing/src/main/java/org/apache/druid/segment/ReferenceCountingSegment.java:
##########
@@ -80,6 +81,12 @@ protected ReferenceCountingSegment(
)
{
super(baseSegment);
+ // In an ideal world, we would ask the baseSegment to be one of
QueryableIndexSegment, IncrementalIndexSegment, LookupSegment, etc.
Review Comment:
IMO, this comment is not super clear since it doesn't explain why it's more
ideal to do it this way. I think we can instead focus on a simple reason for
this check: double-wrapping references causes the inner reference to be
ignored, which is weird.
Suggested new comment:
```
// ReferenceCountingSegment should not wrap another SegmentReference,
because the inner
// reference would effectively be ignored.
```
##########
extensions-core/multi-stage-query/src/main/java/org/apache/druid/msq/dart/worker/DartDataSegmentProvider.java:
##########
@@ -95,7 +96,10 @@ public Supplier<ResourceHolder<CompleteSegment>>
fetchSegment(
final PhysicalSegmentInspector inspector =
segment.as(PhysicalSegmentInspector.class);
channelCounters.addFile(inspector != null ? inspector.getNumRows() :
0, 0);
});
- return new ReferenceCountingResourceHolder<>(new CompleteSegment(null,
segment), closer);
+ return new ReferenceCountingResourceHolder<>(new CompleteSegment(
+ null,
+ Objects.requireNonNull(segment.getBaseSegment())
+ ), closer);
Review Comment:
IMO a clearer comment would be:
```
// It isn't necessary to pass down the SegmentReference to CompleteSegment,
because we've
// already called segment.acquireReferences() and have attached the
reference to "closer".
```
Btw, a side note I have realized as part of thinking about this: in native,
we apply the segment map fn first and then acquire a segment reference; whereas
with Dart we acquire the reference first and then apply the segment map fn.
IMO, the way we do it in Dart makes more sense. If native worked the same way,
then segment map fns could use `Segment` rather than `SegmentReference`, and
the call to `ReferenceCountingSegment.wrapRootGenerationSegment` in
`BaseLeafFrameProcessor` wouldn't necessary. (There is no need to change this
right now, it's fine, I'm just making an observation.)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
