leventov commented on a change in pull request #7185: Avoid many unnecessary
materializations of collections of 'all segments in cluster' cardinality
URL: https://github.com/apache/incubator-druid/pull/7185#discussion_r264326529
##########
File path:
server/src/main/java/org/apache/druid/server/coordinator/helper/DruidCoordinatorSegmentInfoLoader.java
##########
@@ -42,8 +43,26 @@ public DruidCoordinatorRuntimeParams
run(DruidCoordinatorRuntimeParams params)
{
log.info("Starting coordination. Getting available segments.");
- // Display info about all available segments
- final Set<DataSegment> availableSegments =
coordinator.getOrderedAvailableDataSegments();
+ // The following transform() call doesn't actually transform the iterable.
It only checks the sizes of the segments
+ // and emits alerts if segments with negative sizes are encountered. In
other words, semantically it's similar to
+ // Stream.peek(). It works as long as
DruidCoordinatorRuntimeParams.createAvailableSegmentsSet() (which is called
+ // below) guarantees to go over the passed iterable exactly once.
+ //noinspection StaticPseudoFunctionalStyleMethod:
https://youtrack.jetbrains.com/issue/IDEA-153047
+ Iterable<DataSegment> availableSegmentsWithSizeChecking =
Iterables.transform(
+ coordinator.iterateAvailableDataSegments(),
+ segment -> {
+ if (segment.getSize() < 0) {
+ log.makeAlert("No size on a segment")
Review comment:
I don't know what is the root cause.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@druid.apache.org
For additional commands, e-mail: commits-h...@druid.apache.org
