Jackie-Jiang commented on code in PR #14170:
URL: https://github.com/apache/pinot/pull/14170#discussion_r1792602414
##########
pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/realtime/segment/SegmentFlushThresholdComputer.java:
##########
@@ -65,6 +67,16 @@ public int computeThreshold(StreamConfig streamConfig,
CommittingSegmentDescript
long optimalSegmentSizeBytesMin = desiredSegmentSizeBytes / 2;
double optimalSegmentSizeBytesMax = desiredSegmentSizeBytes * 1.5;
+ double segmentSizeVariation =
streamConfig.getFlushThresholdVarianceFraction();
+ // Valid value of Segment size variation should be between 0 and 0.5
+ if (segmentSizeVariation < 0.0 || segmentSizeVariation >= 0.5) {
Review Comment:
Consider moving this validation inside
`StreamConfig.extractFlushThresholdVarianceFraction()`
##########
pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/realtime/segment/SegmentFlushThresholdComputer.java:
##########
@@ -164,6 +177,17 @@ public int computeThreshold(StreamConfig streamConfig,
CommittingSegmentDescript
return (int) targetSegmentNumRows;
}
+ private long applySegmentSizeVariation(double segmentSizeVariation, long
targetSegmentNumRows) {
+ if (segmentSizeVariation > 0.0 && segmentSizeVariation < 1.0) {
+ double variation = (1 - segmentSizeVariation) + 2 * segmentSizeVariation
* RANDOM.nextDouble();
+ // Unnecessary to check for negative variation as it qwill always be
positive
+ if (variation > 0) {
+ return (long) (targetSegmentNumRows * variation);
+ }
+ }
Review Comment:
Can be simplified since we already bound the variation:
```suggestion
if (segmentSizeVariation > 0.0) {
double variation = (1 - segmentSizeVariation) + 2 *
segmentSizeVariation * RANDOM.nextDouble();
return (long) (targetSegmentNumRows * variation);
}
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
