[
https://issues.apache.org/jira/browse/GOBBLIN-2187?focusedWorklogId=951023&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-951023
]
ASF GitHub Bot logged work on GOBBLIN-2187:
-------------------------------------------
Author: ASF GitHub Bot
Created on: 06/Jan/25 12:52
Start Date: 06/Jan/25 12:52
Worklog Time Spent: 10m
Work Description: Blazer-007 commented on code in PR #4090:
URL: https://github.com/apache/gobblin/pull/4090#discussion_r1904115989
##########
gobblin-temporal/src/main/java/org/apache/gobblin/temporal/ddm/activity/impl/GenerateWorkUnitsImpl.java:
##########
@@ -95,7 +95,11 @@ public WorkUnitsSizeSummary asSizeSummary(int numQuantiles) {
private static List<Double> getQuantiles(TDigest digest, int numQuantiles)
{
List<Double> quantileMinSizes = Lists.newArrayList();
for (int i = 1; i <= numQuantiles; i++) {
- quantileMinSizes.add(digest.quantile((i * 1.0) / numQuantiles));
+ double currQuantileMinSize = digest.quantile((i * 1.0) / numQuantiles);
+ if (Double.isNaN(currQuantileMinSize)) {
+ currQuantileMinSize = 0.0;
+ }
+ quantileMinSizes.add(currQuantileMinSize);
Review Comment:
No, if numQuantiles > 0 then this code gives NaN if there is no workunit and
by default numQuantiles is 10 so in this case all values will be filled with
NaN and then this will give issue when fetching medianSize which will be NaN
Issue Time Tracking
-------------------
Worklog Id: (was: 951023)
Time Spent: 20m (was: 10m)
> Fix NaN issue while generating WUSizeSummary
> --------------------------------------------
>
> Key: GOBBLIN-2187
> URL: https://issues.apache.org/jira/browse/GOBBLIN-2187
> Project: Apache Gobblin
> Issue Type: Bug
> Components: gobblin-core
> Reporter: Vivek Rai
> Assignee: Abhishek Tiwari
> Priority: Major
> Time Spent: 20m
> Remaining Estimate: 0h
>
> Recently GoT executions were enhanced to return WUSizeSummary with more info,
> but in case workunits are zero those have NaN value due to divide by zero
> case,
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
