[email protected] has uploaded this change for review. ( http://gerrit.cloudera.org:8080/24532
Change subject: KUDU-3790: Fix NPE in HdrHistogramAccumulator.isZero() ...................................................................... KUDU-3790: Fix NPE in HdrHistogramAccumulator.isZero() Spark's executor-heartbeater thread periodically calls isZero() on every running task's accumulators. The executor-side copy of KuduContext's "kudu.write_duration" accumulator can have its internal histogram field observed as null before the task records its first write (unsafe publication across threads, or constructor-bypassing deserialization such as Kryo). isZero() then threw a NullPointerException on the heartbeater thread, which Spark treats as fatal, killing the executor. In this change, isZero() (and toString()) was made null-tolerant: a null/absent histogram is logically zero. isZero() is also now lock-free so it cannot depend on any field being published. Additionally HistogramWrapper now synchronizes its mutators on `this` rather than on the reassignable `innerHistogram` var, which previously changed the monitor object over time and provided no real mutual exclusion. Adds HdrHistogramAccumulatorTest covering the zero semantics, a serialization round-trip, and regression tests that force a null field and assert isZero() does not throw. Change-Id: Ia3d556aaf1051464303f35281d1d698b20d5ef20 Reviewed-on: http://gerrit.cloudera.org:8080/24527 Reviewed-by: Marton Greber <[email protected]> Tested-by: Marton Greber <[email protected]> Reviewed-by: Zoltan Martonka <[email protected]> (cherry picked from commit 3a4068a26bd919021345b4adf2e89ab7d6da531b) --- M java/kudu-spark/src/main/scala/org/apache/kudu/spark/kudu/HdrHistogramAccumulator.scala A java/kudu-spark/src/test/scala/org/apache/kudu/spark/kudu/HdrHistogramAccumulatorTest.scala 2 files changed, 178 insertions(+), 12 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/32/24532/1 -- To view, visit http://gerrit.cloudera.org:8080/24532 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: branch-1.18.x Gerrit-MessageType: newchange Gerrit-Change-Id: Ia3d556aaf1051464303f35281d1d698b20d5ef20 Gerrit-Change-Number: 24532 Gerrit-PatchSet: 1 Gerrit-Owner: Anonymous Coward <[email protected]> Gerrit-Reviewer: Zoltan Chovan <[email protected]>
