Ma77Ball commented on code in PR #5150:
URL: https://github.com/apache/texera/pull/5150#discussion_r3290677617
##########
amber/src/main/scala/org/apache/texera/amber/engine/architecture/worker/managers/StatisticsManager.scala:
##########
@@ -30,11 +30,11 @@ import
org.apache.texera.amber.engine.architecture.worker.statistics.{
import scala.collection.mutable
class StatisticsManager {
- // DataProcessor
+ // Plain maps (no withDefaultValue) so they survive Kryo round-trip.
private val inputStatistics: mutable.Map[PortIdentity, (Long, Long)] =
- mutable.Map.empty.withDefaultValue((0L, 0L))
+ mutable.Map.empty
Review Comment:
After looking into it:
- withDefaultValue only changes lookups for missing keys (it doesn't add the
values to the db)
- getStatistics iterates the map's real entries (as the values are empty,
not `(0L, 0L)`)
The bug is that after a Kryo restore, the map loses its in-memory
withDefaultValue wrapper, so inputStatistics(portId) on an unseen port throws a
NoSuchElementException.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
