Hello,
We recently encountered an weird exception:
Caused by: java.lang.IllegalStateException: Unexpected empty state address
window set for ACTIVE window
[2021-02-10T17:25:10.715Z..2021-02-10T20:25:10.715Z)
at
org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkState(Preconditions.java:588)
~[beam-vendor-guava-26_0-jre-0.1.jar:?]
at
org.apache.beam.runners.core.MergingActiveWindowSet.checkInvariants(MergingActiveWindowSet.java:329)
~[beam-runners-core-java-2.20.7.9.jar:?]
at
org.apache.beam.runners.core.MergingActiveWindowSet.persist(MergingActiveWindowSet.java:88)
~[beam-runners-core-java-2.20.7.9.jar:?]
at
org.apache.beam.runners.core.ReduceFnRunner.persist(ReduceFnRunner.java:380)
~[beam-runners-core-java-2.20.7.9.jar:?]
at
org.apache.beam.runners.core.GroupAlsoByWindowViaWindowSetNewDoFn.processElement(GroupAlsoByWindowViaWindowSetNewDoFn.java:138)
~[beam-runners-core-java-2.20.7.9.jar:?]
which suggests there is an empty window in ReduceFnRunner#activeWindows
After some more digging and we found there could be two possibilities that
could result in such state:
1. ReduceFnRunner#onTimers is clearing entry value, causing an empty value
entry in ReduceFnRunner#activeWindows
2. ReduceFnRunner#activeWindows initialized with empty value in
MergingActiveWindowSet#emptyIfNull, then empty iterable windowed values is
passed in the subsequent ReduceFnRunner#processElements, where
cleanupTemporaryWindows() is not invoked at all before ReduceFnRunner#persist
Questions are, are #1 and/or #2 possible to happen? If Yes, under what
circumstances?
Best,
Ke
