Hello, I am on java sdk 2.19 and using dataflow for beam job. I use Timers for my stateful transformations, but recently I started seeing the following exception on DRAINING a job. It used to work fine and not sure what changed.
java.lang.UnsupportedOperationException:
1.
1. atorg.apache.beam.runners.dataflow.worker.
StreamingSideInputDoFnRunner.onTimer (
StreamingSideInputDoFnRunner.java:86
<https://console.cloud.google.com/debug/fromlog?appModule=Dataflow%20Jobs&appVersion=2020-07-24_14_14_42-5619042994908733980&file=org%2Fapache.beam.runners.dataflow.worker%2FStreamingSideInputDoFnRunner.java&line=86&project=prosimo-test>
)
2. atorg.apache.beam.runners.dataflow.worker.
SimpleParDoFn.processUserTimer (SimpleParDoFn.java:360
<https://console.cloud.google.com/debug/fromlog?appModule=Dataflow%20Jobs&appVersion=2020-07-24_14_14_42-5619042994908733980&file=org%2Fapache.beam.runners.dataflow.worker%2FSimpleParDoFn.java&line=360&project=prosimo-test>
)
3. atorg.apache.beam.runners.dataflow.worker.SimpleParDoFn.access$600
(SimpleParDoFn.java:73
<https://console.cloud.google.com/debug/fromlog?appModule=Dataflow%20Jobs&appVersion=2020-07-24_14_14_42-5619042994908733980&file=org%2Fapache.beam.runners.dataflow.worker%2FSimpleParDoFn.java&line=73&project=prosimo-test>
)
4. atorg.apache.beam.runners.dataflow.worker.
SimpleParDoFn$TimerType$1.processTimer (SimpleParDoFn.java:444
<https://console.cloud.google.com/debug/fromlog?appModule=Dataflow%20Jobs&appVersion=2020-07-24_14_14_42-5619042994908733980&file=org%2Fapache.beam.runners.dataflow.worker%2FSimpleParDoFn.java&line=444&project=prosimo-test>
)
5. atorg.apache.beam.runners.dataflow.worker.
SimpleParDoFn.processTimers (SimpleParDoFn.java:473
<https://console.cloud.google.com/debug/fromlog?appModule=Dataflow%20Jobs&appVersion=2020-07-24_14_14_42-5619042994908733980&file=org%2Fapache.beam.runners.dataflow.worker%2FSimpleParDoFn.java&line=473&project=prosimo-test>
)
6. atorg.apache.beam.runners.dataflow.worker.
SimpleParDoFn.processTimers (SimpleParDoFn.java:353
<https://console.cloud.google.com/debug/fromlog?appModule=Dataflow%20Jobs&appVersion=2020-07-24_14_14_42-5619042994908733980&file=org%2Fapache.beam.runners.dataflow.worker%2FSimpleParDoFn.java&line=353&project=prosimo-test>
)
7. atorg.apache.beam.runners.dataflow.worker.util.common.worker.
ParDoOperation.finish (ParDoOperation.java:52
<https://console.cloud.google.com/debug/fromlog?appModule=Dataflow%20Jobs&appVersion=2020-07-24_14_14_42-5619042994908733980&file=org%2Fapache.beam.runners.dataflow.worker.util.common.worker%2FParDoOperation.java&line=52&project=prosimo-test>
)
8. atorg.apache.beam.runners.dataflow.worker.util.common.worker.
MapTaskExecutor.execute (MapTaskExecutor.java:85
<https://console.cloud.google.com/debug/fromlog?appModule=Dataflow%20Jobs&appVersion=2020-07-24_14_14_42-5619042994908733980&file=org%2Fapache.beam.runners.dataflow.worker.util.common.worker%2FMapTaskExecutor.java&line=85&project=prosimo-test>
)
9. atorg.apache.beam.runners.dataflow.worker.
StreamingDataflowWorker.process (StreamingDataflowWorker.java:1350
<https://console.cloud.google.com/debug/fromlog?appModule=Dataflow%20Jobs&appVersion=2020-07-24_14_14_42-5619042994908733980&file=org%2Fapache.beam.runners.dataflow.worker%2FStreamingDataflowWorker.java&line=1350&project=prosimo-test>
)
10. atorg.apache.beam.runners.dataflow.worker.
StreamingDataflowWorker.access$1100 (StreamingDataflowWorker.java:152
<https://console.cloud.google.com/debug/fromlog?appModule=Dataflow%20Jobs&appVersion=2020-07-24_14_14_42-5619042994908733980&file=org%2Fapache.beam.runners.dataflow.worker%2FStreamingDataflowWorker.java&line=152&project=prosimo-test>
)
2. Sometime back I opened jira for an issue which is related to this
while doing DRAIN: https://issues.apache.org/jira/browse/BEAM-10053 (Looks
like no one has taken a stab on this Jira)
3. Not sure if the reason is same and due to multiple side inputs that i
use in my PTransforms.
Any help would be appreciated.
Thanks and Regards
Mohil
