Hi Devs,
@Users: I'm cc'ing the user ML to see if there are any users that are
relying on this feature. Please comment here if that is the case.
I'd like to discuss the deprecation and eventual removal of UnionList
Operator State, aka Operator State with Union Redistribution. If you
don't kn
+1 to getting rid of non-keyed state as is in general and for union state
in particular. I had a hard time to wrap my head around the semantics of
non-keyed state when designing the rescale of unaligned checkpoint.
The only plausible use cases are legacy source and sinks. Both should also
be rewor
Generally +1
The one use case I've seen of union state I've seen in production (outside
of sources and sinks) is as a "poor mans" broadcast state. This was
obviously before that feature was added which is now a few years ago so I
don't know if those pipelines still exist. FWIW, if they do the stat
Subject: Re: [DISCUSS] Deprecate and remove UnionList OperatorState
Generally +1
The one use case I've seen of union state I've seen in production (outside of
sources and sinks) is as a "poor mans" broadcast state. This was obviously
before that feature was added which is now
:* Seth Wiesman
> *Sent:* Wednesday, September 9, 2020 9:37:03 AM
> *To:* dev
> *Cc:* Aljoscha Krettek ; user
> *Subject:* Re: [DISCUSS] Deprecate and remove UnionList OperatorState
>
> Generally +1
>
> The one use case I've seen of union state I've seen in prod
On 14.09.20 02:20, Steven Wu wrote:
Right now, we use UnionState to store the `nextCheckpointId` in the Iceberg
sink use case, because we can't retrieve the checkpointId from
the FunctionInitializationContext during the restore case. But we can move
away from it if the restore context provides th