RE: Re: Re: Should we always mark ValueState as "transient" for RichFunctions

2023-02-24 Thread Zhongpu Chen
Hi Shammon, Sorry for the inaccurate description of my last reply. Let me restate my question again: Fact 1: we know that ValueState here should not serialized/de-serialized, so it is a good practice to mark it with "transient". Fact 2: on the other hand, if we don't mark it with "transien

Re: Re: Should we always mark ValueState as "transient" for RichFunctions

2023-02-24 Thread Shammon FY
Hi Sorry that I don't quite understand your question. I think the above functions will only be deserialized when the job is submitted, do you want to test the impact of this on submission throughput? Best, Shammon On Fri, Feb 24, 2023 at 3:04 PM Zhongpu Chen wrote: > Hi Gen, > > Thanks for yo

RE: Re: Should we always mark ValueState as "transient" for RichFunctions

2023-02-23 Thread Zhongpu Chen
Hi Gen, Thanks for your explanation. Back to this code snippet, since they are not marked with "transient" now, I suppose Flink will use avro to serialize them (null values). Is there any benchmark to show the performance test between null values serialization and "transient"? I mean, it is i

Re: Should we always mark ValueState as "transient" for RichFunctions

2023-02-23 Thread Gen Luo
Hi, ValueState is a handle rather than an actual value. So it should never be serialized. In fact, ValueState itself is not a Serializable. It should be ok to always mark it as transient. In this case, I suppose it works because the ValueState is not set (which happens during the runtime) when th

Should we always mark ValueState as "transient" for RichFunctions

2023-02-23 Thread Zhongpu Chen
Hi, When I am reading the code from flink-training-repo [1], I noticed the following code: ```java public static class EnrichmentFunction extends RichCoFlatMapFunction { private ValueStaterideState; private ValueStatefareState; ... } ``` From my understanding, since ValueState