Hi Alex,
It is a little weird.
Would you please provide the program which could reproduce the problem,
including DataStream job code and related classes code. I need some debug
to find out the reason.
Best,
JING ZHANG
Alex Drobinsky <alex.drobin...@gmail.com> 于2021年10月11日周一 下午5:50写道：
> Hi Jing Zhang,
>
> I'm using the FileSystem backend. I also implemented ReadObject function
> to support proper restart procedure:
>
> private void readObject(ObjectInputStream ois)
> throws ClassNotFoundException, IOException {
> ois.defaultReadObject();
> logger.info("Deserialized MultiStorePacketState: " + this.toString());
>
> // No need to do anything in case of empty file
> if (fileName.isEmpty()) {
> return;
> }
> currentFile = new RandomAccessFile(fileName,"rw");
> currentFile.seek(fileOffset);
> }
>
> However, according to logs this function wasn't called.
>
> Btw, it could be beneficial to add this kind of State object e.g. FileState
> which will encapsulate serialization / deserialization for RandomAccessFile
> although the concept itself is a bit contradictory to regular state.
>
> Currently, I implemented and tested a workaround via addition of the boolean
> variable isFileOpened, however it's awkward because I need to check the state
> of the transient variable every time I use state.value().
>
> So should it be expected that transient variables in state would be resetted
> to default values ?
>
>
> пн, 11 окт. 2021 г. в 12:33, JING ZHANG <beyond1...@gmail.com>:
>
>> Hi, Alex
>> What state backend do you choose?
>> If you choose MemoryStateBackend or FsStateBackend, `transient` keyword
>> may not have effect because MemoryStateBackend does not serialize state for
>> regular read/write accesses but keeps it as objects on the heap.
>> If you choose RocksDBStateBackend, I thought it was expected behavior
>> because RocksDBStateBackend stores all state as byte arrays in embedded
>> RocksDB instances. Therefore, it de/serializes the state of a key for every
>> read/write access. CurrentFile is null because the transient variable
>> would not be serialized by default.
>>
>> Best,
>> JING ZHANG
>>
>>
>> Alex Drobinsky <alex.drobin...@gmail.com> 于2021年10月11日周一 下午4:33写道：
>>
>>> Dear flink community,
>>>
>>> I have following state class ( irrelevant fields removed )
>>> public class MultiStorePacketState implements Serializable {
>>>
>>> public transient RandomAccessFile currentFile = null;
>>> public long timerValue;
>>> public String fileName;
>>> public String exportedFileName;
>>> public String sessionKey;
>>> public long fileOffset = 0;
>>>
>>> }
>>>
>>> Once in a while, currentFile became *nullified, *this happens after I
>>> extract state via
>>>
>>> MultiStorePacketState so = state.value();
>>>
>>> The frequency of this behaviour is similar to checkpoint interval (
>>> checkpoint interval defined as 5 seconds and first occurence of this
>>> problem is also 5 seconds), otherwise I don't have any clues to a possible
>>> explanation.
>>>
>>> Is it an expected side effect of checkpoint procedure ?
>>>
>>> Best regards,
>>> Alex
>>>
>>>
