Re: Savepoints and migrating value state data types

Actually, Flink 1.4 will come with improved Avro support. See especially: - https://issues.apache.org/jira/browse/FLINK-7420: Move All Avro Code to flink-avro - https://issues.apache.org/jira/browse/FLINK-7997:

Re: Savepoints and migrating value state data types

Hi Gordon I've been looking into creating a custom AvroSerializer without Kryo which would support Avro schemas and I'm starting to wonder if this is actually the most straightforward way to do it. If I extend a class from TypeSerializer I would also need to implement a TypeInformation class to

Re: Savepoints and migrating value state data types

Hi, Yes, the AvroSerializer currently partially still uses Kryo for object copying. Also, right now, I think the AvroSerializer is only used when the type is recognized as a POJO, and that `isForceAvroEnabled` is set on the job configuration. I’m not sure if that is always possible. As

Re: Savepoints and migrating value state data types

Hi! The exception that you have bumped into indicates that on the restore of the savepoint, the serializer for that registered state in the savepoint no longer exists. This prevents restoring savepoints taken with memory state backends because there will be no serializer available to deserialize

Re: Savepoints and migrating value state data types

Hi Marc, I assume you have set a UID for your CoProcessFunction as described in [1]? Also, can you provide the Flink version you are working with and the serializer you are using? If you have the UID set, your strategy seems to be the same as proposed by [2]: "Although it is not possible to

Savepoints and migrating value state data types

Hi We've got a situation where we're merging several Kafka streams and for certain streams, we want to retain up to 6 days of history. We're trying to figure out how we can migrate savepoint data between application updates when the data type for a certain state buffer updates. Let's assume that