Why does Spark allows only Java Serializable in Checkpointing ? I see in Checkpoint.serialize() that it doesn’t even try to load a serializer from the configuration and uses Java’s ObjectOutputStream.
This means that I can’t use Avro (fro eaxmple) in updateStateByKey, right ? Is there a reason for that ? Are there plans to change that ? Thanks, Amit
