I had a reply prepped yesterday but lost the draft (argh offline editing!).
Back to the topic though:
First of all, thank you very much for the elaborate reply. I guess I'll
get hacking away at some prototypes soon and so some performance testing close
to our use-case. Looking forward to it, too :-)
Great to hear, let us know about your findings please :-)
So what you suggest is that the serializer first deserializes a stored event
"forgivingly", without throwing an exception for for example dropping removed
field and initializing new fields with 0/null/default values, and then I
compare the version and apply additional conversion logic? Or do you mean
something more complex like storing a "dehydrated" version of the event and
re-hydrating it every time an event is loaded from storage, promoting it to the
latest version in this process?
This very much depends on what serialiser you use, because they inherently
support some kind of schema evolution.
I’ve been mostly using protocol buffers in my career. There, what you do is:
mostly use optional fields so these can be null, and you have to be prepped for
this anyway.
And then you can reserve some space for extensions, using that you’re able to
apply this trick: http://www.indelible.org/ink/protobuf-polymorphism/
Because of how proto works, renames are safe for example - because it’s not
using the name in the binary format, but the ID. etc etc…
This should give you a feel how “bound” your evolution strategies will become
to the serialisation format.
Other basic tips are nicely summarised here:
http://martin.kleppmann.com/2012/12/05/schema-evolution-in-avro-protocol-buffers-thrift.html
Again though, dig deeper into the details of each serialisation format before
you decide to go for it (we don’t really have a “use that one” opinion on this).
Other serialisation formats, such as Simple Binary Encoding (designed to be low
latency envs - check benchmarks) support this via keeping version numbers in
the message header:
https://github.com/real-logic/simple-binary-encoding/wiki/Message-Versioning
In essence - version number => some number of extension fields, you cannot
delete fields in this one.
One other format I’d like to point to here is Cap n’ Proto (same author as
original protobufs AFAIK), and their description on this (as it’s listing it up
quite nicely what is safe and what is not):
http://kentonv.github.io/capnproto/language.html#evolving_your_protocol
Summing up… These strategies allow you to change your events and stay
compatible with the same type - to still be able to deserialise.
If you however at some point make some huge change, and need to get events B
but you have events A serialised, or you need to add additional data you can
add another layer
that would promote A events to B events, and then emit them to the actor. This
gets more complicated, but the point is - you’re not doomed even if you have to
do bigger changes (try to avoid this ofc ;-)).
I know I’ve just grown your to-read list by quite a bit, but I hope this helps!
:-)
--
Konrad 'ktoso' Malawski
hAkker @ typesafe
http://akka.io
--
>>>>>>>>>> Read the docs: http://akka.io/docs/
>>>>>>>>>> Check the FAQ:
>>>>>>>>>> http://doc.akka.io/docs/akka/current/additional/faq.html
>>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user
---
You received this message because you are subscribed to the Google Groups "Akka
User List" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to akka-user+unsubscr...@googlegroups.com.
To post to this group, send email to akka-user@googlegroups.com.
Visit this group at http://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.
