I thought I read/heard somewhere that Pickling is aiming to become the default serialization mechanism in Scala, but I cannot find any proof (like a SIP) for it (yet).
Integrating Pickling in Spark is currently work in progress: https://groups.google.com/forum/#!msg/sbt-dev/7IpZrq79oz0/UunWL0GLYjsJ (see Heather Millers posts) -Jeroen Op woensdag 22 oktober 2014 18:10:28 UTC+2 schreef rrodseth: > > I'm hoping Typesafe will come up with a great solution. I noticed that > Scala Pickling is making its way into Spark > > At the very end of this good talk, he mentions plans to use Pickling in > Spark. > > http://functional.tv/post/97699944449/scala-by-the-bay2014-matei-zaharia-next-generation-langu > > <http://www.google.com/url?q=http%3A%2F%2Ffunctional.tv%2Fpost%2F97699944449%2Fscala-by-the-bay2014-matei-zaharia-next-generation-langu&sa=D&sntz=1&usg=AFQjCNGpo4CrFZFa7z2YEwiJI91Mvn7bNg> > > and Heather Miller discusses it on Scalawags > https://www.youtube.com/watch?v=uCpw9eBYbOw > > I also came across this ticket: > https://github.com/scala/pickling/issues/39 > > > On Tue, Oct 21, 2014 at 5:17 PM, Richard Rodseth <rrod...@gmail.com > <javascript:>> wrote: > >> Thanks for the input. I wonder if anyone has tried Scala Pickling. At >> first glance, it seems like it would be a great solution. >> >> On Tue, Oct 21, 2014 at 1:47 PM, Jeroen Gordijn <jeroen....@gmail.com >> <javascript:>> wrote: >> >>> Hi, >>> >>> I'm not really sure yet about the best solution. Currently using json. >>> It starts out being really straight forward when using spray-json. Just >>> create the formatters. But the moment you have to deal with evolution, you >>> have to deal with the Json AST and you loose all help from your compiler. >>> I'm looking into a Protobuf implementation of the serializer. It will give >>> the overhead of having to specify a definition of each persistable message >>> in a proto file, but I guess it will help a lot with evolution. >>> >>> I found that I have to wrap my messages in a general envelope to >>> indicate which message was serialized. I could not use the Akka Persistence >>> default manifest as it indicates which Domain Class was persisted as >>> opposed to the protobuff class. >>> >>> Cheers, >>> Jeroen >>> >>> Op dinsdag 21 oktober 2014 17:31:32 UTC+2 schreef rrodseth: >>>> >>>> Thanks for the references, Patrik. I hope others will share their >>>> experiences. >>>> >>>> On Tue, Oct 21, 2014 at 4:57 AM, Patrik Nordwall <patrik....@gmail.com> >>>> wrote: >>>> >>>>> Hi Richard, >>>>> >>>>> I'm also interested in what others are currently using. >>>>> >>>>> On Sun, Oct 19, 2014 at 3:35 AM, Richard Rodseth <rrod...@gmail.com> >>>>> wrote: >>>>> >>>>>> I'm hoping to build a case for using Akka Persistence, and expect the >>>>>> event schema evolution to be a sticking point. People are quite used to >>>>>> their database migration scripts. >>>>>> >>>>>> I've seen the threads on this list, but I'm wondering what the >>>>>> current thinking is on the least painful option, and would also like >>>>>> pointers to sample code. >>>>>> >>>>> >>>>> I have not heard of any new ideas. Migration and data evolution can be >>>>> implemented in a custom serializer. >>>>> >>>>> >>>>>> Do Protobuf, Avro and Thrift all require an IDL compiler? >>>>>> >>>>> >>>>> Protobuf and Thrift, yes. I'm not sure about Avro. I think it can be >>>>> used without generated code. >>>>> >>>>> >>>>>> Is Cap'n Proto a viable option? >>>>>> >>>>> >>>>> Yes, it looks interesting, especially for high performance, but I have >>>>> no experience of it. >>>>> In same category you also have SBE and FlatBuffers. Here is a nice >>>>> comparison >>>>> <https://kentonv.github.io/capnproto/news/2014-06-17-capnproto-flatbuffers-sbe.html> >>>>> . >>>>> >>>>> >>>>>> Would something based on Scala macros provide a way to avoid an extra >>>>>> compilation? >>>>>> >>>>> >>>>> Scala Pickling <https://github.com/scala/pickling>. I don't have much >>>>> experience of it, and I'm uncertain of how mature it is. >>>>> >>>>> >>>>>> >>>>>> Anything on the horizon I should know about? >>>>>> >>>>> >>>>> JSON is well known and has the nice property of being schema-less. >>>>> CBOR <https://www.rfc-editor.org/info/rfc7049> has similar >>>>> characteristics but has the advantages of a binary format. I have played >>>>> with the jackson-dataformat-cbor >>>>> <https://github.com/FasterXML/jackson-dataformat-cbor> tool and here >>>>> is an example of an Akka serializer >>>>> <https://github.com/typesafehub/activator-akka-cluster-sharding-scala/pull/8/files>. >>>>> >>>>> That sample is using a low level API, with all the advantages of full >>>>> control and possibilities of data migration. The drawback is of course >>>>> the >>>>> amount of hand written mapping code. I expect that it will be possible to >>>>> also use other high level tools with CBOR, such as Jackson Databinding >>>>> and >>>>> Play-json >>>>> <https://www.playframework.com/documentation/2.3.x/ScalaJson>. >>>>> >>>>> Cheers, >>>>> Patrik >>>>> >>>>> >>>>>> >>>>>> Thanks. >>>>>> >>>>>> -- >>>>>> >>>>>>>>>> Read the docs: http://akka.io/docs/ >>>>>> >>>>>>>>>> Check the FAQ: http://doc.akka.io/docs/akka/ >>>>>> current/additional/faq.html >>>>>> >>>>>>>>>> Search the archives: https://groups.google.com/ >>>>>> group/akka-user >>>>>> --- >>>>>> You received this message because you are subscribed to the Google >>>>>> Groups "Akka User List" group. >>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>> send an email to akka-user+...@googlegroups.com. >>>>>> To post to this group, send email to akka...@googlegroups.com. >>>>>> Visit this group at http://groups.google.com/group/akka-user. >>>>>> For more options, visit https://groups.google.com/d/optout. >>>>>> >>>>> >>>>> >>>>> >>>>> -- >>>>> >>>>> Patrik Nordwall >>>>> Typesafe <http://typesafe.com/> - Reactive apps on the JVM >>>>> Twitter: @patriknw >>>>> >>>>> -- >>>>> >>>>>>>>>> Read the docs: http://akka.io/docs/ >>>>> >>>>>>>>>> Check the FAQ: http://doc.akka.io/docs/akka/ >>>>> current/additional/faq.html >>>>> >>>>>>>>>> Search the archives: https://groups.google.com/ >>>>> group/akka-user >>>>> --- >>>>> You received this message because you are subscribed to the Google >>>>> Groups "Akka User List" group. >>>>> To unsubscribe from this group and stop receiving emails from it, send >>>>> an email to akka-user+...@googlegroups.com. >>>>> To post to this group, send email to akka...@googlegroups.com. >>>>> Visit this group at http://groups.google.com/group/akka-user. >>>>> For more options, visit https://groups.google.com/d/optout. >>>>> >>>> >>>> -- >>> >>>>>>>>>> Read the docs: http://akka.io/docs/ >>> >>>>>>>>>> Check the FAQ: >>> http://doc.akka.io/docs/akka/current/additional/faq.html >>> >>>>>>>>>> Search the archives: >>> https://groups.google.com/group/akka-user >>> --- >>> You received this message because you are subscribed to the Google >>> Groups "Akka User List" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to akka-user+...@googlegroups.com <javascript:>. >>> To post to this group, send email to akka...@googlegroups.com >>> <javascript:>. >>> Visit this group at http://groups.google.com/group/akka-user. >>> For more options, visit https://groups.google.com/d/optout. >>> >> >> > -- >>>>>>>>>> Read the docs: http://akka.io/docs/ >>>>>>>>>> Check the FAQ: >>>>>>>>>> http://doc.akka.io/docs/akka/current/additional/faq.html >>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user --- You received this message because you are subscribed to the Google Groups "Akka User List" group. To unsubscribe from this group and stop receiving emails from it, send an email to akka-user+unsubscr...@googlegroups.com. To post to this group, send email to akka-user@googlegroups.com. Visit this group at http://groups.google.com/group/akka-user. For more options, visit https://groups.google.com/d/optout.
