By the way, I did a benchmark on JSON parsing performance recently. Based on that, spray-json was about 10x slower than the Jackson-based parsers. I recommend json4s-jackson, because jackson is almost certainly already a dependency of Sparks (many other Java libraries use it), so the dependencies are very lightweight. I didn't benchmark Argonaut or play-json, partly because play-json pulled in all the Play dependencies, tho as someone else commented in this thread, they plan to split it out.
On Mon, Feb 10, 2014 at 12:22 AM, Pascal Voitot Dev <pascal.voitot....@gmail.com> wrote: > Hi again, > > If spark just need Json with serialization/deserialization of basic > structures and some potential simple validations for webUI, let's remind > that play/json (without any other dependency than Jackson) is about 200-300 > lines of code... the only dependency is jackson which is the best json > parser that I know. The rest of code is about typesafety & composition... > If Spark need some json veryveryvery performant JSON (de)serialization, we > will have to look on things like pickling and potentially some streaming > parsers (I think this is a domain under work right now...) > > Pascal > > > On Mon, Feb 10, 2014 at 1:50 AM, Will Benton <wi...@redhat.com> wrote: > >> Matei, sorry if I was unclear: I'm referring to downstream operating >> system distributions (like Fedora or Debian) that have policies requiring >> that all packages are built from source (using only tools already packaged >> in the distribution). So end-users (and distributions with different >> policies) don't have to build Lift to get the lift-json artifact, but it is >> a concern for many open-source communities. >> >> best, >> wb >> >> >> ----- Original Message ----- >> > From: "Matei Zaharia" <matei.zaha...@gmail.com> >> > To: dev@spark.incubator.apache.org >> > Sent: Sunday, February 9, 2014 4:29:20 PM >> > Subject: Re: proposal: replace lift-json with spray-json >> > >> > Will, why are you saying that downstream distributes need to build all of >> > Lift to package lift-json? Spark just downloads it from Maven Central, >> where >> > it's a JAR with no external dependencies. We don't have any dependency on >> > the rest of lift. >> > >> > Matei >> > >> > On Feb 9, 2014, at 11:28 AM, Will Benton <wi...@redhat.com> wrote: >> > >> > > lift-json is a nice library, but Lift is a pretty heavyweight >> dependency to >> > > track just for its JSON support. (lift-json is relatively >> self-contained >> > > as a dependency from an end-user's perspective, but downstream >> > > distributors need to build all of Lift in order to package the JSON >> > > support.) I understand that this has come up before (cf. SPARK-883) >> and >> > > that the uncertain future of JSON support in the Scala standard >> library is >> > > the motivator for relying on an external library. >> > > >> > > I'm proposing replacing lift-json in Spark with something more >> lightweight. >> > > I've evaluated apparent project liveness and dependency scope for most >> of >> > > the current Scala JSON libraries and believe the best candidate is >> > > spray-json (https://github.com/spray/spray-json), the JSON library >> used by >> > > the Spray HTTP toolkit. spray-json is Apache-licensed, actively >> developed, >> > > and builds and works independently of Spray with only one external >> > > dependency. >> > > >> > > It looks to me like a pretty straightforward change (although >> > > JsonProtocol.scala would be a little more verbose since it couldn't use >> > > the Lift JSON DSL), and I'd like to do it. I'm writing now to ask for >> > > some community feedback before making the change (and submitting a JIRA >> > > and PR). If no one has any serious objections (to the effort in >> general >> > > or to to the choice of spark-json in particular), I'll go ahead and do >> it, >> > > but if anyone has concerns, I'd be happy to discuss and address them >> > > before getting started. >> > > >> > > >> > > thanks, >> > > wb >> > >> > >> -- -- Evan Chan Staff Engineer e...@ooyala.com |
