On Tue, Jan 05, 2010 at 09:50:07PM -0600, Peter Karman wrote: > Marvin Humphrey wrote on 1/5/10 9:28 PM: > > There are still a couple of TODO items: > > > > * JSON::XS is not thread safe, and we need to replace it with our own > > custom JSON parsing module. > > isn't that necessary anyway for non-Perl host languages?
Yes, it's just been a low priority because JSON::XS is a solid and high-performing module. I also don't have a burning desire to implement threads, especially since Perl's ithreads have a lot of downsides and aren't very popular. Thread support will be crucial for Java bindings, though -- JVM startup time is significant enough that using processes as your concurrency model has serious drawbacks compared to threads. And other languages have better thread support than Perl. The main reason I went and worked on threads now was because McCandless and I arrived at something like consensus over in the LUCENE-2026 thread. I've always thought it was important to have a concurrency plan. Now that plan is firming up. > Is there enough C in JSON::XS we could borrow as a starting point? Possible, but if we roll our own I think we should just work from scratch off the spec, avoiding potential IP issues. > Or, is the MIT license compatible with the Apache license, making it > possible to use something like http://oss.metaparadigm.com/json-c/ ? Possible. It wouldn't necessarily be tailored to our data types, so we might not be able to squeeze out absolute maximum performance. However, it's the next logical intermediate step after moving away from host-specific libraries like JSON::XS. OTOH, I hate dependencies, because it so hard to find something suitable that's at least as platform compatible or more so than the library it would be supporting. (CPAN::Testers results for KinoSearch 0.30_072 so far: 32 passes, 0 fails.) And in the C world, build scripts are wildly diverse and wildly platform incompatible. > Just seems like there should be a standalone, Apache-friendly JSON > reader/writer available in C. There probably is, but the JSON spec is awesome for its simplicity. It's not that hard to write a parser. > >In my opinion, we should stop there. > > ++ Cool, looks like consensus on the way. Marvin Humphrey
