On Tue, Jul 2, 2013 at 2:25 PM, Randall Leeds <randall.le...@gmail.com>wrote:
> Very cool experiments there. > > Have you taken the time to understand JSONiq well enough to say > whether that would be a good fit? > > http://www.jsoniq.org/ > > On Sun, Jun 30, 2013 at 2:37 PM, Russell Branca <chewbra...@apache.org> > wrote: > > The discussion of alternative approaches to view engines is one that > > bubbles up semi regularly, with the latest addition for a Lua native > query > > server described in COUCHDB-1842 by Alexander. Lua is a great language > for > > embedding into systems and provides powerful sandboxing facilities. > > > > I'm very intrigued by optimizing for the standard use case, where a user > > wants to build a simple secondary index on their data, and then use a > built > > in reduce function. I think we can find a solution that allows a user to > > define a doc level transformation in a DSL or query language or some > other > > approach that allows us to keep the view generation functionality within > > the Erlang VM and avoid the overhead costs of using an external engine. > > > > I do think it makes sense to have an external engine for flexibility, and > > allowing us to focus on the simple cases while providing a fallback for > > more complex user defined functions. > > > > To experiment with different approaches, I built a Lisp interpreter on > top > > of Erlang with the premise of white listing the entire language, allowing > > explicit control over what the user can and cannot do in view functions. > > You can see it here: [Lispenport]( > https://github.com/chewbranca/lispenport). > > It's by no means a full solution, but it has some interesting properties > > such as really just being syntactic sugar on top of Erlang and all > > constructs are direct Erlang terms, even lambdas are just Erlang funs. > > > > Now, while I would be intrigued by a Lisp DSL for user defined functions > in > > CouchDB, I didn't expect that to be well received by everyone, so I've > > considered this just an experiment. If we were going to take this > approach, > > I would rather take Lisp Flavored Erlang (LFE, another project by Robert > > Virding along with Luerl, and also erlog, a Prolog interpreter in > Erlang), > > and rip out all the pieces we would not want a user to access and use LFE > > as a base starting point. LFE compiles down to intermediate Erlang > bytecode > > and is designed to follow Erlang functionality, making it a nice option > for > > building a view engine to execute in the Erlang VM. > > > > I've also toyed around with the idea of building a NIF around [JQ]( > > http://stedolan.github.io/jq/) which is a great application for slicing > and > > dicing json data structures written in C. > > > > So my general proposal for discussion is that we build a minimal DSL of > > some sort, providing fast and simple doc manipulations that executes > > securely in the Erlang VM, and then we abstract out all functionality > for a > > "full" view engine, list functions, show functions, etc to a separate > > engine that is easily swappable and not required for standard > functionality. > > > > Thoughts? > > > > > > -Russell > Wow JSONiq looks cool! I hadn't seen that before, and it definitely seems like an intriguing option. Would be a solid chunk of development to make a new implementation of it for Erlang, but that could work. We would also want to take a subset of the spec, and not include the pieces of JSONiq that provide IO, but it definitely looks like it's on the right track for what we want. -Russell
