I assume you know that writing a robust, persistent datastore is a big undertaking. If what you're aiming for here is a memory-only db, then draft any API that suits you. (And maybe consider a synchronous API to a memory-only SQLite db?) I'd guess you'll need a draft API to attract feedback. Also keep in mind that there's overhead in crossing the Javascript/C++ boundary in V8.
For a persistent db, you're more likely to make something generally useful by building on an existing system. I'm sure you can bundle Redis with your native module and make configuration easy. On Feb 21, 2:56 pm, Juraj Vitko <juraj.vi...@gmail.com> wrote: > Liam, as I describe in my original post, I think Redis is too verbose > already, and many of the commands would not be necessary for an in- > process Node.js addon DB. > > Having a shmem access to Redis would be interesting, as would be to > actually measure the overhead of the domain socket interface to Redis, > however even if the speed would be the same as with embedded DB -- the > speed is not everything I'm aiming for here - but things like zero- > config, the simplicity of software stack, and easy replacement of > various implementations having the same base API. (Check the > nodejsdb.com page where I write about this as well.) > > On Feb 21, 10:40 pm, Liam <networkimp...@gmail.com> wrote: > > > For API, I'd suggest mirroring the Redis API for a first draft. Are > > there particular features you have in mind which it doesn't provide? > > > Also, before investing a ton of time, it'd be wise to explore just how > > much more performance can be obtained via shmem access to cached data > > vs domain sockets. > > > Assuming it's a lot, then I'd guess you'd want to query the Redis guys > > as to whether they've considered a shmem IPC scheme. > > > On Feb 21, 12:08 pm, Juraj Vitko <juraj.vi...@gmail.com> wrote: > > > > Redis is a standalone DB, accessed either by TCP/IP or Unix domain > > > sockets (I haven't seen a Node.js driver for the latter though). > > > > Quick google revealed (glad you asked, I did not know about > > > these):http://code.google.com/p/redis/issues/detail?id=276[Feature > > > Request] > > > redis as embedded database > > > [Open]http://code.google.com/p/redis/issues/detail?id=231Featurerequest: > > > support for unix domain sockets [Fixed] > > > > I think it would not be entirely straightforward to use Redis as an > > > embedded DB, because it uses fork() to implement the snapshotting, but > > > that's just my guess. > > > > Anyway, I'm kind of trying to see what would people consider a good > > > base API. I assume this API would be much leaner than the one in Redis > > > (http://redis.io/commands), as in Node one would be able to manipulate > > > data and/or Buffers more directly, not needing e.g. BRPOPLPUSH and the > > > like. > > > > On Feb 21, 9:53 pm, Liam <networkimp...@gmail.com> wrote: > > > > > Any insight on how Redis deployments enable "intrinsic" access to > > > > cached data currently? Is there a shmem implementation of the API? > > > > > On Feb 21, 1:09 am, Juraj Vitko <juraj.vi...@gmail.com> wrote: > > > > > >https://github.com/ypocat/nodejsdb(orhttp://nodejsdb.com) > > > > > > tl;dr - There are standalone database products (free or not), and > > > > > that's perfectly cool, but we already know how that works, so let's > > > > > try something different now. > > > > > > The general idea is to get Node.js and a data storage engine into a > > > > > tighter relationship, primarily to have more control of the data, but > > > > > also simpler stack, and even higher performance in accessing the data. > > > > > > I'm using the name "Intrinsic" because "In-process" is not exactly > > > > > accurate. E.g. there may be a shared-memory implementation shared by > > > > > multiple Nodes, or synchronized in-process implementation shared by > > > > > different Node Isolates (if these make it into Node), etc. > > > > > > I really like the base concept of Redis, because it provides simple, > > > > > reliable, predictable and fast primitive building blocks (in the form > > > > > of commands) which can support various app logic strategies, and it's > > > > > not hiding the complexities and overheads of storing and querying > > > > > data, that more complex DB's do. (So you are more likely to have more > > > > > stable production in the end, instead of fiascos with overflowing > > > > > shards etc.) > > > > > > This is also a vague follow-up to this discussion (in this > > > > > group)http://goo.gl/mDWqR-althoughIbelievewe should not insist only on > > > > > in-memory implementations at this time. > > > > > > As for the basic set of basic data structures and operations, that I > > > > > believe would support the above, I think we need: > > > > > > 1) fast unordered Hash Map (key, value) > > > > > (candidate:http://code.google.com/p/sparsehash/) > > > > > > 2) Ordered Map (with minimal empty 'value' overhead to allow for > > > > > Ordered Set implementation if someone wants it) > > > > > (candidate:http://www.cs.princeton.edu/~rs/talks/LLRB/08Penn.pdf) > > > > > > 3) a list that can be used for FIFO, LIFO, stack, etc. - probably > > > > > something close STL's Deque. (http://www.cplusplus.com/reference/stl/ > > > > > deque/) > > > > > > I think the API for the above should be as simple as possible, so that > > > > > we can have multiple implementations and various optimizations later, > > > > > while keeping the amount of needed work down. Also, terse API is > > > > > simple to use. > > > > > > From Node, we could do something like: > > > > > > require('a-nodejsdb-impl').open('/path/db', function(err, db) { > > > > > var users = db.map('users'); > > > > > var users_ordered_by_email = db.smap('users_by_email'); > > > > > users.on('put', function(k, v) { > > > > > users_ordered_by_email.put(v.email, k); > > > > > }); > > > > > users.put(1234, { fname: 'john', lname: 'smith', email: '...@b.c' > > > > > }); > > > > > > } > > > > > > ..which implements a basic User table with primary key on ID and > > > > > ordered index on Email. (The difference being that it gives you 200k > > > > > operations per second and you don't need a separate DB server.) > > > > > > So if you guys have any constructive input regarding this, please post > > > > > it here. -- Job Board: http://jobs.nodejs.org/ Posting guidelines: https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines You received this message because you are subscribed to the Google Groups "nodejs" group. To post to this group, send email to nodejs@googlegroups.com To unsubscribe from this group, send email to nodejs+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/nodejs?hl=en?hl=en
