When in doubt, throw us a build at Cloudant with ebtree maps and we'll see if it comes close to the crazy fast KV map queries.
Kyle On Thu, Jul 23, 2020, 2:17 PM Robert Samuel Newson <rnew...@apache.org> wrote: > I (perhaps obviously) don't agree that I'm tying myself to old CouchDB or > failing to embrace FDB. FDB is a toolkit and does not, to my mind, force us > down any particular path. > > I haven't sat down to modify couch_views in the manner I've suggested > (where ebtree is used as designed; being fed the output of the emit() calls > and calculating reductions as it does so) but I think it's a worthwhile > exercise. I'd be surprised if performance of map-only traversals would be > disappointing but who knows? I also expect it would allow for significant > simplification of the code, which is one of the higher virtues. > > Adam, can you describe in a little more detail how you picture "b+tree is > only used for incremental aggregations,"? It's implied in your reply that > it would preserve the "interesting property" of keeping user data out of > FDB Keys (for casual readers: the new native database encryption, called > "aegis", only encrypts the FDB value. It can't encrypt the key as this > would change the order of keys, which the current code depends on). Did I > misread you? > > B. > > > On 23 Jul 2020, at 20:11, Adam Kocoloski <kocol...@apache.org> wrote: > > > > OK thanks for the clarification. As I said I wasn’t all that confident I > understood the design :) > > > > I like the idea that the b+tree is only used for incremental > aggregations, rather than storing the entire materialized view, for the > same reasons that Garren stated. > > > > An index maintained entirely in ebtree has the interesting property that > it does not leak any user data into FDB Keys, which could be attractive for > security reasons. > > > > Adam > > > >> On Jul 23, 2020, at 1:54 PM, Garren Smith <gar...@apache.org> wrote: > >> > >> On Thu, Jul 23, 2020 at 6:55 PM Paul Davis <paul.joseph.da...@gmail.com > > > >> wrote: > >> > >>>> I would like to keep ebtree to use just for the reduce index. > >>> > >>> Could you expand on your reasoning here, Garren? I haven't done any > >>> experiments on my own to understand if I'm missing something > >>> important. My initial reaction is to not diverge too far from the > >>> previous shape of the implementation since we have a decent idea of > >>> how that behaves already but perhaps you've seen or measured something > >>> I'm not thinking of? > >>> > >> > >> I think this must have been a misunderstanding on my part. I always > thought > >> of using ebtree to solve reduce and wasn't planning to use it for the > map > >> index. > >> I don't like the idea that we have ordered distributed key/value store > and > >> then implementing a b-tree on top of that for indexing. Especially > since we > >> know that the current map index is fast, > >> easy to use, and follows recommended practices in the FDB community on > how > >> to do indexing. ebtree makes sense for reduce and is a means to an end > to > >> give us CouchDB's reduce api, which is heavily reliant on a b-tree, with > >> CouchDB on FDB. This feels like a step backward and I worry we are tying > >> ourselves heavily to old CouchDB instead of using the fact that we > moving > >> to FDB which then allows us to design new api's and functionality. > > > >
