Hi All, I think sticking with 3.x is the better way to go for the CouchDB community. If anyone wants to continue with the FDB-CouchDB work. I think the first place to start there is to change the user-facing API to better fit with FDB. We added some really nice unit tests for FDB-CouchDB, it might be worth investigating if we could move some of the unit tests to 3.x. The one feature I loved with FDB-CouchDB was that the mango indexes were updated in the transaction that the document was added. Meaning the mango indexes were always up to date. I will really miss that feature.
Finally, I've seen some mention around the pluggable storage engine for 3.x. I wrote a PoC Rockdbs version of it that passed most the pluggable storage tests https://github.com/garrensmith/couch_rocks With the pluggable storage engine you will most likely run into the same issue we had with FDB. CouchDB relies on a B-Tree to do anything reduce related. I think what could be possible with the pluggable storage engine is to write the couch_store in something like Rust to see if you get a performance improvement. Or to write an in-memory one. However, I don't think the storage engine is the real bottleneck in CouchDB. Cheers Garren On Fri, Mar 18, 2022 at 9:44 AM Johs Ensby <j...@b2w.com> wrote: > Hi, > I want to second the statements (Fynn, Ronny and others) that I see as > support for option 1 in Jan’s three ways forward: > > 1. Follow IBM in abandoning FDB-Couch, refocus all effort on Erlang-Couch > (3.x). > > “Follow IBM” in the meaning “staying together and have the benefit of a > big brand supporter” is a good thing, also. > > I still think couchDB is unique, but simplicity for devops is the key > benefit. > Replication, offline first, distributed systems, built-in blockchain, > attachments and designdocs that allow full-fledged apps to travel with the > data, > … the feature list goes on and on, and the heritage from the project’s > start is beautiful and still very future-oriented. > > Best regards > Johs > > > On 17 Mar 2022, at 21:25, Jan Lehnardt <j...@apache.org> wrote: > > > > Heya Alex, > > > > I think all these are fair assessments and I can’t think of anything > atop. > > > > Best > > Jan > > — > > > > > >> On 16. Mar 2022, at 23:54, Alex Miller <millerde...@gmail.com> wrote: > >> > >> (With my hat on as a liaison from the FoundationDB community...) > >> > >> I imagine that, like all corporate decisions, stopping work on > >> CouchDB-FDB was some part business reasons (which are entirely out of > >> scope for this discussion) and some part technical reasons. I wanted > >> to try and make sure to collect the feedback about any technical > >> motivations that might have dissuaded the continued work on > >> CouchDB-FDB as CouchDB 4.x. Thus far from watching the thread, I'm > >> at: > >> > >> * Long lived read-only snapshots are required to maintain CouchDB 3.x > >> semantics, and that feature has yet to be implemented in FDB.[1] > >> And this sounds like probably the largest concern. > >> * FDB cluster administration at scale isn't well documented.[2] > >> * The governance model isn't inspiring confidence, though the project > >> decisions haven't inspired distrust either.[3] > >> * The skill sets required to modify a layer (CouchDB) is different > >> than modifying the underlying database (FDB), and it's unclear what > >> degree of features can be requested and completed from the latter.[4] > >> Additionally, there's no consulting/FDB-as-a-service company from > >> which one could potentially pay for core changes. > >> * FoundationDB has been eager to utilize new hardware features for > >> performance, and doesn't degrade gracefully on older hardware.[5] > >> > >> Is there anything else that one might highlight as "If XYZ was > >> improved, a finished CouchDB-FDB would look like a more tenable > >> CouchDB 4.x"? > >> > >> Thanks, > >> Alex > >> > >> > >> [1]: > >> On Sat, Mar 12, 2022 at 1:26 AM Jan Lehnardt <j...@apache.org> wrote: > >>> I know for some the 4.x release is highly anticipated and we as a > project hoped to make a generational jump for our underlying storage and > distribution technologies. During initial discussions about FDB-Couch and > during its development, we anticipated certain developments on the FDB side > (especially allowing longer transactions for consistent _changes responses > with their new Redwood storage engine). It is my understanding that these > developments have not materialised in the way we would like them. > >> > >> [2]: > >> On Sat, Mar 12, 2022 at 1:26 AM Jan Lehnardt <j...@apache.org> wrote: > >>> We also learned that operating a FDB cluster is a significant effort > that somewhat goes against CouchDB’s mostly “just works” nature. We had > asked the IBM team to share their operational FDB learnings with the > CouchDB project, so we can build up community knowledge around this, but > this has not materialised either. > >> > >> [3]: > >>>> On 13 Mar 2022, at 11:17, Reddy B. <redd...@live.fr> wrote: > >>>> And this fact is also the cause of the unease I personally have this > FoundationDB migration: it looks like CouchDB will have much less control > over its destiny and even philosophy. This is different from say an > encrypted messaging app deciding to replace its home-made encryption with > an established and more robust open-source solution. From day 1, I feel > like this project will end up in FoundationDB integrating CouchDB rather > than the other way around. I even suspected that maybe the dev team was no > longer interested in CouchDB and wanted to find it a new home. > >>>> > >>>> My friendly feedback as a user is that I trust the Apache governance > model much more than I trust Apple, especially when the welcome meal they > have offer me is that features will be removed and limitations introduced. > The political background and what I would call "corporate risk" (key > capabilities not implemented by upstream, change in priority or vision, > difficulty to affect the roadmap of upstream etc...), is also a key factor > when choosing a DB solution as a user. > >>> > >>> On Sun, Mar 13, 2022 at 4:39 AM Robert Newson <rnew...@apache.org> > wrote: > >>> I think it’s reasonable to worry about tying CouchDB to FoundationDB > for some of the reasons you mentioned, but not all of them. We did worry, > at the start, at the lack of a governance policy around FoundationDB; > something that would help ensure the project is not beholden to a single > corporate entity that might abandon the project or take it in places that > make it unsuitable for CouchDB in the future. There hasn’t been much > progress on that, but likewise the project has stayed true to form. > >> > >> [4]: > >>>> On 13 Mar 2022, at 11:17, Reddy B. <redd...@live.fr> wrote: > >>>> If even you guys weren't treated as a priority, I doubt that my > feature requests and other input will matter even one bit as a user. And I > would have zero chance of having the expertize required to modify the FDB > core myself and get my changes approved to make my CouchDB Layer- related > request possible. While right now I get can get my hands dirty and > eventually get something done if I really want to. The governance here is > very friendly, welcoming and inspiring trust. > >> > >> I do feel the need to call out that there were a few contributions to > >> FDB from CouchDB folk, and IBM Research Zurich (Diego Didona, et. al) > >> in particular contributed some great storage engine improvements. > >> > >> [5]: > >> On Tue, Mar 15, 2022 at 5:33 AM Will Young <lostnetwork...@gmail.com> > wrote: > >>> foundationdb's requirement for avx was a headache for me as I have > >>> some older x86 as well as Arm, where avx2neon seemed experimental. > > > > >
