Thanks Bob for passing this along. I’m looking forward to renewed interest in the 3.x codebase :)
For our 4.x plans, we’ll have to discuss here what we want to do with it and I’m looking at everyone for input here. Even if you’ve never spoken up on this list before, I’d lie to hear from you. * * * First off, as a project, CouchDB is not obliged to follow IBMs lead and abandon the FDB-CouchDB effort. At the same time, it is not obliged to take what they leave behind and finish it. I know for some the 4.x release is highly anticipated and we as a project hoped to make a generational jump for our underlying storage and distribution technologies. During initial discussions about FDB-Couch and during its development, we anticipated certain developments on the FDB side (especially allowing longer transactions for consistent _changes responses with their new Redwood storage engine). It is my understanding that these developments have not materialised in the way we would like them. The consequence is that there are certain API guarantees that 3.x CouchDB gives (consistent full-database snapshots in _changes) are not possible to build with native FDB features. — I can’t speak to the very specifics of this, and I hope we can dig into all this together in this thread, but my takeaway from this is that *if* we continue with FDB-Couch, I think we will have to reevaluate its compatibility story, as we had hoped to make it mainly a seamless (but better) API upgrade from 3.x. We also learned that operating a FDB cluster is a significant effort that somewhat goes against CouchDB’s mostly “just works” nature. We had asked the IBM team to share their operational FDB learnings with the CoucHDB project, so we can build up community knowledge around this, but this has not materialised either. I’m personally still excited about the opportunities we have with FDB-Couch, but as a project, we might have to come up with a more realistic positioning of FDB-CouchDB. Less a “new and improved drop-in replacement” and maybe more a “if you exceed the scale/capacity of 3.x CouchDB, you can upgrade to FDB-CouchDB at the expense of a few API differences and higher operational cost”. This might be worth a trade-off for large users of CouchDB and thus it might be worth having both of these codebases live alongside each other. However, that comes with a number of consequences: - The 3.x/4.x naming doesn’t quite work if these are meant to continue alongside each other. - Maybe FDB-Couch gets its own separate project name and versioning, with a clear delineation between them. - We would have to maintain two projects complete with release management, vulnerability management, the lot. At the moment, CouchDB has just about enough folks contributing to move forward at a reasonable pace. Doubling that effort might be tricky. While we had an influx of contributors recently, this would probably need more dedicated planning and outreach. - New API features would have to be implemented twice, if we want to keep a majority API overlap. This is not a fun proposition for folks who add features, which is hard enough, but now they have to do it twice, onto two different subsystems. Some features (say multi-doc-transactions) would only be possible in one of the projects (FDB-Couch), what would our policy be for deliberate API feature divergence? - probably more that elude me at the moment. While there are non-trivial points among these, they are not impossible tasks *if* we find enough and the right folks to carry the work forward. * * * For myself, I still see a lot of potential in the 3.x codebase and I’m looking forward to renewed roadmap discussions there. I know I have a long list of things I’d like to see added. From my professional observation, the thing that our (Neighbourhoodie) customers tend to run into the most is the scaling limits of the database-per-user pattern. We have a proposal for per-doc-authentication that helps mitigate a subset of those use-cases, which would be a great help overall. I have worked on a draft PR of this over the years, but it mostly stalled out during the pandemic. I’m planning to restart work on this shortly. If anyone wants to contribute with time and/or money, please do get in touch. The other major issue with 3.x as reported by IBM is _changes feed rewinds when nodes are rotated in and out of clusters. We already fixed a number of changes rewind bugs relatively recently. I don’t know if we got them all now, or if there are theoretical limits to how far we can take this given our consistency model, but it’d be worth spending some time on at least getting rid of all rewind-to-zero cases. * * * I’m also looking forward to all your input on the discussion here. I’m sure this will explode into a lot of detailed discussions quickly, so maybe as a guide to come back to when get closer to having to make a decision, here are three ways forward that I see: 1. Follow IBM in abandoning FDB-Couch, refocus all effort on Erlang-Couch (3.x). 2. Take FDB-Couch development over fully, come up with a story for how FDB-Couch and Erlang-Couch can coexist and when users should choose which one. 3. Hand over the FDB-Couch codebase to an independent team that then can do what they like with it (if this materialises from this discussion). * * * Best Jan — > On 10. Mar 2022, at 17:24, Robert Newson <rnew...@apache.org> wrote: > > Hi, > > For those that are following closely, and particularly those that build or > use CouchDB from our git repo, you'll be aware that CouchDB embarked on an > attempt to build a next-generation version of CouchDB using the FoundationDB > database engine as its new base. > > The principal sponsors of this work, the Cloudant team at IBM, have informed > us that, unfortunately, they will not be continuing to fund the development > of this version and are refocusing their efforts on CouchDB 3.x. > > Cloudant developers will continue to contribute as they always have done and > the CouchDB PMC thanks them for their efforts. > > As the Project Management Committee for the CouchDB project, we are now > asking the developer community how we’d like to proceed in light of this new > information. > > Regards, > Robert Newson > Apache CouchDB PMC >
