Replying to myself... On 1/26/18 11:48 PM, Lionel Debroux wrote: > Hi Scott, > > On 1/26/18 7:05 AM, Scott Kitterman wrote: > > On Thursday, January 25, 2018 11:59:06 PM Lionel Debroux wrote: > > > > > > [...] > > > --- > > > Do you think we should start the journey of getting rid of > > > libdb5.3 at a wide scale ? And if so, how to optimize resource > > > usage in general ? :) > > > --- > > > > Ultimately BDB is a dead end for non-AGPL projects. So my answer to > > your first question is a definite yes. > > > > I'd like to know what the preferred replacement is. I maintain a > > few less heavily used packages that use libdb5.3 and I need to know > > what to tell upstream they should port to. I don't know enough to > > have a real technical opinion. Is lmdb the general solution? > In some (most ?) cases, it would seem so, indeed. > But there's now a wider variety of key/value databases (even those > which don't fork and communicate through *nix or network sockets) > than, say, a decade ago. I remember that several years ago, > bitcoind/bitcoin-qt switched from BDB to LevelDB. > > > As far as postfix goes (which I also co-maintain) that is a two > > release cycle project (it's complicated, but upgrades don't work > > otherwise - if anyone cares see what we did for postfix-sqlite. > > It's no problem to switch to a difference default map type, but > > it'd be nice if we could switch it once to something that was > > otherwise already likely to be installed. > liblmdb* or libleveldb* are much less popular in popcon by_inst than > libdb, yeah... > > > Do we know whether LMDB, and other candidate databases for replacing > BDB, have received suitable hardening against data corruption and > fuzzing ? Well, now I know: some parts of LMDB have not received the appropriate amount of hardening against data corruption.
A direct LMDB translation of the simple BDB fuzzing run I posted in Debian BTS #652036: # apt install lmdb-utils $ rm -f input* $ echo -e "key\nvalue" | mdb_load -T -n input $ zzuf -qcs 0:1000 -C 10 -U 3 mdb_dump -n -l input $ zzuf -qcs 0:1000 -C 10 -U 3 mdb_dump -n -a input $ zzuf -qcs 0:1000 -C 10 -U 3 mdb_stat -n -e -fff -r -a input yields an interesting assortment of SIGFPE, SIGABRT and SIGSEGV... That's under sid amd64, with lmdb-utils / liblmdb0 0.9.21-1, which packages the latest upstream release. I replaced the sources with the latest ones from https://github.com/LMDB/lmdb , since the commit logs mention e.g. "fix FIRST_DUP/LAST_DUP cursor bounds check". AFAICS, it changes the set of symbols, but not the outcome of zzuf... and unsurprisingly, afl-fuzz following a build with afl-clang-fast destroys both mdb_dump and mdb_stat within 2 seconds. Oh well... time to report the issues, after letting afl run for a little while longer, probably until tomorrow morning. Bye, Lionel.