No, I used the correct acronyms in my email. DBM is a general term referring to (at least) three different products: dbm, ndbm and gdbm. All of these use a similar API.
BDB is a different product, and has similarities with DBM in the sense that it too is a key/value store. It has a different API (although it is generally similar) and is a bigger product. DBM is certainly much more lightweight and is more like using "plain files" if you like. Regards, Elias On 11 July 2014 13:10, David B. Lamkins <dlamk...@gmail.com> wrote: > Did you mean to name BDB in the second sentence, or is that a limitation > of DBM as compared to GDBM. Either way, thanks. I'll start with BDB and > compare it to SQLite. > > On Fri, 2014-07-11 at 13:07 +0800, Elias Mårtenson wrote: > > BDB is much heavier than DBM (of which GDBM is an implementation). DBM > > only allows a single process to open the database for writing. > > > > > > I'd like to start with DBM support, as it is simple and efficient, and > > also because I have more direct experience with it. BDB is cool > > though, and I can add that as a second phase. > > > > > > Regards, > > Elias > > > > > > On 11 July 2014 13:04, David B. Lamkins <dlamk...@gmail.com> wrote: > > Could you also do BDB? I'd like to try both GDBM and BDB. > > > > I've quickly skimmed both manuals; it looks like GDBM provides > > no > > support for transactions or journals. I'd like to be able to > > keep that > > functionality without having to write it myself (at least in > > the initial > > implementation). > > > > SQL is optional to BDB on Linux builds. I can live without the > > SQL > > support... > > > > > > On Fri, 2014-07-11 at 12:24 +0800, Elias Mårtenson wrote: > > > > > Would you be willing to move the CF stuff on top of GDBM? > > It's a much > > > better fit for it, and will give you everything you do with > > SQLite > > > today, without the extra overhead (SQLite is also not very > > efficient > > > if you open multiple databases). > > > > > > > > > Regards, > > > Elias > > > > > > > > > On 11 July 2014 12:21, David B. Lamkins <dlamk...@gmail.com> > > wrote: > > > On Thu, 2014-07-10 at 18:52 -0500, Blake McBride > > wrote: > > > > > > > > > > > > > > > The component file system is for APL-only > > applications. The > > > databse > > > > API is to communicate with the rest of the world. > > It > > > depends on what > > > > you need. > > > > > > > > > > > > > and... > > > > > > > > > > > The problem is that you are creating a component > > file system > > > that > > > > _depends_ on features of SQLite and is > > _needlessly_ > > > dependent on those > > > > features. I understand it works on other > > databases, but it > > > is an > > > > unreasonable burden on those systems. You could > > just as > > > easily create > > > > the same system that doesn't depend on particulars > > of > > > SQLite. That > > > > would work everywhere in a way that is reasonable > > to any of > > > the > > > > systems. > > > > > > > > > > > > > Ahhh... OK. Perhaps I've had an "aha!" moment. :) > > > > > > If I understand you correctly, generalizing the > > current > > > component file > > > implementation to work with databases other than > > SQLite is > > > problematic > > > for all of the reasons you've discussed: poor > > generality, poor > > > use of > > > resources, and - in general - a poor fit to SQL. > > > > > > Therefore, here's my proposal: > > > > > > I'll drop all further work on generalizing the > > current > > > component file > > > API to work with SQL databases. IOW, the component > > file system > > > will > > > remain - literally - a component *file* system. The > > quick > > > patch that I > > > worked up last night will never become part of my > > code. > > > > > > Programmers who want a way to persist data for an > > APL-only > > > single-user > > > application and a shallow learning curve and no > > sysadmin or > > > dbadmin > > > concerns will (I hope) appreciate and use the simple > > > component-file API. > > > Programmers who need the functionality of a real > > database - > > > and all that > > > it offers in terms of access control, concurrency > > and remote > > > access - > > > can use lib_sql with a PostgreSQL backend. > > > > > > The presence of SQLite will remain an incidental > > > implementation detail > > > of the component file library. All further concerns > > specific > > > to the use > > > of SQLite as an implementation vehicle will be > > treated as > > > out-of-scope > > > for the project. > > > > > > The current component file code can serve as a > > reference > > > implementation > > > to validate the design choices not implicitly > > required by the > > > ISO APL > > > Annex. There are a number of such choices that I was > > forced to > > > make in > > > the absence of detailed guidance from the spec. For > > example: > > > the choice > > > to ensure that component IDs assigned by CF_APPEND > > are always > > > nondecreasing for a given component file. There are > > many other > > > detailed > > > choices that had to be made. If you have a concern > > with how > > > I've > > > implemented the API to meet the spec, we can work > > out the > > > details > > > together. > > > > > > Meanwhile, if someone feels that it's important to > > layer a > > > component > > > file API on top of lib_sql *and* allow a choice of > > database > > > backends, > > > they may either start from scratch or fork the > > existing code. > > > Personally, I don't see the need for this particular > > layering > > > of > > > abstractions; for anything more complex than an > > > integer-indexed file of > > > APL objects, I'd prefer to work directly with > > lib_sql. But if > > > someone > > > else sees value in the layered approach, by all > > means pursue > > > it as a > > > separate project. > > > > > > Meanwhile, I'll continue working on design sketches > > for a > > > component file > > > system that doesn't build upon SQLite. The primary > > reason for > > > doing so > > > will be to increase performance while not requiring > > use of > > > explicit > > > transactions to reduce single-record insert times > > below the > > > 100ms mark. > > > Of course the content of the component file will > > need to > > > remain reliable > > > in the face of software and hardware failures; > > that's one of > > > the > > > implementation attributes that I got "for free" by > > choosing > > > SQLite as a > > > short-term implementation tactic. An alternative > > > implementation will be > > > challenging, no doubt, but worth the effort in the > > long run. > > > Whether > > > such an implementation should be done as a native > > library or > > > as a hybrid > > > is not clear to me at this time. > > > > > > The take-away from all of this should be the > > following: if you > > > use the > > > existing component file library (and I certainly > > hope that you > > > will do > > > so in cases where it best serves your needs), you > > can be > > > assured that > > > any future work that I do with the component file > > library will > > > preserve > > > your data. Any change in the file format will be > > accompanied > > > by a > > > utility workspace to migrate your existing component > > files. > > > > > > > > > > > This is the most I've been involved with a > > community > > > project. Fun! > > > > > > > > > > > > > Indeed. :) > > > > > > Also, it's worth once again acknowledging Jüergen's > > work in > > > making all > > > of this possible. All of these little libraries and > > utilities > > > are > > > penny-ante stuff compared to the core interpreter. I > > just want > > > to say, > > > again, that I am grateful for the ability to work > > with a > > > full-featured > > > open-source APL interpreter for the first time ever. > > If you > > > look at the > > > history of free APL systems, they've traditionally > > been thin > > > on the > > > ground, undermaintained and not very interesting. > > > > > > A lot of technologies have developed or matured in > > the past 30 > > > years to > > > get to the point where we are today with free APL: > > Unicode, > > > Linux, > > > Emacs, gcc, fast hires video and the x86 hardware > > platform > > > being > > > foremost among a plethora of candidates. None of us > > would be > > > here on > > > this mailing list today, however, had Jüergen not > > put in an > > > immense > > > amount of design and implementation effort to create > > GNU APL. > > > I > > > sincerely appreciate his continuing support and > > involvement. > > > > > > > > > > > > > > > > > > > > > >
