BDB is much heavier than DBM (of which GDBM is an implementation). DBM only allows a single process to open the database for writing.
I'd like to start with DBM support, as it is simple and efficient, and also because I have more direct experience with it. BDB is cool though, and I can add that as a second phase. Regards, Elias On 11 July 2014 13:04, David B. Lamkins <dlamk...@gmail.com> wrote: > Could you also do BDB? I'd like to try both GDBM and BDB. > > I've quickly skimmed both manuals; it looks like GDBM provides no > support for transactions or journals. I'd like to be able to keep that > functionality without having to write it myself (at least in the initial > implementation). > > SQL is optional to BDB on Linux builds. I can live without the SQL > support... > > > On Fri, 2014-07-11 at 12:24 +0800, Elias Mårtenson wrote: > > Would you be willing to move the CF stuff on top of GDBM? It's a much > > better fit for it, and will give you everything you do with SQLite > > today, without the extra overhead (SQLite is also not very efficient > > if you open multiple databases). > > > > > > Regards, > > Elias > > > > > > On 11 July 2014 12:21, David B. Lamkins <dlamk...@gmail.com> wrote: > > On Thu, 2014-07-10 at 18:52 -0500, Blake McBride wrote: > > > > > > > > > > > The component file system is for APL-only applications. The > > databse > > > API is to communicate with the rest of the world. It > > depends on what > > > you need. > > > > > > > > > and... > > > > > > > > The problem is that you are creating a component file system > > that > > > _depends_ on features of SQLite and is _needlessly_ > > dependent on those > > > features. I understand it works on other databases, but it > > is an > > > unreasonable burden on those systems. You could just as > > easily create > > > the same system that doesn't depend on particulars of > > SQLite. That > > > would work everywhere in a way that is reasonable to any of > > the > > > systems. > > > > > > > > > Ahhh... OK. Perhaps I've had an "aha!" moment. :) > > > > If I understand you correctly, generalizing the current > > component file > > implementation to work with databases other than SQLite is > > problematic > > for all of the reasons you've discussed: poor generality, poor > > use of > > resources, and - in general - a poor fit to SQL. > > > > Therefore, here's my proposal: > > > > I'll drop all further work on generalizing the current > > component file > > API to work with SQL databases. IOW, the component file system > > will > > remain - literally - a component *file* system. The quick > > patch that I > > worked up last night will never become part of my code. > > > > Programmers who want a way to persist data for an APL-only > > single-user > > application and a shallow learning curve and no sysadmin or > > dbadmin > > concerns will (I hope) appreciate and use the simple > > component-file API. > > Programmers who need the functionality of a real database - > > and all that > > it offers in terms of access control, concurrency and remote > > access - > > can use lib_sql with a PostgreSQL backend. > > > > The presence of SQLite will remain an incidental > > implementation detail > > of the component file library. All further concerns specific > > to the use > > of SQLite as an implementation vehicle will be treated as > > out-of-scope > > for the project. > > > > The current component file code can serve as a reference > > implementation > > to validate the design choices not implicitly required by the > > ISO APL > > Annex. There are a number of such choices that I was forced to > > make in > > the absence of detailed guidance from the spec. For example: > > the choice > > to ensure that component IDs assigned by CF_APPEND are always > > nondecreasing for a given component file. There are many other > > detailed > > choices that had to be made. If you have a concern with how > > I've > > implemented the API to meet the spec, we can work out the > > details > > together. > > > > Meanwhile, if someone feels that it's important to layer a > > component > > file API on top of lib_sql *and* allow a choice of database > > backends, > > they may either start from scratch or fork the existing code. > > Personally, I don't see the need for this particular layering > > of > > abstractions; for anything more complex than an > > integer-indexed file of > > APL objects, I'd prefer to work directly with lib_sql. But if > > someone > > else sees value in the layered approach, by all means pursue > > it as a > > separate project. > > > > Meanwhile, I'll continue working on design sketches for a > > component file > > system that doesn't build upon SQLite. The primary reason for > > doing so > > will be to increase performance while not requiring use of > > explicit > > transactions to reduce single-record insert times below the > > 100ms mark. > > Of course the content of the component file will need to > > remain reliable > > in the face of software and hardware failures; that's one of > > the > > implementation attributes that I got "for free" by choosing > > SQLite as a > > short-term implementation tactic. An alternative > > implementation will be > > challenging, no doubt, but worth the effort in the long run. > > Whether > > such an implementation should be done as a native library or > > as a hybrid > > is not clear to me at this time. > > > > The take-away from all of this should be the following: if you > > use the > > existing component file library (and I certainly hope that you > > will do > > so in cases where it best serves your needs), you can be > > assured that > > any future work that I do with the component file library will > > preserve > > your data. Any change in the file format will be accompanied > > by a > > utility workspace to migrate your existing component files. > > > > > > > > This is the most I've been involved with a community > > project. Fun! > > > > > > > > > Indeed. :) > > > > Also, it's worth once again acknowledging Jüergen's work in > > making all > > of this possible. All of these little libraries and utilities > > are > > penny-ante stuff compared to the core interpreter. I just want > > to say, > > again, that I am grateful for the ability to work with a > > full-featured > > open-source APL interpreter for the first time ever. If you > > look at the > > history of free APL systems, they've traditionally been thin > > on the > > ground, undermaintained and not very interesting. > > > > A lot of technologies have developed or matured in the past 30 > > years to > > get to the point where we are today with free APL: Unicode, > > Linux, > > Emacs, gcc, fast hires video and the x86 hardware platform > > being > > foremost among a plethora of candidates. None of us would be > > here on > > this mailing list today, however, had Jüergen not put in an > > immense > > amount of design and implementation effort to create GNU APL. > > I > > sincerely appreciate his continuing support and involvement. > > > > > > > > >
