Re: [Bug-apl] ISO Component File API preview

No, I used the correct acronyms in my email. DBM is a general term referring to (at least) three different products: dbm, ndbm and gdbm. All of these use a similar API. BDB is a different product, and has similarities with DBM in the sense that it too is a key/value store. It has a different API (

Re: [Bug-apl] ISO Component File API preview

Did you mean to name BDB in the second sentence, or is that a limitation of DBM as compared to GDBM. Either way, thanks. I'll start with BDB and compare it to SQLite. On Fri, 2014-07-11 at 13:07 +0800, Elias Mårtenson wrote: > BDB is much heavier than DBM (of which GDBM is an implementation). DBM

Re: [Bug-apl] ISO Component File API preview

BDB is much heavier than DBM (of which GDBM is an implementation). DBM only allows a single process to open the database for writing. I'd like to start with DBM support, as it is simple and efficient, and also because I have more direct experience with it. BDB is cool though, and I can add that as

Re: [Bug-apl] ISO Component File API preview

Are we talking about the same thing? You wrote: "SQLite is also not very efficient if you open multiple databases." I was asking about SQLite's performance with multiple dbs. On Fri, 2014-07-11 at 12:58 +0800, Elias Mårtenson wrote: > No, it does not. At least not as far as I know. Each file cont

Re: [Bug-apl] ISO Component File API preview

Could you also do BDB? I'd like to try both GDBM and BDB. I've quickly skimmed both manuals; it looks like GDBM provides no support for transactions or journals. I'd like to be able to keep that functionality without having to write it myself (at least in the initial implementation). SQL is optio

Re: [Bug-apl] Extension proposal: ⍵⍵ to access outer lambda

On 10 July 2014 08:03, Kacper Gutowski wrote: I think the main problem isn't the length of variable's name but the > fact that regular variable is neither lexically scoped nor localized. > This is exactly my concern. > Using ⍵⍵ might be confusing for Dyalog users where ⍺⍺ and ⍵⍵ are used > as

Re: [Bug-apl] ISO Component File API preview

No, it does not. At least not as far as I know. Each file contains a single key-value store, and there is great flexibility in how to store the keys and values (both are simply an arbitrarily long binary string of bytes). All I can find in the GDBM manual about it is the following, which is only b

Re: [Bug-apl] ISO Component File API preview

I'm certainly willing to give GDBM a try as a replacement for SQLite. Does SQLite bog down with multiple databases? Any idea why? A global lock, perhaps...? On Fri, 2014-07-11 at 12:24 +0800, Elias Mårtenson wrote: > Would you be willing to move the CF stuff on top of GDBM? It's a much > better f

Re: [Bug-apl] ISO Component File API preview

I'll probably put together an initial implementation of a GDBM interface tonight. I'll update you guys with the Github address once the project is created. Regards, Elias On 11 July 2014 12:22, David B. Lamkins wrote: > Thank you for the offer. I'll take a look at both and let you know. :) > >

Re: [Bug-apl] ISO Component File API preview

Would you be willing to move the CF stuff on top of GDBM? It's a much better fit for it, and will give you everything you do with SQLite today, without the extra overhead (SQLite is also not very efficient if you open multiple databases). Regards, Elias On 11 July 2014 12:21, David B. Lamkins w

Re: [Bug-apl] ISO Component File API preview

Thank you for the offer. I'll take a look at both and let you know. :) On Fri, 2014-07-11 at 11:16 +0800, Elias Mårtenson wrote: > On 11 July 2014 10:23, Elias Mårtenson wrote: > > I think what David really wants is an interface to Berkeley > DB. I can build that if you want. >

Re: [Bug-apl] ISO Component File API preview

On Thu, 2014-07-10 at 18:52 -0500, Blake McBride wrote: > > > The component file system is for APL-only applications. The databse > API is to communicate with the rest of the world. It depends on what > you need. > and... > > The problem is that you are creating a component file system th

Re: [Bug-apl] ISO Component File API preview

All right, I can fairly easily build a GNU APL interface to it. Regards, Elias On 11 July 2014 11:40, Blake McBride wrote: > GDBM is interesting. It would be a very fast solution and have a few > advantages over the other stuff we've talked about - i.e. more native. > Interesting! > > > On T

Re: [Bug-apl] ISO Component File API preview

GDBM is interesting. It would be a very fast solution and have a few advantages over the other stuff we've talked about - i.e. more native. Interesting! On Thu, Jul 10, 2014 at 10:16 PM, Elias Mårtenson wrote: > On 11 July 2014 10:23, Elias Mårtenson wrote: > > I think what David really want

Re: [Bug-apl] ISO Component File API preview

On 11 July 2014 10:23, Elias Mårtenson wrote: I think what David really wants is an interface to Berkeley DB > . I can build that if you want. > And, to reply to myself, an even better choice (and more appropriate for this project) is probably GDBM

Re: [Bug-apl] ISO Component File API preview

On Thu, Jul 10, 2014 at 9:23 PM, Elias Mårtenson wrote: > On 11 July 2014 07:52, Blake McBride wrote: > > >> On Thu, Jul 10, 2014 at 6:28 PM, David Lamkins wrote: >> > > >> That's a matter of opinion. I happen to believe that satisfying the spec >>> is an excellent starting position. >>> >> >>

Re: [Bug-apl] ISO Component File API preview

On 11 July 2014 07:52, Blake McBride wrote: > On Thu, Jul 10, 2014 at 6:28 PM, David Lamkins wrote: > > That's a matter of opinion. I happen to believe that satisfying the spec >> is an excellent starting position. >> > > I live in Tenessee. One could make an equal argument that heading east

Re: [Bug-apl] Error building on OSX

On 2014-07-08 18:03:21, Juergen Sauermann wrote: > SOL_SOCKET is 1 on linux but according to the man page of TCP(7) SOL_TCP (=6) > should be used. > I have changed the code to use he number 6 instead of SOL_TCP. SVN 368. Acctually, it should be IPPROTO_TCP as both mandated by POSIX and described i

Re: [Bug-apl] ISO Component File API preview

The short answer is that I'd like to retain compliance with the ISO spec. That means either coming up with conforming extensions (e.g. parsing the "filename" to infer database type) or adding new functions (as has already been done for transaction support). Mixing API's to explicitly accept a data

Re: [Bug-apl] Extension proposal: ⍵⍵ to access outer lambda

I know, but it's much more ugly than my proposal, don't you think? Regards, Elias On 9 July 2014 22:06, Juergen Sauermann wrote: > Hi Elias, > > that would be very easy to implement: > > > > * { ⍵ + { ⍵ × OUTER_OMEGA } 10 ⊣ OUTER_OMEGA←⍵ } 100 1100 * > /// Jürgen > > > > On 07/09/2014 10

Re: [Bug-apl] ISO Component File API preview

Dear David, I will answer in-line: On Thu, Jul 10, 2014 at 6:28 PM, David Lamkins wrote: > No reason to not speak plainly. We're all working toward similar goals. > Only the details differ. But the devil is in the details, isn't it...? :) > > For brevity, I've taken the liberty of excerpting y

Re: [Bug-apl] ISO Component File API preview

On Thu, Jul 10, 2014 at 3:07 PM, Blake McBride wrote: > So, SQL only sees varying length strings. APL sees arbitrary APL nested > (or not nested) arrays. > > > That's what my code does, too. It's not space-efficient, but it's easy to write and virtually impossible to get wrong (unless there's a

Re: [Bug-apl] ISO Component File API preview

No reason to not speak plainly. We're all working toward similar goals. Only the details differ. But the devil is in the details, isn't it...? :) For brevity, I've taken the liberty of excerpting your responses. On Thu, Jul 10, 2014 at 2:22 PM, Blake McBride wrote: > > Okay, I read the spec. I

Re: [Bug-apl] ISO Component File API preview

The SQL layer is independent of APL or what you are storing on it. All SQL sees is a bunch of strings. The interpretation of those strings is GNU APL specific - just like STSC's. No vendor's low level ISAM or VSAM file can communicate with any other vendor's APL ISAM or VSAM files without the sp

Re: [Bug-apl] ISO Component File API preview

The IPSA/STSC file system, also know as BSS (Backing Store (sub)System) ran as an independent process. It so happened that it was only used via the APL interpreter. But were it available to the world at large other programs could have utilized it given the API and the code to access it. IFRC it

Re: [Bug-apl] ISO Component File API preview

On Thu, Jul 10, 2014 at 4:49 PM, Peter Teeson wrote: > > On 2014-07-10, at 5:22 PM, Blake McBride wrote: > > What I've been trying to convey (and clearly have failed on every attempt >> so far) is that an APL component file system must be file-centric. Every >> APL component file system that I'v

Re: [Bug-apl] ISO Component File API preview

On 2014-07-10, at 5:22 PM, Blake McBride wrote: > What I've been trying to convey (and clearly have failed on every attempt so > far) is that an APL component file system must be file-centric. Every APL > component file system that I've worked with or have read about has created a > file to co

Re: [Bug-apl] ISO Component File API preview

Dear David, I will answer in-line below. On Thu, Jul 10, 2014 at 3:16 PM, David Lamkins wrote: > Hi Blake, > > I feel that we're talking at cross purposes. > Yes, I thought of using those exact words. > > What I've been trying to convey (and clearly have failed on every attempt > so far) is

Re: [Bug-apl] Extension proposal: ⍵⍵ to access outer lambda

Note that this would conflict with the use of ⍺⍺ and ⍵⍵ in NARS2000 and Dyalog, where they refer to the operands of a defined operator. With three levels of nested lambdas would you also want to be able to use ⍵⍵⍵ in the innermost one, to refer to the value of ⍵ in the outermost one, and so on? J

Re: [Bug-apl] ISO Component File API preview

Hi Blake, I feel that we're talking at cross purposes. What I've been trying to convey (and clearly have failed on every attempt so far) is that an APL component file system must be file-centric. Every APL component file system that I've worked with or have read about has created a file to contai

Re: [Bug-apl] Extension proposal: ⍵⍵ to access outer lambda

On 9 July 2014 18:08, Jay Foad wrote: Note that this would conflict with the use of ⍺⍺ and ⍵⍵ in NARS2000 > and Dyalog, where they refer to the operands of a defined operator. > Yes, but those are called ⍹ and ⍶ on GNU APL, which makes more sense. > With three levels of nested lambdas would yo

Re: [Bug-apl] ISO Component File API preview

Additionally, the advantage of using an SQL database over writing your own native component file system is that the SQL database handles all the underlying structure (pointers and layout) as well as ordering and access for you. A component file system on top of SQL is utterly trivial compared to t

Re: [Bug-apl] ISO Component File API preview

Dear David, I haven't actually had time to look at your work yet. I am going off of what Elias said, my fair amount of experience with SQL databases, my experience writing the keyed files system, and the comments you have made. Forgive me if I am wrong, but based on your comments, it seems like

Re: [Bug-apl] ISO Component File API preview

Since my last post I've fixed four small bugs (all involving return codes and error signals) and have provided both a functional test and a performance test. This is ready for general use. On Tue, Jul 8, 2014 at 10:22 AM, David Lamkins wrote: > I haven't yet written test scripts, but I've info

[Bug-apl] Fwd: ISO Component File API preview

Oops. I forgot to do a reply all... Also: taking this opportunity to trim replies. :) -- Forwarded message -- From: David Lamkins Date: Thu, Jul 10, 2014 at 9:45 AM Subject: Re: [Bug-apl] ISO Component File API preview To: Blake McBride Please keep in mind that one of the key

Re: [Bug-apl] ISO Component File API preview

David, I feel very strongly for many reasons that creating a new database for each component file is a big, big mistake. The overhead on this has got to be inordinately expensive. Did you see what I did with the keyed file system? It wouldn't be hard to convert that to a component file system at

Re: [Bug-apl] ISO Component File API preview

David's implementation opens a new connection (in SQLite, creates a new database file) for each CF "file". This is not a particularly good idea, and even using SQLite is extremely wasteful in terms of resources. You really only want a single database connection. Even if your'e using a single data

Re: [Bug-apl] ISO Component File API preview

I am not sure where you are going with this, but I thought it worked like Elias said - multiple files per database. This means that once you form the connection with the database - a one time event - no component file operation has to worry about any database or SQL related stuff. It is all autom

Re: [Bug-apl] ISO Component File API preview

Here are some of my own personal notes on PostgreSQL. Hope they help. http://wiki.arahant.com/wiki/Wiki.jsp?page=PostgreSQL On Thu, Jul 10, 2014 at 12:30 AM, David B. Lamkins wrote: > Well, it seems simple enough, but I have zero experience with PostgreSQL > and it's going to take me a while

Re: [Bug-apl] Filter fns with ⍙ (delta-underbar) in the name by default?

Yes, good point. I will drop this. :-) Regards, Elias On 10 July 2014 18:04, Juergen Sauermann wrote: > Hi, > > as to opinions, I believe I don't like that either. > > Please remember that we have user-defined commands now, so you can easily > create your own commands that do things in the way

Re: [Bug-apl] Filter fns with ⍙ (delta-underbar) in the name by default?

Hi, as to opinions, I believe I don't like that either. Please remember that we have user-defined commands now, so you can easily create your own commands that do things in the way you prefer. /// Jürgen On 07/10/2014 03:43 AM, David B. Lamkins wrote: Personally, I'm not a fan of having tool