In my own tests, I drew the conclusion that simply paralellising functions over the arrays, and doing so on a function by function basis results in too much time wasted on bookkeeping and waiting for each step to finish.
In my own prototype implementation, I implemented this by having functions not return a result, but rather a descriptor. These descriptors can then be merged together into a single job that can easily be parallelised. The drawback offers this approach is that setting up the job takes much more time, meaning that small operations becomes slower. It's also a lot of more complex. Finally, in my prototype all arrays are immutable, which neatly fixes the copying problem, but this is approach isn't compatible with APL semantics, so it's effectively a different language. Before anyone asks, no, my prototype is exactly that. Just an implementation of some of the fundamental functions so I could test my idea's. It's not an actual implementation. Regards, Elias On 5 Jul 2017 00:15, "Blake McBride" <blake1...@gmail.com> wrote: > Hi Ellas, > > I remember your earlier work and comments. In fact, that is the source of > my knowledge of this. > > Putting more features on a base with known issues is problematic for the > following reasons: > > 1. The time and effort it would take to secure the base is being spent > elsewhere. > > 2. Adding new features is, in effect, ignoring the problem and moving on. > > 3. APL is good and useful as it is. If it's not being used as-is, then > there is some other issue (like efficiency). I remember another post about > unnecessary bloat because of the unnecessary copying. This was causing his > machine to page unnecessarily. The unnecessary coping is major in terms of > performance and memory efficiency. This is big. > > 4. APL was developed with the promise of parallel processing. This is > something that wasn't practical back then, but is now. I am under the > impression that this was one of the big reasons Juergen created GNU APL - > to take advantage of parallel processing. > > Just one opinion. > > Blake > > > On Tue, Jul 4, 2017 at 11:01 AM, Elias Mårtenson <loke...@gmail.com> > wrote: > >> Hello Blake, >> >> I agree those are important points too. Some time ago, I spent time >> trying to improve those things. Unfortunately, my approach turned out to be >> a dead end. I have some other ideas but those require a significantly >> different underlying architecture. Essentially, it requires a garbage >> collector, which changes things quite a bit. >> >> I'm sure Jürgen can go into more detail, but my impression is that >> avoiding copying is much harder than it seems. >> >> Regards, >> Elias >> >> On 4 Jul 2017 22:57, "Blake McBride" <blake1...@gmail.com> wrote: >> >>> My list would be different: >>> >>> 1. don't clone arrays unnecessarily >>> >>> 2. improve support for parallel processing >>> >>> With these, GNU APL would be much more efficient. I think moving the >>> focus to CPU and memory efficiency is much more important than adding >>> extensions of any sort. >>> >>> --blake >>> >>> >>> On Tue, Jul 4, 2017 at 1:59 AM, Elias Mårtenson <loke...@gmail.com> >>> wrote: >>> >>>> Thank you Jürgen, >>>> >>>> I think we understand each others positions, and agree that they are >>>> not entirely the same. >>>> >>>> That said, your points are very well taken, and for the most part I >>>> actually agree with you. >>>> >>>> I have a wishlist of features that I personally believe are important. >>>> For the most part, these have been discussed previously but here they are >>>> for completeness sake: >>>> >>>> 1. Bignums >>>> 2. Lexical binding. >>>> 3. First-class functions. >>>> 4. Rational numbers >>>> 5. Some kind of easy-to-use imperative structure (i.e. something >>>> better than the horrific :If :Then :Else structure in Dyalog) >>>> 6. Some kind of complex datastructure (again, something better than >>>> the Dyalog classes) >>>> >>>> Note that out of these, the only feature that would change the >>>> semantics compared to the ISO spec is the first, bignums. At least if it's >>>> implemented in the "natural" way. >>>> >>>> I've considered working on some of these myself, but I have no >>>> intention of doing so if you're against these ideas in principle. I >>>> certainly have no desire to maintain my own version. >>>> >>>> Regards, >>>> Elias >>>> >>>> On 4 July 2017 at 03:21, Juergen Sauermann < >>>> juergen.sauerm...@t-online.de> wrote: >>>> >>>>> Hi Elias, >>>>> >>>>> thanks for explaining your position. >>>>> >>>>> My concern about free software is not so much political but more >>>>> practical. >>>>> >>>>> If I look at programming languages, then my impression is that those >>>>> languages that make the >>>>> exchange of software simple are more successful than those that do not. >>>>> >>>>> Historically it has always been possible to exchange APL software from >>>>> one interpreter to another, >>>>> but it was never easy. Most of the code can be exchanged via *.ATF* >>>>> files, but the problems were >>>>> often tiny incompatibilities. These incompatibilities are spread all >>>>> over the code, so getting some >>>>> APL workspace to work on a different machine is still an adventure. >>>>> >>>>> That is why I prefer to stick to the ISO standard, no matter how bad >>>>> it is. As long as you use only >>>>> standardised APL functions you have very few compatibility problems. >>>>> There are some, but they >>>>> are well known. But every new function that is not standardised moves >>>>> you away from portability. >>>>> If I object to implementing some new function than this is not for >>>>> political reasons, but because I >>>>> am afraid that the additional incompatibility makes the exchange of >>>>> APL software more difficult. >>>>> >>>>> In some cases the function is so important that the incompatibility >>>>> has to be accepted. Examples >>>>> for that are certainly ⎕SQL, ⎕FIO, and maybe dyadic ⎕CR and ⎕DLX. >>>>> These functions are almost >>>>> impossible to implement efficiently by APL's own means. >>>>> >>>>> On the other end (from my point of view) we have things like the KEY >>>>> function. I still believe that it >>>>> rather fits into the FinAPL Idiom Library than into GNU APL. It is >>>>> shorter than one APL line and if >>>>> you make it an idiom then it remains portable between all APL >>>>> interpreters while otherwise it is only >>>>> portable between GNU APL and Dyalog APL. >>>>> >>>>> I am open to implementing a feature if it is really useful, but only >>>>> then. Becoming a leader in >>>>> implementing new feature is not one of my priorities. There are enough >>>>> other APLs that are >>>>> keen on that (e.g. Dyalog and NARS, see http://www.nars2000.org). The >>>>> ambition of GNU APL >>>>> has always been to become a stable standard interpreter some day. That >>>>> is difficult enough, and >>>>> we have learned from PL/I how too many features can kill a language. >>>>> And I have seen too many >>>>> software projects that failed due to being overly ambitious. I simply >>>>> do not want to share their fate. >>>>> >>>>> Regarding emacs, I can't help to note that I am not using it, because >>>>> it is, for my taste, too complex. >>>>> I rather prefer something simpler like vi. Sometimes less is just more. >>>>> >>>>> Best Regards, >>>>> /// Jürgen >>>>> >>>>> >>>>> On 07/03/2017 04:00 PM, Elias Mårtenson wrote: >>>>> >>>>> Hello Jürgen, and thanks for your thorough reply. >>>>> >>>>> In terms of the usefulness of Key, I don't disagree with you. I'd >>>>> certainly like to see even more flexible solutions. >>>>> >>>>> Where we do disagree is what the goal of free software is. Arguably >>>>> there are probably as many goals as there are people. >>>>> >>>>> What follows below is an explanation as to why I disagree with your >>>>> assessment as to what is the best for Free Software. Please don't take it >>>>> as personal criticism. You know that I have the deepest respect for you as >>>>> the maintainer and author of GNU APL. >>>>> >>>>> After spending quite some time on the Emacs Development mailing list, >>>>> I have learned quite a bit about what the FSF's goals are with regards to >>>>> what they call "Free Software". Time and time again, RMS has stated that >>>>> the goal of GNU is to make people use commercial software less. In order >>>>> words, if a project can implement a feature that draws people away from >>>>> commercial software towards Free Software, then that is what the project >>>>> should do. >>>>> >>>>> At this point, I'd like to clarify that I am not completely in >>>>> agreement with RMS on this. In the Emacs project, this position has >>>>> prevented Emacs from gaining certain important features, simply because >>>>> they would have made it easier to use "non-free" software together with >>>>> Emacs. This is a position I don't agree with. >>>>> >>>>> I'd really like to see GNU APL become a leader in implementing new >>>>> features. That way perhaps we get more people to switch. The point I'm >>>>> making here is that by implementing useful features that would make people >>>>> choose GNU APL before any alternative, then the project would better serve >>>>> the GNU goals. >>>>> >>>>> Regards, >>>>> Elias >>>>> >>>>> On 3 July 2017 at 21:36, Juergen Sauermann < >>>>> juergen.sauerm...@t-online.de> wrote: >>>>> >>>>>> Hi Elias, >>>>>> >>>>>> thaks. The explanation is a bit clearer but the problems remain. >>>>>> >>>>>> Key is a non-standard APL function and we should be careful with the >>>>>> implementation >>>>>> of non-standard functions. >>>>>> >>>>>> Every function in GNU APL is an invitation to use it. If the function >>>>>> is obviously useful then it improves >>>>>> the language. If it merely solves a particular programming case, then >>>>>> it may improve GNU APL a little, >>>>>> but at the price of incompatibility. Programs using it become less >>>>>> portable and that undermines the >>>>>> goal of free software. >>>>>> >>>>>> So the question in such cases is how useful is a function and is that >>>>>> usefulness worth the incompatibility? >>>>>> >>>>>> In the case of the key function I would say no. >>>>>> >>>>>> First of all the key function can only be used if the data it >>>>>> operates on is organized in a specific way: that >>>>>> the first column is the key. That may be the case but the fact that >>>>>> this is needed is somewhat contrary to >>>>>> how other APL function work. You could also call that arbitrary. >>>>>> >>>>>> That goal can easily achieved by other means. If I have a single >>>>>> *KEY* then something along the lines of >>>>>> >>>>>> *((DATA[1;]≡KEY)⌿KEY)[1;]* >>>>>> >>>>>> will give me the first row (or all rows if I remove the right [1;]) >>>>>> in an array that has that KEY. I suppose that is >>>>>> more or less what the key function does (plus applying some function >>>>>> on that expression). The expression is >>>>>> even superior to a function because it can be used at the left side >>>>>> of an assignment. >>>>>> >>>>>> If that is so then the key function is only one of several APL idioms >>>>>> (see http://aplwiki.com/FinnAplIdiomLibrary >>>>>> for a rather famous list of more than 700 such idioms). Each of the >>>>>> 700+ idioms is useful and would deserver >>>>>> its own symbol, but if we would do so (which is technically possible >>>>>> due to Unicode) then we would have turned >>>>>> GNU APL into an unreadable mess. >>>>>> >>>>>> Best Regards, >>>>>> Jürgen Sauermann >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> On 07/03/2017 05:50 AM, Elias Mårtenson wrote: >>>>>> >>>>>> The key function is better described in the Dyalog reference manual, >>>>>> on page 153 here: http://docs.dyalog.com/1 >>>>>> 6.0/Dyalog%20APL%20Language%20Reference%20Guide.pdf >>>>>> >>>>>> Essentially, it's a grouping function. It's used to create groups of >>>>>> similar things, and apply a function on the individual instances. The >>>>>> examples in the section I referenced above should be pretty clear, I >>>>>> think. >>>>>> >>>>>> Regards, >>>>>> Elias >>>>>> >>>>>> On 3 July 2017 at 00:51, Juergen Sauermann < >>>>>> juergen.sauerm...@t-online.de> wrote: >>>>>> >>>>>>> Hi Elias, >>>>>>> >>>>>>> I am not quite in favour of it and it has problems. >>>>>>> >>>>>>> It is not on my keyboard (even though I am using a Dyalog keyboard). >>>>>>> Not to talk about other keyboards. >>>>>>> >>>>>>> It does not really look like need-to-have function and I suppose it >>>>>>> can be >>>>>>> efficiently performed by a short combination of other APL primitives. >>>>>>> >>>>>>> In my opinion adding primitives for every imaginable use case (and >>>>>>> there are certainly use cases for the key function) leads to an >>>>>>> overloading >>>>>>> of the APL language in the long run and does not improve the >>>>>>> language. >>>>>>> >>>>>>> Another problem is that after reading the description several times, >>>>>>> I still >>>>>>> can't explain in simple terms what the function is actually doing. >>>>>>> That makes it >>>>>>> a good candidate for a never used function if it should ever be >>>>>>> implemented. >>>>>>> >>>>>>> Best Regards, >>>>>>> Jürgen Sauermann >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> On 07/02/2017 06:24 PM, Elias Mårtenson wrote: >>>>>>> >>>>>>> How about implementing the key function, ⌸? >>>>>>> >>>>>>> It's described in this article on the Dyalog site: >>>>>>> https://www.dyalog.com/blog/2015/04/exploring-key/ >>>>>>> >>>>>>> Jürgen, are you in favour of this function? >>>>>>> >>>>>>> Regards, >>>>>>> Elias >>>>>>> >>>>>>> >>>>>>> >>>>>> >>>>>> >>>>> >>>>> >>>> >>> >
