It's a little while since I played with Erlang, but the approach used struck me as pretty generic (if unusual), rather than being 'niche'. I think it's early on in Cesarini's Erlang book that he describes how while still a young programmer he had a task which seemed to require a "shared mutable state" that was straightforward in Erlang. He then tried the same thing in C++ and found it cripplingly difficult. I also found the Erlang microthreading approach (and the associated benchmarks) quite impressive, with Java falling over with ten thousand concurrent processes, while Erlang managed 100,000 with ease. This was some years ago. I agree with Raul that you need to pick your language carefully. My 2c is that I've seen any number of distributed projects founder, and in such circumstances Erlang may be worth a second glance (even before you go belly-up :) Another thing to look at is Stackless Python. Jo.
On 28 September 2015 at 16:20, Devon McCormick <[email protected]> wrote: > But does that mean you have to adapt your algo to fit particular > requirements? > > > On Sun, Sep 27, 2015 at 3:59 PM, Jo van Schalkwyk <[email protected] > > > wrote: > > > Re the horrors of "shared mutable state", one consideration is to look at > > Erlang. Joe Armstrong seems to have it about right. What Erlang does is > to > > use 'variables' that can only be written once, and then build from there > > up. It's elegant and seems to prevents the nightmare that Devon > describes. > > > > Regards Jo. > > > > On 28 September 2015 at 07:59, Don Guinn <[email protected]> wrote: > > > > > Why would array programming require a large team? The problem with > large > > > teams is coordination of many people. I would think that if anything > > array > > > programming groups would be smaller. > > > On Sep 27, 2015 11:41 AM, "Devon McCormick" <[email protected]> > wrote: > > > > > > > At a Kdb talk I attended recently, Fintan Quill mentioned that K does > > > > multi-processing but not multi-threading. He didn't go into details > on > > > > this but, based on what they say in this talk and what I've heard > > > > elsewhere, the reason seems obvious: multi-threading is nearly > > impossible > > > > to get to work properly. > > > > > > > > As they say in this talk about how "Shared Mutable State" in > > > multi-threaded > > > > programming should be some of the most feared words in computing, > > Martin > > > > Thompson elaborates "It's a complete nightmare even with really, > really > > > > good people." He then goes on to add "I haven't come across any > > > > multi-threaded application where people are mutating the same space, > > that > > > > isn't bug-ridden." > > > > > > > > So, in short, don't do this. Don't even try to do this: > fine-grained, > > > > multi-threaded parallelism is probably a waste of time with the > current > > > > level of tools we have for it. > > > > > > > > Multi-processing, on the other hand, is dead-easy to do in J for > > anything > > > > that can be parallelized at a coarse-grained level, which is many if > > not > > > > most things. I've written about this extensively over the past few > > years > > > > (all at jcode.jsoftware.com/wiki/): > > > > User:Devon_McCormick/ParallelizedJCodeExamples, > > > > Community/Conference2012/Talks/ParallelSimulationInJ, > > > > NYCJUG/2011-04-12/RedOrBlackGameSimulation > > > > . These latter two give detailed examples on how to achieve > > > > multi-processor parallelization. > > > > > > > > On another note, there's been some mention of processes like Agile > for > > > > speeding up the development process. Keep in mind that the context > for > > > > these kinds of "methodologies" is an idea that a small team might be > > five > > > > people. In the array-programming world, this would be a large team. > > > > > > > > > > > > On Sat, Sep 26, 2015 at 7:19 PM, Don Guinn <[email protected]> > wrote: > > > > > > > > > Given a fork (f g h) f and h can be processed in parallel. But if f > > > and h > > > > > have side effects (shared global names that are modified) then they > > > > should > > > > > not run in parallel. A programming practice that should be avoided > > > > > anyway. This is not a pipeline, but multiple processors could be > > used. > > > > But > > > > > even not running them in parallel I see nothing stating which, f or > > h, > > > is > > > > > run first. So one should avoid code that depended on the order of f > > > and h > > > > > execution anyway. > > > > > > > > > > On Sat, Sep 26, 2015 at 4:42 PM, Raul Miller < > [email protected]> > > > > > wrote: > > > > > > > > > > > On Sat, Sep 26, 2015 at 3:04 PM, Vijay Lulla < > [email protected] > > > > > > > > wrote: > > > > > > > quite easy. But what I'm very unclear about is how does one do > > > > > > > pipelining in J? Say we have functions f, g, and h (all used > > > > > > > monadically) and it is applied like f@g@h y and function g was > > > > > > > particularly costly how can we parallelize (maybe the user has > to > > > > > > > program it himself or the interpreter can do some cost analysis > > > [like > > > > > > > query planning in SQL databases]) it to make it faster? Isn't > > this > > > > > > > what the presenters were mentioning when they were using the > > > example > > > > > > > of airbus pipeline system? > > > > > > > > > > > > "Pipelining" seems to describe a variety of topics. > > > > > > > > > > > > https://en.wikipedia.org/wiki/Pipeline_(computing) > > > > > > > > > > > > So, I would have to say that there is no general technique. If > f, g > > > > > > and h are black boxes, you cannot pipeline them. If you want to > > > > > > reschedule g or make it more efficient, you'll need to know > details > > > > > > about g. The more you know, the greater the odds are that you can > > do > > > > > > it (or the important parts of it) differently, in a more > efficient > > > > > > manner. > > > > > > > > > > > > That said, I should also point out that a lot of the automated > > query > > > > > > planning systems are workarounds for bogus constraints underlying > > the > > > > > > sql standard. That effort could have gone in much more useful > > > > > > directions if people hadn't bought into those ideas. (But at this > > > > > > point, it has turned into a multi-billion dollar industry, so > it's > > > not > > > > > > going away. And there are some applications where the flaws are > not > > > > > > all that important.) > > > > > > > > > > > > Thanks, > > > > > > > > > > > > -- > > > > > > Raul > > > > > > > > > ---------------------------------------------------------------------- > > > > > > For information about J forums see > > > http://www.jsoftware.com/forums.htm > > > > > > > > > > > > > ---------------------------------------------------------------------- > > > > > For information about J forums see > > http://www.jsoftware.com/forums.htm > > > > > > > > > > > > > > > > > > > > > -- > > > > Devon McCormick, CFA > > > > > ---------------------------------------------------------------------- > > > > For information about J forums see > http://www.jsoftware.com/forums.htm > > > > > > > ---------------------------------------------------------------------- > > > For information about J forums see http://www.jsoftware.com/forums.htm > > > > > ---------------------------------------------------------------------- > > For information about J forums see http://www.jsoftware.com/forums.htm > > > > > > -- > Devon McCormick, CFA > ---------------------------------------------------------------------- > For information about J forums see http://www.jsoftware.com/forums.htm > ---------------------------------------------------------------------- For information about J forums see http://www.jsoftware.com/forums.htm
