But does that mean you have to adapt your algo to fit particular requirements?
On Sun, Sep 27, 2015 at 3:59 PM, Jo van Schalkwyk <[email protected]> wrote: > Re the horrors of "shared mutable state", one consideration is to look at > Erlang. Joe Armstrong seems to have it about right. What Erlang does is to > use 'variables' that can only be written once, and then build from there > up. It's elegant and seems to prevents the nightmare that Devon describes. > > Regards Jo. > > On 28 September 2015 at 07:59, Don Guinn <[email protected]> wrote: > > > Why would array programming require a large team? The problem with large > > teams is coordination of many people. I would think that if anything > array > > programming groups would be smaller. > > On Sep 27, 2015 11:41 AM, "Devon McCormick" <[email protected]> wrote: > > > > > At a Kdb talk I attended recently, Fintan Quill mentioned that K does > > > multi-processing but not multi-threading. He didn't go into details on > > > this but, based on what they say in this talk and what I've heard > > > elsewhere, the reason seems obvious: multi-threading is nearly > impossible > > > to get to work properly. > > > > > > As they say in this talk about how "Shared Mutable State" in > > multi-threaded > > > programming should be some of the most feared words in computing, > Martin > > > Thompson elaborates "It's a complete nightmare even with really, really > > > good people." He then goes on to add "I haven't come across any > > > multi-threaded application where people are mutating the same space, > that > > > isn't bug-ridden." > > > > > > So, in short, don't do this. Don't even try to do this: fine-grained, > > > multi-threaded parallelism is probably a waste of time with the current > > > level of tools we have for it. > > > > > > Multi-processing, on the other hand, is dead-easy to do in J for > anything > > > that can be parallelized at a coarse-grained level, which is many if > not > > > most things. I've written about this extensively over the past few > years > > > (all at jcode.jsoftware.com/wiki/): > > > User:Devon_McCormick/ParallelizedJCodeExamples, > > > Community/Conference2012/Talks/ParallelSimulationInJ, > > > NYCJUG/2011-04-12/RedOrBlackGameSimulation > > > . These latter two give detailed examples on how to achieve > > > multi-processor parallelization. > > > > > > On another note, there's been some mention of processes like Agile for > > > speeding up the development process. Keep in mind that the context for > > > these kinds of "methodologies" is an idea that a small team might be > five > > > people. In the array-programming world, this would be a large team. > > > > > > > > > On Sat, Sep 26, 2015 at 7:19 PM, Don Guinn <[email protected]> wrote: > > > > > > > Given a fork (f g h) f and h can be processed in parallel. But if f > > and h > > > > have side effects (shared global names that are modified) then they > > > should > > > > not run in parallel. A programming practice that should be avoided > > > > anyway. This is not a pipeline, but multiple processors could be > used. > > > But > > > > even not running them in parallel I see nothing stating which, f or > h, > > is > > > > run first. So one should avoid code that depended on the order of f > > and h > > > > execution anyway. > > > > > > > > On Sat, Sep 26, 2015 at 4:42 PM, Raul Miller <[email protected]> > > > > wrote: > > > > > > > > > On Sat, Sep 26, 2015 at 3:04 PM, Vijay Lulla <[email protected] > > > > > > wrote: > > > > > > quite easy. But what I'm very unclear about is how does one do > > > > > > pipelining in J? Say we have functions f, g, and h (all used > > > > > > monadically) and it is applied like f@g@h y and function g was > > > > > > particularly costly how can we parallelize (maybe the user has to > > > > > > program it himself or the interpreter can do some cost analysis > > [like > > > > > > query planning in SQL databases]) it to make it faster? Isn't > this > > > > > > what the presenters were mentioning when they were using the > > example > > > > > > of airbus pipeline system? > > > > > > > > > > "Pipelining" seems to describe a variety of topics. > > > > > > > > > > https://en.wikipedia.org/wiki/Pipeline_(computing) > > > > > > > > > > So, I would have to say that there is no general technique. If f, g > > > > > and h are black boxes, you cannot pipeline them. If you want to > > > > > reschedule g or make it more efficient, you'll need to know details > > > > > about g. The more you know, the greater the odds are that you can > do > > > > > it (or the important parts of it) differently, in a more efficient > > > > > manner. > > > > > > > > > > That said, I should also point out that a lot of the automated > query > > > > > planning systems are workarounds for bogus constraints underlying > the > > > > > sql standard. That effort could have gone in much more useful > > > > > directions if people hadn't bought into those ideas. (But at this > > > > > point, it has turned into a multi-billion dollar industry, so it's > > not > > > > > going away. And there are some applications where the flaws are not > > > > > all that important.) > > > > > > > > > > Thanks, > > > > > > > > > > -- > > > > > Raul > > > > > > > ---------------------------------------------------------------------- > > > > > For information about J forums see > > http://www.jsoftware.com/forums.htm > > > > > > > > > > ---------------------------------------------------------------------- > > > > For information about J forums see > http://www.jsoftware.com/forums.htm > > > > > > > > > > > > > > > > -- > > > Devon McCormick, CFA > > > ---------------------------------------------------------------------- > > > For information about J forums see http://www.jsoftware.com/forums.htm > > > > > ---------------------------------------------------------------------- > > For information about J forums see http://www.jsoftware.com/forums.htm > > > ---------------------------------------------------------------------- > For information about J forums see http://www.jsoftware.com/forums.htm > -- Devon McCormick, CFA ---------------------------------------------------------------------- For information about J forums see http://www.jsoftware.com/forums.htm
