My point was perhaps the opposite of how you appear to be taking it. To re-state it: a large team for an array-programming project might be "more than one".
On Sun, Sep 27, 2015 at 2:59 PM, Don Guinn <[email protected]> wrote: > Why would array programming require a large team? The problem with large > teams is coordination of many people. I would think that if anything array > programming groups would be smaller. > On Sep 27, 2015 11:41 AM, "Devon McCormick" <[email protected]> wrote: > > > At a Kdb talk I attended recently, Fintan Quill mentioned that K does > > multi-processing but not multi-threading. He didn't go into details on > > this but, based on what they say in this talk and what I've heard > > elsewhere, the reason seems obvious: multi-threading is nearly impossible > > to get to work properly. > > > > As they say in this talk about how "Shared Mutable State" in > multi-threaded > > programming should be some of the most feared words in computing, Martin > > Thompson elaborates "It's a complete nightmare even with really, really > > good people." He then goes on to add "I haven't come across any > > multi-threaded application where people are mutating the same space, that > > isn't bug-ridden." > > > > So, in short, don't do this. Don't even try to do this: fine-grained, > > multi-threaded parallelism is probably a waste of time with the current > > level of tools we have for it. > > > > Multi-processing, on the other hand, is dead-easy to do in J for anything > > that can be parallelized at a coarse-grained level, which is many if not > > most things. I've written about this extensively over the past few years > > (all at jcode.jsoftware.com/wiki/): > > User:Devon_McCormick/ParallelizedJCodeExamples, > > Community/Conference2012/Talks/ParallelSimulationInJ, > > NYCJUG/2011-04-12/RedOrBlackGameSimulation > > . These latter two give detailed examples on how to achieve > > multi-processor parallelization. > > > > On another note, there's been some mention of processes like Agile for > > speeding up the development process. Keep in mind that the context for > > these kinds of "methodologies" is an idea that a small team might be five > > people. In the array-programming world, this would be a large team. > > > > > > On Sat, Sep 26, 2015 at 7:19 PM, Don Guinn <[email protected]> wrote: > > > > > Given a fork (f g h) f and h can be processed in parallel. But if f > and h > > > have side effects (shared global names that are modified) then they > > should > > > not run in parallel. A programming practice that should be avoided > > > anyway. This is not a pipeline, but multiple processors could be used. > > But > > > even not running them in parallel I see nothing stating which, f or h, > is > > > run first. So one should avoid code that depended on the order of f > and h > > > execution anyway. > > > > > > On Sat, Sep 26, 2015 at 4:42 PM, Raul Miller <[email protected]> > > > wrote: > > > > > > > On Sat, Sep 26, 2015 at 3:04 PM, Vijay Lulla <[email protected]> > > > wrote: > > > > > quite easy. But what I'm very unclear about is how does one do > > > > > pipelining in J? Say we have functions f, g, and h (all used > > > > > monadically) and it is applied like f@g@h y and function g was > > > > > particularly costly how can we parallelize (maybe the user has to > > > > > program it himself or the interpreter can do some cost analysis > [like > > > > > query planning in SQL databases]) it to make it faster? Isn't this > > > > > what the presenters were mentioning when they were using the > example > > > > > of airbus pipeline system? > > > > > > > > "Pipelining" seems to describe a variety of topics. > > > > > > > > https://en.wikipedia.org/wiki/Pipeline_(computing) > > > > > > > > So, I would have to say that there is no general technique. If f, g > > > > and h are black boxes, you cannot pipeline them. If you want to > > > > reschedule g or make it more efficient, you'll need to know details > > > > about g. The more you know, the greater the odds are that you can do > > > > it (or the important parts of it) differently, in a more efficient > > > > manner. > > > > > > > > That said, I should also point out that a lot of the automated query > > > > planning systems are workarounds for bogus constraints underlying the > > > > sql standard. That effort could have gone in much more useful > > > > directions if people hadn't bought into those ideas. (But at this > > > > point, it has turned into a multi-billion dollar industry, so it's > not > > > > going away. And there are some applications where the flaws are not > > > > all that important.) > > > > > > > > Thanks, > > > > > > > > -- > > > > Raul > > > > > ---------------------------------------------------------------------- > > > > For information about J forums see > http://www.jsoftware.com/forums.htm > > > > > > > ---------------------------------------------------------------------- > > > For information about J forums see http://www.jsoftware.com/forums.htm > > > > > > > > > > > -- > > Devon McCormick, CFA > > ---------------------------------------------------------------------- > > For information about J forums see http://www.jsoftware.com/forums.htm > > > ---------------------------------------------------------------------- > For information about J forums see http://www.jsoftware.com/forums.htm > -- Devon McCormick, CFA ---------------------------------------------------------------------- For information about J forums see http://www.jsoftware.com/forums.htm
