Knowing where to put the primitives is indeed the issue. That's why
they're primitives, basically.
Conceptually, the primitives are simple. T. is for configuration and
setup. You always have one thread (the master thread), and you
probably should have one additional thread for each cpu "core" (which
you might think of as a register set -- a small set of variables that
represent the machine itself).
To accomplish this, you might use a routine like
setthreadcount=: {{
change=. y-1 T.''
T.&''"0 (|change)#(*change){0 0 55
1 T.''
}}
setthreadcount <:{.8T.''
Note that the value returned by this routine may be higher than
expected if you had threads in use running tasks at the time. For
example, if you had 32 threads running tasks on a 28 core machine, you
might expect this routine to return 27. I could have named it
'setfuturethreadcount" to try to convey that it would eventually
settle down to that number of threads, but as it is the name is
already longer than I really like.
---------------
But threads are not tasks. To fire off tasks you use (t.), and
(besides timing issues), (t.'') is basically the same as (<@).
To get a handle on the timing side of things, it's probably best to
use timex. That said, I haven't found any good candidates for
threading, yet. (By this I mean: simple code which gets improved
performance using (t.'') (or (t.{{)n}})) vs using (<@)
(That said, tasks can also be useful in some academic contexts, even
without a performance gain.)
I hope this helps,
--
Raul
On Tue, Dec 20, 2022 at 2:06 PM Ak O <akin...@gmail.com> wrote:
>
> Hi Devon,
>
> Thank you for the iterations of your exercise. I have studied these posts
> along with your earliest proto examples in the multi-threading forum posts.
> My challenge is understanding which Primitives go where in the context of
> building up a threading routine for a directed task.
>
> Considering your photo flipping example:
>
> 1. I might structure a scheduling thread that:
>
> a) Assigns threads to process the files within a folder. Each of which
> checking if there are remaining files within the folder. Alternatively
> spinning down if none remain.
>
> b) Assigns threads to process folders. Each processing folders and
> returning to check the scheduler for remaining unprocessed folders.
> Alternatively spinning down in the absence of unprocessed folders.
>
> c) Batches files into groups of a given size class. Assigning subsets of
> threads to unprocessed size classes.
>
> d) Along with many others.
>
> All of which having trade-offs based on the processing question.
>
>
>
> So knowing what the pieces are and how to properly place them is my issue.
>
>
> Ak
>
>
>
>
> On Tue., DecHi . 20, 2022, 10:34 Devon McCormick, <devon...@gmail.com>
> wrote:
>
> > Hi Ak,
> >
> > That's a lot of questions. I, too, would like to better understand pyxs.
> >
> > In the meantime, here are some things I've written up on multithreading:
> > https://code.jsoftware.com/wiki/NYCJUG/2022-05-10#Trying_to_Multithread,
> >
> > https://code.jsoftware.com/wiki/NYCJUG/2022-07-12#Multi-threaded_Photo_Flipping
> > ,
> > and
> >
> > https://code.jsoftware.com/wiki/NYCJUG/2022-08-09#A_New_Multi-threading_Project
> > .
> >
> > It looks like I'm using the "Manager Model" in these exercises.
> >
> > Good luck!
> >
> > Devon
> >
> > On Tue, Dec 20, 2022 at 12:13 PM Ak O <akin...@gmail.com> wrote:
> >
> > > I hope you are all well. Thank you for your hard work on this subject.
> > >
> > > I am having difficulty understanding how to correctly apply the T. and t.
> > > (tcapdot/tdot) Primitives based on the vocabulary documentation pages.
> > >
> > >
> > > The concept of threading is new to me. The documentation pages (T./t.)
> > use
> > > many terms that I am unclear on.
> > >
> > > I appreciate your efforts to explain them. Thank you for your help.
> > >
> > >
> > > Ak.
> > >
> > >
> > >
> > > Vocabulary_tdot Page
> > >
> > >
> > > Reading from the Vocabulary documentation page tdot (at indentation).
> > >
> > >
> > > i. t. ''"0 i. 5
> > > ++-+---+-----+-------+
> > >
> > > ||0|0 1|0 1 2|0 1 2 3|
> > > ++-+---+-----+-------+
> > >
> > > In the context of threading, what is this statement meant to
> > characterize?
> > > Is there a sentence of the form ,'execute some operation in some distinct
> > > thread (having thread attribute(s)..).'?
> > >
> > > [x] u t. n y runs ([x] <@:u y) in an idle thread and returns its
> > > result, which is a pyx. A pyx is a special box. It looks like an atomic
> > box
> > > and can be used like any other box. When the pyx is opened, the opening
> > > thread will block until the value is available. If the creation of the
> > pyx
> > > ended in error, that error will be signaled when the pyx is opened.
> > >
> > >
> > > What is:
> > > ->the purpose of a 'pyx'?
> > > Where does a 'pyx' fit into the threading concept?
> > > What is:
> > > ->thread?
> > > ->idle thread?
> > > How is a thread identified?
> > >
> > >
> > > The n argument to t. controls execution of the task. The format is
> > >
> > > [[<]important parameters][,< keyword[;value]]...
> > > The important parameters are: [threadpool#]
> > >
> > > What is:
> > > -> a task?
> > > -> important parameters?
> > > -> [threadpool#]
> > > -> threadpool
> > > -> i. t.(0;('worker'))"_1 i.5
> > >
> > >
> > > threadpool# indicates which threadpool the task should be assigned
> > to.
> > >
> > > How is a threadpool assigned?
> > > How is a threadpool identified?
> > >
> > >
> > > Threads are assigned to threadpools when they are created.
> > >
> > > How are assigned threadpools identified for created threads?
> > >
> > >
> > > By default, threadpool 0 is used.
> > >
> > > What is the equivalent representation for this statement (i.e [x] u t. n
> > y
> > > runs ([x] <@:u y))?
> > >
> > >
> > > One keyword has been defined.
> > >
> > > What is a 'keyword' (conceptually and type)?
> > > Is it predefined?
> > >
> > > The worker keyword controls whether the task should be executed in
> > the
> > > originating thread if there is no waiting thread for it to run in.
> > >
> > > How does the keyword control the task executing thread?
> > >
> > >
> > > u t. 'worker'
> > > u t. (<'worker')
> > > u t. ('worker';1)
> > >
> > > What differentiates these forms?
> > > Is it always 'worker' a standin?
> > > A few words on cases where one form might be chosen instead of another.
> > >
> > >
> > >
> > > --->
> > > Reference questions
> > >
> > > In trying to understand how to build threading structures (threaded
> > > program), three basic models are given:
> > > (https://hpc-tutorials.llnl.gov/posix/designing_threaded_programs)
> > > Quote:
> > > Several common models for threaded programs exist:
> > >
> > > Manager/worker: a single thread, the manager assigns work to
> > other
> > > threads, the workers.
> > > Typically, the manager handles all input and parcels out work to the
> > > other
> > > tasks. At least two forms of the manager/worker model are common:
> > > static
> > > worker pool and dynamic worker pool.
> > >
> > > Pipeline: a task is broken into a series of suboperations, each
> > of
> > > which is handled
> > > in series, but concurrently, by a different thread. An automobile
> > > assembly
> > > line best describes this model.
> > >
> > > Peer: similar to the manager/worker model, but after the main
> > > thread creates other
> > > threads, it participates in the work.
> > >
> > > Can you please show or diagram the J primitives used to compose the three
> > > models?
> > >
> > >
> > > --->
> > >
> > >
> > > - - - Please give a Primitives translation from C to J under the Creating
> > > Pthreads heading, Reference Source (https://randu.org/tutorials/threads/
> > ).
> > > - - -
> > >
> > > - - - Please give a Primitives translation from C to J under the Pthreads
> > > Attributes heading, Reference Source (
> > https://randu.org/tutorials/threads/
> > > ).
> > > - -
> > > ----------------------------------------------------------------------
> > > For information about J forums see http://www.jsoftware.com/forums.htm
> > >
> >
> >
> > --
> >
> > Devon McCormick, CFA
> >
> > Quantitative Consultant
> > ----------------------------------------------------------------------
> > For information about J forums see http://www.jsoftware.com/forums.htm
> >
> ----------------------------------------------------------------------
> For information about J forums see http://www.jsoftware.com/forums.htm
----------------------------------------------------------------------
For information about J forums see http://www.jsoftware.com/forums.htm
