There are a couple of logic flaws in your job pool example which are causing
your issues. Nothing particularly Nim specific, but here's a breakdown.
while workID < NSUBMISSIONS: # This will run ceil(NSUBMISSIONS / NTHREADS)
* NTHREADS items
for thread in pool.mitems:
let channel = thread.output.tryRecv() # Need to retry this until done
for each loop or block
if channel.dataAvailable: # assume state == ThreadComm.Ready if
anything received
inc workID
thread.input.send(ThreadComm.Begin)
echo "workId: ", workId # added this to print actual number of jobs run
Run
The first is how you're divvying up the number of jobs. You have a thread pool
size of 8, which doesn't fit evenly into your number of submissions of 100. The
logic in your loop will try and submit `ceil(100 / 8).int * 8 ≈ 104` jobs.
The second logic error introduces the stochastic failure. The
`thread.output.tryRecv()` can run _before_ individual threads are finished. You
would need to keep track of the finished threads, and keep retrying the
unfinished threads. Alternatively you can use `thread.output.recv()` instead
which will block until each thread is done per cycle. Modifying your code to
use `recv` it then outputs 1040 every time with 104 work units done.
In other words, thread pools are tricky and you'd need to rethink your logic
for how you submit work to threads and wait for them for finish. Usually you'll
have one channel (queue) shared between multiple workers which will use system
primitives or other methods to let only one thread grab a work chunk at a time.
