Hi all, Apologies for the delay in sending this email but I have been trying to implement and test an alternative and wanted to be sure it works before sending this off.
So, as Matthew suggested this problem has to do with memory allocation. The --no-alloc option in Matthew's suggested snippet does not show the delay I usually see in the thread CPU usage although thread creation is still quite slow past around 20 places. I started developing loci [1] to solve this problem instance yesterday and I got it to a point where I can prove that subprocesses solve the problem I am seeing. No point attaching a screenshot of htop with all bars full to 100%... that's what happens. Also, process creation is almost instantaneous and there's no delay compared to threads. In the evening after I had almost everything sorted, Sam suggested on Slack that I try distributed-places and use them locally. I haven't tried this and I cannot say if it works better or worse but it seems certainly harder to use than loci as my library uses the same API as places. Part of the development was pretty quick because I noticed Matthew had been playing with this before: https://github.com/racket/racket/blob/master/pkgs/racket-benchmarks/tests/racket/benchmarks/places/place-processes.rkt (might be worth noting that the code doesn't work with current racket) I will adding contracts, tests and documentation throughout the week and then replace places in my system with loci so I can dog-food the library. Next step is to add remote loci at which point I will want to compare with distributed-places and possibly improve on it. If anyone has comments, suggestions or complaints on the library please let me know but keep in mind it's barely a day old. Paulo Matos 1: https://github.com/LinkiTools/racket-loci https://pkgd.racket-lang.org/pkgn/search?q=loci On 05/10/2018 19:23, Matthew Flatt wrote: > At Fri, 5 Oct 2018 17:55:47 +0200, Paulo Matos wrote: >> Matthew, Sam, do you understand why this is happening? > > I still think it's probably allocation, and probably specifically > content on the process's page table. Do you see different behavior with > a non-allocating variant (via `--no-alloc` below)? > > We should certainly update the documentation with information about the > limits of parallelism via places. > > ---------------------------------------- > > #lang racket > > (define (go n alloc?) > (place/context p > (let ([v (vector (if alloc? 0.0 0))] > [inc (if alloc? 1.0 1)]) > (let loop ([i 3000000000]) > (unless (zero? i) > (vector-set! v 0 (+ (vector-ref v 0) inc)) > (loop (sub1 i))))) > (printf "Place ~a done~n" n) > n)) > > (module+ main > (define alloc? #t) > (define cores > (command-line > #:once-each > [("--no-alloc") "Non-allocating variant" (set! alloc? #f)] > #:args (cores) > (string->number cores))) > > (time > (map place-wait > (for/list ([i (in-range cores)]) > (printf "Starting core ~a~n" i) > (go i alloc?))))) > -- Paulo Matos -- You received this message because you are subscribed to the Google Groups "Racket Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to racket-users+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
