Pascal discussed "... the lack of good parallel, concurrent garbage collectors in Common Lisp implementations".
We were using Lispworks 7 and retrieving lots of data via Oracle's SQL driver prior to building the application. I introduced multi-thread parallelism, attempting to reduce overall time. The times were the same. I then tried multi-image parallelism and total wall time was cut by half for two parallel images, to one third for three, and close to one fourth for four. That was the number of CPUs our build system had. The single-threaded processes were CPU bound. I didn't experiment further to localize the bottleneck. I was thinking of a global lock around allocating memory on the heap or a GC issue. (Surely each thread could run on its own CPU?) I used multiple parallel images to write Lisp data structures in key/value stores to disk or to generate Lisp code from the data to incorporate in the application build. The application's run time was blazingly fast, albeit in single-threaded mode. Clearly, not every application can take advantage of those techniques. Jeff On Thu, Dec 3, 2020 at 7:58 AM Pascal Costanza <p...@p-cos.net> wrote: > > This was primarily for the lack of good parallel, concurrent garbage > collectors in Common Lisp implementations. The CL version of elPrep was > actually still a tad faster than any of the C++, Go, or Java versions, but we > had to work hard to avoid long GC pauses. elPrep allocates a lot of memory, > and the pause time hurts a lot. We solved this by, basically, disabling the > garbage collector, and reusing memory manually as much as possible, which > turned the program into almost a manually memory-managed affair. > > Manual memory management became a huge burden because we wanted to add more > and more components to the software, and then it becomes almost impossible to > predict object lifetimes. > > We evaluated Go and Java for their concurrent, parallel GCs, and C++ for its > reference counting. Interestingly, reference counting is often described as > more efficient than GC, but in our case that’s not true: Because there is a > huge object graph at some stage that needs to be deallocated, reference > counting incurs more or less the same pause that a non-concurrent GC does. > That’s why we don’t expect Rust to fare better here either. > > Again, we’re still prototyping in Common Lisp, which is a huge win, because > this makes us much more productive. > > Pascal > > > On 3 Dec 2020, at 12:16, Svante Carl v. Erichsen <svante.v.erich...@web.de> > > wrote: > > > > Hi! > > > > I vaguely remember having read that you do that. I'm still wondering > > why, though. I guess that you wrote about it, but I can't find it right > > now. > > > > So, if it's not because Common Lisp is not seen as “production ready”, > > why rewrite instead of just adding the production parts (I guess > > hardening, monitoring, logging, documentation etc.)? > > > > Yours aye > > > > Svante > > > > > > Pascal Costanza writes: > > > >> In my opinion, prototyping in Common Lisp, and then translating to a > >> different programming language for creating the final product, is a > >> perfectly valid professional use of Common Lisp. It’s useful to know > >> which programming languages may be good targets for such an approach. > >> > >> This is, of course, not ideal, because this can easily be > >> misunderstood as a statement that Common Lisp is not fit for > >> purpose. However, I don’t see it that way, and you cannot control > >> people’s perceptions. > >> > >> In our particular case, our manager is on board with this approach, > >> and this allows us to pay for regular licenses for LispWorks. The > >> approach works really well for us. > >> > >> Pascal > >> > >> Sent from my iPad > >> > >>> On 3 Dec 2020, at 05:29, Dave Cooper <david.coo...@genworks.com> wrote: > >>> > >>> > >>> > >>>> Where else do Common Lispers go to talk shop, whether CL or something > >>>> else? > >>> > >>> > >>> To me, Common Lispers "talking shop" by definition means talking about CL > >>> or related topics, not an open-ended "something else." I would turn that > >>> question around and ask "where else do Common Lispers go for unapologetic > >>> mutual support for their chosen or imposed computing platform, which is > >>> Common Lisp?" If groups such as this mailing list become diluted with > >>> hand wringing, naysaying, and negativity, then you tell me Tim, where do > >>> actual Common Lispers go? > >>> > >>> > >>>> CL is very good but it is not perfect. Debating the relative merits of > >>>> various languages can lead to cross-pollination of ideas. It appears > >>>> that > >>>> most innovation is happening elsewhere, and I hope this community can > >>>> bring the best of CL into a worthy successor, whatever it may be called. > >>>> > >>> > >>> If "most innovation is happening elsewhere" then those of us who have the > >>> propensity to look into other languages can serve the community here by > >>> reporting back the cool things they find and discussing how we may or may > >>> not be able to co-opt such things into CL. If such is the perspective and > >>> purpose of "debating the merits of various languages," then indeed, such > >>> debate can result in productive cross-pollination, and this is needed and > >>> wanted. > >>> > >>> If the intention and focus is instead to sing the praises of other > >>> environments in order to seek fellow converts or validation for > >>> converting, and doing this while specifically targeting a group set up to > >>> support "professional common lispers," then I consider such efforts to be > >>> unhelpful in the context of this group and I would invite you to take > >>> such discussions into the forums of those other environments or into some > >>> general language discussion forums. > >>> > >>> Understand that not all of us have the "luxury" on the one hand, nor the > >>> desire on the other hand, to chase the dragon of the latest cool thing, > >>> and we look to groups such as this one specifically to support our crusty > >>> old entrenched mentality -- and to improve our environment as best we > >>> can, understanding the inherent limitations that exist. This is the life > >>> we have chosen. > >>> > >>> > > > > > > -- > Pascal Costanza > > >
