> > I understand one needs to get some sort of invitation or make some sort of > > application? > > You need to create a pull request here: <https://github.com/nim-lang/website>
**Before polluting the blog space with some inconclusive ranting ;-)** , I want to continue trying to resolve the issues that I think we agree Nim version 2.0 must stabilize, and that includes **the poor story on threading.** Since I've started researching Nim issues again, my news feed seems to be flooded with Nim articles from up to a couple of years ago (some from before version 1.0), mostly positive reviews but also now no longer correct - the Garbage Collection is always mentioned, and for any reviewer that digs into it, the poor threading model in conjunction with the old (but still default) memory Garbage Collection comes up. @shirleyquirk obviously read the same things and corrected at least one of the more recent reviewers, but there are so many out there getting/giving the wrong impressions. It seems to me we need the stability of a version 2.0 and right quickly, as hopefully its introduction will garner some more publicity where we can announce all the new features and capabilites including Arc/Orc and an easy to use threading story as well as all the things you mention in the OP of this thread. **To that end, I have been trying to push the implementation of the new `std/threadpool` library forward; however, in order to do that I need a working `std/channels` and therein lies the problem** : as [I mentioned in the code I posted further up the thread](https://forum.nim-lang.org/t/7983#52214), there are issues with that code and the clues (and issues on that code filed by others) would indicate a data race, which is confirmed by a few minutes spent reading the code - **there are two locks that can be placed on a channel, one each for sending and receiving, but neither disallow critical variant cases for a small window in each of their execution.** That wasn't the big problem; the big problem is reviewing the code looking into what can be done about it, as follows: 1. The code is purportedly based on a carefully researched and successful implementation of queue/channels from a thirty year old paper, which I read and is in fact good work. However, that paper describes an infinite list based queue and the reason it works without races is that the authors are very clear on how they access the head and tail so as not to cause data races. 2. The `std/channels` implementation (with an intermediate implementation or two) basically throws away all of the things that made the queue in the paper work by implementing a finite array-backed buffer based queue with the resulting data races. 3. If one accepts infinite length channels using lists, then the implementation is easy to fix just by using the true algorithms from the paper. 4. However, if the current implementation using buffer arrays is to be preserved, then the separate locks isn't going to work, or a third lock is needed so as to be likely less efficient that the original (and common) single lock scheme (which is what Go channels use). 5. I normally don't like to "dis" other people's work, but this was really poorly thought out, with premature optimizations before the basic design was actually proven. **My preferred option is to replace the whole file with something that works** (hopefully proving that it is at least as fast as the old implementation and competitive with the fastest of the competition such as that of the Go language. That leads to my rant about the implementation of **what is called a "cache" but is actually allocation from a localized memory pool.** Now I don't say that this application doesn't need help from something similar, especially if we are to use unbuffered channels (elements = 1) in place of `FlowVar[T]` where they will require a new instance every time we `spawn` a thread no matter how small; I object to its **specialized** use. I also object to the `freeChannelCache()` proc, since the memory pool can be created on a per thread basis (essential globals are `{.threadvar.}`) as it just arbitrarily deletes the cache no matter that there may still be running channel instances (channels are now "multi-owner" since they can be pseudo-copied using atomic reference counting). We have had a conversation before about memory allocation where you mentioned that you were thinking of a better memory allocator for Nim (although back then you were still tuning garbage collection ideas); currently it seems that this has evolved into just providing a choice between `malloc` and the default allocator, which is fine. However, **the need for allocation from a sized memory pool is frequent enough that I think that there should exist a `sys/mempool` library** which programmers can use where such use is appropriate and instead of just using raw pointers as here, it should be wrapped in our "destructors" so that we can free the allocation pool without deleting the memory already assigned, leaving their freeing up to their own destructor procs. This `std/channels` library is one place where such a library would be used, but it is a useful optimization for many other applications, whether one selects `malloc` or the default allocator to do the actual allocation. Languages without memory pool allocation are often penalized in competitive benchmarks due to allocation time crippling their performance - this is the one area where C++ benchmarks are often slower than comparable benchmarks using other languages; algorithms that are somewhat functional in their implementation such as the Hamming Numbers one you have included as an Arc/Orc test is such a one as there are many small allocations/deallocations as in several for each element of the "lazy list" of Hamming Numbers. Now, I could just file some issues or contribute to the already existing issues, but **I think that these two issues of what the API of the `std/channels` library should be and whether it makes sense to separate out a `std/mempool` library with its own API need some thought and guidance from you.** The proper procedures were used in the thinking and design that went into the production of the `std/isolation` library via a RFC; I don't think the same processing went into this `std/channels` library, which is a bit of a mess. **I could write a RFC (or two) covering the API of `std/channels` and that of a `std/mempool`, but I don 't want to waste anyone's time if my ideas aren't the direction you and the community thinks we should go...** This is a "vision" of what I think Nim version 2.0 needs, but more than that, it is something to which I can contribute given the go-ahead ;-)
