When I think about how this plays out in a large multithreaded program, it 
worries me.

For example, I used to work on a program that served queries pertaining to 
products for a shopping website. This program used to load tons of data at 
program initialization time: it would load up the database of product 
categories, it loaded tables of statistics about how often various products 
were purchased, it loaded a script that helped guide serving, and so forth. 
There was a lot of data, in hundreds of classes, and it took up gigabytes, so 
keeping multiple copies in RAM wouldn't have been an option.

The libraries that handled this data weren't necessarily written _for_ this 
multithreaded server. For example, the product category database was used in 
dozens of programs, most of which were single threaded. The scripting language 
was just a scripting language - it was used in many programs, again, most of 
which were single-threaded.

Now, I think: what if I had tried to implement this in Nim? The team that 
implemented the product category database would probably have been written it 
in the "normal" Nim style, with refs and garbage collection. Likewise for the 
scripting language. Then, the team that wrote the product server would have 
looked at these libraries and realized that they couldn't use them in a 
multithreaded server.

Of course, I can do true sharing using Boehm (ie, using established libraries 
in a multithreaded program), but the compiler is trying very hard to stop me 
from doing this. I can currently circumvent it with clever use of typecasts, 
but will future versions of the compiler be even more aggressive in trying to 
stop me? Will it always be feasible to use real shared memory in Nim?

I would urge the Nim designers to think of this as the real challenge for 
multithreading: if I already have a library, can I then take the data from that 
library and put it in shared memory? Or do I have to rewrite the library? If 
the latter, then that's seriously limiting to how realistic it is to craft 
large multithreaded programs. 

Reply via email to