Re: Garbage Collection Issue
On Monday, 1 June 2020 at 12:37:05 UTC, Steven Schveighoffer wrote: I was under the impression that TLS works by altering a global pointer during the context switch. I didn't think accessing a variable involved a system call. For sure they are slower than "normal" variables, but how much slower? I'm not sure. It depends, there several different optimizations possible. This is essentially the difference between the -fPIC and -fpie flag GNU compilers. -fpie can optimize TLS so that it is an offset from a certain register (fs or gs with x86). Otherwise the compiler insert __tls_get_addr. Typically shared objects gets this call, but the executable can optimize. So if druntime is a shared object, it will use __tls_get_addr. TLS variables will not be major hit if used moderately, used in a loop, then you will certainly see a performance hit. This can only take you so far, when the language uses TLS by default. The GC has to support scanning TLS and so it uses TLS to track thread-specific data. Yes, this was some of the annoyance I had when porting druntime. The thread startup code needed to use link library (like elf/link.h for linux) in order to obtain the entire TLS area (areas because there a several of them). This includes scanning sections during startup and it becomes even more complicated with runtime loaded modules. Basically there is a lot of boiler plate in druntime just for reading the executable format. druntime has tons of elf stuff in it just to load a program something I'm not too keen on, because it's a lot of code and you need to support all the quirks with different CPU archs and operating systems. You'd want druntime to be more OS agnostic a let the OS services deal with the TLS stuff. The only upside can be that you can have a full symbolic stack trace during aborts when a poking in the executable formats. Well, that's how it is because of GC and there is not really any way around it. A non tracing GC would not have this requirement though. When you dig into these details you realize how heavy the D language really is and some solutions get negative beauty points.
Re: Garbage Collection Issue
On 6/1/20 6:51 AM, IGotD- wrote: On Sunday, 31 May 2020 at 16:57:06 UTC, Steven Schveighoffer wrote: I can't imagine much of druntime working at all without TLS. Indeed, it is a requirement these days. I believe that's where these roots are being stored. I would really like if druntime could remove its TLS variables as much as possible. TLS is really a complicated solution underneath and druntime makes it even more complicated. It requires a hook in thread creation since the raw TLS specification only applies simple variables that can be initialized using memcpy/memset. Any thread that is created outside the druntime will fail if D supports "complex" TLS variables. D can use non-D created threads, but they will not be scanned by the GC, or run thread static constructors or destructors. TLS variables are also slower that normal variables since it often requires a system call in order to obtain the variable. I was under the impression that TLS works by altering a global pointer during the context switch. I didn't think accessing a variable involved a system call. For sure they are slower than "normal" variables, but how much slower? I'm not sure. druntime should use stack variables much it can and/or shared variables. druntime does not needlessly use TLS as far as I know. If you find a case that can be switched please file a bug report. If you ever encounter a TLS variable which is global variable in D, try to see if you can solve it with a stack or shared variable. This can only take you so far, when the language uses TLS by default. The GC has to support scanning TLS and so it uses TLS to track thread-specific data. What it sounds like to me is that the OP implemented a "get it to compile" solution for TLS, and this is not working for him. There is no removing TLS, because the language uses it directly for global variables, and many guarantees are enabled by it. For instance the array append runtime uses a lock-free TLS cache to ensure speedy appending. Without TLS, the global lock would be needed for every append. -Steve
Re: Garbage Collection Issue
On 6/1/20 5:53 AM, a11e99z wrote: On Sunday, 31 May 2020 at 16:57:06 UTC, Steven Schveighoffer wrote: I can't imagine much of druntime working at all without TLS. Indeed, it is a requirement these days. TLS is evil for async/await when any thread can execute any fiber (case where fiber tied to thread is wrong/dead version of async/await cuz 1st thread has 1000 fibers and 2nd only 10) That would require fiber local storage, which I don't know if that is supported. I think fibers jumping between threads is also not supported, but I don't know for certain. -Steve
Re: Garbage Collection Issue
On Sunday, 31 May 2020 at 16:57:06 UTC, Steven Schveighoffer wrote: I can't imagine much of druntime working at all without TLS. Indeed, it is a requirement these days. I believe that's where these roots are being stored. -Steve I would really like if druntime could remove its TLS variables as much as possible. TLS is really a complicated solution underneath and druntime makes it even more complicated. It requires a hook in thread creation since the raw TLS specification only applies simple variables that can be initialized using memcpy/memset. Any thread that is created outside the druntime will fail if D supports "complex" TLS variables. TLS variables are also slower that normal variables since it often requires a system call in order to obtain the variable. druntime should use stack variables much it can and/or shared variables. If you ever encounter a TLS variable which is global variable in D, try to see if you can solve it with a stack or shared variable.
Re: Garbage Collection Issue
On Sunday, 31 May 2020 at 16:57:06 UTC, Steven Schveighoffer wrote: I can't imagine much of druntime working at all without TLS. Indeed, it is a requirement these days. TLS is evil for async/await when any thread can execute any fiber (case where fiber tied to thread is wrong/dead version of async/await cuz 1st thread has 1000 fibers and 2nd only 10)
Re: Garbage Collection Issue
On 5/30/20 9:51 PM, Marius Cristian Baciu wrote: I am encountering a strange problem with the GC on a specific platform: at the first attempt to clear the current memory pool to make room for a new allocation, the GC considers that the page in which the main thread resides (the one created in the init function of the GC) can be freed.. therefore, frees the entire pool and reallocates at the same location; later, when accessing thread's address, it stumbles upon garbage data. The question is: where does the GC expects the address of the thread to be found so that it takes it into consideration? A relevant mention would be that the platform doesn't support TLS so it won't find anything when trying to access that data. Could it be related to this? I can't imagine much of druntime working at all without TLS. Indeed, it is a requirement these days. I believe that's where these roots are being stored. -Steve
Garbage Collection Issue
I am encountering a strange problem with the GC on a specific platform: at the first attempt to clear the current memory pool to make room for a new allocation, the GC considers that the page in which the main thread resides (the one created in the init function of the GC) can be freed.. therefore, frees the entire pool and reallocates at the same location; later, when accessing thread's address, it stumbles upon garbage data. The question is: where does the GC expects the address of the thread to be found so that it takes it into consideration? A relevant mention would be that the platform doesn't support TLS so it won't find anything when trying to access that data. Could it be related to this?
