Re: Reload a DLL and _getptd returns zero
On 01/15/13 22:36, Michael Ost wrote: Do you know what the implications are of using this hack? Would it simply be that any windows code that wants to use FlsAlloc would fail to load with a wine linkage error? Yes, any code that uses the function will fail while trying to call it. Also, is this what the VC runtimes that are statically compiled into a windows DLL do if FlsAlloc is unavailable? For backwards compatibility, I presume...? I guess it's for backwards compatibility. Is the hacky implementation the line: if (NtCurrentTeb()-FlsSlots) NtCurrentTeb()-FlsSlots[index] = 0; It only clears the data in thread that calls FlsFree. It needs to be cleared in every thread that was using it. that I see in FlsFree? Because, if so, that doesn't fix the crash. If not, I'd be interested in seeing your hack because we might just use it. It was written just to check if it helps. It's not usable for anyone.
Re: Reload a DLL and _getptd returns zero
On 01/15/13 00:32, Michael Ost wrote: Here's a link to a zip file that includes the prebuilt .exe and .dll windows binaries that crash in wine. Plus the code. https://dl.dropbox.com/u/97386125/msvcrt-dll-problem.zip The crash is caused by incomplete FlsFree implementation. There's a comment in it's code that says what needs to be added: /* FIXME: add equivalent of ThreadZeroTlsCell here */ I guess it would be best to create a bug describing this issue (http://bugs.winehq.org/). Cheers, Piotr
Re: Reload a DLL and _getptd returns zero
On 01/15/2013 03:39 AM, Piotr Caban wrote: On 01/15/13 00:32, Michael Ost wrote: Here's a link to a zip file that includes the prebuilt .exe and .dll windows binaries that crash in wine. Plus the code. https://dl.dropbox.com/u/97386125/msvcrt-dll-problem.zip The crash is caused by incomplete FlsFree implementation. There's a comment in it's code that says what needs to be added: /* FIXME: add equivalent of ThreadZeroTlsCell here */ Really? Are you sure that's it? By the way, to aid my future Wine sleuthing, can you share any hints on how you figured this out? I guess it would be best to create a bug describing this issue (http://bugs.winehq.org/). OK. Do you know if it is acceptable form to post a windows binary as an example case on the bug? The DLL part of the bug has to be a windows binary. Thanks for the tip! -- mo
Re: Reload a DLL and _getptd returns zero
On 1/15/13 7:59 PM, Michael Ost wrote: On 01/15/2013 03:39 AM, Piotr Caban wrote: The crash is caused by incomplete FlsFree implementation. There's a comment in it's code that says what needs to be added: /* FIXME: add equivalent of ThreadZeroTlsCell here */ Really? Are you sure that's it? Yes. If you need a workaround to run you app you can comment FlsAlloc export. If this function is not available msvcrt will use TlsAlloc. I've also confirmed it by adding a hacky implementation of zeroing the memory on free. By the way, to aid my future Wine sleuthing, can you share any hints on how you figured this out? There's no easy answer to this question. I've looked on the relay log and after figuring out it may be related to Fls* functions I have added some more traces to it. Afterwards it was quite obvious that application was using a value that was supposed to be no longer valid (FlsGetValue was returning old pointer instead of NULL). OK. Do you know if it is acceptable form to post a windows binary as an example case on the bug? The DLL part of the bug has to be a windows binary. I think it's ok if it's your code and compiler license is allowing to publish produced executable. I've already seen some executables attached to bugs. Piotr
Re: Reload a DLL and _getptd returns zero
On 01/15/2013 01:08 PM, Piotr Caban wrote: On 1/15/13 7:59 PM, Michael Ost wrote: On 01/15/2013 03:39 AM, Piotr Caban wrote: The crash is caused by incomplete FlsFree implementation. There's a comment in it's code that says what needs to be added: /* FIXME: add equivalent of ThreadZeroTlsCell here */ Really? Are you sure that's it? Yes. If you need a workaround to run you app you can comment FlsAlloc export. If this function is not available msvcrt will use TlsAlloc. I've I can confirm that commenting out FlsAlloc does fix this crash. Wow! Do you know what the implications are of using this hack? Would it simply be that any windows code that wants to use FlsAlloc would fail to load with a wine linkage error? Also, is this what the VC runtimes that are statically compiled into a windows DLL do if FlsAlloc is unavailable? For backwards compatibility, I presume...? also confirmed it by adding a hacky implementation of zeroing the memory on free. Is the hacky implementation the line: if (NtCurrentTeb()-FlsSlots) NtCurrentTeb()-FlsSlots[index] = 0; that I see in FlsFree? Because, if so, that doesn't fix the crash. If not, I'd be interested in seeing your hack because we might just use it. By the way, to aid my future Wine sleuthing, can you share any hints on how you figured this out? There's no easy answer to this question. I've looked on the relay log and after figuring out it may be related to Fls* functions I have added some more traces to it. Afterwards it was quite obvious that application was using a value that was supposed to be no longer valid (FlsGetValue was returning old pointer instead of NULL). Thanks for this work! -- mo
Re: Reload a DLL and _getptd returns zero
On 01/12/2013 05:21 PM, Dan Kegel wrote: The library in question is using static vc runtime linkage, so _getptd() is linked in That's somewhat worrisome. So you have multiple instances of the C runtime library active in the same app? Maybe you're not unloading that dll gracefully enough. Might be heap corruption. I wonder if Valgrind might be helpful. I second the suggestion to post a minimal testcase. Here's a link to a zip file that includes the prebuilt .exe and .dll windows binaries that crash in wine. Plus the code. https://dl.dropbox.com/u/97386125/msvcrt-dll-problem.zip So, again, the bug is that MSVCRT calls are failing when called from a DLL that has been reloaded. If the calls come from a thread and if the DLL is built with static MSVCRT runtimes. There is no crash for the program in Windows. I also tried rebuilding the code for winelib, and it doesn't appear to have the same behavior. So the problem doesn't happen if Wine's MSVCRT is used, only if a windows DLL that uses static MSVCRT linkage is reloaded. Thanks -- mo
Re: Reload a DLL and _getptd returns zero
On Fri, Jan 11, 2013 at 11:24:55AM -0800, Michael Ost wrote: Hi list, Does anyone know why _getptd() calls would return zero on a DLL that has been reloaded? Does something happen to the TLS used by _getptd? At Muse Research we have been struggling with a deep bug for years where there is a crash _sometimes_ if you load a DLL a second time. Finally (with a big tip of the hat to Julien Pommier at PianoTeq!) we have a test case that demonstrates a (if not _the_) crash. The sequence is: 1. [main] start a thread 2. [main] load a library 3. [thread] call std::cout from the library 4. [main] free the library 5. [main] reload the library 6. [thread] call std::cout from the library - crash! It looks like the crash happens because the MSVCRT function _getptd() is returning zero in step 6. This is dereferenced and crashes. In windows, there is no crash. The library in question is using static vc runtime linkage, so _getptd() is linked in and I don't know exactly what it is doing. But Wine's MSVCRT implementation of _getptd() gets its data from TlsGetValue. And I can see that just before the crash there is a TlsGetValue call that returns zero. Looking at TlsGetValue(), there must be something wrong with the values in TlsSlots. Maybe they persist for DLLs in Windows in a way they don't for Wine...? Or maybe Wine doesn't reinitialize them the same way when the DLL is reloaded? Any hints, thoughts? clues? Thanks! Is this statically linked msvcrt from Wine or Windows? And why free/reload a statically linked library? Ciao, Marcus
Re: Reload a DLL and _getptd returns zero
On 1/13/13 1:17 PM, Marcus Meissner wrote: On Fri, Jan 11, 2013 at 11:24:55AM -0800, Michael Ost wrote: Hi list, Does anyone know why _getptd() calls would return zero on a DLL that has been reloaded? Does something happen to the TLS used by _getptd? At Muse Research we have been struggling with a deep bug for years where there is a crash _sometimes_ if you load a DLL a second time. Finally (with a big tip of the hat to Julien Pommier at PianoTeq!) we have a test case that demonstrates a (if not _the_) crash. The sequence is: 1. [main] start a thread 2. [main] load a library 3. [thread] call std::cout from the library 4. [main] free the library 5. [main] reload the library 6. [thread] call std::cout from the library - crash! It looks like the crash happens because the MSVCRT function _getptd() is returning zero in step 6. This is dereferenced and crashes. In windows, there is no crash. The library in question is using static vc runtime linkage, so _getptd() is linked in and I don't know exactly what it is doing. But Wine's MSVCRT implementation of _getptd() gets its data from TlsGetValue. And I can see that just before the crash there is a TlsGetValue call that returns zero. Looking at TlsGetValue(), there must be something wrong with the values in TlsSlots. Maybe they persist for DLLs in Windows in a way they don't for Wine...? Or maybe Wine doesn't reinitialize them the same way when the DLL is reloaded? Any hints, thoughts? clues? Thanks! Is this statically linked msvcrt from Wine or Windows? It's statically linked in Windows. And why free/reload a statically linked library? We load and unload the DLL that has MSCVRT statically linked into it. Our program is a plugin player, that loads (and unloads) VST sound plugins for musicians to perform with. -- mo
Re: Reload a DLL and _getptd returns zero
On 1/14/2013 01:59, Michael Ost wrote: On 1/13/13 1:17 PM, Marcus Meissner wrote: On Fri, Jan 11, 2013 at 11:24:55AM -0800, Michael Ost wrote: Hi list, Does anyone know why _getptd() calls would return zero on a DLL that has been reloaded? Does something happen to the TLS used by _getptd? At Muse Research we have been struggling with a deep bug for years where there is a crash _sometimes_ if you load a DLL a second time. Finally (with a big tip of the hat to Julien Pommier at PianoTeq!) we have a test case that demonstrates a (if not _the_) crash. The sequence is: 1. [main] start a thread 2. [main] load a library 3. [thread] call std::cout from the library 4. [main] free the library 5. [main] reload the library 6. [thread] call std::cout from the library - crash! It looks like the crash happens because the MSVCRT function _getptd() is returning zero in step 6. This is dereferenced and crashes. In windows, there is no crash. The library in question is using static vc runtime linkage, so _getptd() is linked in and I don't know exactly what it is doing. But Wine's MSVCRT implementation of _getptd() gets its data from TlsGetValue. And I can see that just before the crash there is a TlsGetValue call that returns zero. Looking at TlsGetValue(), there must be something wrong with the values in TlsSlots. Maybe they persist for DLLs in Windows in a way they don't for Wine...? Or maybe Wine doesn't reinitialize them the same way when the DLL is reloaded? Any hints, thoughts? clues? Thanks! Is this statically linked msvcrt from Wine or Windows? It's statically linked in Windows. And why free/reload a statically linked library? We load and unload the DLL that has MSCVRT statically linked into it. Our program is a plugin player, that loads (and unloads) VST sound plugins for musicians to perform with. Not sure if it's related but wine's msvcrt doesn't like to be unloaded, so LdrAddRefDll() is used to prevent that. If I remember correctly there was a problem with messed up standard handles or something like that. -- mo
Re: Reload a DLL and _getptd returns zero
Am 11.01.2013 20:24, schrieb Michael Ost: Hi list, Does anyone know why _getptd() calls would return zero on a DLL that has been reloaded? Does something happen to the TLS used by _getptd? At Muse Research we have been struggling with a deep bug for years where there is a crash _sometimes_ if you load a DLL a second time. Finally (with a big tip of the hat to Julien Pommier at PianoTeq!) we have a test case that demonstrates a (if not _the_) crash. The sequence is: 1. [main] start a thread 2. [main] load a library 3. [thread] call std::cout from the library 4. [main] free the library 5. [main] reload the library 6. [thread] call std::cout from the library - crash! Hi Michael, developers are sometimes quite lazy, provide some testcode (maybe shipping with an executable) and you have better chances, maybe file a bug, not sure. -- Best Regards, André Hentschel
