Re: Valgrind
Hi, well I just found my old post and have the same question again. Any news / new insights on this? On 2015-04-20 13:16:23 +, Robert M. Münch said: Hi, I just found quite old posts about Valgrind and D. Can someone give me a short update, what the state of support for D is and if there is anythings special to take into account. Thanks a lot. -- Robert M. Münch http://www.saphirion.com smarter | better | faster
Re: Valgrind
On Monday, 20 April 2015 at 13:29:58 UTC, John Colvin wrote: On Monday, 20 April 2015 at 13:28:57 UTC, John Colvin wrote: On Monday, 20 April 2015 at 13:16:23 UTC, Robert M. Münch wrote: Hi, I just found quite old posts about Valgrind and D. Can someone give me a short update, what the state of support for D is and if there is anythings special to take into account. Thanks a lot. The only special thing to take in to account is that valgrind will choke on DMD generated floating point code, so really you have to use GDC or LDC if you want to use valgrind. Correction: The only special thing I know of. AFAIK, this has been fixed a long time ago.
Re: Valgrind
On Monday, 20 April 2015 at 17:25:55 UTC, John Colvin wrote: On Monday, 20 April 2015 at 16:58:18 UTC, Robert M. Münch wrote: On 2015-04-20 13:29:57 +, John Colvin said: Were the causes ever analyzed? I'm a bit wondering why it happens on floating point stuff... valgrind doesn't have full support for x87 code, which dmd emits all over the place. There is company is Germany, which does Valgrind consultancy, http://www.open-works.net/contact.html which could fix this issue, if you are prepared to throw some money their way. Nick
Re: Valgrind
On Monday, 20 April 2015 at 13:28:57 UTC, John Colvin wrote: The only special thing to take in to account is that valgrind will choke on DMD generated floating point code I actually fixed this problem a while ago. https://github.com/D-Programming-Language/dmd/pull/4368 An actual problem with valgrind is the GC, because most of it's operations appear to valgrind as memory corruptions. You can GC.disable() collections, use gcstub.d, or help Vladimir with his valgrind/GC support to make things work. http://dlang.org/phobos/core_memory.html#.GC.disable https://github.com/D-Programming-Language/druntime/blob/master/src/gcstub/gc.d https://github.com/CyberShadow/druntime/commits/valgrind
Re: Valgrind
Valgrind has a mechanism for teaching it how to ignore certain patterns. A long time ago I setup suppressions for the gc, but the code has changed out from under that version so the work would need to be redone. On 4/20/2015 7:23 PM, Martin Nowak via Digitalmars-d-learn wrote: On Monday, 20 April 2015 at 13:28:57 UTC, John Colvin wrote: The only special thing to take in to account is that valgrind will choke on DMD generated floating point code I actually fixed this problem a while ago. https://github.com/D-Programming-Language/dmd/pull/4368 An actual problem with valgrind is the GC, because most of it's operations appear to valgrind as memory corruptions. You can GC.disable() collections, use gcstub.d, or help Vladimir with his valgrind/GC support to make things work. http://dlang.org/phobos/core_memory.html#.GC.disable https://github.com/D-Programming-Language/druntime/blob/master/src/gcstub/gc.d https://github.com/CyberShadow/druntime/commits/valgrind
Valgrind
Hi, I just found quite old posts about Valgrind and D. Can someone give me a short update, what the state of support for D is and if there is anythings special to take into account. Thanks a lot. -- Robert M. Münch http://www.saphirion.com smarter | better | faster
Re: Valgrind
On Monday, 20 April 2015 at 13:28:57 UTC, John Colvin wrote: On Monday, 20 April 2015 at 13:16:23 UTC, Robert M. Münch wrote: Hi, I just found quite old posts about Valgrind and D. Can someone give me a short update, what the state of support for D is and if there is anythings special to take into account. Thanks a lot. The only special thing to take in to account is that valgrind will choke on DMD generated floating point code, so really you have to use GDC or LDC if you want to use valgrind. Correction: The only special thing I know of.
Re: Valgrind
On Monday, 20 April 2015 at 13:16:23 UTC, Robert M. Münch wrote: Hi, I just found quite old posts about Valgrind and D. Can someone give me a short update, what the state of support for D is and if there is anythings special to take into account. Thanks a lot. The only special thing to take in to account is that valgrind will choke on DMD generated floating point code, so really you have to use GDC or LDC if you want to use valgrind.
Re: Valgrind
On Monday, 20 April 2015 at 16:58:18 UTC, Robert M. Münch wrote: On 2015-04-20 13:29:57 +, John Colvin said: On Monday, 20 April 2015 at 13:28:57 UTC, John Colvin wrote: On Monday, 20 April 2015 at 13:16:23 UTC, Robert M. Münch wrote: Hi, I just found quite old posts about Valgrind and D. Can someone give me a short update, what the state of support for D is and if there is anythings special to take into account. Thanks a lot. The only special thing to take in to account is that valgrind will choke on DMD generated floating point code, so really you have to use GDC or LDC if you want to use valgrind. Correction: The only special thing I know of. Ok, thanks. Were the causes ever analyzed? I'm a bit wondering why it happens on floating point stuff... valgrind doesn't have full support for x87 code, which dmd emits all over the place.
Re: Valgrind
On 2015-04-20 13:29:57 +, John Colvin said: On Monday, 20 April 2015 at 13:28:57 UTC, John Colvin wrote: On Monday, 20 April 2015 at 13:16:23 UTC, Robert M. Münch wrote: Hi, I just found quite old posts about Valgrind and D. Can someone give me a short update, what the state of support for D is and if there is anythings special to take into account. Thanks a lot. The only special thing to take in to account is that valgrind will choke on DMD generated floating point code, so really you have to use GDC or LDC if you want to use valgrind. Correction: The only special thing I know of. Ok, thanks. Were the causes ever analyzed? I'm a bit wondering why it happens on floating point stuff... -- Robert M. Münch http://www.saphirion.com smarter | better | faster
Re: valgrind
11-Mar-2013 04:43, Druzhinin Alexandr пишет: On 11.03.2013 02:32, Dmitry Olshansky wrote: I didn't bother to check the text dump in depth but here is one slice: n6: 41816 (heap allocation functions) malloc/new/new[], --alloc-fns, etc. n1: 8208 0x540209: _D2gc3gcx4Pool10initializeMFmbZv (in /home/dmitry/Dscanner/dscanner-dmd) n1: 8208 0x53EBEB: _D2gc3gcx3Gcx7newPoolMFmbZPS2gc3gcx4Pool (in /home/dmitry/Dscanner/dscanner-dmd) n1: 8208 0x53C33B: _D2gc3gcx2GC12mallocNoSyncMFmkPmZPv (in /home/dmitry/Dscanner/dscanner-dmd) n1: 8208 0x53C1F1: _D2gc3gcx2GC6mallocMFmkPmZPv (in /home/dmitry/Dscanner/dscanner-dmd) n1: 8208 0x51CABB: gc_malloc (in /home/dmitry/Dscanner/dscanner-dmd) n1: 8208 0x51E7A6: _d_newclass (in /home/dmitry/Dscanner/dscanner-dmd) n1: 8208 0x545123: thread_attachThis (in /home/dmitry/Dscanner/dscanner-dmd) n1: 8208 0x5450B5: thread_init (in /home/dmitry/Dscanner/dscanner-dmd) n1: 8208 0x53B72B: gc_init (in /home/dmitry/Dscanner/dscanner-dmd) n1: 8208 0x51E710: _ So I guess your Mr. X is D's GC that reserves about 1Mb at start. Thanks for answer, Dmitry! My Mr.X was opengl driver - I just checked my code that using 3rd libs without debug symbols and found who it was. Massif shows me gc the same way as you, I just didn't show it. What visualizer do you use? I think it's deceptively called massif-visualizer :) -- Dmitry Olshansky
Re: valgrind
On Sunday, 10 March 2013 at 11:06:41 UTC, simendsjo wrote: I haven't used valgrind/cachegrind on C/C++ programs before, but I'm pretty sure it doesn't behave as expected on D code. I found this page to fix the mangling issue: http://wiki.dlang.org/Other_Dev_Tools, but all callgraphs is empty. If you're on Linux I'd recommend that you also look at Perf[1], as Valgrind is sometimes wrong about where your code is slow. [1] https://perf.wiki.kernel.org/index.php/Tutorial
Re: valgrind
On 12.03.2013 01:28, Dmitry Olshansky wrote: Thanks for answer, Dmitry! My Mr.X was opengl driver - I just checked my code that using 3rd libs without debug symbols and found who it was. Massif shows me gc the same way as you, I just didn't show it. What visualizer do you use? I think it's deceptively called massif-visualizer :) very, very amazing and so unexpected! :)
Re: valgrind
On 12.03.2013 01:39, Brian Schott wrote: On Sunday, 10 March 2013 at 11:06:41 UTC, simendsjo wrote: I haven't used valgrind/cachegrind on C/C++ programs before, but I'm pretty sure it doesn't behave as expected on D code. I found this page to fix the mangling issue: http://wiki.dlang.org/Other_Dev_Tools, but all callgraphs is empty. If you're on Linux I'd recommend that you also look at Perf[1], as Valgrind is sometimes wrong about where your code is slow. [1] https://perf.wiki.kernel.org/index.php/Tutorial Thanks for link!
valgrind
I haven't used valgrind/cachegrind on C/C++ programs before, but I'm pretty sure it doesn't behave as expected on D code. I found this page to fix the mangling issue: http://wiki.dlang.org/Other_Dev_Tools, but all callgraphs is empty.
Re: valgrind
On Sunday, 10 March 2013 at 11:06:41 UTC, simendsjo wrote: I haven't used valgrind/cachegrind on C/C++ programs before, but I'm pretty sure it doesn't behave as expected on D code. I found this page to fix the mangling issue: http://wiki.dlang.org/Other_Dev_Tools, but all callgraphs is empty. Ah, nevermind. Just found callgrind..
Re: valgrind
On 10.03.2013 18:11, simendsjo wrote: On Sunday, 10 March 2013 at 11:06:41 UTC, simendsjo wrote: I haven't used valgrind/cachegrind on C/C++ programs before, but I'm pretty sure it doesn't behave as expected on D code. I found this page to fix the mangling issue: http://wiki.dlang.org/Other_Dev_Tools, but all callgraphs is empty. Ah, nevermind. Just found callgrind.. Why are you pretty sure? The difference is garbage collector only that scans memory and causes wrong read from valgrind POV and mangling makes work with it harder a little bit. But profiling, heap usage and memory leaks are the same I'm sure . I use it recently but I found one bug and it was simple and clear - I mean bug finding.
Re: valgrind
On Sunday, 10 March 2013 at 11:58:39 UTC, Druzhinin Alexandr wrote: On 10.03.2013 18:11, simendsjo wrote: On Sunday, 10 March 2013 at 11:06:41 UTC, simendsjo wrote: I haven't used valgrind/cachegrind on C/C++ programs before, but I'm pretty sure it doesn't behave as expected on D code. I found this page to fix the mangling issue: http://wiki.dlang.org/Other_Dev_Tools, but all callgraphs is empty. Ah, nevermind. Just found callgrind.. Why are you pretty sure? The difference is garbage collector only that scans memory and causes wrong read from valgrind POV and mangling makes work with it harder a little bit. But profiling, heap usage and memory leaks are the same I'm sure . I use it recently but I found one bug and it was simple and clear - I mean bug finding. My pretty sure was referring to the fact that the callgraph only printed the function I was looking at. As I said, --tool=callgrind was the option I was looking for.
Re: valgrind
10-Mar-2013 15:06, simendsjo пишет: I haven't used valgrind/cachegrind on C/C++ programs before, but I'm pretty sure it doesn't behave as expected on D code. I found this page to fix the mangling issue: http://wiki.dlang.org/Other_Dev_Tools, but all callgraphs is empty. I've used it just yesterday with great success. At least callgrind and massif definitely work. -- Dmitry Olshansky
Re: valgrind
On 10.03.2013 19:45, Dmitry Olshansky wrote: 10-Mar-2013 15:06, simendsjo пишет: I haven't used valgrind/cachegrind on C/C++ programs before, but I'm pretty sure it doesn't behave as expected on D code. I found this page to fix the mangling issue: http://wiki.dlang.org/Other_Dev_Tools, but all callgraphs is empty. I've used it just yesterday with great success. At least callgrind and massif definitely work. Dmitry, using massif I get tree like this (there are normal informative branches too, of course): | -26.85% (10,364,525B) 0xD418247: ??? | -09.86% (3,807,325B) 0x1442E08E: ??? | | -02.99% (1,155,831B) 0x145188A6: ??? | | | -02.99% (1,155,831B) 0x3E: ??? | | | -02.99% (1,155,831B) 0xEB70BC7: ??? | | | -02.99% (1,155,831B) 0x145187CE: ??? | | | -02.99% (1,155,831B) 0x145188A6: ??? how to know who the hell is used my memory? who is Mr. '???'? can building phobos and druntime help with it?
Re: valgrind
10-Mar-2013 17:04, Druzhinin Alexandr пишет: On 10.03.2013 19:45, Dmitry Olshansky wrote: 10-Mar-2013 15:06, simendsjo пишет: I haven't used valgrind/cachegrind on C/C++ programs before, but I'm pretty sure it doesn't behave as expected on D code. I found this page to fix the mangling issue: http://wiki.dlang.org/Other_Dev_Tools, but all callgraphs is empty. I've used it just yesterday with great success. At least callgrind and massif definitely work. Dmitry, using massif I get tree like this (there are normal informative branches too, of course): | -26.85% (10,364,525B) 0xD418247: ??? | -09.86% (3,807,325B) 0x1442E08E: ??? | | -02.99% (1,155,831B) 0x145188A6: ??? | | | -02.99% (1,155,831B) 0x3E: ??? | | | -02.99% (1,155,831B) 0xEB70BC7: ??? | | | -02.99% (1,155,831B) 0x145187CE: ??? | | | -02.99% (1,155,831B) 0x145188A6: ??? how to know who the hell is used my memory? who is Mr. '???'? can building phobos and druntime help with it? Here is what I see when I run it on optimized binary compiled with ldc: http://sdrv.ms/ZrFgh9 Same compiled with gdc: http://sdrv.ms/12FPPF3 And dmd also optimized: http://sdrv.ms/15Gj7RF And it looks good. I didn't bother to check the text dump in depth but here is one slice: n6: 41816 (heap allocation functions) malloc/new/new[], --alloc-fns, etc. n1: 8208 0x540209: _D2gc3gcx4Pool10initializeMFmbZv (in /home/dmitry/Dscanner/dscanner-dmd) n1: 8208 0x53EBEB: _D2gc3gcx3Gcx7newPoolMFmbZPS2gc3gcx4Pool (in /home/dmitry/Dscanner/dscanner-dmd) n1: 8208 0x53C33B: _D2gc3gcx2GC12mallocNoSyncMFmkPmZPv (in /home/dmitry/Dscanner/dscanner-dmd) n1: 8208 0x53C1F1: _D2gc3gcx2GC6mallocMFmkPmZPv (in /home/dmitry/Dscanner/dscanner-dmd) n1: 8208 0x51CABB: gc_malloc (in /home/dmitry/Dscanner/dscanner-dmd) n1: 8208 0x51E7A6: _d_newclass (in /home/dmitry/Dscanner/dscanner-dmd) n1: 8208 0x545123: thread_attachThis (in /home/dmitry/Dscanner/dscanner-dmd) n1: 8208 0x5450B5: thread_init (in /home/dmitry/Dscanner/dscanner-dmd) n1: 8208 0x53B72B: gc_init (in /home/dmitry/Dscanner/dscanner-dmd) n1: 8208 0x51E710: _ So I guess your Mr. X is D's GC that reserves about 1Mb at start. -- Dmitry Olshansky
Re: valgrind
On 11.03.2013 02:32, Dmitry Olshansky wrote: I didn't bother to check the text dump in depth but here is one slice: n6: 41816 (heap allocation functions) malloc/new/new[], --alloc-fns, etc. n1: 8208 0x540209: _D2gc3gcx4Pool10initializeMFmbZv (in /home/dmitry/Dscanner/dscanner-dmd) n1: 8208 0x53EBEB: _D2gc3gcx3Gcx7newPoolMFmbZPS2gc3gcx4Pool (in /home/dmitry/Dscanner/dscanner-dmd) n1: 8208 0x53C33B: _D2gc3gcx2GC12mallocNoSyncMFmkPmZPv (in /home/dmitry/Dscanner/dscanner-dmd) n1: 8208 0x53C1F1: _D2gc3gcx2GC6mallocMFmkPmZPv (in /home/dmitry/Dscanner/dscanner-dmd) n1: 8208 0x51CABB: gc_malloc (in /home/dmitry/Dscanner/dscanner-dmd) n1: 8208 0x51E7A6: _d_newclass (in /home/dmitry/Dscanner/dscanner-dmd) n1: 8208 0x545123: thread_attachThis (in /home/dmitry/Dscanner/dscanner-dmd) n1: 8208 0x5450B5: thread_init (in /home/dmitry/Dscanner/dscanner-dmd) n1: 8208 0x53B72B: gc_init (in /home/dmitry/Dscanner/dscanner-dmd) n1: 8208 0x51E710: _ So I guess your Mr. X is D's GC that reserves about 1Mb at start. Thanks for answer, Dmitry! My Mr.X was opengl driver - I just checked my code that using 3rd libs without debug symbols and found who it was. Massif shows me gc the same way as you, I just didn't show it. What visualizer do you use?
dmd output executable vs valgrind --tool=massif
Massif (and valgrind in general) is a nice tool and using it I tried to investigate where memory leaks. I found that more than 50% of heap consumed by unknown code. How can I get what those symbols is? May be there is some option to hint valgrind where to find symbols or may be I should rebuild, for example, druntime with debug info and so on?
Valgrind D
Hi, has anybody experience with the usage of valgrind and D? We are here using DMD (dmd 1.033/tango 0.997 but also some older versions) and got problems using valgrind on this. It seems that the _d_newarrayT routine or something under it jumps or moves on unitialized value(s). Same with the _adDupT and _d_arrayappendT-method, which all call the _D3gcx3Gcx11fullcollectMFPvZk method. Has anybody the same experiences? Is it a bug within the Compiler or should be something avoided? Stefan [1] attached resource: valgrind-output ==19023== Conditional jump or move depends on uninitialised value(s) ==19023==at 0x85273FF: _D3gcx3Gcx11fullcollectMFPvZk (in /home/stefan/workspace/DDev/build/default/valgrindproject/test_executable) ==19023==by 0x85270F9: _D3gcx3Gcx16fullcollectshellMFZk (in /home/stefan/workspace/DDev/build/default/valgrindproject/test_executable) ==19023==by 0x8525729: _D3gcx2GC12mallocNoSyncMFkkZPv (in /home/stefan/workspace/DDev/build/default/valgrindproject/test_executable) ==19023==by 0x8525646: _D3gcx2GC6mallocMFkkZPv (in /home/stefan/workspace/DDev/build/default/valgrindproject/test_executable) ==19023==by 0x85250B6: gc_malloc (in /home/stefan/workspace/DDev/build/default/valgrindproject/test_executable) ==19023==by 0x8523893: _d_newarrayT (in /home/stefan/workspace/DDev/build/default/valgrindproject/test_executable) ==19023==by 0x85201C1: _aaGet (in /home/stefan/workspace/DDev/build/default/valgrindproject/test_executable) ==19023==by 0x8546035: _D5tango4text5Regex11__T4TDFATwZ4TDFA14epsilonClosureMFC5tango4text5Regex11_ _T4TDFATwZ4TDFA11SubsetStateC5tango4text5Regex11__T4TDFATwZ4TDFA11SubsetStat eZC5tango4text5Regex11__T4TDFATwZ4TDFA11SubsetState15__foreachbody57MFKC5tan go4text5Regex22__T14TNFATransitionTwZ14TNFATransitionZi (in /home/stefan/workspace/DDev/build/default/valgrindproject/test_executable) ==19023==by 0x8059513: _D5tango4text5Regex68__T4ListTC5tango4text5Regex22__T14TNFATransitionTwZ14TN FATransitionZ4List7opApplyMFDFKC5tango4text5Regex22__T14TNFATransitionTwZ14T NFATransitionZiZi (/home/stefan/workspace/DDev/dev_env/tango/dmd/include/tango/text/Regex.d:28 8) ==19023==by 0x8545E52: _D5tango4text5Regex11__T4TDFATwZ4TDFA14epsilonClosureMFC5tango4text5Regex11_ _T4TDFATwZ4TDFA11SubsetStateC5tango4text5Regex11__T4TDFATwZ4TDFA11SubsetStat eZC5tango4text5Regex11__T4TDFATwZ4TDFA11SubsetState (in /home/stefan/workspace/DDev/build/default/valgrindproject/test_executable) ==19023==by 0x8544AC7: _D5tango4text5Regex11__T4TDFATwZ4TDFA5_ctorMFC5tango4text5Regex11__T4TNFATwZ 4TNFAZC5tango4text5Regex11__T4TDFATwZ4TDFA (in /home/stefan/workspace/DDev/build/default/valgrindproject/test_executable) ==19023==by 0x854294B: _D5tango4text5Regex14__T7RegExpTTaZ7RegExpT5_ctorMFAabbbZC5tango4text5Regex1 4__T7RegExpTTaZ7RegExpT (in /home/stefan/workspace/DDev/build/default/valgrindproject/test_executable)
Re: Valgrind D
Stefan Rohe wrote: Hi, has anybody experience with the usage of valgrind and D? We are here using DMD (dmd 1.033/tango 0.997 but also some older versions) and got problems using valgrind on this. It seems that the _d_newarrayT routine or something under it jumps or moves on unitialized value(s). The last time I looked into this, Valgrind was wrong about these cases--initialization is actually logically guaranteed. You can check for yourself though. See lib/compiler/dmd/lifetime.d.
Re: Valgrind D
Sean Kelly wrote: Stefan Rohe wrote: Hi, has anybody experience with the usage of valgrind and D? We are here using DMD (dmd 1.033/tango 0.997 but also some older versions) and got problems using valgrind on this. It seems that the _d_newarrayT routine or something under it jumps or moves on unitialized value(s). The last time I looked into this, Valgrind was wrong about these cases--initialization is actually logically guaranteed. You can check for yourself though. See lib/compiler/dmd/lifetime.d. I spent a good looking at similar reports a few years ago. The cause, at the time, was that the gc code walks the stack(s) during collections and has to examine every stack slot since it has no way of knowing anything about which slots are valid and which aren't. Valgrind, on the other hand, does know what stack slots have been assigned values and which haven't so can accurately report that the program has looked at memory that hasn't been initialized yet. One 'fix' might be to pre-initialize the stack with 0's upon function entry. I'm not even terribly convinced that'd work. Later, Brad
