Re: DWARF support
Thanks everyone! A blog post would be awesome, indeed. Ben, I didn't have anything particular in mind, just curious about possibly more ergonomic alternatives to profiling builds for getting this sort of information. (Motivation similar to Richard's, I think.) On Fri, Nov 19, 2021, 1:57 AM Sven Tennie wrote: > Am Fr., 19. Nov. 2021 um 01:09 Uhr schrieb Ben Gamari < > b...@smart-cactus.org>: > >> Artem Pelenitsyn writes: >> >> > Another question would be where do I read about Haskell-native stack >> > unwinder. The issue and MR Ben referenced have descriptions, but the MR >> > didn't touch anything inside `docs` which is a bit scary. Are there any >> > good recourses to dive into it besides the source code in the MR? >> > >> Indeed the user's guide documentation surrounding info table provenance >> should be updated to note this new capability. >> > > Hey Ben, Hey Artem, Hey all, > > Thanks for bringing this up! I'll add some documentation to the user's > guide and will likely write a blog post covering the "material" from notes > in a more approachable way. > > Two details to note regarding the IPE based backtrace mechanism: > > - It's based on return frames on the STG stack. So, it doesn't work for > trivial programs. A return frame is produced when the scrutinee of a case > expression is evaluated. > The test [1] shows a working example. > > - To get backtraces containing references in libraries, those have to be > compiled with `-finfo-table-map`, too. > > Now that I've got the attention of many GHC-Devs: A big Thank-You! goes to > Ben and Matthew that were always reachable and always very helpful during > the whole development process of this feature! > > Best regards, > > Sven > > [1] - > https://gitlab.haskell.org/ghc/ghc/-/blob/aed98ddaf72cc38fb570d8415cac5de9d818/testsuite/tests/rts/decodeMyStack.hs > ___ ghc-devs mailing list ghc-devs@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
Re: DWARF support
Am Fr., 19. Nov. 2021 um 01:09 Uhr schrieb Ben Gamari : > Artem Pelenitsyn writes: > > > Another question would be where do I read about Haskell-native stack > > unwinder. The issue and MR Ben referenced have descriptions, but the MR > > didn't touch anything inside `docs` which is a bit scary. Are there any > > good recourses to dive into it besides the source code in the MR? > > > Indeed the user's guide documentation surrounding info table provenance > should be updated to note this new capability. > Hey Ben, Hey Artem, Hey all, Thanks for bringing this up! I'll add some documentation to the user's guide and will likely write a blog post covering the "material" from notes in a more approachable way. Two details to note regarding the IPE based backtrace mechanism: - It's based on return frames on the STG stack. So, it doesn't work for trivial programs. A return frame is produced when the scrutinee of a case expression is evaluated. The test [1] shows a working example. - To get backtraces containing references in libraries, those have to be compiled with `-finfo-table-map`, too. Now that I've got the attention of many GHC-Devs: A big Thank-You! goes to Ben and Matthew that were always reachable and always very helpful during the whole development process of this feature! Best regards, Sven [1] - https://gitlab.haskell.org/ghc/ghc/-/blob/aed98ddaf72cc38fb570d8415cac5de9d818/testsuite/tests/rts/decodeMyStack.hs ___ ghc-devs mailing list ghc-devs@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
Re: DWARF support
Artem Pelenitsyn writes: > Another question would be where do I read about Haskell-native stack > unwinder. The issue and MR Ben referenced have descriptions, but the MR > didn't touch anything inside `docs` which is a bit scary. Are there any > good recourses to dive into it besides the source code in the MR? > Indeed the user's guide documentation surrounding info table provenance should be updated to note this new capability. There is a long Note in compiler/GHC/Driver/GenerateCgIPEStub.hs which describes the implementation. As far as the interface is concerned, it's quite straightforward: * One can call GHC.Stack.CloneStack.cloneMyStack, which returns a "frozen" copy of the calling context's stack * One can then call GHC.Stack.CloneStack.decode to decode the stack into a list of frames. Did you have any particular questions? Cheers, - Ben signature.asc Description: PGP signature ___ ghc-devs mailing list ghc-devs@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
Re: DWARF support
Another question would be where do I read about Haskell-native stack unwinder. The issue and MR Ben referenced have descriptions, but the MR didn't touch anything inside `docs` which is a bit scary. Are there any good recourses to dive into it besides the source code in the MR? -- Best, Artem On Thu, Nov 18, 2021, 11:31 AM Chris Smith wrote: > Just to satisfy my curiosity here, when talking about backtraces here, are > you talking about a lexical call stack, or an execution stack? > > On Thu, Nov 18, 2021 at 11:24 AM Richard Eisenberg > wrote: > >> >> >> On Nov 18, 2021, at 10:29 AM, Ben Gamari wrote: >> >> At this point, for backtrace support I would rather put my money is on a >> native Haskell stack unwinder (such as Sven Tennie's work [3,4]). Not only >> is it more portable but it is also more robust (whereas with DWARF any >> single object lacking debug information would break unwinding), and is >> significantly less costly since we know much more about the structure of >> our stack than a DWARF unwinder would. >> >> >> Interesting -- this is helpful to know. I had heard about DWARF support >> for some years and thought that it would deliver stack traces. Now I will >> look for other sources. All good -- I understand how this is hard! -- and >> nice to know about. >> >> Thanks for the writeup, Ben. >> >> Richard >> ___ >> ghc-devs mailing list >> ghc-devs@haskell.org >> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs >> > ___ > ghc-devs mailing list > ghc-devs@haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs > ___ ghc-devs mailing list ghc-devs@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
Re: DWARF support
Just to satisfy my curiosity here, when talking about backtraces here, are you talking about a lexical call stack, or an execution stack? On Thu, Nov 18, 2021 at 11:24 AM Richard Eisenberg wrote: > > > On Nov 18, 2021, at 10:29 AM, Ben Gamari wrote: > > At this point, for backtrace support I would rather put my money is on a > native Haskell stack unwinder (such as Sven Tennie's work [3,4]). Not only > is it more portable but it is also more robust (whereas with DWARF any > single object lacking debug information would break unwinding), and is > significantly less costly since we know much more about the structure of > our stack than a DWARF unwinder would. > > > Interesting -- this is helpful to know. I had heard about DWARF support > for some years and thought that it would deliver stack traces. Now I will > look for other sources. All good -- I understand how this is hard! -- and > nice to know about. > > Thanks for the writeup, Ben. > > Richard > ___ > ghc-devs mailing list > ghc-devs@haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs > ___ ghc-devs mailing list ghc-devs@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
Re: DWARF support
> On Nov 18, 2021, at 10:29 AM, Ben Gamari wrote: > > At this point, for backtrace support I would rather put my money is on a > native Haskell stack unwinder (such as Sven Tennie's work [3,4]). Not only > is it more portable but it is also more robust (whereas with DWARF any > single object lacking debug information would break unwinding), and is > significantly less costly since we know much more about the structure of > our stack than a DWARF unwinder would. Interesting -- this is helpful to know. I had heard about DWARF support for some years and thought that it would deliver stack traces. Now I will look for other sources. All good -- I understand how this is hard! -- and nice to know about. Thanks for the writeup, Ben. Richard___ ghc-devs mailing list ghc-devs@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
Re: DWARF support
Richard Eisenberg writes: > Thanks for this! > >> On Nov 17, 2021, at 7:27 AM, Moritz Angermann >> wrote: >> >> For Linux and most BSDs, we have settled on the Executable and Linking >> Format (ELF) as the container format for >> your machine code. And you might see where the inspiration for DWARF might >> come from. > > This suggests to me that DWARF is the canonical format for debugging > information on Linux and most BSDs. Is that statement correct? If so, how is > that different to "platform-native"? Actually, the precise wording doesn't > matter: I think I'm just requesting for a more direct relationship between > "DWARF" and "compatibility with all the debugging and profiling tools you use > for other languages". > >> >> For macOS, we have mach object (mach-o) as the container format. Its >> distinctly different to ELF, and also the >> reason why Linux/BSD and macOS are sometimes substantially different, wrt to >> executable packaging and linking. > > OK. So there is no macOS support here. That's fine -- I'm just trying to > understand the status quo. > >> >> For windows we have Portable Executable (PE) as the container format. > > This implies that the DWARF work is (unsurprisingly) completely inapplicable > for Windows. > >> Depending on how familiar you are with development on macOS, you might know >> the notion of dSYM folders, >> where macOS usually separate the application binary into the binary, and >> then stores the (d)ebug (SYM)bols in >> a separate folder. Those are iirc DWARF objects in the end. > > This suggests to me that the DWARF work is applicable to the macOS use case, > but much more work still needs to be done. OK. > > Looking for more information, I checked the manual. And I found this: > https://downloads.haskell.org/ghc/latest/docs/html/users_guide/debug-info.html > It's really helpful! And it suggests that I actually *can* do this on > macOS. Perhaps that video will really happen after all. > I should clarify here that there are really two pieces to "debug information support": * the code generation logic responsible for producing the DWARF metadata * the RTS support for using that information to unwind the stack at runtime That section really describes the former. As noted in my earlier email and in #20702, the library used by the RTS to implement unwinding only targets ELF platforms. Cheers, - Ben signature.asc Description: PGP signature ___ ghc-devs mailing list ghc-devs@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
Re: DWARF support
Richard Eisenberg writes: > Hi devs, > > I was intrigued by Bodigrim's comment about HasCallStack in base > (https://github.com/haskell/core-libraries-committee/issues/5#issuecomment-970942580) > that there are other alternatives, such as DWARF. Over the years, I > had tuned out every time I saw the word DWARF: it was (and is!) an > unknown acronym and seems like a low-level detail. But Bodigrim's > comment made me want to re-think this stance. > > I found Ben's series of blog posts on DWARF, starting with > https://www.haskell.org/ghc/blog/20200403-dwarf-1.html. These are very > helpful! In particular, they taught me that DWARF = platform-native > debugging metadata. Is that translation accurate? If so, perhaps we > should use both names: if I see that GHC x.y.z has DWARF support, I > quickly scroll to the next bullet. If I see that GHC x.y.z has support > for platform-native debugging metadata and is now compatible with e.g. > gdb, I'm interested. > > Going further, I have a key question for my use case: is this support > available on Mac? The first post in the series describes support for > "Linux and several BSDs" and the last post says that "Windows PDB > support" is future work. (Is "PDB" platform-native debugging metadata > for Windows? I don't know.) But I don't see any mention of Mac. What's > the status here? > The problem is that DWARF is only one format for debugging information. Windows uses PDB, which is practically undocumented and consequently essentially unsupportable, AFAICT. I believe some other platforms still use the STABS format, which I do not believe we will ever support. Sadly, Darwin is, as usual, a bit of a tricky case. The platform seems to use a variant of the typical "split-debug" packaging configuration for debug information. Apple calls this separate debug information `dsym` files. In typical Apple fasion, the documentation surrounding this mechanism is very poor. It appears that the user or packager must generate these files manually using dsymutils but it's quite unclear how a consuming library is to find the dsym for a particular object. To make matters worse, the library that we use for unwinding, libdw, has absolutely no support for it. Ultimately, my goals for working on DWARF were: 1. To ensure that we had *some* baseline symbol and line information support on platforms for low-level debugging. I have found even basic line information invaluable when looking at code generation issues and thankfully gdb can always use inline DWARF information, even on platforms like on Windows where it isn't quite "native" 2. Providing an interface for users to unwind the stack from Haskell on the "easy" platforms (e.g. the ELF-based BSDs and Linux). I was hoping someone would express interest in picking up Darwin and Windows, at which point we could start to push this as a universal backtrace mechanism, but sadly this hasn't yet happened. 3. Provide a basis for statistic profiling support. This is sadly complicated by the stack pointer issues that Andreas and I note in our respective blog posts [1,2]. Sadly, overcoming this without breaking potential users is harder than we thought due to limitations in LLVM. At this point, for backtrace support I would rather put my money is on a native Haskell stack unwinder (such as Sven Tennie's work [3,4]). Not only is it more portable but it is also more robust (whereas with DWARF any single object lacking debug information would break unwinding), and is significantly less costly since we know much more about the structure of our stack than a DWARF unwinder would. Cheers, - Ben [1] https://well-typed.com/blog/2020/04/dwarf-4/ [2] https://well-typed.com/blog/2021/07/ghc-sp-profiling/ [3] https://gitlab.haskell.org/ghc/ghc/-/issues/18163 [4] https://gitlab.haskell.org/ghc/ghc/-/merge_requests/5456 signature.asc Description: PGP signature ___ ghc-devs mailing list ghc-devs@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
Re: DWARF support
Am 17/11/2021 um 17:08 schrieb Richard Eisenberg: For windows we have Portable Executable (PE) as the container format. This implies that the DWARF work is (unsurprisingly) completely inapplicable for Windows. It's not quite as simple. Dwarf info can be embedded into windows executables/libraries. Tools that understand dwarf can read that information and use it. However many tools on windows don't understand dwarf debugging information since windows has it's own debugging format. So it's not quite as helpful as on linux where everyone agreed to use dwarf. But it still can be used for some things. E.g. there are debuggers on windows (lldbg, gdb) that can read and use this information. Further we could in theory start emitting the same information in the windows native format. In that case we could re-use much of the work that went into GHC to allows us to collect debugging information. Putting into the right format comes fairly late in the pipeline and all steps up to that could be shared. Cheers Andreas ___ ghc-devs mailing list ghc-devs@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
Re: DWARF support
Thanks for this! > On Nov 17, 2021, at 7:27 AM, Moritz Angermann > wrote: > > For Linux and most BSDs, we have settled on the Executable and Linking Format > (ELF) as the container format for > your machine code. And you might see where the inspiration for DWARF might > come from. This suggests to me that DWARF is the canonical format for debugging information on Linux and most BSDs. Is that statement correct? If so, how is that different to "platform-native"? Actually, the precise wording doesn't matter: I think I'm just requesting for a more direct relationship between "DWARF" and "compatibility with all the debugging and profiling tools you use for other languages". > > For macOS, we have mach object (mach-o) as the container format. Its > distinctly different to ELF, and also the > reason why Linux/BSD and macOS are sometimes substantially different, wrt to > executable packaging and linking. OK. So there is no macOS support here. That's fine -- I'm just trying to understand the status quo. > > For windows we have Portable Executable (PE) as the container format. This implies that the DWARF work is (unsurprisingly) completely inapplicable for Windows. > Depending on how familiar you are with development on macOS, you might know > the notion of dSYM folders, > where macOS usually separate the application binary into the binary, and then > stores the (d)ebug (SYM)bols in > a separate folder. Those are iirc DWARF objects in the end. This suggests to me that the DWARF work is applicable to the macOS use case, but much more work still needs to be done. OK. Looking for more information, I checked the manual. And I found this: https://downloads.haskell.org/ghc/latest/docs/html/users_guide/debug-info.html It's really helpful! And it suggests that I actually *can* do this on macOS. Perhaps that video will really happen after all. Forgive the noise -- I should have checked the manual first. :) Richard ___ ghc-devs mailing list ghc-devs@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
Re: DWARF support
Thanks Carter! Yes I completely forgot about the unwinding librarys. Sorry. My bad! Best, Moritz On Wed 17. Nov 2021 at 21:08, Carter Schonwald wrote: > My understanding is that the platform specific part of ghc dwarf support > atm is the stack walking to generate dwarf data in stack traces. This is > because the dwarf stack walking Libs that are relatively mature are mostly > centered around elf? > > It should still be possible with some work to use perf and gdb style > tools, though the complications are > > a) you have to make sure all the Libs are built with dwarf > > b) there’s some complications around loading / placing the dwarf files > adjacent to the object code files on Darwin (at least last time I checked > which was years ago following the wiki entry johan tibbel wrote up I > think?) > > C) scheduler yields make gdb stepping through a program a tad more > annoying, I think the “setting the yield timer to zero” is the work around > > D) the “source” you step through is essentially the c— z-encoded code? So > you still need to do some puzzling out of stuff > > > On Wed, Nov 17, 2021 at 7:28 AM Moritz Angermann < > moritz.angerm...@gmail.com> wrote: > >> Hi Richard, >> >> I’m not sure using platform native AND the term DWARF would help rather >> than add to confusion. Let me still try to >> help a bit with context here. >> >> For Linux and most BSDs, we have settled on the Executable and Linking >> Format (ELF) as the container format for >> your machine code. And you might see where the inspiration for DWARF >> might come from. >> >> For macOS, we have mach object (mach-o) as the container format. Its >> distinctly different to ELF, and also the >> reason why Linux/BSD and macOS are sometimes substantially different, wrt >> to executable packaging and linking. >> >> For windows we have Portable Executable (PE) as the container format. >> >> My recollection is that we implemented DWARF in the NCG only for ELF. >> I've always wanted to scratch an itch >> and try to make it work for mach-o as well, but never got around to it >> (yet?). The NCGs have flags that specify >> if we want to emit debug info or not. I believe most codegens except for >> x86_64/elf ignore that flag. >> >> This is a non-trivial engineering effort to get done properly, I believe. >> And we all spend time on many other things. >> >> Depending on how familiar you are with development on macOS, you might >> know the notion of dSYM folders, >> where macOS usually separate the application binary into the binary, and >> then stores the (d)ebug (SYM)bols in >> a separate folder. Those are iirc DWARF objects in the end. >> >> Hope this helps a bit; my recollection might be a bit rusty. >> >> Best, >> Moritz >> >> >> >> On Wed 17. Nov 2021 at 20:02, Richard Eisenberg >> wrote: >> >>> Hi devs, >>> >>> I was intrigued by Bodigrim's comment about HasCallStack in base ( >>> https://github.com/haskell/core-libraries-committee/issues/5#issuecomment-970942580) >>> that there are other alternatives, such as DWARF. Over the years, I had >>> tuned out every time I saw the word DWARF: it was (and is!) an unknown >>> acronym and seems like a low-level detail. But Bodigrim's comment made me >>> want to re-think this stance. >>> >>> I found Ben's series of blog posts on DWARF, starting with >>> https://www.haskell.org/ghc/blog/20200403-dwarf-1.html. These are very >>> helpful! In particular, they taught me that DWARF = platform-native >>> debugging metadata. Is that translation accurate? If so, perhaps we should >>> use both names: if I see that GHC x.y.z has DWARF support, I quickly scroll >>> to the next bullet. If I see that GHC x.y.z has support for platform-native >>> debugging metadata and is now compatible with e.g. gdb, I'm interested. >>> >>> Going further, I have a key question for my use case: is this support >>> available on Mac? The first post in the series describes support for "Linux >>> and several BSDs" and the last post says that "Windows PDB support" is >>> future work. (Is "PDB" platform-native debugging metadata for Windows? I >>> don't know.) But I don't see any mention of Mac. What's the status here? >>> >>> It would be very cool if this conversation ends with me making a video >>> on how a few simple GHC flags can allow us to, say, get a stack trace on a >>> pattern-match failure in a Haskell program. >>> >>> Thanks! >>> Richard >>> >>> ___ >>> ghc-devs mailing list >>> ghc-devs@haskell.org >>> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs >>> >> ___ >> ghc-devs mailing list >> ghc-devs@haskell.org >> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs >> > ___ ghc-devs mailing list ghc-devs@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
Re: DWARF support
My understanding is that the platform specific part of ghc dwarf support atm is the stack walking to generate dwarf data in stack traces. This is because the dwarf stack walking Libs that are relatively mature are mostly centered around elf? It should still be possible with some work to use perf and gdb style tools, though the complications are a) you have to make sure all the Libs are built with dwarf b) there’s some complications around loading / placing the dwarf files adjacent to the object code files on Darwin (at least last time I checked which was years ago following the wiki entry johan tibbel wrote up I think?) C) scheduler yields make gdb stepping through a program a tad more annoying, I think the “setting the yield timer to zero” is the work around D) the “source” you step through is essentially the c— z-encoded code? So you still need to do some puzzling out of stuff On Wed, Nov 17, 2021 at 7:28 AM Moritz Angermann wrote: > Hi Richard, > > I’m not sure using platform native AND the term DWARF would help rather > than add to confusion. Let me still try to > help a bit with context here. > > For Linux and most BSDs, we have settled on the Executable and Linking > Format (ELF) as the container format for > your machine code. And you might see where the inspiration for DWARF > might come from. > > For macOS, we have mach object (mach-o) as the container format. Its > distinctly different to ELF, and also the > reason why Linux/BSD and macOS are sometimes substantially different, wrt > to executable packaging and linking. > > For windows we have Portable Executable (PE) as the container format. > > My recollection is that we implemented DWARF in the NCG only for ELF. > I've always wanted to scratch an itch > and try to make it work for mach-o as well, but never got around to it > (yet?). The NCGs have flags that specify > if we want to emit debug info or not. I believe most codegens except for > x86_64/elf ignore that flag. > > This is a non-trivial engineering effort to get done properly, I believe. > And we all spend time on many other things. > > Depending on how familiar you are with development on macOS, you might > know the notion of dSYM folders, > where macOS usually separate the application binary into the binary, and > then stores the (d)ebug (SYM)bols in > a separate folder. Those are iirc DWARF objects in the end. > > Hope this helps a bit; my recollection might be a bit rusty. > > Best, > Moritz > > > > On Wed 17. Nov 2021 at 20:02, Richard Eisenberg > wrote: > >> Hi devs, >> >> I was intrigued by Bodigrim's comment about HasCallStack in base ( >> https://github.com/haskell/core-libraries-committee/issues/5#issuecomment-970942580) >> that there are other alternatives, such as DWARF. Over the years, I had >> tuned out every time I saw the word DWARF: it was (and is!) an unknown >> acronym and seems like a low-level detail. But Bodigrim's comment made me >> want to re-think this stance. >> >> I found Ben's series of blog posts on DWARF, starting with >> https://www.haskell.org/ghc/blog/20200403-dwarf-1.html. These are very >> helpful! In particular, they taught me that DWARF = platform-native >> debugging metadata. Is that translation accurate? If so, perhaps we should >> use both names: if I see that GHC x.y.z has DWARF support, I quickly scroll >> to the next bullet. If I see that GHC x.y.z has support for platform-native >> debugging metadata and is now compatible with e.g. gdb, I'm interested. >> >> Going further, I have a key question for my use case: is this support >> available on Mac? The first post in the series describes support for "Linux >> and several BSDs" and the last post says that "Windows PDB support" is >> future work. (Is "PDB" platform-native debugging metadata for Windows? I >> don't know.) But I don't see any mention of Mac. What's the status here? >> >> It would be very cool if this conversation ends with me making a video on >> how a few simple GHC flags can allow us to, say, get a stack trace on a >> pattern-match failure in a Haskell program. >> >> Thanks! >> Richard >> >> ___ >> ghc-devs mailing list >> ghc-devs@haskell.org >> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs >> > ___ > ghc-devs mailing list > ghc-devs@haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs > ___ ghc-devs mailing list ghc-devs@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
Re: DWARF support
Hi Richard, I’m not sure using platform native AND the term DWARF would help rather than add to confusion. Let me still try to help a bit with context here. For Linux and most BSDs, we have settled on the Executable and Linking Format (ELF) as the container format for your machine code. And you might see where the inspiration for DWARF might come from. For macOS, we have mach object (mach-o) as the container format. Its distinctly different to ELF, and also the reason why Linux/BSD and macOS are sometimes substantially different, wrt to executable packaging and linking. For windows we have Portable Executable (PE) as the container format. My recollection is that we implemented DWARF in the NCG only for ELF. I've always wanted to scratch an itch and try to make it work for mach-o as well, but never got around to it (yet?). The NCGs have flags that specify if we want to emit debug info or not. I believe most codegens except for x86_64/elf ignore that flag. This is a non-trivial engineering effort to get done properly, I believe. And we all spend time on many other things. Depending on how familiar you are with development on macOS, you might know the notion of dSYM folders, where macOS usually separate the application binary into the binary, and then stores the (d)ebug (SYM)bols in a separate folder. Those are iirc DWARF objects in the end. Hope this helps a bit; my recollection might be a bit rusty. Best, Moritz On Wed 17. Nov 2021 at 20:02, Richard Eisenberg wrote: > Hi devs, > > I was intrigued by Bodigrim's comment about HasCallStack in base ( > https://github.com/haskell/core-libraries-committee/issues/5#issuecomment-970942580) > that there are other alternatives, such as DWARF. Over the years, I had > tuned out every time I saw the word DWARF: it was (and is!) an unknown > acronym and seems like a low-level detail. But Bodigrim's comment made me > want to re-think this stance. > > I found Ben's series of blog posts on DWARF, starting with > https://www.haskell.org/ghc/blog/20200403-dwarf-1.html. These are very > helpful! In particular, they taught me that DWARF = platform-native > debugging metadata. Is that translation accurate? If so, perhaps we should > use both names: if I see that GHC x.y.z has DWARF support, I quickly scroll > to the next bullet. If I see that GHC x.y.z has support for platform-native > debugging metadata and is now compatible with e.g. gdb, I'm interested. > > Going further, I have a key question for my use case: is this support > available on Mac? The first post in the series describes support for "Linux > and several BSDs" and the last post says that "Windows PDB support" is > future work. (Is "PDB" platform-native debugging metadata for Windows? I > don't know.) But I don't see any mention of Mac. What's the status here? > > It would be very cool if this conversation ends with me making a video on > how a few simple GHC flags can allow us to, say, get a stack trace on a > pattern-match failure in a Haskell program. > > Thanks! > Richard > > ___ > ghc-devs mailing list > ghc-devs@haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs > ___ ghc-devs mailing list ghc-devs@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
DWARF support
Hi devs, I was intrigued by Bodigrim's comment about HasCallStack in base (https://github.com/haskell/core-libraries-committee/issues/5#issuecomment-970942580) that there are other alternatives, such as DWARF. Over the years, I had tuned out every time I saw the word DWARF: it was (and is!) an unknown acronym and seems like a low-level detail. But Bodigrim's comment made me want to re-think this stance. I found Ben's series of blog posts on DWARF, starting with https://www.haskell.org/ghc/blog/20200403-dwarf-1.html. These are very helpful! In particular, they taught me that DWARF = platform-native debugging metadata. Is that translation accurate? If so, perhaps we should use both names: if I see that GHC x.y.z has DWARF support, I quickly scroll to the next bullet. If I see that GHC x.y.z has support for platform-native debugging metadata and is now compatible with e.g. gdb, I'm interested. Going further, I have a key question for my use case: is this support available on Mac? The first post in the series describes support for "Linux and several BSDs" and the last post says that "Windows PDB support" is future work. (Is "PDB" platform-native debugging metadata for Windows? I don't know.) But I don't see any mention of Mac. What's the status here? It would be very cool if this conversation ends with me making a video on how a few simple GHC flags can allow us to, say, get a stack trace on a pattern-match failure in a Haskell program. Thanks! Richard ___ ghc-devs mailing list ghc-devs@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
Re: How's the integration of DWARF support coming along?
Er, yes. Copied it from the wrong browser window… Greetings, Peter ___ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs
Re: How's the integration of DWARF support coming along?
Peter Wortmann writes: > Okay, I have uploaded the “core” set of patches to Phab: > > https://phabricator.haskell.org/D155 > Surely you mean D169 [1]? Cheers, - Ben [1] https://phabricator.haskell.org/D169 pgppRR0lUBx18.pgp Description: PGP signature ___ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs
Re: How's the integration of DWARF support coming along?
Okay, I have uploaded the “core” set of patches to Phab: https://phabricator.haskell.org/D155 Not entirely sure this is the best way to go about it - even though this barely covers the essentials, it is still a huge patch. If that makes more sense, I might try to set it up as a series of dependant diffs. As usual, the “full” stack of patches is on GitHub: http://github.com/scpmw/ghc/commits/profiling-import Greetings, Peter ___ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs
Re: How's the integration of DWARF support coming along?
inally agreed on a simple and sensible way to implement `catchWithStack` that have all most good properties you would like. I just need to implement it and test it. I can definitely man up and implement this. :) Here's my master thesis btw [1], it should answer Ömer's question of how we retrieve a stack from a language you think won't have a stack. :) Cheers, Arash [1]: http://arashrouhani.com/papers/master-thesis.pdf On 2014-08-13 17:02, Johan Tibell wrote: Hi, How's the integration of DWARF support coming along? It's probably one of the most important improvements to the runtime in quite some time since unlocks *two* important features, namely * trustworthy profiling (using e.g. Linux perf events and other low-overhead, code preserving, sampling profilers), and * stack traces. The former is really important to move our core libraries performance up a notch. Right now -prof is too invasive for it to be useful when evaluating the hotspots in these libraries (which are already often heavily tuned). The latter one is really important for real life Haskell on the server, where you can sometimes can get some crash that only happens once a day under very specific conditions. Knowing where the crash happens is then *very* useful. -- Johan ___ ghc-devs mailing list ghc-devs@haskell.org<mailto:ghc-devs@haskell.org><mailto:ghc-devs@haskell.org<mailto:ghc-devs@haskell.org>> http://www.haskell.org/mailman/listinfo/ghc-devs ___ ghc-devs mailing list ghc-devs@haskell.org<mailto:ghc-devs@haskell.org><mailto:ghc-devs@haskell.org<mailto:ghc-devs@haskell.org>> http://www.haskell.org/mailman/listinfo/ghc-devs ___ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs
Re: How's the integration of DWARF support coming along?
Seeing the code on Phab it two weeks sounds great. Do you mind expanding on what tick scopes are. It sounds scarily like something that happens at runtime. :) On Wed, Aug 13, 2014 at 8:49 PM, Peter Wortmann wrote: > > > At this point I have a bit more time on my hands again (modulo post-thesis > vacations), but we are basically still in “review hell”. > > I think “just” for perf_events support we’d need the following patches[1]: > 1. Source notes (Core support) > 2. Source notes (CorePrep & Stg support) > 3. Source notes (Cmm support) > 4. Tick scopes > 5. Debug data extraction (NCG support) > 6. Generate .loc/.file directives > > We have a basic “okay” from the Simons up to number 2 (conditional on > better documentation). Number 4 sticks out because Simon Marlow wanted to > have a closer look at it - this is basically about how to maintain source > ticks in a robust fashion on the Cmm level (see also section 5.5 of my > thesis[2]). > > Meanwhile I have ported NCG DWARF generation over to Mac Os, and am > working on reviving LLVM support. My plan was to check that I didn’t > accidentally break Linux support, then push for review again in a week or > so (Phab?). > > Greetings, > Peter > > [1] https://github.com/scpmw/ghc/commits/profiling-import > [2] http://www.personal.leeds.ac.uk/~scpmw/static/thesis.pdf > > On 13 Aug 2014, at 20:01, Johan Tibell johan.tib...@gmail.com>> wrote: > > What's the minimal amount of work we need to do to just get the dwarf data > in the codegen by 7.10 (RC late december) so we can start using e.g. linux > perf events to profile Haskell programs? > > > On Wed, Aug 13, 2014 at 7:31 PM, Arash Rouhani <mailto:rar...@student.chalmers.se>> wrote: > Hi Johan! > > I haven't done much (just been lazy) lately, I've tried to benchmark my > results but I don't get any sensible results at all yet. > > Last time Peter said he's working on a more portable way to read dwarf > information that doesn't require Linux. But I'm sure he'll give a more > acurate update than me soon in this mail thread. > > As for stack traces, I don't think there's any big tasks left, but I > summarize what I have in mind: > > * The haskell interface is done and I've iterated on it a bit, so it's > in a decent shape at least. Some parts still need testing. > * I wish I could implement the `forceCaseContinuation` that I've > described in my thesis. If someone is good with code generation (I just > suck at it, it's probably simple) and is willing to assist me a bit, please > say so. :) > * I tried benchmarking, I gave up after not getting any useful results. > * I'm unfortunately totally incapable to help out with dwarf debug data > generation, only Peter knows that part, particularly I never grasped his > theoretical framework of causality in Haskell. > * Peter and I have finally agreed on a simple and sensible way to > implement `catchWithStack` that have all most good properties you would > like. I just need to implement it and test it. I can definitely man up and > implement this. :) > > Here's my master thesis btw [1], it should answer Ömer's question of how > we retrieve a stack from a language you think won't have a stack. :) > > Cheers, > Arash > > [1]: http://arashrouhani.com/papers/master-thesis.pdf > > > > > > On 2014-08-13 17:02, Johan Tibell wrote: > Hi, > > How's the integration of DWARF support coming along? It's probably one of > the most important improvements to the runtime in quite some time since > unlocks *two* important features, namely > > * trustworthy profiling (using e.g. Linux perf events and other > low-overhead, code preserving, sampling profilers), and > * stack traces. > > The former is really important to move our core libraries performance up a > notch. Right now -prof is too invasive for it to be useful when evaluating > the hotspots in these libraries (which are already often heavily tuned). > > The latter one is really important for real life Haskell on the server, > where you can sometimes can get some crash that only happens once a day > under very specific conditions. Knowing where the crash happens is then > *very* useful. > > -- Johan > > > > > ___ > ghc-devs mailing list > ghc-devs@haskell.org<mailto:ghc-devs@haskell.org> > http://www.haskell.org/mailman/listinfo/ghc-devs > > > > ___ > ghc-devs mailing list > ghc-devs@haskell.org<mailto:ghc-devs@haskell.org> > http://www.haskell.org/mailman/listinfo/ghc-devs > > > > ___ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs
Re: How's the integration of DWARF support coming along?
At this point I have a bit more time on my hands again (modulo post-thesis vacations), but we are basically still in “review hell”. I think “just” for perf_events support we’d need the following patches[1]: 1. Source notes (Core support) 2. Source notes (CorePrep & Stg support) 3. Source notes (Cmm support) 4. Tick scopes 5. Debug data extraction (NCG support) 6. Generate .loc/.file directives We have a basic “okay” from the Simons up to number 2 (conditional on better documentation). Number 4 sticks out because Simon Marlow wanted to have a closer look at it - this is basically about how to maintain source ticks in a robust fashion on the Cmm level (see also section 5.5 of my thesis[2]). Meanwhile I have ported NCG DWARF generation over to Mac Os, and am working on reviving LLVM support. My plan was to check that I didn’t accidentally break Linux support, then push for review again in a week or so (Phab?). Greetings, Peter [1] https://github.com/scpmw/ghc/commits/profiling-import [2] http://www.personal.leeds.ac.uk/~scpmw/static/thesis.pdf On 13 Aug 2014, at 20:01, Johan Tibell mailto:johan.tib...@gmail.com>> wrote: What's the minimal amount of work we need to do to just get the dwarf data in the codegen by 7.10 (RC late december) so we can start using e.g. linux perf events to profile Haskell programs? On Wed, Aug 13, 2014 at 7:31 PM, Arash Rouhani mailto:rar...@student.chalmers.se>> wrote: Hi Johan! I haven't done much (just been lazy) lately, I've tried to benchmark my results but I don't get any sensible results at all yet. Last time Peter said he's working on a more portable way to read dwarf information that doesn't require Linux. But I'm sure he'll give a more acurate update than me soon in this mail thread. As for stack traces, I don't think there's any big tasks left, but I summarize what I have in mind: * The haskell interface is done and I've iterated on it a bit, so it's in a decent shape at least. Some parts still need testing. * I wish I could implement the `forceCaseContinuation` that I've described in my thesis. If someone is good with code generation (I just suck at it, it's probably simple) and is willing to assist me a bit, please say so. :) * I tried benchmarking, I gave up after not getting any useful results. * I'm unfortunately totally incapable to help out with dwarf debug data generation, only Peter knows that part, particularly I never grasped his theoretical framework of causality in Haskell. * Peter and I have finally agreed on a simple and sensible way to implement `catchWithStack` that have all most good properties you would like. I just need to implement it and test it. I can definitely man up and implement this. :) Here's my master thesis btw [1], it should answer Ömer's question of how we retrieve a stack from a language you think won't have a stack. :) Cheers, Arash [1]: http://arashrouhani.com/papers/master-thesis.pdf On 2014-08-13 17:02, Johan Tibell wrote: Hi, How's the integration of DWARF support coming along? It's probably one of the most important improvements to the runtime in quite some time since unlocks *two* important features, namely * trustworthy profiling (using e.g. Linux perf events and other low-overhead, code preserving, sampling profilers), and * stack traces. The former is really important to move our core libraries performance up a notch. Right now -prof is too invasive for it to be useful when evaluating the hotspots in these libraries (which are already often heavily tuned). The latter one is really important for real life Haskell on the server, where you can sometimes can get some crash that only happens once a day under very specific conditions. Knowing where the crash happens is then *very* useful. -- Johan ___ ghc-devs mailing list ghc-devs@haskell.org<mailto:ghc-devs@haskell.org> http://www.haskell.org/mailman/listinfo/ghc-devs ___ ghc-devs mailing list ghc-devs@haskell.org<mailto:ghc-devs@haskell.org> http://www.haskell.org/mailman/listinfo/ghc-devs ___ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs
Re: How's the integration of DWARF support coming along?
Peter will have to answer that. But it seemed to me that it has been working fine all the time. I suppose it's just to resolve merge conflicts. There were some refactorings he wanted to do. In addition to this it will also be some packaging issues I suppose. I'm hoping Peter will answer in this mail thread soon, since he knows this much better. /Arash On 2014-08-13 20:01, Johan Tibell wrote: What's the minimal amount of work we need to do to just get the dwarf data in the codegen by 7.10 (RC late december) so we can start using e.g. linux perf events to profile Haskell programs? On Wed, Aug 13, 2014 at 7:31 PM, Arash Rouhani mailto:rar...@student.chalmers.se>> wrote: Hi Johan! I haven't done much (just been lazy) lately, I've tried to benchmark my results but I don't get any sensible results at all yet. Last time Peter said he's working on a more portable way to read dwarf information that doesn't require Linux. But I'm sure he'll give a more acurate update than me soon in this mail thread. As for stack traces, I don't think there's any big tasks left, but I summarize what I have in mind: * The haskell interface is done and I've iterated on it a bit, so it's in a decent shape at least. Some parts still need testing. * I wish I could implement the `forceCaseContinuation` that I've described in my thesis. If someone is good with code generation (I just suck at it, it's probably simple) and is willing to assist me a bit, please say so. :) * I tried benchmarking, I gave up after not getting any useful results. * I'm unfortunately totally incapable to help out with dwarf debug data generation, only Peter knows that part, particularly I never grasped his theoretical framework of causality in Haskell. * Peter and I have finally agreed on a simple and sensible way to implement `catchWithStack` that have all most good properties you would like. I just need to implement it and test it. I can definitely man up and implement this. :) Here's my master thesis btw [1], it should answer Ömer's question of how we retrieve a stack from a language you think won't have a stack. :) Cheers, Arash [1]: http://arashrouhani.com/papers/master-thesis.pdf On 2014-08-13 17:02, Johan Tibell wrote: Hi, How's the integration of DWARF support coming along? It's probably one of the most important improvements to the runtime in quite some time since unlocks *two* important features, namely * trustworthy profiling (using e.g. Linux perf events and other low-overhead, code preserving, sampling profilers), and * stack traces. The former is really important to move our core libraries performance up a notch. Right now -prof is too invasive for it to be useful when evaluating the hotspots in these libraries (which are already often heavily tuned). The latter one is really important for real life Haskell on the server, where you can sometimes can get some crash that only happens once a day under very specific conditions. Knowing where the crash happens is then *very* useful. -- Johan ___ ghc-devs mailing list ghc-devs@haskell.org <mailto:ghc-devs@haskell.org> http://www.haskell.org/mailman/listinfo/ghc-devs ___ ghc-devs mailing list ghc-devs@haskell.org <mailto:ghc-devs@haskell.org> http://www.haskell.org/mailman/listinfo/ghc-devs ___ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs
Re: How's the integration of DWARF support coming along?
What's the minimal amount of work we need to do to just get the dwarf data in the codegen by 7.10 (RC late december) so we can start using e.g. linux perf events to profile Haskell programs? On Wed, Aug 13, 2014 at 7:31 PM, Arash Rouhani wrote: > Hi Johan! > > I haven't done much (just been lazy) lately, I've tried to benchmark my > results but I don't get any sensible results at all yet. > > Last time Peter said he's working on a more portable way to read dwarf > information that doesn't require Linux. But I'm sure he'll give a more > acurate update than me soon in this mail thread. > > As for stack traces, I don't think there's any big tasks left, but I > summarize what I have in mind: > >- The haskell interface is done and I've iterated on it a bit, so it's >in a decent shape at least. Some parts still need testing. >- I wish I could implement the `forceCaseContinuation` that I've >described in my thesis. If someone is good with code generation (I just >suck at it, it's probably simple) and is willing to assist me a bit, please >say so. :) >- I tried benchmarking, I gave up after not getting any useful results. > - I'm unfortunately totally incapable to help out with dwarf debug >data generation, only Peter knows that part, particularly I never grasped >his theoretical framework of causality in Haskell. >- Peter and I have finally agreed on a simple and sensible way to >implement `catchWithStack` that have all most good properties you would >like. I just need to implement it and test it. I can definitely man up and >implement this. :) > > Here's my master thesis btw [1], it should answer Ömer's question of how > we retrieve a stack from a language you think won't have a stack. :) > > Cheers, > Arash > > [1]: http://arashrouhani.com/papers/master-thesis.pdf > > > > > > On 2014-08-13 17:02, Johan Tibell wrote: > > Hi, > > How's the integration of DWARF support coming along? It's probably one > of the most important improvements to the runtime in quite some time since > unlocks *two* important features, namely > > * trustworthy profiling (using e.g. Linux perf events and other > low-overhead, code preserving, sampling profilers), and > * stack traces. > > The former is really important to move our core libraries performance up > a notch. Right now -prof is too invasive for it to be useful when > evaluating the hotspots in these libraries (which are already often heavily > tuned). > > The latter one is really important for real life Haskell on the server, > where you can sometimes can get some crash that only happens once a day > under very specific conditions. Knowing where the crash happens is then > *very* useful. > > -- Johan > > > > ___ > ghc-devs mailing > listghc-devs@haskell.orghttp://www.haskell.org/mailman/listinfo/ghc-devs > > > > ___ > ghc-devs mailing list > ghc-devs@haskell.org > http://www.haskell.org/mailman/listinfo/ghc-devs > > ___ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs
Re: How's the integration of DWARF support coming along?
Hi Johan! I haven't done much (just been lazy) lately, I've tried to benchmark my results but I don't get any sensible results at all yet. Last time Peter said he's working on a more portable way to read dwarf information that doesn't require Linux. But I'm sure he'll give a more acurate update than me soon in this mail thread. As for stack traces, I don't think there's any big tasks left, but I summarize what I have in mind: * The haskell interface is done and I've iterated on it a bit, so it's in a decent shape at least. Some parts still need testing. * I wish I could implement the `forceCaseContinuation` that I've described in my thesis. If someone is good with code generation (I just suck at it, it's probably simple) and is willing to assist me a bit, please say so. :) * I tried benchmarking, I gave up after not getting any useful results. * I'm unfortunately totally incapable to help out with dwarf debug data generation, only Peter knows that part, particularly I never grasped his theoretical framework of causality in Haskell. * Peter and I have finally agreed on a simple and sensible way to implement `catchWithStack` that have all most good properties you would like. I just need to implement it and test it. I can definitely man up and implement this. :) Here's my master thesis btw [1], it should answer Ömer's question of how we retrieve a stack from a language you think won't have a stack. :) Cheers, Arash [1]: http://arashrouhani.com/papers/master-thesis.pdf On 2014-08-13 17:02, Johan Tibell wrote: Hi, How's the integration of DWARF support coming along? It's probably one of the most important improvements to the runtime in quite some time since unlocks *two* important features, namely * trustworthy profiling (using e.g. Linux perf events and other low-overhead, code preserving, sampling profilers), and * stack traces. The former is really important to move our core libraries performance up a notch. Right now -prof is too invasive for it to be useful when evaluating the hotspots in these libraries (which are already often heavily tuned). The latter one is really important for real life Haskell on the server, where you can sometimes can get some crash that only happens once a day under very specific conditions. Knowing where the crash happens is then *very* useful. -- Johan ___ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs ___ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs
Re: How's the integration of DWARF support coming along?
Without any overhead we'll get the runtime stack trace, which isn't exactly the same as what we can get with emulation, but has the benefit that we can leave it on in all of our shipped code if we like. This latter is a really crucial property for stack traces to be widely useful. On Wed, Aug 13, 2014 at 7:13 PM, Ömer Sinan Ağacan wrote: > Sorry for my previous email. (used a gmail shortcut by mistake) > > We won't have stacks as we have in imperative(without TCO) and strict > languages. So we still need some kind of emulation and I think this > means some extra run-time operations. I'm wondering about two things: > > 1) Do we still get same traces as we get using GHC.Stack right now? > 2) If yes, then how can we have that without any runtime costs? > > Thanks and sorry again for my previous email. > > --- > Ömer Sinan Ağacan > http://osa1.net > > > 2014-08-13 20:08 GMT+03:00 Ömer Sinan Ağacan : > > Will generated stack traces be different that > > > > --- > > Ömer Sinan Ağacan > > http://osa1.net > > > > > > 2014-08-13 19:56 GMT+03:00 Johan Tibell : > >> Yes, it doesn't use any code modification so it doesn't have runtime > >> overhead (except when generating the actual trace) or interfere with > >> compiler optimizations. In other words you can actually have it enabled > at > >> all time. It only requires that you compile with -g, just like with a C > >> compiler. > >> > >> > >> On Wed, Aug 13, 2014 at 6:45 PM, Ömer Sinan Ağacan < > omeraga...@gmail.com> > >> wrote: > >>> > >>> Is this stack trace support different than what we have currently? > >>> (e.g. the one implemented with GHC.Stack and cost centers) > >>> > >>> --- > >>> Ömer Sinan Ağacan > >>> http://osa1.net > >>> > >>> > >>> 2014-08-13 18:02 GMT+03:00 Johan Tibell : > >>> > Hi, > >>> > > >>> > How's the integration of DWARF support coming along? It's probably > one > >>> > of > >>> > the most important improvements to the runtime in quite some time > since > >>> > unlocks *two* important features, namely > >>> > > >>> > * trustworthy profiling (using e.g. Linux perf events and other > >>> > low-overhead, code preserving, sampling profilers), and > >>> > * stack traces. > >>> > > >>> > The former is really important to move our core libraries > performance up > >>> > a > >>> > notch. Right now -prof is too invasive for it to be useful when > >>> > evaluating > >>> > the hotspots in these libraries (which are already often heavily > tuned). > >>> > > >>> > The latter one is really important for real life Haskell on the > server, > >>> > where you can sometimes can get some crash that only happens once a > day > >>> > under very specific conditions. Knowing where the crash happens is > then > >>> > *very* useful. > >>> > > >>> > -- Johan > >>> > > >>> > > >>> > ___ > >>> > ghc-devs mailing list > >>> > ghc-devs@haskell.org > >>> > http://www.haskell.org/mailman/listinfo/ghc-devs > >>> > > >> > >> > ___ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs
Re: How's the integration of DWARF support coming along?
Sorry for my previous email. (used a gmail shortcut by mistake) We won't have stacks as we have in imperative(without TCO) and strict languages. So we still need some kind of emulation and I think this means some extra run-time operations. I'm wondering about two things: 1) Do we still get same traces as we get using GHC.Stack right now? 2) If yes, then how can we have that without any runtime costs? Thanks and sorry again for my previous email. --- Ömer Sinan Ağacan http://osa1.net 2014-08-13 20:08 GMT+03:00 Ömer Sinan Ağacan : > Will generated stack traces be different that > > --- > Ömer Sinan Ağacan > http://osa1.net > > > 2014-08-13 19:56 GMT+03:00 Johan Tibell : >> Yes, it doesn't use any code modification so it doesn't have runtime >> overhead (except when generating the actual trace) or interfere with >> compiler optimizations. In other words you can actually have it enabled at >> all time. It only requires that you compile with -g, just like with a C >> compiler. >> >> >> On Wed, Aug 13, 2014 at 6:45 PM, Ömer Sinan Ağacan >> wrote: >>> >>> Is this stack trace support different than what we have currently? >>> (e.g. the one implemented with GHC.Stack and cost centers) >>> >>> --- >>> Ömer Sinan Ağacan >>> http://osa1.net >>> >>> >>> 2014-08-13 18:02 GMT+03:00 Johan Tibell : >>> > Hi, >>> > >>> > How's the integration of DWARF support coming along? It's probably one >>> > of >>> > the most important improvements to the runtime in quite some time since >>> > unlocks *two* important features, namely >>> > >>> > * trustworthy profiling (using e.g. Linux perf events and other >>> > low-overhead, code preserving, sampling profilers), and >>> > * stack traces. >>> > >>> > The former is really important to move our core libraries performance up >>> > a >>> > notch. Right now -prof is too invasive for it to be useful when >>> > evaluating >>> > the hotspots in these libraries (which are already often heavily tuned). >>> > >>> > The latter one is really important for real life Haskell on the server, >>> > where you can sometimes can get some crash that only happens once a day >>> > under very specific conditions. Knowing where the crash happens is then >>> > *very* useful. >>> > >>> > -- Johan >>> > >>> > >>> > ___ >>> > ghc-devs mailing list >>> > ghc-devs@haskell.org >>> > http://www.haskell.org/mailman/listinfo/ghc-devs >>> > >> >> ___ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs
Re: How's the integration of DWARF support coming along?
Will generated stack traces be different that --- Ömer Sinan Ağacan http://osa1.net 2014-08-13 19:56 GMT+03:00 Johan Tibell : > Yes, it doesn't use any code modification so it doesn't have runtime > overhead (except when generating the actual trace) or interfere with > compiler optimizations. In other words you can actually have it enabled at > all time. It only requires that you compile with -g, just like with a C > compiler. > > > On Wed, Aug 13, 2014 at 6:45 PM, Ömer Sinan Ağacan > wrote: >> >> Is this stack trace support different than what we have currently? >> (e.g. the one implemented with GHC.Stack and cost centers) >> >> --- >> Ömer Sinan Ağacan >> http://osa1.net >> >> >> 2014-08-13 18:02 GMT+03:00 Johan Tibell : >> > Hi, >> > >> > How's the integration of DWARF support coming along? It's probably one >> > of >> > the most important improvements to the runtime in quite some time since >> > unlocks *two* important features, namely >> > >> > * trustworthy profiling (using e.g. Linux perf events and other >> > low-overhead, code preserving, sampling profilers), and >> > * stack traces. >> > >> > The former is really important to move our core libraries performance up >> > a >> > notch. Right now -prof is too invasive for it to be useful when >> > evaluating >> > the hotspots in these libraries (which are already often heavily tuned). >> > >> > The latter one is really important for real life Haskell on the server, >> > where you can sometimes can get some crash that only happens once a day >> > under very specific conditions. Knowing where the crash happens is then >> > *very* useful. >> > >> > -- Johan >> > >> > >> > ___ >> > ghc-devs mailing list >> > ghc-devs@haskell.org >> > http://www.haskell.org/mailman/listinfo/ghc-devs >> > > > ___ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs
Re: How's the integration of DWARF support coming along?
Yes, it doesn't use any code modification so it doesn't have runtime overhead (except when generating the actual trace) or interfere with compiler optimizations. In other words you can actually have it enabled at all time. It only requires that you compile with -g, just like with a C compiler. On Wed, Aug 13, 2014 at 6:45 PM, Ömer Sinan Ağacan wrote: > Is this stack trace support different than what we have currently? > (e.g. the one implemented with GHC.Stack and cost centers) > > --- > Ömer Sinan Ağacan > http://osa1.net > > > 2014-08-13 18:02 GMT+03:00 Johan Tibell : > > Hi, > > > > How's the integration of DWARF support coming along? It's probably one of > > the most important improvements to the runtime in quite some time since > > unlocks *two* important features, namely > > > > * trustworthy profiling (using e.g. Linux perf events and other > > low-overhead, code preserving, sampling profilers), and > > * stack traces. > > > > The former is really important to move our core libraries performance up > a > > notch. Right now -prof is too invasive for it to be useful when > evaluating > > the hotspots in these libraries (which are already often heavily tuned). > > > > The latter one is really important for real life Haskell on the server, > > where you can sometimes can get some crash that only happens once a day > > under very specific conditions. Knowing where the crash happens is then > > *very* useful. > > > > -- Johan > > > > > > ___ > > ghc-devs mailing list > > ghc-devs@haskell.org > > http://www.haskell.org/mailman/listinfo/ghc-devs > > > ___ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs
Re: How's the integration of DWARF support coming along?
Is this stack trace support different than what we have currently? (e.g. the one implemented with GHC.Stack and cost centers) --- Ömer Sinan Ağacan http://osa1.net 2014-08-13 18:02 GMT+03:00 Johan Tibell : > Hi, > > How's the integration of DWARF support coming along? It's probably one of > the most important improvements to the runtime in quite some time since > unlocks *two* important features, namely > > * trustworthy profiling (using e.g. Linux perf events and other > low-overhead, code preserving, sampling profilers), and > * stack traces. > > The former is really important to move our core libraries performance up a > notch. Right now -prof is too invasive for it to be useful when evaluating > the hotspots in these libraries (which are already often heavily tuned). > > The latter one is really important for real life Haskell on the server, > where you can sometimes can get some crash that only happens once a day > under very specific conditions. Knowing where the crash happens is then > *very* useful. > > -- Johan > > > ___ > ghc-devs mailing list > ghc-devs@haskell.org > http://www.haskell.org/mailman/listinfo/ghc-devs > ___ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs
Re: How's the integration of DWARF support coming along?
On Wed, Aug 13, 2014 at 5:07 PM, Tuncer Ayaz wrote: > On Wed, Aug 13, 2014 at 5:02 PM, Johan Tibell wrote: > > Hi, > > > > How's the integration of DWARF support coming along? It's probably > > one of the most important improvements to the runtime in quite some > > time since unlocks *two* important features, namely > > > > * trustworthy profiling (using e.g. Linux perf events and other > > low-overhead, code preserving, sampling profilers), and > > * stack traces. > > > > The former is really important to move our core libraries > > performance up a notch. Right now -prof is too invasive for it to be > > useful when evaluating the hotspots in these libraries (which are > > already often heavily tuned). > > > > The latter one is really important for real life Haskell on the > > server, where you can sometimes can get some crash that only happens > > once a day under very specific conditions. Knowing where the crash > > happens is then *very* useful. > > Doesn't it also enable using gdb and lldb, or is there another missing > piece? > No, those should also work. It enables *a lot* of generic infrastructure that programmers has written over the years. ___ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs
Re: How's the integration of DWARF support coming along?
On Wed, Aug 13, 2014 at 5:02 PM, Johan Tibell wrote: > Hi, > > How's the integration of DWARF support coming along? It's probably > one of the most important improvements to the runtime in quite some > time since unlocks *two* important features, namely > > * trustworthy profiling (using e.g. Linux perf events and other > low-overhead, code preserving, sampling profilers), and > * stack traces. > > The former is really important to move our core libraries > performance up a notch. Right now -prof is too invasive for it to be > useful when evaluating the hotspots in these libraries (which are > already often heavily tuned). > > The latter one is really important for real life Haskell on the > server, where you can sometimes can get some crash that only happens > once a day under very specific conditions. Knowing where the crash > happens is then *very* useful. Doesn't it also enable using gdb and lldb, or is there another missing piece? ___ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs
How's the integration of DWARF support coming along?
Hi, How's the integration of DWARF support coming along? It's probably one of the most important improvements to the runtime in quite some time since unlocks *two* important features, namely * trustworthy profiling (using e.g. Linux perf events and other low-overhead, code preserving, sampling profilers), and * stack traces. The former is really important to move our core libraries performance up a notch. Right now -prof is too invasive for it to be useful when evaluating the hotspots in these libraries (which are already often heavily tuned). The latter one is really important for real life Haskell on the server, where you can sometimes can get some crash that only happens once a day under very specific conditions. Knowing where the crash happens is then *very* useful. -- Johan ___ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs
