Re: DWARF support
Am Fr., 19. Nov. 2021 um 01:09 Uhr schrieb Ben Gamari : > Artem Pelenitsyn writes: > > > Another question would be where do I read about Haskell-native stack > > unwinder. The issue and MR Ben referenced have descriptions, but the MR > > didn't touch anything inside `docs` which is a bit scary. Are there any > > good recourses to dive into it besides the source code in the MR? > > > Indeed the user's guide documentation surrounding info table provenance > should be updated to note this new capability. > Hey Ben, Hey Artem, Hey all, Thanks for bringing this up! I'll add some documentation to the user's guide and will likely write a blog post covering the "material" from notes in a more approachable way. Two details to note regarding the IPE based backtrace mechanism: - It's based on return frames on the STG stack. So, it doesn't work for trivial programs. A return frame is produced when the scrutinee of a case expression is evaluated. The test [1] shows a working example. - To get backtraces containing references in libraries, those have to be compiled with `-finfo-table-map`, too. Now that I've got the attention of many GHC-Devs: A big Thank-You! goes to Ben and Matthew that were always reachable and always very helpful during the whole development process of this feature! Best regards, Sven [1] - https://gitlab.haskell.org/ghc/ghc/-/blob/aed98ddaf72cc38fb570d8415cac5de9d818/testsuite/tests/rts/decodeMyStack.hs ___ ghc-devs mailing list ghc-devs@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
Re: DWARF support
Artem Pelenitsyn writes: > Another question would be where do I read about Haskell-native stack > unwinder. The issue and MR Ben referenced have descriptions, but the MR > didn't touch anything inside `docs` which is a bit scary. Are there any > good recourses to dive into it besides the source code in the MR? > Indeed the user's guide documentation surrounding info table provenance should be updated to note this new capability. There is a long Note in compiler/GHC/Driver/GenerateCgIPEStub.hs which describes the implementation. As far as the interface is concerned, it's quite straightforward: * One can call GHC.Stack.CloneStack.cloneMyStack, which returns a "frozen" copy of the calling context's stack * One can then call GHC.Stack.CloneStack.decode to decode the stack into a list of frames. Did you have any particular questions? Cheers, - Ben signature.asc Description: PGP signature ___ ghc-devs mailing list ghc-devs@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
Re: How to build Haddock documentation quickly?
Hi Norman, Could you see if the command-line described here is of any help? https://gitlab.haskell.org/ghc/ghc/-/issues/17929#building-the-docs Particularly the "--freeze1 --flavour=Quick" bit. Cheers! Le 18/11/2021 à 21:06, Norman Ramsey a écrit : In service of #20528, I'm trying to clean up some Haddock documentation in the GHC sources. The only command I know is ./hadrian/build -j _build/docs/html/index.html It turns out that if I change Haddock comments in one file (GHC/Tc/Gen/Foreign.hs), rebuilding the HTML takes a minute and a quarter. That's a little long. (And I had one build take over four minutes, which is even longer.) What tricks can I use to speed up this process? Might there be a way to run Haddock directly until things are to my liking? Norman ___ ghc-devs mailing list ghc-devs@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs -- Hécate ✨ 🐦: @TechnoEmpress IRC: Hecate WWW: https://glitchbra.in RUN: BSD ___ ghc-devs mailing list ghc-devs@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
How to build Haddock documentation quickly?
In service of #20528, I'm trying to clean up some Haddock documentation in the GHC sources. The only command I know is ./hadrian/build -j _build/docs/html/index.html It turns out that if I change Haddock comments in one file (GHC/Tc/Gen/Foreign.hs), rebuilding the HTML takes a minute and a quarter. That's a little long. (And I had one build take over four minutes, which is even longer.) What tricks can I use to speed up this process? Might there be a way to run Haddock directly until things are to my liking? Norman ___ ghc-devs mailing list ghc-devs@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
Re: DWARF support
Another question would be where do I read about Haskell-native stack unwinder. The issue and MR Ben referenced have descriptions, but the MR didn't touch anything inside `docs` which is a bit scary. Are there any good recourses to dive into it besides the source code in the MR? -- Best, Artem On Thu, Nov 18, 2021, 11:31 AM Chris Smith wrote: > Just to satisfy my curiosity here, when talking about backtraces here, are > you talking about a lexical call stack, or an execution stack? > > On Thu, Nov 18, 2021 at 11:24 AM Richard Eisenberg > wrote: > >> >> >> On Nov 18, 2021, at 10:29 AM, Ben Gamari wrote: >> >> At this point, for backtrace support I would rather put my money is on a >> native Haskell stack unwinder (such as Sven Tennie's work [3,4]). Not only >> is it more portable but it is also more robust (whereas with DWARF any >> single object lacking debug information would break unwinding), and is >> significantly less costly since we know much more about the structure of >> our stack than a DWARF unwinder would. >> >> >> Interesting -- this is helpful to know. I had heard about DWARF support >> for some years and thought that it would deliver stack traces. Now I will >> look for other sources. All good -- I understand how this is hard! -- and >> nice to know about. >> >> Thanks for the writeup, Ben. >> >> Richard >> ___ >> ghc-devs mailing list >> ghc-devs@haskell.org >> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs >> > ___ > ghc-devs mailing list > ghc-devs@haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs > ___ ghc-devs mailing list ghc-devs@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
Re: DWARF support
Just to satisfy my curiosity here, when talking about backtraces here, are you talking about a lexical call stack, or an execution stack? On Thu, Nov 18, 2021 at 11:24 AM Richard Eisenberg wrote: > > > On Nov 18, 2021, at 10:29 AM, Ben Gamari wrote: > > At this point, for backtrace support I would rather put my money is on a > native Haskell stack unwinder (such as Sven Tennie's work [3,4]). Not only > is it more portable but it is also more robust (whereas with DWARF any > single object lacking debug information would break unwinding), and is > significantly less costly since we know much more about the structure of > our stack than a DWARF unwinder would. > > > Interesting -- this is helpful to know. I had heard about DWARF support > for some years and thought that it would deliver stack traces. Now I will > look for other sources. All good -- I understand how this is hard! -- and > nice to know about. > > Thanks for the writeup, Ben. > > Richard > ___ > ghc-devs mailing list > ghc-devs@haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs > ___ ghc-devs mailing list ghc-devs@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
Re: DWARF support
> On Nov 18, 2021, at 10:29 AM, Ben Gamari wrote: > > At this point, for backtrace support I would rather put my money is on a > native Haskell stack unwinder (such as Sven Tennie's work [3,4]). Not only > is it more portable but it is also more robust (whereas with DWARF any > single object lacking debug information would break unwinding), and is > significantly less costly since we know much more about the structure of > our stack than a DWARF unwinder would. Interesting -- this is helpful to know. I had heard about DWARF support for some years and thought that it would deliver stack traces. Now I will look for other sources. All good -- I understand how this is hard! -- and nice to know about. Thanks for the writeup, Ben. Richard___ ghc-devs mailing list ghc-devs@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
Re: DWARF support
Richard Eisenberg writes: > Thanks for this! > >> On Nov 17, 2021, at 7:27 AM, Moritz Angermann >> wrote: >> >> For Linux and most BSDs, we have settled on the Executable and Linking >> Format (ELF) as the container format for >> your machine code. And you might see where the inspiration for DWARF might >> come from. > > This suggests to me that DWARF is the canonical format for debugging > information on Linux and most BSDs. Is that statement correct? If so, how is > that different to "platform-native"? Actually, the precise wording doesn't > matter: I think I'm just requesting for a more direct relationship between > "DWARF" and "compatibility with all the debugging and profiling tools you use > for other languages". > >> >> For macOS, we have mach object (mach-o) as the container format. Its >> distinctly different to ELF, and also the >> reason why Linux/BSD and macOS are sometimes substantially different, wrt to >> executable packaging and linking. > > OK. So there is no macOS support here. That's fine -- I'm just trying to > understand the status quo. > >> >> For windows we have Portable Executable (PE) as the container format. > > This implies that the DWARF work is (unsurprisingly) completely inapplicable > for Windows. > >> Depending on how familiar you are with development on macOS, you might know >> the notion of dSYM folders, >> where macOS usually separate the application binary into the binary, and >> then stores the (d)ebug (SYM)bols in >> a separate folder. Those are iirc DWARF objects in the end. > > This suggests to me that the DWARF work is applicable to the macOS use case, > but much more work still needs to be done. OK. > > Looking for more information, I checked the manual. And I found this: > https://downloads.haskell.org/ghc/latest/docs/html/users_guide/debug-info.html > It's really helpful! And it suggests that I actually *can* do this on > macOS. Perhaps that video will really happen after all. > I should clarify here that there are really two pieces to "debug information support": * the code generation logic responsible for producing the DWARF metadata * the RTS support for using that information to unwind the stack at runtime That section really describes the former. As noted in my earlier email and in #20702, the library used by the RTS to implement unwinding only targets ELF platforms. Cheers, - Ben signature.asc Description: PGP signature ___ ghc-devs mailing list ghc-devs@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
Re: DWARF support
Richard Eisenberg writes: > Hi devs, > > I was intrigued by Bodigrim's comment about HasCallStack in base > (https://github.com/haskell/core-libraries-committee/issues/5#issuecomment-970942580) > that there are other alternatives, such as DWARF. Over the years, I > had tuned out every time I saw the word DWARF: it was (and is!) an > unknown acronym and seems like a low-level detail. But Bodigrim's > comment made me want to re-think this stance. > > I found Ben's series of blog posts on DWARF, starting with > https://www.haskell.org/ghc/blog/20200403-dwarf-1.html. These are very > helpful! In particular, they taught me that DWARF = platform-native > debugging metadata. Is that translation accurate? If so, perhaps we > should use both names: if I see that GHC x.y.z has DWARF support, I > quickly scroll to the next bullet. If I see that GHC x.y.z has support > for platform-native debugging metadata and is now compatible with e.g. > gdb, I'm interested. > > Going further, I have a key question for my use case: is this support > available on Mac? The first post in the series describes support for > "Linux and several BSDs" and the last post says that "Windows PDB > support" is future work. (Is "PDB" platform-native debugging metadata > for Windows? I don't know.) But I don't see any mention of Mac. What's > the status here? > The problem is that DWARF is only one format for debugging information. Windows uses PDB, which is practically undocumented and consequently essentially unsupportable, AFAICT. I believe some other platforms still use the STABS format, which I do not believe we will ever support. Sadly, Darwin is, as usual, a bit of a tricky case. The platform seems to use a variant of the typical "split-debug" packaging configuration for debug information. Apple calls this separate debug information `dsym` files. In typical Apple fasion, the documentation surrounding this mechanism is very poor. It appears that the user or packager must generate these files manually using dsymutils but it's quite unclear how a consuming library is to find the dsym for a particular object. To make matters worse, the library that we use for unwinding, libdw, has absolutely no support for it. Ultimately, my goals for working on DWARF were: 1. To ensure that we had *some* baseline symbol and line information support on platforms for low-level debugging. I have found even basic line information invaluable when looking at code generation issues and thankfully gdb can always use inline DWARF information, even on platforms like on Windows where it isn't quite "native" 2. Providing an interface for users to unwind the stack from Haskell on the "easy" platforms (e.g. the ELF-based BSDs and Linux). I was hoping someone would express interest in picking up Darwin and Windows, at which point we could start to push this as a universal backtrace mechanism, but sadly this hasn't yet happened. 3. Provide a basis for statistic profiling support. This is sadly complicated by the stack pointer issues that Andreas and I note in our respective blog posts [1,2]. Sadly, overcoming this without breaking potential users is harder than we thought due to limitations in LLVM. At this point, for backtrace support I would rather put my money is on a native Haskell stack unwinder (such as Sven Tennie's work [3,4]). Not only is it more portable but it is also more robust (whereas with DWARF any single object lacking debug information would break unwinding), and is significantly less costly since we know much more about the structure of our stack than a DWARF unwinder would. Cheers, - Ben [1] https://well-typed.com/blog/2020/04/dwarf-4/ [2] https://well-typed.com/blog/2021/07/ghc-sp-profiling/ [3] https://gitlab.haskell.org/ghc/ghc/-/issues/18163 [4] https://gitlab.haskell.org/ghc/ghc/-/merge_requests/5456 signature.asc Description: PGP signature ___ ghc-devs mailing list ghc-devs@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
