[Lldb-commits] [PATCH] D48393: Make DWARFParsing more thread-safe

2018-09-24 Thread Jonas Devlieghere via Phabricator via lldb-commits
JDevlieghere added a comment. In https://reviews.llvm.org/D48393#1139327, @labath wrote: > It's even more complicated than that, in case you really have reference > cycles, you can have multiple threads starting parsing from different points > in that cycle, and getting deadlocked waiting for t

[Lldb-commits] [PATCH] D48393: Make DWARFParsing more thread-safe

2018-09-24 Thread Greg Clayton via Phabricator via lldb-commits
clayborg added a comment. A little background might help here: The lldb_private::Module lock is used to prevent multiple queries into the DWARF from stomping on each other. Multi-threaded DWARF parsing was primarily added to speed up indexing and the only place it is used. Is that not true? Ind

[Lldb-commits] [PATCH] D48393: Make DWARFParsing more thread-safe

2018-09-25 Thread Jonas Devlieghere via Phabricator via lldb-commits
JDevlieghere added a comment. Thanks for the information, Greg! In https://reviews.llvm.org/D48393#1243588, @clayborg wrote: > A little background might help here: The lldb_private::Module lock is used to > prevent multiple queries into the DWARF from stomping on each other. > > Multi-threaded

[Lldb-commits] [PATCH] D48393: Make DWARFParsing more thread-safe

2018-09-25 Thread Pavel Labath via Phabricator via lldb-commits
labath added a comment. In https://reviews.llvm.org/D48393#1243195, @JDevlieghere wrote: > In https://reviews.llvm.org/D48393#1139327, @labath wrote: > > > The only sane algorithm I can come up right now is to make the list of > > parsed dies local to each thread/parsing entity (e.g. via a "visi

[Lldb-commits] [PATCH] D48393: Make DWARFParsing more thread-safe

2018-09-25 Thread Greg Clayton via Phabricator via lldb-commits
clayborg added a comment. In https://reviews.llvm.org/D48393#1244426, @JDevlieghere wrote: > Thanks for the information, Greg! > > In https://reviews.llvm.org/D48393#1243588, @clayborg wrote: > > > A little background might help here: The lldb_private::Module lock is used > > to prevent multiple

[Lldb-commits] [PATCH] D48393: Make DWARFParsing more thread-safe

2018-09-25 Thread Greg Clayton via Phabricator via lldb-commits
clayborg added a comment. In https://reviews.llvm.org/D48393#1244649, @labath wrote: > I agree with Greg that it would be best to restrict things such that there is > only one instance of parsing going on at any given time for a single module. > I think this was pretty much the state we reached

[Lldb-commits] [PATCH] D48393: Make DWARFParsing more thread-safe

2018-09-25 Thread Frederic Riss via Phabricator via lldb-commits
friss added a comment. In https://reviews.llvm.org/D48393#1245049, @clayborg wrote: > In https://reviews.llvm.org/D48393#1244649, @labath wrote: > > > I agree with Greg that it would be best to restrict things such that there > > is only one instance of parsing going on at any given time for a s

[Lldb-commits] [PATCH] D48393: Make DWARFParsing more thread-safe

2018-09-25 Thread Greg Clayton via Phabricator via lldb-commits
clayborg added a comment. In https://reviews.llvm.org/D48393#1245342, @friss wrote: > In https://reviews.llvm.org/D48393#1245049, @clayborg wrote: > > > In https://reviews.llvm.org/D48393#1244649, @labath wrote: > > > > > I agree with Greg that it would be best to restrict things such that > > >

[Lldb-commits] [PATCH] D48393: Make DWARFParsing more thread-safe

2018-06-20 Thread Frederic Riss via Phabricator via lldb-commits
friss created this revision. friss added reviewers: clayborg, labath, jingham. Herald added subscribers: JDevlieghere, aprantl. Debug information is read lazily so a lot of the state of the reader is constructred incrementally. There's also nothing preventing symbol queries to be performed at the

[Lldb-commits] [PATCH] D48393: Make DWARFParsing more thread-safe

2018-06-20 Thread Zachary Turner via Phabricator via lldb-commits
zturner added a comment. Long term I think one potentially interesting possibility for solving a lot of these threading and lazy evaluation issues is to make a task queue that runs all related operations on a single thread and returns a future you can wait on. This way, the core data structure

[Lldb-commits] [PATCH] D48393: Make DWARFParsing more thread-safe

2018-06-20 Thread Frederic Riss via Phabricator via lldb-commits
friss added a comment. In https://reviews.llvm.org/D48393#1138398, @zturner wrote: > Long term I think one potentially interesting possibility for solving a lot > of these threading and lazy evaluation issues is to make a task queue that > runs all related operations on a single thread and retu

[Lldb-commits] [PATCH] D48393: Make DWARFParsing more thread-safe

2018-06-21 Thread Pavel Labath via Phabricator via lldb-commits
labath added a comment. I am not sure this will actually solve the problems you are seeing. This may avoid corrupting the internal DenseMap data structures, but it will not make the algorithm using them actually correct. For example the pattern in `ParseTypeFromDWARF` is: 1. check the "already

[Lldb-commits] [PATCH] D48393: Make DWARFParsing more thread-safe

2018-06-21 Thread Greg Clayton via Phabricator via lldb-commits
clayborg added a comment. In https://reviews.llvm.org/D48393#1138989, @labath wrote: > I am not sure this will actually solve the problems you are seeing. This may > avoid corrupting the internal DenseMap data structures, but it will not make > the algorithm using them actually correct. > For

[Lldb-commits] [PATCH] D48393: Make DWARFParsing more thread-safe

2018-06-21 Thread Zachary Turner via Phabricator via lldb-commits
zturner added a subscriber: friss. zturner added a comment. Related question: Is the laziness done to save memory, startup time, or both? https://reviews.llvm.org/D48393 ___ lldb-commits mailing list lldb-commits@lists.llvm.org http://lists.llvm.org

[Lldb-commits] [PATCH] D48393: Make DWARFParsing more thread-safe

2018-06-21 Thread Pavel Labath via Phabricator via lldb-commits
labath added a comment. In https://reviews.llvm.org/D48393#1139270, @clayborg wrote: > In https://reviews.llvm.org/D48393#1138989, @labath wrote: > > > I am not sure this will actually solve the problems you are seeing. This > > may avoid corrupting the internal DenseMap data structures, but it

Re: [Lldb-commits] [PATCH] D48393: Make DWARFParsing more thread-safe

2018-06-20 Thread Jim Ingham via lldb-commits
> On Jun 20, 2018, at 3:14 PM, Frederic Riss via Phabricator > wrote: > > friss added a comment. > > In https://reviews.llvm.org/D48393#1138398, @zturner wrote: > >> Long term I think one potentially interesting possibility for solving a lot >> of these threading and lazy evaluation issues

Re: [Lldb-commits] [PATCH] D48393: Make DWARFParsing more thread-safe

2018-06-21 Thread Zachary Turner via lldb-commits
Related question: Is the laziness done to save memory, startup time, or both? On Thu, Jun 21, 2018 at 7:36 AM Greg Clayton via Phabricator < revi...@reviews.llvm.org> wrote: > clayborg added a comment. > > In https://reviews.llvm.org/D48393#1138989, @labath wrote: > > > I am not sure this will act

Re: [Lldb-commits] [PATCH] D48393: Make DWARFParsing more thread-safe

2018-06-21 Thread Greg Clayton via lldb-commits
> On Jun 21, 2018, at 7:47 AM, Zachary Turner wrote: > > Related question: Is the laziness done to save memory, startup time, or both? Both. It allows us to fetch only what we need when we need it. Time to break at main.cpp:123 is much quicker. Using LLDB for symbolication is much quicker as

Re: [Lldb-commits] [PATCH] D48393: Make DWARFParsing more thread-safe

2018-06-21 Thread Zachary Turner via lldb-commits
Performance i get. Gdb is almost unusable for large programs because of how long it takes to load debug info. Do you have specific numbers on memory usage? How much memory (absolute and %) is saved by loading debug info lazily on a relatively large project? On Thu, Jun 21, 2018 at 7:54 AM Greg Cla

Re: [Lldb-commits] [PATCH] D48393: Make DWARFParsing more thread-safe

2018-06-21 Thread Greg Clayton via lldb-commits
> On Jun 21, 2018, at 7:58 AM, Zachary Turner wrote: > > Performance i get. Gdb is almost unusable for large programs because of how > long it takes to load debug info. Agreed. With our new DWARF5 tables, we will be even better. Or test on a mac with dSYM files and you will get similar numbe

Re: [Lldb-commits] [PATCH] D48393: Make DWARFParsing more thread-safe

2018-06-21 Thread Frédéric Riss via lldb-commits
> On Jun 21, 2018, at 3:18 AM, Pavel Labath via Phabricator > wrote: > > labath added a comment. > > I am not sure this will actually solve the problems you are seeing. This may > avoid corrupting the internal DenseMap data structures, but it will not make > the algorithm using them actuall

Re: [Lldb-commits] [PATCH] D48393: Make DWARFParsing more thread-safe

2018-06-21 Thread Frédéric Riss via lldb-commits
> On Jun 21, 2018, at 8:03 AM, Pavel Labath via Phabricator > wrote: > > labath added a comment. > > In https://reviews.llvm.org/D48393#1139270, @clayborg wrote: > >> In https://reviews.llvm.org/D48393#1138989, @labath wrote: >> >>> I am not sure this will actually solve the problems you ar

Re: [Lldb-commits] [PATCH] D48393: Make DWARFParsing more thread-safe

2018-06-21 Thread Greg Clayton via lldb-commits
> On Jun 21, 2018, at 8:32 AM, Frédéric Riss wrote: > > > >> On Jun 21, 2018, at 3:18 AM, Pavel Labath via Phabricator >> wrote: >> >> labath added a comment. >> >> I am not sure this will actually solve the problems you are seeing. This may >> avoid corrupting the internal DenseMap data

Re: [Lldb-commits] [PATCH] D48393: Make DWARFParsing more thread-safe

2018-06-21 Thread Jim Ingham via lldb-commits
Part of gdb's slow startup used to be because it built its indexes manually. But also, while gdb does do lazy ingestion of debug information it's laziness is (or was last time I looked at it) on a compile-unit boundary, so when you stop in a file that includes a many complex types you can end

Re: [Lldb-commits] [PATCH] D48393: Make DWARFParsing more thread-safe

2018-06-22 Thread Pavel Labath via lldb-commits
On Wed, 20 Jun 2018 at 23:21, Jim Ingham wrote: > > It is not uncommon that you would be parsing the DWARF for module A and find > a type that is only known as a forward declaration. In that case, lldb will > look through the other Modules' debug info for a real definition, parse that > and im

Re: [Lldb-commits] [PATCH] D48393: Make DWARFParsing more thread-safe

2018-06-22 Thread Jim Ingham via lldb-commits
> On Jun 22, 2018, at 4:05 AM, Pavel Labath wrote: > > On Wed, 20 Jun 2018 at 23:21, Jim Ingham wrote: >> >> It is not uncommon that you would be parsing the DWARF for module A and find >> a type that is only known as a forward declaration. In that case, lldb will >> look through the other

Re: [Lldb-commits] [PATCH] D48393: Make DWARFParsing more thread-safe

2018-06-25 Thread Pavel Labath via lldb-commits
On Fri, 22 Jun 2018 at 18:42, Jim Ingham wrote: > > > > > On Jun 22, 2018, at 4:05 AM, Pavel Labath wrote: > > > > On Wed, 20 Jun 2018 at 23:21, Jim Ingham wrote: > >> > >> It is not uncommon that you would be parsing the DWARF for module A and > >> find a type that is only known as a forward d