Re: [Lldb-commits] [PATCH] D132624: [LLDB] Devirtualize coroutine promise types for `std::coroutine_handle`

[Adrian Vogelsgesang via lldb-commits]

Thank you for that hint, Jim! This infrastructure indeed looks very
interesting for my use case.
I will most likely use the HistoryThread class to represent the backtrace
of the logical coroutine threads.

Is my understanding correct, that `GetExtendedBacktraceThreads` only allows
me to append additional stack trace frames to existing, physical
threads, though? I want to expose completely new threads instead. In a
sense, I want to display each pending request inside my asynchronous
multiplexer as its own thread, such that I can see not only the requests
which an OS-level thread is currently working on, but also the onse which
are currently queued/not yet processed/blocked on IO

On Fri, Aug 26, 2022 at 6:38 PM Jim Ingham  wrote:

>
>
> > On Aug 26, 2022, at 7:05 AM, Adrian Vogelsgesang via Phabricator via
> lldb-commits  wrote:
> >
> > avogelsgesang added a comment.
> >
> >> I don't know much about coroutines, but it seems like your goal is to
> format them like a linked list
> >
> > actually, my preferred goal would be to show them as a logical,
> user-level thread. Such that you can type
> >
> >  thread backtrace cxxcoro:0xb2a0
> >
> > to get the backtrace of the logical coroutine thread routed at the
> coroutine at address `0xb2a0`, or maybe even
> >
> >  thread backtrace cxxcoro:hdl
> >
> > where `hdl` is evaluated as an expression to identify the coroutine
> handle from where to dump the backtrace.
> >
> > Also, it would be neat if those logical threads show up in `thread
> list`...
> >
> > But it seems there is currently no infrastructure yet in LLDB for
> logical threads provided by `LanguageRuntime` plugins.
> >
> > I guess at some point, I will write an RFC about that on discourse. But
> before that, I will first do some more exploration on how LLDB works and I
> will first grab the low-hanging fruits (like a data formatter for
> `std::coroutine_handle` and patching LLVM to emit the necessary debug info)
>
> lldb has the notion of "extended backtrace threads" - backed by lldb's
> "History" threads - that it uses in a similar circumstance handling Darwin
> dispatch queues.  If you have a thread that is serving a Darwin "dispatch
> queue" SBThread.GetExtendedBacktraceThreads will return the backtrace of
> the thread that enqueued the work, at the point where the enqueuing is
> done.  I bet you could make the same setup work for these coroutines.
>
> Jim
>
>
> >
> >
> > Repository:
> >  rG LLVM Github Monorepo
> >
> > CHANGES SINCE LAST ACTION
> >  https://reviews.llvm.org/D132624/new/
> >
> > https://reviews.llvm.org/D132624
> >
> > ___
> > lldb-commits mailing list
> > lldb-commits@lists.llvm.org
> > https://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-commits
>
>
___
lldb-commits mailing list
lldb-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-commits

Re: [Lldb-commits] [PATCH] D132624: [LLDB] Devirtualize coroutine promise types for `std::coroutine_handle`

[Jim Ingham via lldb-commits]

> On Aug 26, 2022, at 5:08 PM, Adrian Vogelsgesang 
>  wrote:
> 
> Thank you for that hint, Jim! This infrastructure indeed looks very 
> interesting for my use case.
> I will most likely use the HistoryThread class to represent the backtrace of 
> the logical coroutine threads.
> 
> Is my understanding correct, that `GetExtendedBacktraceThreads` only allows 
> me to append additional stack trace frames to existing, physical threads, 
> though? I want to expose completely new threads instead. In a sense, I want 
> to display each pending request inside my asynchronous multiplexer as its own 
> thread, such that I can see not only the requests which an OS-level thread is 
> currently working on, but also the onse which are currently queued/not yet 
> processed/blocked on IO

I don't know how the std::coroutine stuff works...  In what sense is a "pending 
request" a Thread?  Isn't it a bunch of state and a function pointer that's 
waiting around to be run?  In which case, maybe we need another entity to 
describe those?  This is something that we don't handle at all in the "dispatch 
queue" interface in lldb.  We don't have a way to say "show me all queued but 
not being serviced" work.  When we get around to supporting that we'd need to 
add something to describe "queue-able work".

OTOH, if you are really dealing something like a scheduler so that the "pending 
work" is threads that were moved off of "system threads" and left in memory 
till they are re-enqueued, then you do need a way to tell lldb about those 
threads.  There are two ways to do that in lldb.  One is the OperatingSystem 
Plugin, which requires only that you provide a "Register Context's" for threads 
that you know about but the system doesn't.  The stack pointer in the context 
you return should point to the stored stack, and then lldb will do the work of 
producing the rest of the stack frame for that thread.  The other way is using 
the ScriptedProcess feature, where you can just make up threads and their stack 
frames out of whole cloth.  If your thread is still backed by the stack memory 
it was using, then the OperatingSystem plugin way is the easiest way to go for 
this.

Jim

> 
> On Fri, Aug 26, 2022 at 6:38 PM Jim Ingham  > wrote:
>> 
>> 
>> > On Aug 26, 2022, at 7:05 AM, Adrian Vogelsgesang via Phabricator via 
>> > lldb-commits > > > wrote:
>> > 
>> > avogelsgesang added a comment.
>> > 
>> >> I don't know much about coroutines, but it seems like your goal is to 
>> >> format them like a linked list
>> > 
>> > actually, my preferred goal would be to show them as a logical, user-level 
>> > thread. Such that you can type
>> > 
>> >  thread backtrace cxxcoro:0xb2a0
>> > 
>> > to get the backtrace of the logical coroutine thread routed at the 
>> > coroutine at address `0xb2a0`, or maybe even
>> > 
>> >  thread backtrace cxxcoro:hdl
>> > 
>> > where `hdl` is evaluated as an expression to identify the coroutine handle 
>> > from where to dump the backtrace.
>> > 
>> > Also, it would be neat if those logical threads show up in `thread list`...
>> > 
>> > But it seems there is currently no infrastructure yet in LLDB for logical 
>> > threads provided by `LanguageRuntime` plugins.
>> > 
>> > I guess at some point, I will write an RFC about that on discourse. But 
>> > before that, I will first do some more exploration on how LLDB works and I 
>> > will first grab the low-hanging fruits (like a data formatter for 
>> > `std::coroutine_handle` and patching LLVM to emit the necessary debug info)
>> 
>> lldb has the notion of "extended backtrace threads" - backed by lldb's 
>> "History" threads - that it uses in a similar circumstance handling Darwin 
>> dispatch queues.  If you have a thread that is serving a Darwin "dispatch 
>> queue" SBThread.GetExtendedBacktraceThreads will return the backtrace of the 
>> thread that enqueued the work, at the point where the enqueuing is done.  I 
>> bet you could make the same setup work for these coroutines.
>> 
>> Jim
>> 
>> 
>> > 
>> > 
>> > Repository:
>> >  rG LLVM Github Monorepo
>> > 
>> > CHANGES SINCE LAST ACTION
>> >  https://reviews.llvm.org/D132624/new/
>> > 
>> > https://reviews.llvm.org/D132624
>> > 
>> > ___
>> > lldb-commits mailing list
>> > lldb-commits@lists.llvm.org 
>> > https://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-commits
>> 

___
lldb-commits mailing list
lldb-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-commits

Re: [Lldb-commits] [PATCH] D132624: [LLDB] Devirtualize coroutine promise types for `std::coroutine_handle`

[Jim Ingham via lldb-commits]

> On Aug 26, 2022, at 7:05 AM, Adrian Vogelsgesang via Phabricator via 
> lldb-commits  wrote:
> 
> avogelsgesang added a comment.
> 
>> I don't know much about coroutines, but it seems like your goal is to format 
>> them like a linked list
> 
> actually, my preferred goal would be to show them as a logical, user-level 
> thread. Such that you can type
> 
>  thread backtrace cxxcoro:0xb2a0
> 
> to get the backtrace of the logical coroutine thread routed at the coroutine 
> at address `0xb2a0`, or maybe even
> 
>  thread backtrace cxxcoro:hdl
> 
> where `hdl` is evaluated as an expression to identify the coroutine handle 
> from where to dump the backtrace.
> 
> Also, it would be neat if those logical threads show up in `thread list`...
> 
> But it seems there is currently no infrastructure yet in LLDB for logical 
> threads provided by `LanguageRuntime` plugins.
> 
> I guess at some point, I will write an RFC about that on discourse. But 
> before that, I will first do some more exploration on how LLDB works and I 
> will first grab the low-hanging fruits (like a data formatter for 
> `std::coroutine_handle` and patching LLVM to emit the necessary debug info)

lldb has the notion of "extended backtrace threads" - backed by lldb's 
"History" threads - that it uses in a similar circumstance handling Darwin 
dispatch queues.  If you have a thread that is serving a Darwin "dispatch 
queue" SBThread.GetExtendedBacktraceThreads will return the backtrace of the 
thread that enqueued the work, at the point where the enqueuing is done.  I bet 
you could make the same setup work for these coroutines.

Jim


> 
> 
> Repository:
>  rG LLVM Github Monorepo
> 
> CHANGES SINCE LAST ACTION
>  https://reviews.llvm.org/D132624/new/
> 
> https://reviews.llvm.org/D132624
> 
> ___
> lldb-commits mailing list
> lldb-commits@lists.llvm.org
> https://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-commits

___
lldb-commits mailing list
lldb-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-commits

Re: [Lldb-commits] [PATCH] D132624: [LLDB] Devirtualize coroutine promise types for `std::coroutine_handle`

[Jim Ingham via lldb-commits]

> On Aug 25, 2022, at 8:11 AM, Pavel Labath via Phabricator via lldb-commits 
>  wrote:
> 
> labath added a comment.
> 
> In D132624#3748434 , @avogelsgesang 
> wrote:
> 
>>> The only concern is that if it would be not so easy to read if there are 
>>> too many levels? (Consider the 30levels example in D132451 
>>> ). If it would be better to print ... at 
>>> certain level? Or this is already handled by lldb?
>> 
>> Agree, it would be better to limit the printing at some point. However, 
>> afaict, limiting the depth to which children are expanded cannot be done 
>> from inside the data formatter/synthetic children frontend. I am not sure if 
>> lldb already has a separate mechanism in place to limit printing in this 
>> case. If it does not, I think it makes sense to add it, but that would be a 
>> separate commit
> 
> The best (I think) mechanism we have is the "pointer depth" limit (defaulting 
> to 1). It works fine on regular objects, but requires some care with 
> synthetic children. The fact that it does not kick in here leads me to 
> believe that the data formatter is creating the synthetic children as non 
> pointer objects (e.g. by automatically dereferencing any nested pointers), 
> even though the structures clearly contain some pointers inside. That is a 
> problem. Not only it creates excessively large outputs, it can even cause 
> lldb to hang (looping endlessly while trying to print "all" children) if the 
> data structures it is trying to print are circular (e.g. due to corruption).

There are actually two separate controls for the depth of child traversal:

(lldb) help v
...
   -D  ( --depth  )
Set the max recurse depth when dumping aggregate types (default is 
infinity).
...
   -P  ( --ptr-depth  )
The number of pointers to be traversed when dumping values (default 
is zero).

There have to be two, because aggregate types can't have cycles, so it's safe 
to set those to a high value, but pointer following can lead to cycles and we 
don't currently do cycle detection so you have to be more careful with this 
setting.

Jim


> 
> 
> Repository:
>  rG LLVM Github Monorepo
> 
> CHANGES SINCE LAST ACTION
>  https://reviews.llvm.org/D132624/new/
> 
> https://reviews.llvm.org/D132624
> 
> ___
> lldb-commits mailing list
> lldb-commits@lists.llvm.org
> https://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-commits

___
lldb-commits mailing list
lldb-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-commits