Documented in https://phabricator.haskell.org/D4893
Ömer Ömer Sinan Ağacan <[email protected]>, 22 Haz 2018 Cum, 09:54 tarihinde şunu yazdı: > > OK, finally everything makes sense I think. I was very confused by the code > and > previous emails where you said: > > > Large objects can only be primitive objects, like MUT_ARR_PTRS, allocated by > > the RTS, and none of these have SRTs. > > I was pointing out that this is not entirely correct; we allocate large > stacks. > But as you say scavenge_one() handles that case by scavenging stack SRTs. > > So in summary: > > - scavenge_one() is called to scavenge mut_lists and large objects. > - When scavenging mut_lists no need to scaveng SRTs (see previous emails) > - When scavenging large objects we know that certain objects can't be large > (i.e. FUN, THUNK), but some others can (i.e. STACK), so scavenge_one() > scavenges stack SRTs but does not scavenge FUN and THUNK SRTs. > > Ömer > > Simon Marlow <[email protected]>, 21 Haz 2018 Per, 21:27 tarihinde şunu > yazdı: > > > > When scavenge_one() sees a STACK, it calls scavenge_stack() which traverses > > the stack frames, including their SRTs. > > > > So I don't understand what's going wrong for you - how are the SRTs not > > being traversed? > > > > Cheers > > Simon > > > > On 21 June 2018 at 11:58, Ömer Sinan Ağacan <[email protected]> wrote: > >> > >> Here's an example where we allocate a large (4K) stack: > >> > >> >>> bt > >> #0 allocateMightFail (cap=0x7f366808cfc0 <MainCapability>, > >> n=4096) at rts/sm/Storage.c:876 > >> #1 0x00007f3667e4a85d in allocate (cap=0x7f366808cfc0 > >> <MainCapability>, n=4096) at rts/sm/Storage.c:849 > >> #2 0x00007f3667e16f46 in threadStackOverflow (cap=0x7f366808cfc0 > >> <MainCapability>, tso=0x4200152a68) at rts/Threads.c:600 > >> #3 0x00007f3667e12a64 in schedule > >> (initialCapability=0x7f366808cfc0 <MainCapability>, task=0x78c970) at > >> rts/Schedule.c:520 > >> #4 0x00007f3667e1215f in scheduleWaitThread (tso=0x4200105388, > >> ret=0x0, pcap=0x7ffef40dce78) at rts/Schedule.c:2533 > >> #5 0x00007f3667e25685 in rts_evalLazyIO (cap=0x7ffef40dce78, > >> p=0x736ef8, ret=0x0) at rts/RtsAPI.c:530 > >> #6 0x00007f3667e25f7a in hs_main (argc=16, argv=0x7ffef40dd0a8, > >> main_closure=0x736ef8, rts_config=...) t rts/RtsMain.c:72 > >> #7 0x00000000004f738f in main () > >> > >> This is based on an old tree so source locations may not be correct, it's > >> this > >> code in threadStackOverflow(): > >> > >> // Charge the current thread for allocating stack. Stack usage is > >> // non-deterministic, because the chunk boundaries might vary from > >> // run to run, but accounting for this is better than not > >> // accounting for it, since a deep recursion will otherwise not be > >> // subject to allocation limits. > >> cap->r.rCurrentTSO = tso; > >> new_stack = (StgStack*) allocate(cap, chunk_size); > >> cap->r.rCurrentTSO = NULL; > >> > >> SET_HDR(new_stack, &stg_STACK_info, old_stack->header.prof.ccs); > >> TICK_ALLOC_STACK(chunk_size); > >> > >> Ömer > >> Ömer Sinan Ağacan <[email protected]>, 21 Haz 2018 Per, 13:42 > >> tarihinde şunu yazdı: > >> > > >> > > Large objects can only be primitive objects, like MUT_ARR_PTRS, > >> > > allocated by > >> > > the RTS, and none of these have SRTs. > >> > > >> > Is is not possible to allocate a large STACK? I'm currently observing > >> > this in > >> > gdb: > >> > > >> > >>> call *Bdescr(0x4200ec9000) > >> > $2 = { > >> > start = 0x4200ec9000, > >> > free = 0x4200ed1000, > >> > link = 0x4200100e80, > >> > u = { > >> > back = 0x4200103980, > >> > bitmap = 0x4200103980, > >> > scan = 0x4200103980 > >> > }, > >> > gen = 0x77b4b8, > >> > gen_no = 1, > >> > dest_no = 1, > >> > node = 0, > >> > flags = 1027, <-- BF_LARGE | BF_EVACUTED | ... > >> > blocks = 8, > >> > _padding = {[0] = 0, [1] = 0, [2] = 0} > >> > } > >> > > >> > >>> call printClosure(0x4200ec9000) > >> > 0x4200ec9000: STACK > >> > > >> > >>> call checkClosure(0x4200ec9000) > >> > $3 = 4096 -- makes sense, larger than 3277 bytes > >> > > >> > So I have a large STACK object, and STACKs can refer to static objects. > >> > But > >> > when we scavenge this object we don't scavenge its SRTs because we use > >> > scavenge_one(). This seems wrong to me. > >> > > >> > Ömer > >> > > >> > Simon Marlow <[email protected]>, 20 Haz 2018 Çar, 14:32 tarihinde şunu > >> > yazdı: > >> > > > >> > > Interesting point. I don't think there are any large objects with > >> > > SRTs, but we should document the invariant because we're relying on it. > >> > > > >> > > Large objects can only be primitive objects, like MUT_ARR_PTRS, > >> > > allocated by the RTS, and none of these have SRTs. > >> > > > >> > > We did have plans to allocate memory for large dynamic objects using > >> > > `allocate()` from compiled code, in which case we could have large > >> > > objects that could be THUNK, FUN, etc. and could have an SRT, in which > >> > > case we would need to revisit this. You might want to take a look at > >> > > Note [big objects] in GCUtils.c, which is relevant here. > >> > > > >> > > Cheers > >> > > Simon > >> > > > >> > > > >> > > On 20 June 2018 at 09:20, Ömer Sinan Ağacan <[email protected]> > >> > > wrote: > >> > >> > >> > >> Hi Simon, > >> > >> > >> > >> I'm confused about this code again. You said > >> > >> > >> > >> > scavenge_one() is only used for a non-major collection, where we > >> > >> > aren't > >> > >> > traversing SRTs. > >> > >> > >> > >> But I think this is not true; scavenge_one() is also used to scavenge > >> > >> large > >> > >> objects (in scavenge_large()), which are scavenged even in major GCs. > >> > >> So it > >> > >> seems like we never really scavenge SRTs of large objects. This > >> > >> doesn't look > >> > >> right to me. Am I missing anything? Can large objects not refer to > >> > >> static > >> > >> objects? > >> > >> > >> > >> Thanks > >> > >> > >> > >> Ömer > >> > >> > >> > >> Ömer Sinan Ağacan <[email protected]>, 2 May 2018 Çar, 09:03 > >> > >> tarihinde şunu yazdı: > >> > >> > > >> > >> > Thanks Simon, this is really helpful. > >> > >> > > >> > >> > > If you look at scavenge_fun_srt() and co, you'll see that they > >> > >> > > return > >> > >> > > immediately if !major_gc. > >> > >> > > >> > >> > Thanks for pointing this out -- I didn't realize it's returning > >> > >> > early when > >> > >> > !major_gc and this caused a lot of confusion. Now everything makes > >> > >> > sense. > >> > >> > > >> > >> > I'll add a note for scavenging SRTs and refer to it in relevant > >> > >> > code and submit > >> > >> > a diff. > >> > >> > > >> > >> > Ömer > >> > >> > > >> > >> > 2018-05-01 22:10 GMT+03:00 Simon Marlow <[email protected]>: > >> > >> > > Your explanation is basically right. scavenge_one() is only used > >> > >> > > for a > >> > >> > > non-major collection, where we aren't traversing SRTs. Admittedly > >> > >> > > this is a > >> > >> > > subtle point that could almost certainly be documented better, I > >> > >> > > probably > >> > >> > > just overlooked it. > >> > >> > > > >> > >> > > More inline: > >> > >> > > > >> > >> > > On 1 May 2018 at 10:26, Ömer Sinan Ağacan <[email protected]> > >> > >> > > wrote: > >> > >> > >> > >> > >> > >> I have an idea but it doesn't explain everything; > >> > >> > >> > >> > >> > >> SRTs are used to collect CAFs, and CAFs are always added to the > >> > >> > >> oldest > >> > >> > >> generation's mut_list when allocated [1]. > >> > >> > >> > >> > >> > >> When we're scavenging a mut_list we know we're not doing a major > >> > >> > >> GC, and > >> > >> > >> because mut_list of oldest generation has all the newly > >> > >> > >> allocated CAFs, > >> > >> > >> which > >> > >> > >> will be scavenged anyway, no need to scavenge SRTs for those. > >> > >> > >> > >> > >> > >> Also, static objects are always evacuated to the oldest gen [2], > >> > >> > >> so any > >> > >> > >> CAFs > >> > >> > >> that are alive but not in the mut_list of the oldest gen will > >> > >> > >> stay alive > >> > >> > >> after > >> > >> > >> a non-major GC, again no need to scavenge SRTs to keep these > >> > >> > >> alive. > >> > >> > >> > >> > >> > >> This also explains why it's OK to not collect static objects > >> > >> > >> (and not > >> > >> > >> treat > >> > >> > >> them as roots) in non-major GCs. > >> > >> > >> > >> > >> > >> However this doesn't explain > >> > >> > >> > >> > >> > >> - Why it's OK to scavenge large objects with scavenge_one(). > >> > >> > > > >> > >> > > > >> > >> > > I don't understand - perhaps you could elaborate on why you think > >> > >> > > it might > >> > >> > > not be OK? Large objects are treated exactly the same as small > >> > >> > > objects with > >> > >> > > respect to their lifetimes. > >> > >> > > > >> > >> > >> > >> > >> > >> - Why we scavenge SRTs in non-major collections in other places > >> > >> > >> (e.g. > >> > >> > >> scavenge_block()). > >> > >> > > > >> > >> > > > >> > >> > > If you look at scavenge_fun_srt() and co, you'll see that they > >> > >> > > return > >> > >> > > immediately if !major_gc. > >> > >> > > > >> > >> > >> > >> > >> > >> Simon, could you say a few words about this? > >> > >> > > > >> > >> > > > >> > >> > > Was that enough words? I have more if necessary :) > >> > >> > > > >> > >> > > Cheers > >> > >> > > Simon > >> > >> > > > >> > >> > > > >> > >> > >> > >> > >> > >> > >> > >> > >> [1]: > >> > >> > >> https://github.com/ghc/ghc/blob/master/rts/sm/Storage.c#L445-L449 > >> > >> > >> [2]: > >> > >> > >> https://github.com/ghc/ghc/blob/master/rts/sm/Scav.c#L1761-L1763 > >> > >> > >> > >> > >> > >> Ömer > >> > >> > >> > >> > >> > >> 2018-03-28 17:49 GMT+03:00 Ben Gamari <[email protected]>: > >> > >> > >> > Hi Simon, > >> > >> > >> > > >> > >> > >> > I'm a bit confused by scavenge_one; namely it doesn't scavenge > >> > >> > >> > SRTs. It > >> > >> > >> > appears that it is primarily used for remembered set entries > >> > >> > >> > but it's > >> > >> > >> > not at all clear why this means that we can safely ignore SRTs > >> > >> > >> > (e.g. in > >> > >> > >> > the FUN and THUNK cases). > >> > >> > >> > > >> > >> > >> > Can you shed some light on this? > >> > >> > >> > > >> > >> > >> > Cheers, > >> > >> > >> > > >> > >> > >> > - Ben > >> > >> > >> > > >> > >> > >> > _______________________________________________ > >> > >> > >> > ghc-devs mailing list > >> > >> > >> > [email protected] > >> > >> > >> > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs > >> > >> > >> > > >> > >> > >> _______________________________________________ > >> > >> > >> ghc-devs mailing list > >> > >> > >> [email protected] > >> > >> > >> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs > >> > >> > > > >> > >> > > > >> > > > >> > > > > > > _______________________________________________ ghc-devs mailing list [email protected] http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
