I have sent a more complete patch that should also address fragmentation issue with requests >= 4K and < 2MB.
On Monday, March 23, 2020 at 6:12:51 PM UTC-4, Waldek Kozaczuk wrote: > > I have just sent a new patch to the mailing list. I am hoping it will > address the OOM crash if my theory of heavy memory fragmentation is right. > It would be nice if Nadav could review it. > > Regardless if you have another crash in production and are able to connect > with gdb, could you run 'osv heap' - it should show free_page_ranges. If > memory is heavily fragmented we should see a long list. > > It would be nice to recreate that load in dev env and capture the memory > trace data (BWT you do not need to enable backtrace to have enough useful > information). It would help us better understand how memory is allocated by > the app. I saw you send me one trace but it does not seem to be revealing > anything interesting. > > Waldek > > On Monday, March 23, 2020 at 1:19:18 AM UTC-4, rickp wrote: >> >> On Sun, 2020-03-22 at 22:08 -0700, Waldek Kozaczuk wrote: >> > >> > >> > On Monday, March 23, 2020 at 12:36:52 AM UTC-4, rickp wrote: >> > > Looks to me like its trying to allocate 40MB but the available >> > > memory >> > > is 10GB, surely? 10933128KB is 10,933MB >> > > >> > >> > I misread the number - forgot about 1K. >> > >> > Any chance you could run the app outside of production with memory >> > tracing enabled - >> > >> https://github.com/cloudius-systems/osv/wiki/Trace-analysis-using-trace.py#tracing-memory-allocations >> >> > (without --trace-backtrace) for while? And then we can have a better >> > sense of what kind of allocations it makes. The output of trace >> > memory-analyzer would be really helpful. >> >> I can certainly run that local with locally generated workloads, which >> should be close enough - but we've never managed to trigger the oom >> condition that way (other than by really constraining the memory >> artificially). It should be close enough though - let me see what I can >> do. >> >> Rick >> >> >> -- You received this message because you are subscribed to the Google Groups "OSv Development" group. To unsubscribe from this group and stop receiving emails from it, send an email to osv-dev+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/osv-dev/e88bb103-ca6f-4f12-b944-e2d1391e2f8e%40googlegroups.com.
