Hi Ticker, I think the cache is not meant to improve run time, it is used to deduplicate and thus reduce memory. Maybe it would be better to use a smaller chunk size and no cache. No idea why I didn't use merged = new ArrayList<>(len);
Gerd ________________________________________ Von: mkgmap-dev <mkgmap-dev-boun...@lists.mkgmap.org.uk> im Auftrag von Ticker Berkin <rwb-mkg...@jagit.co.uk> Gesendet: Dienstag, 11. Mai 2021 19:19 An: Development list for mkgmap Betreff: Re: [mkgmap-dev] MDR building out-of-memory Hi Gerd I've looked at this, and if Mdr5 space becomes a problem again, I'll consider converting to it. A couple of comments: My map had 2909735 poi so the sort chunk size was ~727433. The cache sizes after each chunk were 563095, 603595, 597239 & 605718. Is the cache worth-while for this low hit-rate? Just running the Gmapsupp combiner on existing tiles (without --route, so no streets), I got a run time of 1 min 44 secs with the cache and 1 min 30 without! However most of this time is copying the tiles into gmapsupp.img so not an accurate statistic. You could pre-allocate List<> "merged" with the correct size. Ticker On Tue, 2021-05-11 at 14:40 +0000, Gerd Petermann wrote: > Hi Ticker, > > I've committed the patch as is. I've not seen big changes in > performance, but I've used a different (already existing) set of > files which was created with my own style. For me, > Mdr11.preWriteImpl() is the most problematic part reg. OOM errors. > > Maybe look at the code which uses LargeListSorter. > > Gerd > > ________________________________________ > Von: mkgmap-dev <mkgmap-dev-boun...@lists.mkgmap.org.uk> im Auftrag > von Ticker Berkin <rwb-mkg...@jagit.co.uk> > Gesendet: Dienstag, 11. Mai 2021 13:27 > An: Development list for mkgmap > Betreff: Re: [mkgmap-dev] MDR building out-of-memory > > Hi Gerd > > Here is updated version of patch. > > Changes from last: > > Uses your cache code for region and country (in 2 places). For > British > Isles, there are 190 regions and 7 countries, so I don't think the > extra memory will be a problem and there should be some performance > benefit. > > Delays allocating cities until it can use sortKeys.size() for initial > allocation. For above map this is 0.07% too big, so I don't think > trimToSize() is worthwhile. > > Shares the Sort object between the 4 methods. > > Ticker > _______________________________________________ > mkgmap-dev mailing list > mkgmap-dev@lists.mkgmap.org.uk > https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev _______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev _______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev