Hi, I could speed it up somewhat, results are available, all seems well on this regression front, see http://people.apache.org/~centic/poi_regression/reports/index400RC1to401RC1.html for comparison to 4.0.0 and http://people.apache.org/~centic/poi_regression/reportsAll/index400RC1to401RC1.html for all failures.
Only 2 errors are shown compared to 4.0 and both are non-issues as far as I see, at least I cannot reproduce the problem when running the file manually so seems to be some issue in the test-driver. OOMs are likely huge files, also there is a chance of one huge file affecting other files processed at the same time and the rate is very low (187 OOM-failures for 1476184 files). Dominik. On Fri, Nov 23, 2018 at 10:16 PM Dominik Stadler <dominik.stad...@gmx.at> wrote: > Hi, > > somehow the run is taking a long time this time around, estimate finish is > Nov. 26 in the afternoon at the moment. > > Until now my runs was quite limited in amount of memory I could use, but I > can now run it with more memory now as I can run on newer hardware, so > hopefully the number of OOMs will decrease somewhat and the remaining ones > are really huge documents which contains excessively large amounts of data. > We'll see. I can easily adjust the reports and provide a separate set of > documents which trigger an OOM after the run finishes. > > Probably more memory means that some huge documents are now processed > instead of aborted, so that might even explain the longer runtime that I > see. > > Dominik. > > On Fri, Nov 23, 2018 at 6:25 PM Andreas Beeker <kiwiwi...@apache.org> > wrote: > >> Maybe its obvious, but before I roll another RC I'm waiting for Dominiks >> common crawl results - as we have the opportunity to add another stability >> fix. >> >> Btw. what makes me "sad", ... is to see those OOMs. >> Can't we simply exclude them ... in the statistics? ;) >> >> But to be serious again - is there a way to identify the source of them >> in the mass test? >> >> My assumptions is, that the OOMs are a result of successive files and not >> the file failing with the OOM. >> >> Andi >> >> >> --------------------------------------------------------------------- >> To unsubscribe, e-mail: dev-unsubscr...@poi.apache.org >> For additional commands, e-mail: dev-h...@poi.apache.org >> >>
