Thanks Anton! Providing a stack trace is great; we can afford the black pixels I think ;)
I found some bugs in a separate effort which might be related, although I saw a different exception so I'm not sure. I'll post a patch soon, and if you are able to re-test and see if you can reproduce, that would be a huge help! -Mike On Thu, Jan 21, 2021 at 4:37 PM Anton Hägerstrand <an...@blunders.io> wrote: > > Hi! > > I recently posted about profiling the nightly benchmarks, and in the process > of doing that I came across an exception that looks rather nasty. I was about > to create a Jira issue, but the nice Jira submit form told me to post it here > or in irc first. > > The Lucene commit where I saw this was > eb24e95731b9f865b95b821c1745264fdc58119, which was the master branch's HEAD > this Saturday. It's is an EOF error, which seems to happen when reading > vector values: > > Exception in thread "Thread-1" java.lang.RuntimeException: > java.lang.RuntimeException: java.io.EOFException: seek past EOF: > MMapIndexInput(path="/home/anton/dev/lucene-bench-home/indices/lucene_bench_2021-01-17_eb24e95_medium_1thread/index/_32.vec") > [slice=vector-data] > at perf.TaskThreads$TaskThread.run(TaskThreads.java:105) > Caused by: java.lang.RuntimeException: java.io.EOFException: seek past EOF: > MMapIndexInput(path="/home/anton/dev/lucene-bench-home/indices/lucene_bench_2021-01-17_eb24e95_medium_1thread/index/_32.vec") > [slice=vector-data] > at perf.SearchTask.go(SearchTask.java:322) > at perf.TaskThreads$TaskThread.run(TaskThreads.java:91) > Caused by: java.io.EOFException: seek past EOF: > MMapIndexInput(path="/home/anton/dev/lucene-bench-home/indices/lucene_bench_2021-01-17_eb24e95_medium_1thread/index/_32.vec") > [slice=vector-data] > at > org.apache.lucene.store.ByteBufferIndexInput.seek(ByteBufferIndexInput.java:255) > at > org.apache.lucene.store.ByteBufferIndexInput$MultiBufferImpl.seek(ByteBufferIndexInput.java:575) > at > org.apache.lucene.codecs.lucene90.Lucene90VectorReader$OffHeapVectorValues.vectorValue(Lucene90VectorReader.java:432) > at org.apache.lucene.util.hnsw.HnswGraph.search(HnswGraph.java:118) > at > org.apache.lucene.codecs.lucene90.Lucene90VectorReader$OffHeapVectorValues.search(Lucene90VectorReader.java:409) > at perf.KnnQuery$KnnWeight.scorer(KnnQuery.java:88) > at org.apache.lucene.search.Weight.bulkScorer(Weight.java:166) > at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:743) > at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:533) > at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:664) > at > org.apache.lucene.search.IndexSearcher.searchAfter(IndexSearcher.java:510) > at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:520) > at perf.SearchTask.go(SearchTask.java:263) > > Sorry about the long stack trace in an email - still not quite up to speed > with etiquette on this mailing list, should I have attached a file instead? > > I have reproduced it 3 times on my machine, running the nightly benchmark's > search benchmark with the competition.WIKI_MEDIUM_ALL source. It does not > occur with e.g. the competition.WIKI_MEDIUM_10M source. The source of the > competition file is in > https://github.com/mikemccand/luceneutil/blob/master/src/python/competition.py. > I run the indexing with only 1 thread - not sure if that matters. > > I'll happily provide more information regarding system setup etc. if that can > help in figuring things out. > > best regards, > Anton Hägerstrand --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
