> > Another thing to try is turning on the infoStream
> > (IndexWriter.setInfoStream(...)) and capture & post the resulting log.
> > It will be very large since it takes quite a while for the error to
> > occur...
>
> I can do that.
Here's a more complete dump. I've modified the code so that I now
remove any existing versions of the document before re-indexing it and
its pages.
Bill
/Library/Java/Home/bin/java '-Dcom.parc.uplib.indexing.debugMode=true'
'-Dcom.parc.uplib.indexing.indexProperties=contents:title:categories$,*:date@:apparent-mime-type*:authors$\sand\s:comment:abstract:email-message-id*:email-guid*:email-subject:email-from-name:email-from-address*:email-attachment-to*:email-thread-index*:email-references$,*:email-in-reply-to$,*:keywords$,*:album:performer:composer:music-genre*:audio-length:accompaniment:paragraph-ids$,*:sha-hash*'
-classpath
"/local/uplib/share/UpLib-1.7/code/lucene-core-2.2.0.jar:/local/uplib/share/UpLib-1.7/code/LuceneIndexing.jar"
-Dorg.apache.lucene.writeLockTimeout=20000
com.parc.uplib.indexing.LuceneIndexing "/local/janssen-uplib/index" update
/local/janssen-uplib/docs 01174-15-2815-270 01174-15-2552-042 01173-98-5675-575
01173-98-4457-188 01173-83-8266-533 01173-80-8759-205
updating
doc_root_dir is /local/janssen-uplib/docs
Working on document /local/janssen-uplib/docs/01174-15-2815-270
Adding header 'apparent-mime-type' I to 01174-15-2815-270
Adding header 'authors' IT to 01174-15-2815-270
Adding header 'categories' I (article) to 01174-15-2815-270
Adding header 'date' I (20070317) to 01174-15-2815-270
Adding header 'sha-hash' I to 01174-15-2815-270
Created empty doc Document<stored/uncompressed,indexed<id:01174-15-2815-270>
stored/uncompressed,indexed<uplibdate:20070317>
stored/uncompressed,indexed<uplibtype:whole>>
Using charset utf8 for contents.txt
Using language en for contents.txt
page 0 (3566): human nature Full-Mental Nudit
page 1 (3100): I know what you're thinking: W
Using charset utf8 for contents.txt
Using language en for contents.txt
Added 01174-15-2815-270 (3 versions)
Working on document /local/janssen-uplib/docs/01174-15-2552-042
Adding header 'abstract' IT to 01174-15-2552-042
Adding header 'apparent-mime-type' I to 01174-15-2552-042
Adding header 'categories' I (photo) to 01174-15-2552-042
Adding header 'date' I (20070316) to 01174-15-2552-042
Adding header 'sha-hash' I to 01174-15-2552-042
Created empty doc Document<stored/uncompressed,indexed<id:01174-15-2552-042>
stored/uncompressed,indexed<uplibdate:20070317>
stored/uncompressed,indexed<uplibtype:whole>>
Added 01174-15-2552-042 (1 versions)
Working on document /local/janssen-uplib/docs/01173-98-5675-575
Adding header 'apparent-mime-type' I to 01173-98-5675-575
Adding header 'authors' IT to 01173-98-5675-575
Adding header 'categories' I (article) to 01173-98-5675-575
Adding header 'categories' I (medical) to 01173-98-5675-575
Adding header 'date' I (20070313) to 01173-98-5675-575
Adding header 'sha-hash' I to 01173-98-5675-575
Created empty doc Document<stored/uncompressed,indexed<id:01173-98-5675-575>
stored/uncompressed,indexed<uplibdate:20070315>
stored/uncompressed,indexed<uplibtype:whole>>
Using charset utf8 for contents.txt
Using language en for contents.txt
page 0 (2730): March 13, 2007 DOW JONES REPRI
page 1 (4445): But just how far -- and how fa
page 2 (2638): "We don't sell snow tires," sa
page 3 (981): A spokeswoman for Rite Aid say
Using charset utf8 for contents.txt
Using language en for contents.txt
Added 01173-98-5675-575 (5 versions)
Working on document /local/janssen-uplib/docs/01173-98-4457-188
Adding header 'apparent-mime-type' I to 01173-98-4457-188
Adding header 'authors' IT to 01173-98-4457-188
Adding header 'categories' I (article) to 01173-98-4457-188
Adding header 'date' I (19911006) to 01173-98-4457-188
Adding header 'sha-hash' I to 01173-98-4457-188
Created empty doc Document<stored/uncompressed,indexed<id:01173-98-4457-188>
stored/uncompressed,indexed<uplibdate:20070315>
stored/uncompressed,indexed<uplibtype:whole>>
Using charset utf8 for contents.txt
Using language en for contents.txt
page 0 (2897): The Economics of the Colonial
merging segments _ram_0 (1 docs) _ram_1 (1 docs) _ram_2 (1 docs) _ram_3 (1
docs) _ram_4 (1 docs) _ram_5 (1 docs) _ram_6 (1 docs) _ram_7 (1 docs) _ram_8 (1
docs) _ram_9 (1 docs) into _1v9 (10 docs)
flush 6 buffered deleted terms on 6 segments.
[EMAIL PROTECTED] main: now checkpoint "segments_3qj" [isCommit = true]
[EMAIL PROTECTED] main: IncRef "_1v4.cfs": pre-incr count is 1
[EMAIL PROTECTED] main: IncRef "_1v4_1.del": pre-incr count is 1
[EMAIL PROTECTED] main: IncRef "_1v5.cfs": pre-incr count is 1
[EMAIL PROTECTED] main: IncRef "_1v5_1.del": pre-incr count is 1
[EMAIL PROTECTED] main: IncRef "_1v6.cfs": pre-incr count is 1
[EMAIL PROTECTED] main: IncRef "_1v6_1.del": pre-incr count is 1
[EMAIL PROTECTED] main: IncRef "_1v7.cfs": pre-incr count is 1
[EMAIL PROTECTED] main: IncRef "_1v7_1.del": pre-incr count is 0
[EMAIL PROTECTED] main: IncRef "_1v8.cfs": pre-incr count is 1
[EMAIL PROTECTED] main: IncRef "_1v8_1.del": pre-incr count is 0
[EMAIL PROTECTED] main: IncRef "_1v9.fnm": pre-incr count is 0
[EMAIL PROTECTED] main: IncRef "_1v9.fdx": pre-incr count is 0
[EMAIL PROTECTED] main: IncRef "_1v9.fdt": pre-incr count is 0
[EMAIL PROTECTED] main: IncRef "_1v9.tii": pre-incr count is 0
[EMAIL PROTECTED] main: IncRef "_1v9.tis": pre-incr count is 0
[EMAIL PROTECTED] main: IncRef "_1v9.frq": pre-incr count is 0
[EMAIL PROTECTED] main: IncRef "_1v9.prx": pre-incr count is 0
[EMAIL PROTECTED] main: IncRef "_1v9.nrm": pre-incr count is 0
[EMAIL PROTECTED] main: deleteCommits: now remove commit "segments_3qi"
[EMAIL PROTECTED] main: DecRef "_1v4.cfs": pre-decr count is 2
[EMAIL PROTECTED] main: DecRef "_1v4_1.del": pre-decr count is 2
[EMAIL PROTECTED] main: DecRef "_1v5.cfs": pre-decr count is 2
[EMAIL PROTECTED] main: DecRef "_1v5_1.del": pre-decr count is 2
[EMAIL PROTECTED] main: DecRef "_1v6.cfs": pre-decr count is 2
[EMAIL PROTECTED] main: DecRef "_1v6_1.del": pre-decr count is 2
[EMAIL PROTECTED] main: DecRef "_1v7.cfs": pre-decr count is 2
[EMAIL PROTECTED] main: DecRef "_1v8.cfs": pre-decr count is 2
[EMAIL PROTECTED] main: DecRef "segments_3qi": pre-decr count is 1
[EMAIL PROTECTED] main: delete "segments_3qi"
[EMAIL PROTECTED] main: now checkpoint "segments_3qk" [isCommit = true]
[EMAIL PROTECTED] main: IncRef "_1v4.cfs": pre-incr count is 1
[EMAIL PROTECTED] main: IncRef "_1v4_1.del": pre-incr count is 1
[EMAIL PROTECTED] main: IncRef "_1v5.cfs": pre-incr count is 1
[EMAIL PROTECTED] main: IncRef "_1v5_1.del": pre-incr count is 1
[EMAIL PROTECTED] main: IncRef "_1v6.cfs": pre-incr count is 1
[EMAIL PROTECTED] main: IncRef "_1v6_1.del": pre-incr count is 1
[EMAIL PROTECTED] main: IncRef "_1v7.cfs": pre-incr count is 1
[EMAIL PROTECTED] main: IncRef "_1v7_1.del": pre-incr count is 1
[EMAIL PROTECTED] main: IncRef "_1v8.cfs": pre-incr count is 1
[EMAIL PROTECTED] main: IncRef "_1v8_1.del": pre-incr count is 1
[EMAIL PROTECTED] main: IncRef "_1v9.cfs": pre-incr count is 0
[EMAIL PROTECTED] main: deleteCommits: now remove commit "segments_3qj"
[EMAIL PROTECTED] main: DecRef "_1v4.cfs": pre-decr count is 2
[EMAIL PROTECTED] main: DecRef "_1v4_1.del": pre-decr count is 2
[EMAIL PROTECTED] main: DecRef "_1v5.cfs": pre-decr count is 2
[EMAIL PROTECTED] main: DecRef "_1v5_1.del": pre-decr count is 2
[EMAIL PROTECTED] main: DecRef "_1v6.cfs": pre-decr count is 2
[EMAIL PROTECTED] main: DecRef "_1v6_1.del": pre-decr count is 2
[EMAIL PROTECTED] main: DecRef "_1v7.cfs": pre-decr count is 2
[EMAIL PROTECTED] main: DecRef "_1v7_1.del": pre-decr count is 2
[EMAIL PROTECTED] main: DecRef "_1v8.cfs": pre-decr count is 2
[EMAIL PROTECTED] main: DecRef "_1v8_1.del": pre-decr count is 2
[EMAIL PROTECTED] main: DecRef "_1v9.fnm": pre-decr count is 1
[EMAIL PROTECTED] main: delete "_1v9.fnm"
[EMAIL PROTECTED] main: DecRef "_1v9.fdx": pre-decr count is 1
[EMAIL PROTECTED] main: delete "_1v9.fdx"
[EMAIL PROTECTED] main: DecRef "_1v9.fdt": pre-decr count is 1
[EMAIL PROTECTED] main: delete "_1v9.fdt"
[EMAIL PROTECTED] main: DecRef "_1v9.tii": pre-decr count is 1
[EMAIL PROTECTED] main: delete "_1v9.tii"
[EMAIL PROTECTED] main: DecRef "_1v9.tis": pre-decr count is 1
[EMAIL PROTECTED] main: delete "_1v9.tis"
[EMAIL PROTECTED] main: DecRef "_1v9.frq": pre-decr count is 1
[EMAIL PROTECTED] main: delete "_1v9.frq"
[EMAIL PROTECTED] main: DecRef "_1v9.prx": pre-decr count is 1
[EMAIL PROTECTED] main: delete "_1v9.prx"
[EMAIL PROTECTED] main: DecRef "_1v9.nrm": pre-decr count is 1
[EMAIL PROTECTED] main: delete "_1v9.nrm"
[EMAIL PROTECTED] main: DecRef "segments_3qj": pre-decr count is 1
[EMAIL PROTECTED] main: delete "segments_3qj"
page 1 (3372): Newsweek. I rely on The Econom
page 2 (3395): print, ÒI rely on the Wall Str
page 3 (3032): British condescension toward t
page 4 (1100): The Òthat being soÓ style of d
Using charset utf8 for contents.txt
Using language en for contents.txt
Added 01173-98-4457-188 (6 versions)
Working on document /local/janssen-uplib/docs/01173-83-8266-533
Adding header 'apparent-mime-type' I to 01173-83-8266-533
Adding header 'categories' I (article) to 01173-83-8266-533
Adding header 'categories' I (medical) to 01173-83-8266-533
Adding header 'categories' I (medical/aging) to 01173-83-8266-533
Adding header 'categories' I (medical) to 01173-83-8266-533
Adding header 'categories' I (medical/exercise) to 01173-83-8266-533
Adding header 'categories' I (medical) to 01173-83-8266-533
Adding header 'date' I (20070313) to 01173-83-8266-533
Adding header 'sha-hash' I to 01173-83-8266-533
Adding header 'source' IT to 01173-83-8266-533
Adding header 'title' IT (Study shows why exercise boosts brainpower) to
01173-83-8266-533
Created empty doc Document<stored/uncompressed,indexed<id:01173-83-8266-533>
stored/uncompressed,indexed<uplibdate:20070313>
stored/uncompressed,indexed<uplibtype:whole>>
Using charset utf8 for contents.txt
Using language en for contents.txt
page 0 (2282): Powered by SAVE THIS | EMAIL T
page 1 (1040): Exercise generated blood flow
Using charset utf8 for contents.txt
Using language en for contents.txt
Added 01173-83-8266-533 (3 versions)
Working on document /local/janssen-uplib/docs/01173-80-8759-205
Adding header 'apparent-mime-type' I to 01173-80-8759-205
Adding header 'categories' I (photo) to 01173-80-8759-205
Adding header 'categories' I (apple) to 01173-80-8759-205
Adding header 'categories' I (apple) to 01173-80-8759-205
Adding header 'date' I (20070312) to 01173-80-8759-205
Adding header 'sha-hash' I to 01173-80-8759-205
Adding header 'title' IT (Apple Store New York showing iPhone ad on March
12th, 2007) to 01173-80-8759-205
Created empty doc Document<stored/uncompressed,indexed<id:01173-80-8759-205>
stored/uncompressed,indexed<uplibdate:20070313>
stored/uncompressed,indexed<uplibtype:whole>>
Added 01173-80-8759-205 (1 versions)
Optimizing...
merging segments _ram_a (1 docs) _ram_b (1 docs) _ram_c (1 docs) _ram_d (1
docs) _ram_e (1 docs) _ram_f (1 docs) _ram_g (1 docs) _ram_h (1 docs) _ram_i (1
docs) into _1va (9 docs)
[EMAIL PROTECTED] main: now checkpoint "segments_3ql" [isCommit = true]
[EMAIL PROTECTED] main: IncRef "_1v4.cfs": pre-incr count is 1
[EMAIL PROTECTED] main: IncRef "_1v4_1.del": pre-incr count is 1
[EMAIL PROTECTED] main: IncRef "_1v5.cfs": pre-incr count is 1
[EMAIL PROTECTED] main: IncRef "_1v5_1.del": pre-incr count is 1
[EMAIL PROTECTED] main: IncRef "_1v6.cfs": pre-incr count is 1
[EMAIL PROTECTED] main: IncRef "_1v6_1.del": pre-incr count is 1
[EMAIL PROTECTED] main: IncRef "_1v7.cfs": pre-incr count is 1
[EMAIL PROTECTED] main: IncRef "_1v7_1.del": pre-incr count is 1
[EMAIL PROTECTED] main: IncRef "_1v8.cfs": pre-incr count is 1
[EMAIL PROTECTED] main: IncRef "_1v8_1.del": pre-incr count is 1
[EMAIL PROTECTED] main: IncRef "_1v9.cfs": pre-incr count is 1
[EMAIL PROTECTED] main: IncRef "_1va.fnm": pre-incr count is 0
[EMAIL PROTECTED] main: IncRef "_1va.fdx": pre-incr count is 0
[EMAIL PROTECTED] main: IncRef "_1va.fdt": pre-incr count is 0
[EMAIL PROTECTED] main: IncRef "_1va.tii": pre-incr count is 0
[EMAIL PROTECTED] main: IncRef "_1va.tis": pre-incr count is 0
[EMAIL PROTECTED] main: IncRef "_1va.frq": pre-incr count is 0
[EMAIL PROTECTED] main: IncRef "_1va.prx": pre-incr count is 0
[EMAIL PROTECTED] main: IncRef "_1va.nrm": pre-incr count is 0
[EMAIL PROTECTED] main: deleteCommits: now remove commit "segments_3qk"
[EMAIL PROTECTED] main: DecRef "_1v4.cfs": pre-decr count is 2
[EMAIL PROTECTED] main: DecRef "_1v4_1.del": pre-decr count is 2
[EMAIL PROTECTED] main: DecRef "_1v5.cfs": pre-decr count is 2
[EMAIL PROTECTED] main: DecRef "_1v5_1.del": pre-decr count is 2
[EMAIL PROTECTED] main: DecRef "_1v6.cfs": pre-decr count is 2
[EMAIL PROTECTED] main: DecRef "_1v6_1.del": pre-decr count is 2
[EMAIL PROTECTED] main: DecRef "_1v7.cfs": pre-decr count is 2
[EMAIL PROTECTED] main: DecRef "_1v7_1.del": pre-decr count is 2
[EMAIL PROTECTED] main: DecRef "_1v8.cfs": pre-decr count is 2
[EMAIL PROTECTED] main: DecRef "_1v8_1.del": pre-decr count is 2
[EMAIL PROTECTED] main: DecRef "_1v9.cfs": pre-decr count is 2
[EMAIL PROTECTED] main: DecRef "segments_3qk": pre-decr count is 1
[EMAIL PROTECTED] main: delete "segments_3qk"
[EMAIL PROTECTED] main: now checkpoint "segments_3qm" [isCommit = true]
[EMAIL PROTECTED] main: IncRef "_1v4.cfs": pre-incr count is 1
[EMAIL PROTECTED] main: IncRef "_1v4_1.del": pre-incr count is 1
[EMAIL PROTECTED] main: IncRef "_1v5.cfs": pre-incr count is 1
[EMAIL PROTECTED] main: IncRef "_1v5_1.del": pre-incr count is 1
[EMAIL PROTECTED] main: IncRef "_1v6.cfs": pre-incr count is 1
[EMAIL PROTECTED] main: IncRef "_1v6_1.del": pre-incr count is 1
[EMAIL PROTECTED] main: IncRef "_1v7.cfs": pre-incr count is 1
[EMAIL PROTECTED] main: IncRef "_1v7_1.del": pre-incr count is 1
[EMAIL PROTECTED] main: IncRef "_1v8.cfs": pre-incr count is 1
[EMAIL PROTECTED] main: IncRef "_1v8_1.del": pre-incr count is 1
[EMAIL PROTECTED] main: IncRef "_1v9.cfs": pre-incr count is 1
[EMAIL PROTECTED] main: IncRef "_1va.cfs": pre-incr count is 0
[EMAIL PROTECTED] main: deleteCommits: now remove commit "segments_3ql"
[EMAIL PROTECTED] main: DecRef "_1v4.cfs": pre-decr count is 2
[EMAIL PROTECTED] main: DecRef "_1v4_1.del": pre-decr count is 2
[EMAIL PROTECTED] main: DecRef "_1v5.cfs": pre-decr count is 2
[EMAIL PROTECTED] main: DecRef "_1v5_1.del": pre-decr count is 2
[EMAIL PROTECTED] main: DecRef "_1v6.cfs": pre-decr count is 2
[EMAIL PROTECTED] main: DecRef "_1v6_1.del": pre-decr count is 2
[EMAIL PROTECTED] main: DecRef "_1v7.cfs": pre-decr count is 2
[EMAIL PROTECTED] main: DecRef "_1v7_1.del": pre-decr count is 2
[EMAIL PROTECTED] main: DecRef "_1v8.cfs": pre-decr count is 2
[EMAIL PROTECTED] main: DecRef "_1v8_1.del": pre-decr count is 2
[EMAIL PROTECTED] main: DecRef "_1v9.cfs": pre-decr count is 2
[EMAIL PROTECTED] main: DecRef "_1va.fnm": pre-decr count is 1
[EMAIL PROTECTED] main: delete "_1va.fnm"
[EMAIL PROTECTED] main: DecRef "_1va.fdx": pre-decr count is 1
[EMAIL PROTECTED] main: delete "_1va.fdx"
[EMAIL PROTECTED] main: DecRef "_1va.fdt": pre-decr count is 1
[EMAIL PROTECTED] main: delete "_1va.fdt"
[EMAIL PROTECTED] main: DecRef "_1va.tii": pre-decr count is 1
[EMAIL PROTECTED] main: delete "_1va.tii"
[EMAIL PROTECTED] main: DecRef "_1va.tis": pre-decr count is 1
[EMAIL PROTECTED] main: delete "_1va.tis"
[EMAIL PROTECTED] main: DecRef "_1va.frq": pre-decr count is 1
[EMAIL PROTECTED] main: delete "_1va.frq"
[EMAIL PROTECTED] main: DecRef "_1va.prx": pre-decr count is 1
[EMAIL PROTECTED] main: delete "_1va.prx"
[EMAIL PROTECTED] main: DecRef "_1va.nrm": pre-decr count is 1
[EMAIL PROTECTED] main: delete "_1va.nrm"
[EMAIL PROTECTED] main: DecRef "segments_3ql": pre-decr count is 1
[EMAIL PROTECTED] main: delete "segments_3ql"
merging segments _1v4 (19680 docs) _1v5 (10 docs) _1v6 (9 docs) _1v7 (10 docs)
_1v8 (9 docs) _1v9 (10 docs) _1va (9 docs)[EMAIL PROTECTED] main: refresh:
removing newly created unreferenced file "_1vb.fdt"
[EMAIL PROTECTED] main: delete "_1vb.fdt"
[EMAIL PROTECTED] main: refresh: removing newly created unreferenced file
"_1vb.fdx"
[EMAIL PROTECTED] main: delete "_1vb.fdx"
[EMAIL PROTECTED] main: refresh: removing newly created unreferenced file
"_1vb.fnm"
[EMAIL PROTECTED] main: delete "_1vb.fnm"
[EMAIL PROTECTED] main: refresh: removing newly created unreferenced file
"_1vb.frq"
[EMAIL PROTECTED] main: delete "_1vb.frq"
[EMAIL PROTECTED] main: refresh: removing newly created unreferenced file
"_1vb.prx"
[EMAIL PROTECTED] main: delete "_1vb.prx"
[EMAIL PROTECTED] main: refresh: removing newly created unreferenced file
"_1vb.tii"
[EMAIL PROTECTED] main: delete "_1vb.tii"
[EMAIL PROTECTED] main: refresh: removing newly created unreferenced file
"_1vb.tis"
[EMAIL PROTECTED] main: delete "_1vb.tis"
java.lang.ArrayIndexOutOfBoundsException: Array index out of range: 23754
at org.apache.lucene.util.BitVector.get(BitVector.java:72)
at
org.apache.lucene.index.SegmentTermDocs.next(SegmentTermDocs.java:118)
at
org.apache.lucene.index.SegmentTermPositions.next(SegmentTermPositions.java:98)
at
org.apache.lucene.index.SegmentMerger.appendPostings(SegmentMerger.java:361)
at
org.apache.lucene.index.SegmentMerger.mergeTermInfo(SegmentMerger.java:325)
at
org.apache.lucene.index.SegmentMerger.mergeTermInfos(SegmentMerger.java:297)
at
org.apache.lucene.index.SegmentMerger.mergeTerms(SegmentMerger.java:261)
at org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:98)
at
org.apache.lucene.index.IndexWriter.mergeSegments(IndexWriter.java:1883)
at org.apache.lucene.index.IndexWriter.optimize(IndexWriter.java:1231)
at
com.parc.uplib.indexing.LuceneIndexing.update(LuceneIndexing.java:419)
at com.parc.uplib.indexing.LuceneIndexing.main(LuceneIndexing.java:664)
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
