Thanks for the report. Would you be so kind as to send as much
information as possible so that we can reproduce the error conditions ?
Toivo Pedaste writes:
> I've been trying the new compression code
>
> After some experimentation (the config file parser doesn't
> seem to reject anything) I settled on the following config
>
> wordlist_compress: 1
> wordlist_compress_debug: 2
> wordlist_cache_size: 40000000
>
> Setting the cache_size seems to be important, without it
> the CPU on my Pentium III Xeon was just grinding.
Yes, it does make a *huge* difference. The rule (should be documented, Marcel)
is that the cache size should be at least 2% of the expected index size. The
Berkeley DB file has 1% of internal pages that *must* be cached for good
performances. Giving an additional 1% leaves room for caching leaf pages.
> Eventually it failed with the following error:
>
> 8782:10873:4:http://www.acs.uwa.edu.au/minutes/a_board/ab19961120/ab19961120.html:
> BitStream::Show: ntags:72 size:2418 buffsize: 612 ::: #
> CMPRTYPE:000:2418 #00# page:lsn.file:002:2418
> #00000000000000000000000000000000# page:lsn.offset:034:2418
> #00000000000000000000000000000000# page:pgno:066:2418
> #00110110010111000000000000000000# page:prev_pgno:098:2418
> #00000000000000000000000000000000# page:next_pgno:130:2418
> #00000000000000000000000000000000# page:entries:162:2418
> #0001110000000000# page:hf_offset:178:2418 #0001110111100000#
> page:level:194:2418 #01000000# page:type:202:2418 #11000000#
> seperatekey_len0:210:2418 #1011000000000000#
> seperatekey_bti_len0:226:2418 #1011000000000000#
> seperatekey_bti_type0:242:2418 #10000000# seperatekey_bti_pgno0:250:2418
> #01111110100011000000000000000000# seperatekey_bti_nrecs0:282:2418
> #00000000000000000000000000000000# seperatekey_btidata0:314:2418
>
>#01000110100000000000000011111000001100000000000000000000111011101110111011101110110001101100111000100110#
> seperatekey_len1:418:2418 #1011000000000000#
> seperatekey_bti_len1:434:2418 #1011000000000000#
> seperatekey_bti_type1:450:2418 #10000000# seperatekey_bti_pgno1:458:2418
> #11010110001111000000000000000000# seperatekey_bti_nrecs1:490:2418
> #00000000000000000000000000000000# seperatekey_btidata1:522:2418
>
>#00000101000000000000000000001001001100000000000000000000111011101110111011101110110001101100111000100110#
> NumField0:626:2418 ## size:626:2418 #0110110000000000#
> put_valsCompType:642:2418 #10# nbits:644:2418 #0010# data:648:2418
>
>#101110101010101010101010101010101011101010101010101010101010101010101011101010101010101010101010101010101011111110111011101111111011101010101010101110111011101110111011101110111011101110101011101110111011101110111011#
> NumField1:864:2418 ## size:864:2418 #0110110000000000#
> put_valsCompType:880:2418 #10# nbits:882:2418 #0111# data:886:2418
>
>#011101010000000001110101000001001111100000010110001000001111000110000010011011100000100111000100001101011000000000001100000000011101010100001110100110000000000000100000000010001000000111111010000010010011100000101100000100000110010100000000001100000000011101010100001110100110000011111111000000000010001000001011111010000000010011100000101000101000001010111010000000101111001000000111110010001010100010100010001100000000011101001011000101100101010010110001000000000111010110000100000010000011000101010000111111001110001010001100000010011101111000010110001110001110110100000000101110110000100111001100001111001001000000111000101000100001100011000000110000000001011110001100101010001110001011011110000000011010011001101011101010000011111010100000100101110100#
> NumField2:1642:2418 ## size:1642:2418 #0100000000000000#
> put_valsCompType:1658:2418 #10# nbits:1660:2418 #1110# data:1664:2418
> #00000010000001# NumField3:1678:2418 ## size:1678:2418
> #0110110000000000# put_valsCompType:1694:2418 #10# nbits:1696:2418
> #0011# data:1700:2418
>
>#110110001000001111110000010010101000001110000000011100000100011010000100100110000100101101010000001001001000111000000010010101010000010010010100100000100000001001001000001010001000011110001000001011010000001001001000111000000010010101010000110011000000110100000000011100000100011000100000010101100000110110001000101011001000011100000000110111010000010110100000100010110000011100000000110010100000111001011000111000100000101001001000110010110000100111000000001100001000100010001000101000100000001001101100010011011100011010010000000010010000110001100000100000110000001000010000110000000100011110010000001010000001000001010100110011110100101011000000#
> NumField4:2348:2418 ## size:2348:2418 #0000000000000000#
> NumField5:2364:2418 ## size:2364:2418 #0000000000000000#
> NumField6:2380:2418 ## size:2380:2418 #0000000000000000#
> NumField7:2396:2418 ## size:2396:2418 #1110110000000000#
> put_valsCompType:2412:2418 #10# nbits:2414:2418 #0000
> ERROR:BitStream:bitpos:2418:check_tag: found tag NumField8 at 3298
> expected it at 2418
>
Cheers,
--
Loic Dachary
ECILA
100 av. du Gal Leclerc
93500 Pantin - France
Tel: 33 1 56 96 10 85
e-mail: [EMAIL PROTECTED]
URL: http://www.senga.org/
------------------------------------
To unsubscribe from the htdig3-dev mailing list, send a message to
[EMAIL PROTECTED]
You will receive a message to confirm this.
