[ https://issues.apache.org/jira/browse/LUCENE-2975?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13008587#comment-13008587 ]
Uwe Schindler edited comment on LUCENE-2975 at 3/18/11 7:20 PM: ---------------------------------------------------------------- I just want to add that on Solaris 64 bit I had initially 1.6.0_21 installed. With that version CheckIndex passed, too (verified in my logs). But the bad thing: The application server logs showed random DataFormatErrors like CheckIndex a few times. This is why I started to debug and then upgraded to _24. Since _24 it fails always on Solaris. Robert does not look into JDK code, I did: They changed some places in NIO's unsafe Bits class (some memory copy methods), so I assume the bug is there. was (Author: thetaphi): I just want to add that on Solaris 64 bit I had initially 1.6.0_21 installed. With that version CheckIndex passed, too (verified in my locks). But the bad thing: The application server logs showed random DataFormatErrors like CheckIndex a few times. This is why I started to debug and then upgraded to _24. Since _24 it fails always on Solaris. Robert does not look into JDK code, I did: They changed some places in NIO's unsafe Bits class (some memory copy methods), so I assume the bug is there. > MMapDirectory on chunk size boundaries broken > --------------------------------------------- > > Key: LUCENE-2975 > URL: https://issues.apache.org/jira/browse/LUCENE-2975 > Project: Lucene - Java > Issue Type: Bug > Affects Versions: 3.1 > Reporter: Uwe Schindler > Priority: Blocker > Fix For: 3.1 > > > When testing the 3.1-RC1 made by Yonik on the PANGAEA (www.pangaea.de) > productive system I figured out that suddenly on a large segment (about 5 > GiB) some stored fiels suddenly produce a strange deflate decompression > problem (CompressionTools) although the stored fields are no longer pre-3.0 > compressed. It seems that the header of the stored field is read incorrectly > at the buffer boundary in MultiMMapDir and then FieldsReader just incorrectly > detects a deflate-compressed field (CompressionTools). > The error occurs reproducible on CheckIndex with MMapDirectory, but not with > NIODir or SimpleDir. The FDT file of that segment is 2.6 GiB, on Solaris the > chunk size is Integer.MAX_VALUE, so we have 2 MultiMMap IndexInputs. > Robert and me have the index ready as a tar file, we will do tests on our > local machines and hopefully solve the bug, maybe introduced by Robert's > recent changes to MMap. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org