On Thu, Jan 30, 2014 at 3:23 AM, Edward Capriolo <edlinuxg...@gmail.com>wrote:
> Is this only a ByteOrderPartitioner problem? > No, see the comments on https://issues.apache.org/jira/browse/CASSANDRA-6638for more details. -- Sylvain > > > On Wed, Jan 29, 2014 at 7:34 PM, Tyler Hobbs <ty...@datastax.com> wrote: > >> Ignace, >> >> Thanks for reporting this. I've been able to reproduce the issue with a >> unit test, so I opened >> https://issues.apache.org/jira/browse/CASSANDRA-6638. I'm not 100% sure >> if your fix is the correct one, but I should be able to get it fixed >> quickly and figure out the full set of cases where a key (or keys) may be >> skipped. >> >> >> On Wed, Jan 29, 2014 at 9:53 AM, Desimpel, Ignace < >> ignace.desim...@nuance.com> wrote: >> >>> Got into a problem when testing a vnode setup. >>> >>> I'm using a byteordered partitioner, linux, code version 2.0.4, >>> replication factor 1, 4 machine >>> >>> All goes ok until I run cleanup, and gets worse when adding / >>> decommissioning nodes. >>> >>> >>> >>> In my opinion the problem can be found in the SSTableScanner:: >>> KeyScanningIterator::computeNext routine at the lines >>> >>> >>> >>> currentRange = rangeIterator.next(); >>> >>> seekToCurrentRangeStart(); >>> >>> if (ifile.isEOF())return endOfData(); >>> >>> >>> >>> To see what is wrong, think of having 3 ranges in the list, and both the >>> first and second range will not produce a valid currentKey. The first time >>> in the loop we get the first range, and then call >>> seekToCurrentRangeStart(). That routine doesn't do anything in that case, >>> so then the first key is read from the sstable. But this first key does not >>> match the first range, so we loop again. We get the second range and call >>> seekToCurrentRangeStart() again. Again this does not do anything, leaving >>> all file pointers. So then a new currentKey is read from the sstable BUT >>> that should not be the case. We should, in that case, continue to test with >>> the 'old' currentKey. So in that case we are SKIPPING (possible) VALID >>> RECORDS !!! >>> >>> >>> >>> To make things worse, in my test case, I only had one key. So when I get >>> into the second loop, the test isEOF() was true, so the routine stopped >>> immediately having 100 ranges still to test. >>> >>> >>> >>> Anyway, attached a new version of the SSTableScanner.java file. Seems to >>> work for me, but I'm sure a more experienced eye should have a look at this >>> problem (and/or possible other scanners and/or situations like scrub, range >>> queries ...?). >>> >>> >>> >>> Well, I hope I'm wrong about this.... >>> >>> >>> >>> Regards, >>> >>> >>> >>> Ignace Desimpel >>> >>> >>> >>> >>> >>> >>> >>> >>> >> >> >> >> -- >> Tyler Hobbs >> DataStax <http://datastax.com/> >> > >
