Yep, already done: https://issues.apache.org/jira/browse/CASSANDRA-12877
On Fri, Nov 4, 2016 at 11:15 PM, Jonathan Haddad <j...@jonhaddad.com> wrote:
> Can you file a Jira for this? Would be good to make sure 3.10 doesn't get
> released with this bug.
> On Fri, Nov 4, 2016 at 6:11 PM Voytek Jarnot <voytek.jar...@gmail.com>
> wrote:
>
>> Thought I'd follow-up to myself, in case anyone else comes across this
>> problem. I found a reasonably easy test case to reproduce the problem:
>>
>> This works in 3.9, but doesn't work in 3.10-snapshot:
>>
>> CREATE KEYSPACE vjtest WITH replication = {'class': 'SimpleStrategy',
>> 'replication_factor': '1'};
>> use vjtest ;
>> create table tester(id1 text, id2 text, id3 text, val1 text, primary
>> key((id1, id2), id3));
>> create custom index tester_idx_val1 on tester(val1) using
>> 'org.apache.cassandra.index.sasi.SASIIndex';
>> insert into tester(id1,id2,id3, val1) values ('1-1','1-2','1-3','asdf');
>> insert into tester(id1,id2,id3, val1) values ('1-1','1-2','2-3','asdf');
>> insert into tester(id1,id2,id3, val1) values ('1-1','1-2','3-3','asdf');
>> insert into tester(id1,id2,id3, val1) values ('1-1','1-2','4-3','asdf');
>> insert into tester(id1,id2,id3, val1) values ('1-1','1-2','5-3','asdf');
>> insert into tester(id1,id2,id3, val1) values ('1-1','1-2','6-3','asdf');
>> insert into tester(id1,id2,id3, val1) values ('1-1','1-2','7-3','asdf');
>> insert into tester(id1,id2,id3, val1) values ('1-1','1-2','8-3','asdf');
>> insert into tester(id1,id2,id3, val1) values ('1-1','1-2','9-3','asdf');
>>
>> That's it - when Cassandra tries to flush all hell breaks loose (well,
>> maaybe not, but an unhandled error gets logged). Also, the index doesn't
>> actually work subsequently.
>>
>> On Fri, Nov 4, 2016 at 3:58 PM, Voytek Jarnot <voytek.jar...@gmail.com>
>> wrote:
>>
>> Wondering if anyone has encountered the same...
>>
>> Full story and stacktraces below, short version is that creating a SASI
>> index fails for me when running a 3.10-SNAPSHOT build. One caveat: creating
>> the index on an empty table doesn't fail; however, soon after I start
>> pumping data into the table similar problems occur.
>>
>> I created CASSANDRA-12877 for this, but am beginning to suspect it might
>> be related to CASSANDRA-11990. The thing that's throwing me is that I
>> can't seem to duplicate this with a simple test table.
>>
>> Background:
>>
>> Ended up building/loading a 3.10-SNAPSHOT to try to get past
>> CASSANDRA-11670, CASSANDRA-12223, and CASSANDRA-12689.
>>
>> 1) built/installed 3.10-SNAPSHOT from git branch cassandra-3.X
>> 2) created keyspace (SimpleStrategy, RF 1)
>> 3) created table: (simplified version below, many more valX columns
>> present)
>>
>> CREATE TABLE test_table (
>> id1 text,
>> id2 text,
>> id3 date,
>> id4 timestamp,
>> id5 text,
>> val1 text,
>> val2 text,
>> val3 text,
>> task_id text,
>> val4 text,
>> val5 text,
>> PRIMARY KEY ((id1, id2), id3, id4, id5)
>> ) WITH CLUSTERING ORDER BY (id3 DESC, id4 DESC, id5 ASC)
>>
>> 4) created materialized view:
>>
>> CREATE MATERIALIZED VIEW test_table_by_task_id AS
>> SELECT *
>> FROM test_table
>> WHERE id1 IS NOT NULL AND id2 IS NOT NULL AND id3 IS NOT NULL AND id4
>> IS NOT NULL AND id5 IS NOT NULL AND task_id IS NOT NULL
>> PRIMARY KEY (task_id, id3, id4, id1, id2, id5)
>> WITH CLUSTERING ORDER BY (id3 DESC, id4 DESC, id1 ASC, id2 ASC, id5
>> ASC)
>>
>> 5) inserted 27 million "rows" (i.e., unique values for id5)
>> 6) create index attempt
>>
>> create custom index idx_test_table_val5 on test_table(val5) using
>> 'org.apache.cassandra.index.sasi.SASIIndex';
>>
>> 7) no error in cqlsh, but system.log shows many of the following:
>>
>> INFO [SASI-General:1] 2016-11-04 13:46:47,578
>> PerSSTableIndexWriter.java:277 - Flushed index segment
>> /mydir/cassandra/apache-cassandra-3.10-SNAPSHOT/data/
>> data/mykeyspace/test_table-133dd090a2b411e6b1bf6df2a1af06
>> f0/mc-149-big-SI_idx_test_table_val5.db_0, took 869 ms.
>> ERROR [SASI-General:1] 2016-11-04 13:46:47,584 CassandraDaemon.java:229 -
>> Exception in thread Thread[SASI-General:1,5,main]
>> java.lang.AssertionError: cannot have more than 8 overflow collisions per
>> leaf, but had: 12
>> at org.apache.cassandra.index.sasi.disk.
>> AbstractTokenTreeBuilder$Leaf.createOverflowEntry(
>> AbstractTokenTreeBuilder.java:357) ~[apache-cassandra-3.10-
>> SNAPSHOT.jar:3.10-SNAPSHOT]
>> at org.apache.cassandra.index.sasi.disk.
>> AbstractTokenTreeBuilder$Leaf.createEntry(AbstractTokenTreeBuilder.java:346)
>> ~[apache-cassandra-3.10-SNAPSHOT.jar:3.10-SNAPSHOT]
>> at org.apache.cassandra.index.sasi.disk.DynamicTokenTreeBuilder$
>> DynamicLeaf.serializeData(DynamicTokenTreeBuilder.java:180)
>> ~[apache-cassandra-3.10-SNAPSHOT.jar:3.10-SNAPSHOT]
>> at org.apache.cassandra.index.sasi.disk.
>> AbstractTokenTreeBuilder$Leaf.serialize(AbstractTokenTreeBuilder.java:306)
>> ~[apache-cassandra-3.10-SNAPSHOT.jar:3.10-SNAPSHOT]
>> at org.apache.cassandra.index.sasi.disk.AbstractTokenTreeBuilder.
>> write(AbstractTokenTreeBuilder.java:90) ~[apache-cassandra-3.10-
>> SNAPSHOT.jar:3.10-SNAPSHOT]
>> at org.apache.cassandra.index.sasi.disk.OnDiskIndexBuilder$
>> MutableDataBlock.flushAndClear(OnDiskIndexBuilder.java:629)
>> ~[apache-cassandra-3.10-SNAPSHOT.jar:3.10-SNAPSHOT]
>> at org.apache.cassandra.index.sasi.disk.OnDiskIndexBuilder$
>> MutableLevel.flush(OnDiskIndexBuilder.java:446) ~[apache-cassandra-3.10-
>> SNAPSHOT.jar:3.10-SNAPSHOT]
>> at org.apache.cassandra.index.sasi.disk.OnDiskIndexBuilder$
>> MutableLevel.add(OnDiskIndexBuilder.java:433) ~[apache-cassandra-3.10-
>> SNAPSHOT.jar:3.10-SNAPSHOT]
>> at org.apache.cassandra.index.sasi.disk.OnDiskIndexBuilder.
>> addTerm(OnDiskIndexBuilder.java:207) ~[apache-cassandra-3.10-
>> SNAPSHOT.jar:3.10-SNAPSHOT]
>> at org.apache.cassandra.index.sasi.disk.OnDiskIndexBuilder.
>> finish(OnDiskIndexBuilder.java:293) ~[apache-cassandra-3.10-
>> SNAPSHOT.jar:3.10-SNAPSHOT]
>> at org.apache.cassandra.index.sasi.disk.OnDiskIndexBuilder.
>> finish(OnDiskIndexBuilder.java:258) ~[apache-cassandra-3.10-
>> SNAPSHOT.jar:3.10-SNAPSHOT]
>> at org.apache.cassandra.index.sasi.disk.OnDiskIndexBuilder.
>> finish(OnDiskIndexBuilder.java:241) ~[apache-cassandra-3.10-
>> SNAPSHOT.jar:3.10-SNAPSHOT]
>> at org.apache.cassandra.index.sasi.disk.PerSSTableIndexWriter$Index.
>> lambda$scheduleSegmentFlush$0(PerSSTableIndexWriter.java:267)
>> ~[apache-cassandra-3.10-SNAPSHOT.jar:3.10-SNAPSHOT]
>> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>> ~[na:1.8.0_101]
>> at
>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>> ~[na:1.8.0_101]
>> at
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>> [na:1.8.0_101]
>> at java.lang.Thread.run(Thread.java:745) [na:1.8.0_101]
>>
>> As well as some of these:
>>
>> ERROR [CompactionExecutor:3] 2016-11-04 13:49:13,142 DataTracker.java:168
>> - Can't open index file at /mydir/cassandra/apache-
>> cassandra-3.10-SNAPSHOT/data/data/mykeyspace/test_table-
>> 133dd090a2b411e6b1bf6df2a1af06f0/mc-300-big-SI_idx_test_table_val5.db,
>> skipping.
>> java.lang.IllegalArgumentException: position: 3472329188772431788,
>> limit: 8180
>> at org.apache.cassandra.index.sasi.utils.MappedBuffer.
>> position(MappedBuffer.java:106) ~[apache-cassandra-3.10-
>> SNAPSHOT.jar:3.10-SNAPSHOT]
>> at
>> org.apache.cassandra.index.sasi.disk.OnDiskIndex.<init>(OnDiskIndex.java:147)
>> ~[apache-cassandra-3.10-SNAPSHOT.jar:3.10-SNAPSHOT]
>> at
>> org.apache.cassandra.index.sasi.SSTableIndex.<init>(SSTableIndex.java:62)
>> ~[apache-cassandra-3.10-SNAPSHOT.jar:3.10-SNAPSHOT]
>> at org.apache.cassandra.index.sasi.conf.DataTracker.
>> getIndexes(DataTracker.java:150) [apache-cassandra-3.10-
>> SNAPSHOT.jar:3.10-SNAPSHOT]
>> at
>> org.apache.cassandra.index.sasi.conf.DataTracker.update(DataTracker.java:69)
>> [apache-cassandra-3.10-SNAPSHOT.jar:3.10-SNAPSHOT]
>> at
>> org.apache.cassandra.index.sasi.conf.ColumnIndex.update(ColumnIndex.java:147)
>> [apache-cassandra-3.10-SNAPSHOT.jar:3.10-SNAPSHOT]
>> at
>> org.apache.cassandra.index.sasi.SASIIndexBuilder.completeSSTable(SASIIndexBuilder.java:156)
>> [apache-cassandra-3.10-SNAPSHOT.jar:3.10-SNAPSHOT]
>> at
>> org.apache.cassandra.index.sasi.SASIIndexBuilder.build(SASIIndexBuilder.java:125)
>> [apache-cassandra-3.10-SNAPSHOT.jar:3.10-SNAPSHOT]
>> at org.apache.cassandra.db.compaction.CompactionManager$
>> 14.run(CompactionManager.java:1583) [apache-cassandra-3.10-
>> SNAPSHOT.jar:3.10-SNAPSHOT]
>> at
>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>> [na:1.8.0_101]
>> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>> [na:1.8.0_101]
>> at
>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>> [na:1.8.0_101]
>> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>> [na:1.8.0_101]
>> at
>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>> [na:1.8.0_101]
>> at
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>> [na:1.8.0_101]
>> at java.lang.Thread.run(Thread.java:745) [na:1.8.0_101]
>>
>>
>>
