Ruslan Fomkin created CASSANDRA-20445:
-----------------------------------------
Summary: Failures on creating index with too long names
Key: CASSANDRA-20445
URL: https://issues.apache.org/jira/browse/CASSANDRA-20445
Project: Apache Cassandra
Issue Type: Bug
Components: CQL/Semantics
Reporter: Ruslan Fomkin
Creating any index with long index name is problematic, since index names are
used in file names, thus leading to too long file name errors.
Creating i2 index with a name of 255 characters or longer fails on the create
index statement.
Creating SAI index with a name of ~222 characters or longer fails on flush.
Creating SASI index with a name of ~241 characters or longer produces incorrect
result after flush, while log contains an exception similar to the SAI flush
error. The incorrect result issue was introduced by ignoring exceptions during
SASI flush in https://issues.apache.org/jira/browse/CASSANDRA-11383
Index name length was never validated to my understanding, however, it is used
in file names. The bug might exist from version 3.
The following test reproduces the problem on trunk:
{code:java}
@RunWith(Parameterized.class)
public class IndexNameTest extends CQLTester
{
@Parameterized.Parameter(0)
public String createIndexQuery;
@Parameterized.Parameter(1)
public int maxLength;
@Parameterized.Parameters(name = "{0}")
public static List<Object[]> parameters()
{
return List.of(
new Object[]{ "CREATE INDEX %s ON %s(%s)", 255 },
new Object[]{ "CREATE CUSTOM INDEX %s ON %s(%s) USING
'org.apache.cassandra.index.sasi.SASIIndex'", 213 },
new Object[]{ "CREATE CUSTOM INDEX %s ON %s(%s) USING
'StorageAttachedIndex'", 190 }
);
}
@Test
public void testMaxAcceptableLongNamesNewIndex() throws Throwable
{
String longName = "a".repeat(maxLength);
createTable("CREATE TABLE %s (" +
"key int PRIMARY KEY," +
"value int)"
);
createIndex(String.format(createIndexQuery, longName, "%s", "value"));
execute(String.format("INSERT INTO %%s (\"key\", %s) VALUES (1, 1)",
"value"));
execute(String.format("INSERT INTO %%s (\"key\", %s) VALUES (2, 2)",
"value"));
assertRows(execute(String.format("SELECT key, %s FROM %%s WHERE %<s = 1",
"value")), row(1, 1));
flush();
assertRows(execute(String.format("SELECT key, %s FROM %%s WHERE %<s = 1",
"value")), row(1, 1));
}
} {code}
For i2 the error is:
{code:java}
ERROR [GlobalLogFollower] 2025-03-17 18:33:55,716 Directories.java:276 - Failed
to create
build/test/cassandra/data/cql_test_keyspace/table_testmaxacceptablelongnamesnewindex_crea_00-1b255f4def2540a60000000000000006/.aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
directory
{code}
{code:java}
java.lang.RuntimeException: Failed to list files in
build/test/cassandra/data/cql_test_keyspace/table_testmaxacceptablelongnamesnewindex_crea_00-1b255f4def2540a60000000000000006/.aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
at
org.apache.cassandra.db.lifecycle.LogAwareFileLister.list(LogAwareFileLister.java:77)
at
org.apache.cassandra.db.lifecycle.LifecycleTransaction.getFiles(LifecycleTransaction.java:636)
at
org.apache.cassandra.db.Directories$SSTableLister.filter(Directories.java:1139)
at
org.apache.cassandra.db.Directories$SSTableLister.list(Directories.java:1086)
at
org.apache.cassandra.db.Directories$SSTableLister.list(Directories.java:1072)
at
org.apache.cassandra.db.Directories.lambda$getUIDGenerator$5(Directories.java:1297)
at
java.base/java.util.stream.AbstractPipeline.sourceSpliterator(AbstractPipeline.java:405)
at
java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
at java.base/java.util.stream.IntPipeline.reduce(IntPipeline.java:496)
at java.base/java.util.stream.IntPipeline.max(IntPipeline.java:459)
at
org.apache.cassandra.io.sstable.SequenceBasedSSTableId$Builder.generator(SequenceBasedSSTableId.java:102)
at
org.apache.cassandra.db.Directories.getUIDGenerator(Directories.java:1302)
at
org.apache.cassandra.db.ColumnFamilyStore.createColumnFamilyStore(ColumnFamilyStore.java:768)
at
org.apache.cassandra.db.ColumnFamilyStore.createColumnFamilyStore(ColumnFamilyStore.java:756)
at
org.apache.cassandra.index.internal.CassandraIndex.setMetadata(CassandraIndex.java:235)
at
org.apache.cassandra.index.internal.CassandraIndex.<init>(CassandraIndex.java:85)
at
org.apache.cassandra.index.internal.composites.RegularColumnIndex.<init>(RegularColumnIndex.java:53)
at
org.apache.cassandra.index.internal.CassandraIndexFunctions$2.newIndexInstance(CassandraIndexFunctions.java:121)
at
org.apache.cassandra.index.internal.CassandraIndex.newIndex(CassandraIndex.java:763)
at
org.apache.cassandra.index.SecondaryIndexManager.createInstance(SecondaryIndexManager.java:894)
at
org.apache.cassandra.index.SecondaryIndexManager.createIndex(SecondaryIndexManager.java:224)
at
org.apache.cassandra.index.SecondaryIndexManager.addIndex(SecondaryIndexManager.java:293)
at
org.apache.cassandra.index.SecondaryIndexManager.reload(SecondaryIndexManager.java:210)
at
org.apache.cassandra.db.ColumnFamilyStore.reload(ColumnFamilyStore.java:392)
at
org.apache.cassandra.schema.DistributedSchema.alterTable(DistributedSchema.java:353)
at
org.apache.cassandra.schema.DistributedSchema.lambda$initializeKeyspaceInstances$8(DistributedSchema.java:217)
at com.google.common.collect.ImmutableList.forEach(ImmutableList.java:422)
at
org.apache.cassandra.schema.DistributedSchema.lambda$initializeKeyspaceInstances$10(DistributedSchema.java:217)
at com.google.common.collect.ImmutableList.forEach(ImmutableList.java:422)
at
org.apache.cassandra.schema.DistributedSchema.initializeKeyspaceInstances(DistributedSchema.java:199)
at
org.apache.cassandra.tcm.listeners.SchemaListener.notifyInternal(SchemaListener.java:50)
at
org.apache.cassandra.tcm.listeners.SchemaListener.notifyPreCommit(SchemaListener.java:43)
at org.apache.cassandra.tcm.log.LocalLog.notifyPreCommit(LocalLog.java:626)
at
org.apache.cassandra.tcm.log.LocalLog.processPendingInternal(LocalLog.java:527)
at
org.apache.cassandra.tcm.log.LocalLog$Async$AsyncRunnable.run(LocalLog.java:819)
at
org.apache.cassandra.concurrent.InfiniteLoopExecutor.loop(InfiniteLoopExecutor.java:121)
at
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.base/java.lang.Thread.run(Thread.java:829)
{code}
For SAI:
{code:java}
ERROR [MemtableFlushWriter:1] 2025-03-17 19:04:01,654
MemtableIndexWriter.java:157 -
[cql_test_keyspace.table_testmaxacceptablelongnamesnewindex_crea_01.aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa]
Error while flushing index java.nio.file.FileSystemException:
/Users/ruslan/GitRepos/Apache/cassandra/build/test/cassandra/data/cql_test_keyspace/table_testmaxacceptablelongnamesnewindex_crea_01-1b255f4def2540a6000000000000000b/oa-1-big-SAI+aa+aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa+ColumnComplete.db:
File name too long
java.lang.RuntimeException: java.nio.file.FileSystemException:
/Users/ruslan/GitRepos/Apache/cassandra/build/test/cassandra/data/cql_test_keyspace/table_testmaxacceptablelongnamesnewindex_crea_01-1b255f4def2540a6000000000000000b/oa-1-big-SAI+aa+aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa+ColumnComplete.db:
File name too long
at
org.apache.cassandra.io.util.SequentialWriter.openChannel(SequentialWriter.java:133)
at
org.apache.cassandra.io.util.SequentialWriter.<init>(SequentialWriter.java:166)
at
org.apache.cassandra.io.util.SequentialWriter.<init>(SequentialWriter.java:155)
at
org.apache.cassandra.index.sai.disk.io.IndexFileUtils$ChecksummingWriter.<init>(IndexFileUtils.java:108)
at
org.apache.cassandra.index.sai.disk.io.IndexFileUtils.openOutput(IndexFileUtils.java:64)
at
org.apache.cassandra.index.sai.disk.format.IndexDescriptor.openPerIndexOutput(IndexDescriptor.java:252)
at
org.apache.cassandra.index.sai.disk.format.IndexDescriptor.openPerIndexOutput(IndexDescriptor.java:242)
at
org.apache.cassandra.index.sai.disk.v1.ColumnCompletionMarkerUtil.create(ColumnCompletionMarkerUtil.java:51)
at
org.apache.cassandra.index.sai.disk.v1.MemtableIndexWriter.completeIndexFlush(MemtableIndexWriter.java:219)
at
org.apache.cassandra.index.sai.disk.v1.MemtableIndexWriter.complete(MemtableIndexWriter.java:151)
at
org.apache.cassandra.index.sai.disk.StorageAttachedIndexWriter.complete(StorageAttachedIndexWriter.java:185)
at java.base/java.util.ArrayList.forEach(ArrayList.java:1541)
at
java.base/java.util.Collections$UnmodifiableCollection.forEach(Collections.java:1085)
at
org.apache.cassandra.io.sstable.format.SSTableWriter.commit(SSTableWriter.java:289)
at
org.apache.cassandra.io.sstable.SimpleSSTableMultiWriter.commit(SimpleSSTableMultiWriter.java:90)
at
org.apache.cassandra.db.ColumnFamilyStore$Flush.flushMemtable(ColumnFamilyStore.java:1352)
at
org.apache.cassandra.db.ColumnFamilyStore$Flush.run(ColumnFamilyStore.java:1251)
at
org.apache.cassandra.concurrent.ExecutionFailure$1.run(ExecutionFailure.java:133)
at
java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at
java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.base/java.lang.Thread.run(Thread.java:829)
{code}
For SASI assertion failure
{code:java}
java.lang.AssertionError: Got less rows than expected. Expected 1 but got 0
at org.junit.Assert.fail(Assert.java:88) at
org.junit.Assert.assertTrue(Assert.java:41) at
org.apache.cassandra.cql3.CQLTester.assertRows(CQLTester.java:2115) at
org.apache.cassandra.index.IndexNameTest.testMaxAcceptableLongNamesNewIndex(IndexNameTest.java:67)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native
Method) at
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566) at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at
org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
at
org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) at
org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:55) at
org.junit.rules.RunRules.evaluate(RunRules.java:20) at
org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325) at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290) at
org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71) at
org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288) at
org.junit.runners.ParentRunner.access$000(ParentRunner.java:58) at
org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268) at
org.junit.runners.ParentRunner.run(ParentRunner.java:363) at
org.junit.runners.Suite.runChild(Suite.java:128) at
org.junit.runners.Suite.runChild(Suite.java:27) at
org.junit.runners.ParentRunner$3.run(ParentRunner.java:290) at
org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71) at
org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288) at
org.junit.runners.ParentRunner.access$000(ParentRunner.java:58) at
org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268) at
org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
at
org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) at
org.junit.runners.ParentRunner.run(ParentRunner.java:363) at
org.junit.runner.JUnitCore.run(JUnitCore.java:137) at
com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:69)
at
com.intellij.rt.junit.IdeaTestRunner$Repeater$1.execute(IdeaTestRunner.java:38)
at
com.intellij.rt.execution.junit.TestsRepeater.repeat(TestsRepeater.java:11) at
com.intellij.rt.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:35)
at
com.intellij.rt.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:232)
at com.intellij.rt.junit.JUnitStarter.main(JUnitStarter.java:55)
{code}
and an exception in the log
{code:java}
ERROR [SASI-Memtable:1] 2025-03-17 19:05:23,968 PerSSTableIndexWriter.java:270
- Failed to build index segment
/Users/ruslan/GitRepos/Apache/cassandra/build/test/cassandra/data/cql_test_keyspace/table_testmaxacceptablelongnamesnewindex_crea_00-1b255f4def2540a60000000000000006/oa-1-big-SI_aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa.db
java.lang.RuntimeException: java.nio.file.FileSystemException:
/Users/ruslan/GitRepos/Apache/cassandra/build/test/cassandra/data/cql_test_keyspace/table_testmaxacceptablelongnamesnewindex_crea_00-1b255f4def2540a60000000000000006/oa-1-big-SI_aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa.db:
File name too long
at
org.apache.cassandra.io.util.SequentialWriter.openChannel(SequentialWriter.java:133)
at
org.apache.cassandra.io.util.SequentialWriter.<init>(SequentialWriter.java:166)
at
org.apache.cassandra.io.util.SequentialWriter.<init>(SequentialWriter.java:155)
at
org.apache.cassandra.index.sasi.disk.OnDiskIndexBuilder.finish(OnDiskIndexBuilder.java:272)
at
org.apache.cassandra.index.sasi.disk.OnDiskIndexBuilder.finish(OnDiskIndexBuilder.java:262)
at
org.apache.cassandra.index.sasi.disk.OnDiskIndexBuilder.finish(OnDiskIndexBuilder.java:242)
at
org.apache.cassandra.index.sasi.disk.PerSSTableIndexWriter$Index.lambda$scheduleSegmentFlush$0(PerSSTableIndexWriter.java:266)
at
org.apache.cassandra.index.sasi.disk.PerSSTableIndexWriter$Index.lambda$complete$1(PerSSTableIndexWriter.java:295)
at org.apache.cassandra.concurrent.FutureTask$2.call(FutureTask.java:124)
at org.apache.cassandra.concurrent.FutureTask.call(FutureTask.java:61)
at org.apache.cassandra.concurrent.FutureTask.run(FutureTask.java:71)
at
java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at
java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.base/java.lang.Thread.run(Thread.java:829)
{code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
