[
https://issues.apache.org/jira/browse/CASSANDRA-20445?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17936268#comment-17936268
]
Ruslan Fomkin commented on CASSANDRA-20445:
-------------------------------------------
I am looking into a solution to add index name length validation. It's
necessary to have different validation length for different index types, i.e,.
i2, SAI, and SASI. For SAI and SASI it's more complicated as the length can
vary depending on index options. I guess it's important to not be restrictive
with a single length limit, since it might break existing indexes.
I am not looking into fixing incorrect results of
https://issues.apache.org/jira/browse/CASSANDRA-11383. I will create a separate
issue for this.
> Failures on creating index with too long names
> ----------------------------------------------
>
> Key: CASSANDRA-20445
> URL: https://issues.apache.org/jira/browse/CASSANDRA-20445
> Project: Apache Cassandra
> Issue Type: Bug
> Components: CQL/Semantics
> Reporter: Ruslan Fomkin
> Assignee: Ruslan Fomkin
> Priority: Normal
>
> Creating any index with long index name is problematic, since index names are
> used in file names, thus leading to too long file name errors.
> Creating i2 index with a name of 255 characters or longer fails on the create
> index statement. (it even hangs in my unit test run on trunk)
> Creating SAI index with a name of ~222 characters or longer fails on flush.
> Creating SASI index with a name of ~241 characters or longer produces
> incorrect result after flush, while log contains an exception similar to the
> SAI flush error. The incorrect result issue was introduced by ignoring
> exceptions during SASI flush in
> https://issues.apache.org/jira/browse/CASSANDRA-11383
> Index name length was never validated to my understanding, however, it is
> used in file names. The bug might exist from version 3.
> The following test reproduces the problem on trunk:
>
> {code:java}
> @RunWith(Parameterized.class)
> public class IndexNameTest extends CQLTester
> {
> @Parameterized.Parameter(0)
> public String createIndexQuery;
> @Parameterized.Parameter(1)
> public int maxLength;
> @Parameterized.Parameters(name = "{0}")
> public static List<Object[]> parameters()
> {
> return List.of(
> new Object[]{ "CREATE INDEX %s ON %s(%s)", 255 },
> new Object[]{ "CREATE CUSTOM INDEX %s ON %s(%s) USING
> 'org.apache.cassandra.index.sasi.SASIIndex'", 213 },
> new Object[]{ "CREATE CUSTOM INDEX %s ON %s(%s) USING
> 'StorageAttachedIndex'", 190 }
> );
> }
> @Test
> public void testMaxAcceptableLongNamesNewIndex() throws Throwable
> {
> String longName = "a".repeat(maxLength);
> createTable("CREATE TABLE %s (" +
> "key int PRIMARY KEY," +
> "value int)"
> );
> createIndex(String.format(createIndexQuery, longName, "%s", "value"));
> execute(String.format("INSERT INTO %%s (\"key\", %s) VALUES (1, 1)",
> "value"));
> execute(String.format("INSERT INTO %%s (\"key\", %s) VALUES (2, 2)",
> "value"));
> assertRows(execute(String.format("SELECT key, %s FROM %%s WHERE %<s = 1",
> "value")), row(1, 1));
> flush();
> assertRows(execute(String.format("SELECT key, %s FROM %%s WHERE %<s = 1",
> "value")), row(1, 1));
> }
> } {code}
> For i2 the error is:
>
>
> {code:java}
> ERROR [GlobalLogFollower] 2025-03-17 18:33:55,716 Directories.java:276 -
> Failed to create
> build/test/cassandra/data/cql_test_keyspace/table_testmaxacceptablelongnamesnewindex_crea_00-1b255f4def2540a60000000000000006/.aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
> directory
> {code}
> {code:java}
> java.lang.RuntimeException: Failed to list files in
> build/test/cassandra/data/cql_test_keyspace/table_testmaxacceptablelongnamesnewindex_crea_00-1b255f4def2540a60000000000000006/.aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
> at
> org.apache.cassandra.db.lifecycle.LogAwareFileLister.list(LogAwareFileLister.java:77)
> at
> org.apache.cassandra.db.lifecycle.LifecycleTransaction.getFiles(LifecycleTransaction.java:636)
> at
> org.apache.cassandra.db.Directories$SSTableLister.filter(Directories.java:1139)
> at
> org.apache.cassandra.db.Directories$SSTableLister.list(Directories.java:1086)
> at
> org.apache.cassandra.db.Directories$SSTableLister.list(Directories.java:1072)
> at
> org.apache.cassandra.db.Directories.lambda$getUIDGenerator$5(Directories.java:1297)
> at
> java.base/java.util.stream.AbstractPipeline.sourceSpliterator(AbstractPipeline.java:405)
> at
> java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
> at java.base/java.util.stream.IntPipeline.reduce(IntPipeline.java:496)
> at java.base/java.util.stream.IntPipeline.max(IntPipeline.java:459)
> at
> org.apache.cassandra.io.sstable.SequenceBasedSSTableId$Builder.generator(SequenceBasedSSTableId.java:102)
> at
> org.apache.cassandra.db.Directories.getUIDGenerator(Directories.java:1302)
> at
> org.apache.cassandra.db.ColumnFamilyStore.createColumnFamilyStore(ColumnFamilyStore.java:768)
> at
> org.apache.cassandra.db.ColumnFamilyStore.createColumnFamilyStore(ColumnFamilyStore.java:756)
> at
> org.apache.cassandra.index.internal.CassandraIndex.setMetadata(CassandraIndex.java:235)
> at
> org.apache.cassandra.index.internal.CassandraIndex.<init>(CassandraIndex.java:85)
> at
> org.apache.cassandra.index.internal.composites.RegularColumnIndex.<init>(RegularColumnIndex.java:53)
> at
> org.apache.cassandra.index.internal.CassandraIndexFunctions$2.newIndexInstance(CassandraIndexFunctions.java:121)
> at
> org.apache.cassandra.index.internal.CassandraIndex.newIndex(CassandraIndex.java:763)
> at
> org.apache.cassandra.index.SecondaryIndexManager.createInstance(SecondaryIndexManager.java:894)
> at
> org.apache.cassandra.index.SecondaryIndexManager.createIndex(SecondaryIndexManager.java:224)
> at
> org.apache.cassandra.index.SecondaryIndexManager.addIndex(SecondaryIndexManager.java:293)
> at
> org.apache.cassandra.index.SecondaryIndexManager.reload(SecondaryIndexManager.java:210)
> at
> org.apache.cassandra.db.ColumnFamilyStore.reload(ColumnFamilyStore.java:392)
> at
> org.apache.cassandra.schema.DistributedSchema.alterTable(DistributedSchema.java:353)
> at
> org.apache.cassandra.schema.DistributedSchema.lambda$initializeKeyspaceInstances$8(DistributedSchema.java:217)
> at com.google.common.collect.ImmutableList.forEach(ImmutableList.java:422)
> at
> org.apache.cassandra.schema.DistributedSchema.lambda$initializeKeyspaceInstances$10(DistributedSchema.java:217)
> at com.google.common.collect.ImmutableList.forEach(ImmutableList.java:422)
> at
> org.apache.cassandra.schema.DistributedSchema.initializeKeyspaceInstances(DistributedSchema.java:199)
> at
> org.apache.cassandra.tcm.listeners.SchemaListener.notifyInternal(SchemaListener.java:50)
> at
> org.apache.cassandra.tcm.listeners.SchemaListener.notifyPreCommit(SchemaListener.java:43)
> at
> org.apache.cassandra.tcm.log.LocalLog.notifyPreCommit(LocalLog.java:626)
> at
> org.apache.cassandra.tcm.log.LocalLog.processPendingInternal(LocalLog.java:527)
> at
> org.apache.cassandra.tcm.log.LocalLog$Async$AsyncRunnable.run(LocalLog.java:819)
> at
> org.apache.cassandra.concurrent.InfiniteLoopExecutor.loop(InfiniteLoopExecutor.java:121)
> at
> io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
> at java.base/java.lang.Thread.run(Thread.java:829)
> {code}
> For SAI:
> {code:java}
> ERROR [MemtableFlushWriter:1] 2025-03-17 19:04:01,654
> MemtableIndexWriter.java:157 -
> [cql_test_keyspace.table_testmaxacceptablelongnamesnewindex_crea_01.aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa]
> Error while flushing index java.nio.file.FileSystemException:
> /Users/ruslan/GitRepos/Apache/cassandra/build/test/cassandra/data/cql_test_keyspace/table_testmaxacceptablelongnamesnewindex_crea_01-1b255f4def2540a6000000000000000b/oa-1-big-SAI+aa+aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa+ColumnComplete.db:
> File name too long
> java.lang.RuntimeException: java.nio.file.FileSystemException:
> /Users/ruslan/GitRepos/Apache/cassandra/build/test/cassandra/data/cql_test_keyspace/table_testmaxacceptablelongnamesnewindex_crea_01-1b255f4def2540a6000000000000000b/oa-1-big-SAI+aa+aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa+ColumnComplete.db:
> File name too long
> at
> org.apache.cassandra.io.util.SequentialWriter.openChannel(SequentialWriter.java:133)
> at
> org.apache.cassandra.io.util.SequentialWriter.<init>(SequentialWriter.java:166)
> at
> org.apache.cassandra.io.util.SequentialWriter.<init>(SequentialWriter.java:155)
> at
> org.apache.cassandra.index.sai.disk.io.IndexFileUtils$ChecksummingWriter.<init>(IndexFileUtils.java:108)
> at
> org.apache.cassandra.index.sai.disk.io.IndexFileUtils.openOutput(IndexFileUtils.java:64)
> at
> org.apache.cassandra.index.sai.disk.format.IndexDescriptor.openPerIndexOutput(IndexDescriptor.java:252)
> at
> org.apache.cassandra.index.sai.disk.format.IndexDescriptor.openPerIndexOutput(IndexDescriptor.java:242)
> at
> org.apache.cassandra.index.sai.disk.v1.ColumnCompletionMarkerUtil.create(ColumnCompletionMarkerUtil.java:51)
> at
> org.apache.cassandra.index.sai.disk.v1.MemtableIndexWriter.completeIndexFlush(MemtableIndexWriter.java:219)
> at
> org.apache.cassandra.index.sai.disk.v1.MemtableIndexWriter.complete(MemtableIndexWriter.java:151)
> at
> org.apache.cassandra.index.sai.disk.StorageAttachedIndexWriter.complete(StorageAttachedIndexWriter.java:185)
> at java.base/java.util.ArrayList.forEach(ArrayList.java:1541)
> at
> java.base/java.util.Collections$UnmodifiableCollection.forEach(Collections.java:1085)
> at
> org.apache.cassandra.io.sstable.format.SSTableWriter.commit(SSTableWriter.java:289)
> at
> org.apache.cassandra.io.sstable.SimpleSSTableMultiWriter.commit(SimpleSSTableMultiWriter.java:90)
> at
> org.apache.cassandra.db.ColumnFamilyStore$Flush.flushMemtable(ColumnFamilyStore.java:1352)
> at
> org.apache.cassandra.db.ColumnFamilyStore$Flush.run(ColumnFamilyStore.java:1251)
> at
> org.apache.cassandra.concurrent.ExecutionFailure$1.run(ExecutionFailure.java:133)
> at
> java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
> at
> java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
> at
> io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
> at java.base/java.lang.Thread.run(Thread.java:829)
> {code}
> For SASI assertion failure
> {code:java}
> java.lang.AssertionError: Got less rows than expected. Expected 1 but got 0
> at org.junit.Assert.fail(Assert.java:88) at
> org.junit.Assert.assertTrue(Assert.java:41) at
> org.apache.cassandra.cql3.CQLTester.assertRows(CQLTester.java:2115) at
> org.apache.cassandra.index.IndexNameTest.testMaxAcceptableLongNamesNewIndex(IndexNameTest.java:67)
> at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native
> Method) at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.base/java.lang.reflect.Method.invoke(Method.java:566) at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
> at
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
> at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:55) at
> org.junit.rules.RunRules.evaluate(RunRules.java:20) at
> org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325) at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290) at
> org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71) at
> org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288) at
> org.junit.runners.ParentRunner.access$000(ParentRunner.java:58) at
> org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268) at
> org.junit.runners.ParentRunner.run(ParentRunner.java:363) at
> org.junit.runners.Suite.runChild(Suite.java:128) at
> org.junit.runners.Suite.runChild(Suite.java:27) at
> org.junit.runners.ParentRunner$3.run(ParentRunner.java:290) at
> org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71) at
> org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288) at
> org.junit.runners.ParentRunner.access$000(ParentRunner.java:58) at
> org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268) at
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
> at
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
> at org.junit.runners.ParentRunner.run(ParentRunner.java:363) at
> org.junit.runner.JUnitCore.run(JUnitCore.java:137) at
> com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:69)
> at
> com.intellij.rt.junit.IdeaTestRunner$Repeater$1.execute(IdeaTestRunner.java:38)
> at
> com.intellij.rt.execution.junit.TestsRepeater.repeat(TestsRepeater.java:11)
> at
> com.intellij.rt.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:35)
> at
> com.intellij.rt.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:232)
> at com.intellij.rt.junit.JUnitStarter.main(JUnitStarter.java:55)
> {code}
> and an exception in the log
> {code:java}
> ERROR [SASI-Memtable:1] 2025-03-17 19:05:23,968
> PerSSTableIndexWriter.java:270 - Failed to build index segment
> /Users/ruslan/GitRepos/Apache/cassandra/build/test/cassandra/data/cql_test_keyspace/table_testmaxacceptablelongnamesnewindex_crea_00-1b255f4def2540a60000000000000006/oa-1-big-SI_aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa.db
> java.lang.RuntimeException: java.nio.file.FileSystemException:
> /Users/ruslan/GitRepos/Apache/cassandra/build/test/cassandra/data/cql_test_keyspace/table_testmaxacceptablelongnamesnewindex_crea_00-1b255f4def2540a60000000000000006/oa-1-big-SI_aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa.db:
> File name too long
> at
> org.apache.cassandra.io.util.SequentialWriter.openChannel(SequentialWriter.java:133)
> at
> org.apache.cassandra.io.util.SequentialWriter.<init>(SequentialWriter.java:166)
> at
> org.apache.cassandra.io.util.SequentialWriter.<init>(SequentialWriter.java:155)
> at
> org.apache.cassandra.index.sasi.disk.OnDiskIndexBuilder.finish(OnDiskIndexBuilder.java:272)
> at
> org.apache.cassandra.index.sasi.disk.OnDiskIndexBuilder.finish(OnDiskIndexBuilder.java:262)
> at
> org.apache.cassandra.index.sasi.disk.OnDiskIndexBuilder.finish(OnDiskIndexBuilder.java:242)
> at
> org.apache.cassandra.index.sasi.disk.PerSSTableIndexWriter$Index.lambda$scheduleSegmentFlush$0(PerSSTableIndexWriter.java:266)
> at
> org.apache.cassandra.index.sasi.disk.PerSSTableIndexWriter$Index.lambda$complete$1(PerSSTableIndexWriter.java:295)
> at org.apache.cassandra.concurrent.FutureTask$2.call(FutureTask.java:124)
> at org.apache.cassandra.concurrent.FutureTask.call(FutureTask.java:61)
> at org.apache.cassandra.concurrent.FutureTask.run(FutureTask.java:71)
> at
> java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
> at
> java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
> at
> io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
> at java.base/java.lang.Thread.run(Thread.java:829)
> {code}
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
