[
https://issues.apache.org/jira/browse/CASSANDRA-17507?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17656119#comment-17656119
]
Andres de la Peña edited comment on CASSANDRA-17507 at 1/9/23 3:17 PM:
-----------------------------------------------------------------------
[This JVM
dtest|https://github.com/apache/cassandra/compare/cassandra-4.0...adelapena:cassandra:17507-4.0]
reproduces the bug. It testes a 3.x -> 4.0 rolling upgrade scenario with a
table with {{COMPACT STORAGE}} and a query over that uses paging. The bug only
seems to manifest itself when the driver uses native protocol v3, instead on
the default (v5 for 4.0 and v4 for 3.11).
The tests results can be found
[here|https://app.circleci.com/pipelines/github/adelapena/cassandra/2536/workflows/5791569d-8ea1-42b5-bacd-bd8716afaee8/jobs/25163].
The artifacts stored for each test contain an identical stacktrace, for
example [this
one|https://output.circle-artifacts.com/output/job/f4cbecbc-92dd-49c8-a75d-a5a7b53bcd21/artifacts/0/stdout/fails/1/org.apache.cassandra.distributed.upgrade.CompactStoragePagingTest%23testPagingWithCompactStorageAndProtocolVersion.txt]
If this is actually caused by the combination of {{{}COMPACT STORAGE{}}},
paging and and old protocol version, probably the easiest workaround until we
get a fix is setting the driver to use a more recent version of the native
transport protocol.
was (Author: adelapena):
[This JVM
dtest|https://github.com/apache/cassandra/compare/trunk...adelapena:cassandra:17507-4.0?expand=1]
reproduces the bug. It testes a 3.x -> 4.0 rolling upgrade scenario with a
table with {{COMPACT STORAGE}} and a query over that uses paging. The bug only
seems to manifest itself when the driver uses native protocol v3, instead on
the default (v5 for 4.0 and v4 for 3.11).
The tests results can be found
[here|https://app.circleci.com/pipelines/github/adelapena/cassandra/2536/workflows/5791569d-8ea1-42b5-bacd-bd8716afaee8/jobs/25163].
The artifacts stored for each test contain an identical stacktrace, for
example [this
one|https://output.circle-artifacts.com/output/job/f4cbecbc-92dd-49c8-a75d-a5a7b53bcd21/artifacts/0/stdout/fails/1/org.apache.cassandra.distributed.upgrade.CompactStoragePagingTest%23testPagingWithCompactStorageAndProtocolVersion.txt]
If this is actually caused by the combination of {{{}COMPACT STORAGE{}}},
paging and and old protocol version, probably the easiest workaround until we
get a fix is setting the driver to use a more recent version of the native
transport protocol.
> IllegalArgumentException in query code path during 3.11.12 => 4.0.3 rolling
> upgrade
> -----------------------------------------------------------------------------------
>
> Key: CASSANDRA-17507
> URL: https://issues.apache.org/jira/browse/CASSANDRA-17507
> Project: Cassandra
> Issue Type: Bug
> Components: Consistency/Coordination
> Reporter: Thomas Steinmaurer
> Priority: Normal
> Fix For: 4.0.x
>
>
> In a 6 node 3.11.12 test cluster - freshly set up, thus no legacy SSTables
> etc. - with ~ 1TB SSTables on disk per node, I have been running a rolling
> upgrade to 4.0.3. On upgraded 4.0.3 nodes I then have seen the following
> exception regularly, which disappeared once all 6 nodes have been on 4.0.3.
> Is this known? Can this be ignored? As said, just a test drive, but not sure
> if we want to have that in production, especially with a larger number of
> nodes, where it could take some time, until all are upgraded. Thanks!
> {code}
> ERROR [Native-Transport-Requests-8] 2022-03-30 11:30:24,057
> ErrorMessage.java:457 - Unexpected exception during request
> java.lang.IllegalArgumentException: newLimit > capacity: (290 > 15)
> at java.base/java.nio.Buffer.createLimitException(Buffer.java:372)
> at java.base/java.nio.Buffer.limit(Buffer.java:346)
> at java.base/java.nio.ByteBuffer.limit(ByteBuffer.java:1107)
> at java.base/java.nio.ByteBuffer.limit(ByteBuffer.java:262)
> at
> org.apache.cassandra.db.marshal.ByteBufferAccessor.slice(ByteBufferAccessor.java:107)
> at
> org.apache.cassandra.db.marshal.ByteBufferAccessor.slice(ByteBufferAccessor.java:39)
> at
> org.apache.cassandra.db.marshal.ValueAccessor.sliceWithShortLength(ValueAccessor.java:225)
> at
> org.apache.cassandra.db.marshal.CompositeType.splitName(CompositeType.java:222)
> at
> org.apache.cassandra.service.pager.PagingState$RowMark.decodeClustering(PagingState.java:434)
> at
> org.apache.cassandra.service.pager.PagingState$RowMark.clustering(PagingState.java:388)
> at
> org.apache.cassandra.service.pager.SinglePartitionPager.nextPageReadQuery(SinglePartitionPager.java:88)
> at
> org.apache.cassandra.service.pager.SinglePartitionPager.nextPageReadQuery(SinglePartitionPager.java:32)
> at
> org.apache.cassandra.service.pager.AbstractQueryPager.fetchPage(AbstractQueryPager.java:69)
> at
> org.apache.cassandra.service.pager.SinglePartitionPager.fetchPage(SinglePartitionPager.java:32)
> at
> org.apache.cassandra.cql3.statements.SelectStatement$Pager$NormalPager.fetchPage(SelectStatement.java:352)
> at
> org.apache.cassandra.cql3.statements.SelectStatement.execute(SelectStatement.java:400)
> at
> org.apache.cassandra.cql3.statements.SelectStatement.execute(SelectStatement.java:250)
> at
> org.apache.cassandra.cql3.statements.SelectStatement.execute(SelectStatement.java:88)
> at
> org.apache.cassandra.cql3.QueryProcessor.processStatement(QueryProcessor.java:244)
> at
> org.apache.cassandra.cql3.QueryProcessor.processPrepared(QueryProcessor.java:723)
> at
> org.apache.cassandra.cql3.QueryProcessor.processPrepared(QueryProcessor.java:701)
> at
> org.apache.cassandra.transport.messages.ExecuteMessage.execute(ExecuteMessage.java:159)
> at
> org.apache.cassandra.transport.Message$Request.execute(Message.java:242)
> at
> org.apache.cassandra.transport.Dispatcher.processRequest(Dispatcher.java:86)
> at
> org.apache.cassandra.transport.Dispatcher.processRequest(Dispatcher.java:106)
> at
> org.apache.cassandra.transport.Dispatcher.lambda$dispatch$0(Dispatcher.java:70)
> at
> java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
> at
> org.apache.cassandra.concurrent.AbstractLocalAwareExecutorService$FutureTask.run(AbstractLocalAwareExecutorService.java:165)
> at org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:119)
> at
> io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
> at java.base/java.lang.Thread.run(Thread.java:829)
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
