Michael Wohlwend created CASSANDRA-14014:
--------------------------------------------
Summary: Inserting of larger columns leads to exceptions and
failing repairs
Key: CASSANDRA-14014
URL: https://issues.apache.org/jira/browse/CASSANDRA-14014
Project: Cassandra
Issue Type: Bug
Components: Repair
Environment: 7 node cluster, debian stretch, 32GB ram, 2TB disk, C*
3.11.1
Reporter: Michael Wohlwend
Attachments: cassandra-log.txt
Inserting records with an unusual large text field (hundreds of MB instead of a
few KB) caused exceptions of the kind "mutation too large":
===========================
ERROR [MutationStage-3] 2017-11-07 08:32:52,712 StorageProxy.java:1414 - Failed
to apply mutation locally : {}
java.lang.IllegalArgumentException: Mutation of 185.439MiB is too large for the
maximum size of 64.000MiB
at org.apache.cassandra.db.commitlog.CommitLog.add(CommitLog.java:255)
~[apache-cassandra-3.11.1.jar:3.11.1]
at org.apache.cassandra.db.Keyspace.applyInternal(Keyspace.java:585)
~[apache-cassandra-3.11.1.jar:3.11.1]
at org.apache.cassandra.db.Keyspace.apply(Keyspace.java:462)
~[apache-cassandra-3.11.1.jar:3.11.1]
at org.apache.cassandra.db.Mutation.apply(Mutation.java:227)
~[apache-cassandra-3.11.1.jar:3.11.1]
at org.apache.cassandra.db.Mutation.apply(Mutation.java:232)
~[apache-cassandra-3.11.1.jar:3.11.1]
at org.apache.cassandra.db.Mutation.apply(Mutation.java:241)
~[apache-cassandra-3.11.1.jar:3.11.1]
at
org.apache.cassandra.service.StorageProxy$8.runMayThrow(StorageProxy.java:1408)
~[apache-cassandra-3.11.1.jar:3.11.1]
at
org.apache.cassandra.service.StorageProxy$LocalMutationRunnable.run(StorageProxy.java:2647)
[apache-cassandra-3.11.1.jar:3.11.1]
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
[na:1.8.0_141]
at
org.apache.cassandra.concurrent.AbstractLocalAwareExecutorService$FutureTask.run(AbstractLocalAwareExecutorService.java:162)
[apache-cassandra-3.11.1.jar:3.11.1]
at
org.apache.cassandra.concurrent.AbstractLocalAwareExecutorService$LocalSessionFutureTask.run(AbstractLocalAwareExecutorService.java:134)
[apache-cassandra-3.11.1.jar:3.11.1]
at org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:109)
[apache-cassandra-3.11.1.jar:3.11.1]
at java.lang.Thread.run(Thread.java:748) [na:1.8.0_141]
WARN [MutationStage-145] 2017-11-07 08:33:08,291
AbstractLocalAwareExecutorService.java:167 - Uncaught exception on thread
Thread[MutationStage-145,5,main]: {}
====================================
Trying to repair the table fails. In the end, the cassandra shuts down on the
node.
The exceptions thrown during repair are "EOF after xxx bytes out of YYY", "leak
detected",
"failed to create merkle tree ... OutOfMemory" and "self-supression not
permitted".
See the attached log file (edited and shortened).
Michael
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
