Nimi Wariboko Jr. created CASSANDRA-11074: ---------------------------------------------
Summary: SSTable Metadata Corruption / ClusteringKey fails Assertion Key: CASSANDRA-11074 URL: https://issues.apache.org/jira/browse/CASSANDRA-11074 Project: Cassandra Issue Type: Bug Reporter: Nimi Wariboko Jr. Fix For: 3.2 I've seen this exact issue twice now in the same keyspace, but different tables - where it seems my SSTable gets corrupted and the metadata composite keys fails an assertion in tiemuuidcomparison. When querying this SSTable for data, the query will timeout, and you get this error in the logs: Here is the full stack trace: {code} java.lang.AssertionError: null at org.apache.cassandra.db.marshal.TimeUUIDType.compareCustom(TimeUUIDType.java:65) ~[apache-cassandra-3.2.jar:3.2-SNAPSHOT] at org.apache.cassandra.db.marshal.AbstractType.compare(AbstractType.java:158) ~[apache-cassandra-3.2.jar:3.2-SNAPSHOT] at org.apache.cassandra.db.ClusteringComparator.compareComponent(ClusteringComparator.java:166) ~[apache-cassandra-3.2.jar:3.2-SNAPSHOT] at org.apache.cassandra.db.Slice$Bound.compareTo(Slice.java:485) ~[apache-cassandra-3.2.jar:3.2-SNAPSHOT] at org.apache.cassandra.db.Slice$Bound.access$100(Slice.java:346) ~[apache-cassandra-3.2.jar:3.2-SNAPSHOT] at org.apache.cassandra.db.Slice.intersects(Slice.java:252) ~[apache-cassandra-3.2.jar:3.2-SNAPSHOT] at org.apache.cassandra.db.filter.ClusteringIndexNamesFilter.shouldInclude(ClusteringIndexNamesFilter.java:212) ~[apache-cassandra-3.2.jar:3.2-SNAPSHOT] at org.apache.cassandra.db.SinglePartitionReadCommand.shouldInclude(SinglePartitionReadCommand.java:630) ~[apache-cassandra-3.2.jar:3.2-SNAPSHOT] at org.apache.cassandra.db.SinglePartitionReadCommand.queryMemtableAndDiskInternal(SinglePartitionReadCommand.java:537) ~[apache-cassandra-3.2.jar:3.2-SNAPSHOT] at org.apache.cassandra.db.SinglePartitionReadCommand.queryMemtableAndDisk(SinglePartitionReadCommand.java:464) ~[apache-cassandra-3.2.jar:3.2-SNAPSHOT] at org.apache.cassandra.db.SinglePartitionReadCommand.queryMemtableAndDisk(SinglePartitionReadCommand.java:456) ~[apache-cassandra-3.2.jar:3.2-SNAPSHOT] at org.apache.cassandra.index.internal.composites.CompositesSearcher$1.prepareNext(CompositesSearcher.java:145) ~[apache-cassandra-3.2.jar:3.2-SNAPSHOT] at org.apache.cassandra.index.internal.composites.CompositesSearcher$1.hasNext(CompositesSearcher.java:78) ~[apache-cassandra-3.2.jar:3.2-SNAPSHOT] at org.apache.cassandra.db.transform.BasePartitions.hasNext(BasePartitions.java:72) ~[apache-cassandra-3.2.jar:3.2-SNAPSHOT] at org.apache.cassandra.db.partitions.UnfilteredPartitionIterators$Serializer.serialize(UnfilteredPartitionIterators.java:287) ~[apache-cassandra-3.2.jar:3.2-SNAPSHOT] at org.apache.cassandra.db.ReadResponse$LocalDataResponse.build(ReadResponse.java:136) ~[apache-cassandra-3.2.jar:3.2-SNAPSHOT] at org.apache.cassandra.db.ReadResponse$LocalDataResponse.<init>(ReadResponse.java:128) ~[apache-cassandra-3.2.jar:3.2-SNAPSHOT] at org.apache.cassandra.db.ReadResponse$LocalDataResponse.<init>(ReadResponse.java:123) ~[apache-cassandra-3.2.jar:3.2-SNAPSHOT] at org.apache.cassandra.db.ReadResponse.createDataResponse(ReadResponse.java:65) ~[apache-cassandra-3.2.jar:3.2-SNAPSHOT] at org.apache.cassandra.db.ReadCommand.createResponse(ReadCommand.java:292) ~[apache-cassandra-3.2.jar:3.2-SNAPSHOT] at org.apache.cassandra.db.ReadCommandVerbHandler.doVerb(ReadCommandVerbHandler.java:50) ~[apache-cassandra-3.2.jar:3.2-SNAPSHOT] at org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:64) ~[apache-cassandra-3.2.jar:3.2-SNAPSHOT] at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) ~[na:1.8.0_45] at org.apache.cassandra.concurrent.AbstractLocalAwareExecutorService$FutureTask.run(AbstractLocalAwareExecutorService.java:164) ~[apache-cassandra-3.2.jar:3.2-SNAPSHOT] at org.apache.cassandra.concurrent.AbstractLocalAwareExecutorService$LocalSessionFutureTask.run(AbstractLocalAwareExecutorService.java:136) [apache-cassandra-3.2.jar:3.2-SNAPSHOT] at org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:105) [apache-cassandra-3.2.jar:3.2-SNAPSHOT] at java.lang.Thread.run(Thread.java:745) [na:1.8.0_45] {code} In both times, the Assertion fails from the timeuuid key read from the metadata. Which is read in ClusteringIndexNamesFilter.java:212 (https://github.com/apache/cassandra/blob/cassandra-3.2.1/src/java/org/apache/cassandra/db/filter/ClusteringIndexNamesFilter.java#L212) {code} public boolean shouldInclude(SSTableReader sstable) { ClusteringComparator comparator = sstable.metadata.comparator; List<ByteBuffer> minClusteringValues = sstable.getSSTableMetadata().minClusteringValues; List<ByteBuffer> maxClusteringValues = sstable.getSSTableMetadata().maxClusteringValues; // If any of the requested clustering is within the bounds covered by the sstable, we need to include the sstable for (Clustering clustering : clusterings) { if (Slice.make(clustering).intersects(comparator, minClusteringValues, maxClusteringValues)) return true; } return false; } {code} The error occurs with either minClusteringValues or maxClusteringValues, I can't tell which, because the comparison happens on the same line. Slice.java:252 (https://github.com/apache/cassandra/blob/cassandra-3.2.1/src/java/org/apache/cassandra/db/Slice.java#L252) which turtle down to TimeUUIDType.compareCustom(a,b) {code} public boolean intersects(ClusteringComparator comparator, List<ByteBuffer> minClusteringValues, List<ByteBuffer> maxClusteringValues) { // If this slice start after max or end before min, it can't intersect if (start.compareTo(comparator, maxClusteringValues) > 0 || end.compareTo(comparator, minClusteringValues) < 0) return false; {code} If I use Cassandra 3.4's sstablemetadata function (which includes minClustringValues/maxClustringValues) it seems that the values are totally valid (they both pass Assertion in a comparable script I wrote). sstableverify completes successfully, and a scrub doesn't seem to help as well - I have to export and reimport the data for another datastore. Here is the output from ./sstablemetadata {code} root@cass1:/tmp/apache-cassandra-3.4-SNAPSHOT/tools/bin# CASSANDRA_HOME=../../ CASSANDRA_INCLUDE=../../bin/cassandra.in.sh ls -d /var/lib/cassandra/data/cmpayments/section_lines-89867550737011e5938129463d90c3f0/* | grep Data | CASSANDRA_HOME=../../ CASSANDRA_INCLUDE=../../bin/cassandra.in.sh xargs -n 1 ./sstablemetadata SSTable: /var/lib/cassandra/data/cmpayments/section_lines-89867550737011e5938129463d90c3f0/ma-4803-big Partitioner: org.apache.cassandra.dht.Murmur3Partitioner Bloom Filter FP chance: 0.010000 Minimum timestamp: 1453375631979368 Maximum timestamp: 1453419499417290 SSTable min local deletion time: 1453375631 SSTable max local deletion time: 2147483647 Compressor: org.apache.cassandra.io.compress.LZ4Compressor Compression ratio: 0.4473869787789238 TTL min: 0 TTL max: 0 minClustringValues: [b88c962a-73ab-11e5-a474-5cf938947aea] maxClustringValues: [4c2688c0-a3d0-11e5-9d05-0242ac110002] Estimated droppable tombstones: 0.043774182787776034 SSTable Level: 0 Repaired at: 1453787106092 ReplayPosition(segmentId=1453356986934, position=1014) totalColumnsSet: 848635 totalRows: 87361 Estimated tombstone drop times: 1453375635: 1000 1453375680: 1000 1453375912: 1000 1453376113: 1000 1453376314: 524 1453376323: 1000 1453376485: 1000 1453376626: 1000 1453376884: 1000 1453377038: 1000 1453377281: 1000 1453377440: 1000 1453377551: 1000 1453377746: 1000 1453377807: 1000 1453377843: 1000 1453377922: 1000 1453377987: 1000 1453378024: 1000 1453378162: 1000 1453378417: 1000 1453378711: 1000 1453378891: 1000 1453379219: 1000 1453379441: 1000 1453379624: 1000 1453379898: 1000 1453380072: 1000 1453380251: 580 1453380258: 420 1453380601: 1000 1453380792: 1000 1453381105: 1000 1453381265: 1000 1453381443: 1000 1453381509: 1000 1453381582: 1000 1453381630: 1000 1453381990: 1000 1453382018: 755 1453382025: 245 1453382189: 1000 1453382207: 1000 1453382306: 1000 1453382328: 1000 1453382369: 1000 1453382434: 1000 1453382453: 1000 1453382587: 1000 1453382607: 1000 1453382727: 1000 1453382752: 1000 1453382825: 1000 1453382863: 1000 1453382882: 1000 1453382921: 1000 1453382983: 1000 1453383027: 499 1453383100: 1000 1453383163: 1000 1453383189: 562 1453383196: 438 1453383339: 760 1453383346: 240 1453383365: 585 1453383372: 415 1453383482: 392 1453383489: 608 1453383507: 1000 1453383678: 1000 1453383834: 1000 1453383909: 1000 1453384010: 580 1453384144: 544 1453384151: 456 1453384438: 1000 1453384532: 1000 1453384768: 653 1453384775: 347 1453384932: 1000 1453384956: 1000 1453385033: 1000 1453385149: 1000 1453385193: 331 1453391997: 669 1453392132: 1000 1453392195: 1000 1453392254: 1000 1453392302: 1000 1453392358: 1000 1453392402: 1000 1453392503: 291 1453392557: 662 1453399174: 1000 1453399235: 1000 1453399296: 1000 1453399341: 1000 1453399363: 1000 1453419498: 805 2147483647: 875972 Count Row Size Cell Count 1 0 0 2 0 0 3 0 0 4 0 0 5 0 0 6 0 0 7 0 0 8 0 0 10 0 0 12 0 0 14 0 0 17 0 0 20 0 0 24 0 0 29 0 0 35 0 0 42 0 0 50 0 0 60 0 0 72 0 0 86 0 0 103 0 0 124 0 0 149 0 0 179 0 0 215 0 0 258 0 0 310 0 0 372 0 0 446 0 0 535 0 0 642 0 0 770 0 0 924 0 0 1109 0 0 1331 0 0 1597 0 0 1916 0 0 2299 0 0 2759 0 0 3311 0 0 3973 0 0 4768 0 0 5722 0 0 6866 0 0 8239 0 0 9887 0 0 11864 0 0 14237 0 0 17084 0 2 20501 0 0 24601 0 0 29521 0 2 35425 0 0 42510 0 4 51012 0 0 61214 0 0 73457 0 0 88148 0 0 105778 0 0 126934 0 0 152321 0 0 182785 0 0 219342 0 0 263210 0 0 315852 0 2 379022 0 0 454826 0 0 545791 0 2 654949 0 0 785939 0 0 943127 0 0 1131752 1 0 1358102 1 0 1629722 0 0 1955666 0 0 2346799 0 0 2816159 2 0 3379391 0 0 4055269 0 0 4866323 0 0 5839588 0 0 7007506 0 0 8409007 0 0 10090808 0 0 12108970 0 0 14530764 0 0 17436917 0 0 20924300 1 0 25109160 0 0 30130992 1 0 36157190 0 0 43388628 0 0 52066354 0 0 62479625 0 0 74975550 0 0 89970660 0 0 107964792 0 0 129557750 0 0 155469300 0 0 186563160 0 0 223875792 0 0 268650950 0 0 322381140 0 0 386857368 0 0 464228842 0 0 557074610 0 0 668489532 0 0 802187438 0 0 962624926 0 0 1155149911 0 0 1386179893 0 0 1663415872 0 0 1996099046 0 0 2395318855 0 0 2874382626 0 3449259151 0 4139110981 0 4966933177 0 5960319812 0 7152383774 0 8582860529 0 10299432635 0 12359319162 0 14831182994 0 17797419593 0 21356903512 0 25628284214 0 30753941057 0 36904729268 0 44285675122 0 53142810146 0 63771372175 0 76525646610 0 91830775932 0 110196931118 0 132236317342 0 158683580810 0 190420296972 0 228504356366 0 274205227639 0 329046273167 0 394855527800 0 473826633360 0 568591960032 0 682310352038 0 818772422446 0 982526906935 0 1179032288322 0 1414838745986 0 Estimated cardinality: 6 EncodingStats minTTL: 0 EncodingStats minLocalDeletionTime: 1453373014 EncodingStats minTimestamp: 1453373014633946 KeyType: org.apache.cassandra.db.marshal.CompositeType(org.apache.cassandra.db.marshal.TimeUUIDType,org.apache.cassandra.db.marshal.TimeUUIDType) ClusteringTypes: [org.apache.cassandra.db.marshal.TimeUUIDType] StaticColumns: {} RegularColumns: {gross:org.apache.cassandra.db.marshal.DecimalType, assetid:org.apache.cassandra.db.marshal.UTF8Type, raw:org.apache.cassandra.db.marshal.BytesType, lineid:org.apache.cassandra.db.marshal.UTF8Type, owners:org.apache.cassandra.db.marshal.MapType(org.apache.cassandra.db.marshal.TimeUUIDType,org.apache.cassandra.db.marshal.DecimalType), unassigned:org.apache.cassandra.db.marshal.BooleanType, type:org.apache.cassandra.db.marshal.UTF8Type, title:org.apache.cassandra.db.marshal.UTF8Type, views:org.apache.cassandra.db.marshal.LongType, platform:org.apache.cassandra.db.marshal.UTF8Type} SSTable: /var/lib/cassandra/data/cmpayments/section_lines-89867550737011e5938129463d90c3f0/ma-4804-big Partitioner: org.apache.cassandra.dht.Murmur3Partitioner Bloom Filter FP chance: 0.010000 Minimum timestamp: 1453373014633946 Maximum timestamp: 1453422073664423 SSTable min local deletion time: 1453373014 SSTable max local deletion time: 2147483647 Compressor: org.apache.cassandra.io.compress.LZ4Compressor Compression ratio: 0.4513946038492647 TTL min: 0 TTL max: 0 minClustringValues: [8b73e5e5-73a9-11e5-811a-5cf938947aea] maxClustringValues: [979d5e82-be45-11e5-8113-0242ac110002] Estimated droppable tombstones: 0.046573551370141054 SSTable Level: 0 Repaired at: 0 ReplayPosition(segmentId=1453356986934, position=1014) totalColumnsSet: 10649142 totalRows: 1100894 Estimated tombstone drop times: 1453373097: 31465 1453373269: 36611 1453373443: 30973 1453373647: 16000 1453373788: 13000 1453373953: 13000 1453374163: 19453 1453374373: 18312 1453374554: 4600 1453374692: 13000 1453374869: 10000 1453375031: 12000 1453375213: 14553 1453375411: 13069 1453375573: 9931 1453375793: 12143 1453376005: 10000 1453376190: 7810 1453376374: 6000 1453376556: 7470 1453376760: 7553 1453376963: 6000 1453377141: 5000 1453377366: 9000 1453377608: 8000 1453377793: 12906 1453377959: 16771 1453378171: 2101 1453378352: 5413 1453378495: 6192 1453378720: 7667 1453378943: 4000 1453379093: 4524 1453379275: 5000 1453379432: 4000 1453379564: 6000 1453379813: 7028 1453380061: 8000 1453380201: 1000 1453380336: 4736 1453380486: 6264 1453380730: 11000 1453380996: 8000 1453381192: 7000 1453381381: 13844 1453381523: 14371 1453381731: 5618 1453381941: 5000 1453382122: 4587 1453382323: 6210 1453382583: 5000 1453382706: 1000 1453382891: 5000 1453383033: 3000 1453383226: 3000 1453383428: 2000 1453383581: 4000 1453383786: 4000 1453383935: 5000 1453384175: 5000 1453384366: 7000 1453384559: 5042 1453384723: 6533 1453384985: 17384 1453385133: 7899 1453392140: 13000 1453392310: 8000 1453392445: 3000 1453392621: 5000 1453392787: 5000 1453392971: 4001 1453393132: 1000 1453393260: 5000 1453393422: 2557 1453393628: 4487 1453393953: 6000 1453394197: 2906 1453394324: 3000 1453394585: 4000 1453394785: 3000 1453394908: 1000 1453395081: 5487 1453395369: 2000 1453399065: 10000 1453399274: 13000 1453419602: 30387 1453419787: 19000 1453419948: 20368 1453420113: 25024 1453420257: 23138 1453420410: 26278 1453420591: 27474 1453420813: 23524 1453421013: 30201 1453421195: 36011 1453421404: 45970 1453421604: 49000 1453421783: 43136 1453421995: 6912 2147483647: 10653789 Count Row Size Cell Count 1 0 0 2 0 0 3 0 0 4 0 0 5 0 0 6 0 0 7 0 0 8 0 0 10 0 0 12 0 0 14 0 0 17 0 0 20 0 0 24 0 0 29 0 0 35 0 0 42 0 0 50 0 0 60 0 0 72 0 0 86 0 0 103 0 0 124 0 0 149 0 0 179 0 0 215 0 0 258 0 0 310 0 2 372 0 0 446 0 0 535 0 0 642 0 0 770 0 0 924 0 0 1109 0 0 1331 0 0 1597 0 0 1916 0 0 2299 0 0 2759 0 2 3311 0 0 3973 0 4 4768 0 8 5722 0 8 6866 0 6 8239 0 2 9887 0 2 11864 1 0 14237 0 0 17084 0 6 20501 0 6 24601 0 0 29521 0 4 35425 0 0 42510 0 26 51012 0 0 61214 0 6 73457 0 4 88148 0 2 105778 1 6 126934 0 2 152321 1 6 182785 3 0 219342 0 34 263210 2 14 315852 6 10 379022 1 6 454826 1 0 545791 0 0 654949 1 0 785939 0 0 943127 2 0 1131752 2 0 1358102 3 2 1629722 1 0 1955666 0 0 2346799 4 0 2816159 9 0 3379391 2 0 4055269 3 0 4866323 0 0 5839588 4 0 7007506 1 0 8409007 2 0 10090808 1 0 12108970 17 0 14530764 7 0 17436917 2 0 20924300 6 0 25109160 0 0 30130992 0 0 36157190 0 0 43388628 0 0 52066354 0 0 62479625 0 0 74975550 0 0 89970660 1 0 107964792 0 0 129557750 0 0 155469300 0 0 186563160 0 0 223875792 0 0 268650950 0 0 322381140 0 0 386857368 0 0 464228842 0 0 557074610 0 0 668489532 0 0 802187438 0 0 962624926 0 0 1155149911 0 0 1386179893 0 0 1663415872 0 0 1996099046 0 0 2395318855 0 0 2874382626 0 3449259151 0 4139110981 0 4966933177 0 5960319812 0 7152383774 0 8582860529 0 10299432635 0 12359319162 0 14831182994 0 17797419593 0 21356903512 0 25628284214 0 30753941057 0 36904729268 0 44285675122 0 53142810146 0 63771372175 0 76525646610 0 91830775932 0 110196931118 0 132236317342 0 158683580810 0 190420296972 0 228504356366 0 274205227639 0 329046273167 0 394855527800 0 473826633360 0 568591960032 0 682310352038 0 818772422446 0 982526906935 0 1179032288322 0 1414838745986 0 Estimated cardinality: 84 EncodingStats minTTL: 0 EncodingStats minLocalDeletionTime: 1453373014 EncodingStats minTimestamp: 1453373014633946 KeyType: org.apache.cassandra.db.marshal.CompositeType(org.apache.cassandra.db.marshal.TimeUUIDType,org.apache.cassandra.db.marshal.TimeUUIDType) ClusteringTypes: [org.apache.cassandra.db.marshal.TimeUUIDType] StaticColumns: {} RegularColumns: {gross:org.apache.cassandra.db.marshal.DecimalType, assetid:org.apache.cassandra.db.marshal.UTF8Type, raw:org.apache.cassandra.db.marshal.BytesType, lineid:org.apache.cassandra.db.marshal.UTF8Type, owners:org.apache.cassandra.db.marshal.MapType(org.apache.cassandra.db.marshal.TimeUUIDType,org.apache.cassandra.db.marshal.DecimalType), unassigned:org.apache.cassandra.db.marshal.BooleanType, type:org.apache.cassandra.db.marshal.UTF8Type, title:org.apache.cassandra.db.marshal.UTF8Type, views:org.apache.cassandra.db.marshal.LongType, platform:org.apache.cassandra.db.marshal.UTF8Type} SSTable: /var/lib/cassandra/data/cmpayments/section_lines-89867550737011e5938129463d90c3f0/ma-4806-big Partitioner: org.apache.cassandra.dht.Murmur3Partitioner Bloom Filter FP chance: 0.010000 Minimum timestamp: 1453373205271670 Maximum timestamp: 1453421821025555 SSTable min local deletion time: 1453373205 SSTable max local deletion time: 2147483647 Compressor: org.apache.cassandra.io.compress.LZ4Compressor Compression ratio: 0.4783642849189682 TTL min: 0 TTL max: 0 minClustringValues: [407c9473-7425-11e5-8e4c-5cf938947aea] maxClustringValues: [00f0b2c2-bdad-11e5-b365-0242ac110002] Estimated droppable tombstones: 0.04462957125056687 SSTable Level: 0 Repaired at: 0 ReplayPosition(segmentId=1453356986934, position=1014) totalColumnsSet: 667751 totalRows: 67717 Estimated tombstone drop times: 1453373205: 1000 1453373220: 1000 1453373238: 1000 1453373255: 853 1453373257: 147 1453373268: 916 1453373270: 84 1453373290: 1000 1453373321: 1000 1453373346: 1000 1453373370: 1000 1453373403: 1000 1453373433: 592 1453373435: 408 1453373473: 702 1453373475: 298 1453373532: 1000 1453373578: 677 1453373580: 323 1453373616: 284 1453373618: 707 1453373620: 9 1453373644: 661 1453373646: 339 1453373685: 1000 1453373773: 321 1453373775: 286 1453373777: 393 1453373904: 252 1453373906: 453 1453373908: 295 1453373973: 244 1453373975: 243 1453385006: 506 1453385008: 380 1453385010: 114 1453385040: 565 1453385042: 279 1453385044: 156 1453385055: 453 1453385057: 547 1453385093: 544 1453385095: 316 1453385097: 140 1453385121: 246 1453385123: 278 1453420037: 1000 1453420099: 1000 1453420114: 609 1453420116: 391 1453420147: 1000 1453420174: 932 1453420176: 68 1453420198: 1000 1453420232: 551 1453420234: 449 1453420282: 1000 1453420323: 1000 1453420360: 1000 1453420401: 684 1453420403: 316 1453420432: 1000 1453420478: 1000 1453420515: 1000 1453420548: 1000 1453420581: 1000 1453420647: 1000 1453420666: 415 1453420668: 585 1453420736: 663 1453420738: 337 1453420765: 487 1453420983: 1000 1453420998: 217 1453421027: 1000 1453421049: 1000 1453421091: 1000 1453421100: 1000 1453421146: 1000 1453421183: 1000 1453421207: 1000 1453421241: 1000 1453421257: 1000 1453421275: 1000 1453421299: 1000 1453421328: 1000 1453421345: 1000 1453421368: 1000 1453421398: 1000 1453421404: 1000 1453421418: 1000 1453421433: 1000 1453421466: 1000 1453421483: 1000 1453421500: 1000 1453421521: 1000 1453421547: 1000 1453421558: 1 1453421821: 1 2147483647: 668181 Count Row Size Cell Count 1 0 0 2 0 0 3 0 0 4 0 0 5 0 0 6 0 0 7 0 0 8 0 0 10 0 2 12 0 0 14 0 0 17 0 0 20 0 0 24 0 0 29 0 0 35 0 0 42 0 0 50 0 0 60 0 0 72 0 0 86 0 0 103 0 0 124 0 0 149 0 0 179 0 0 215 0 0 258 0 0 310 0 0 372 0 0 446 0 0 535 0 0 642 0 0 770 0 0 924 1 0 1109 0 0 1331 0 0 1597 0 0 1916 0 0 2299 0 0 2759 0 0 3311 0 0 3973 0 0 4768 0 0 5722 0 0 6866 0 0 8239 0 0 9887 0 0 11864 0 0 14237 0 2 17084 0 0 20501 0 0 24601 0 0 29521 0 0 35425 0 0 42510 0 2 51012 0 0 61214 0 0 73457 0 0 88148 0 0 105778 0 0 126934 0 0 152321 0 0 182785 0 0 219342 0 4 263210 0 2 315852 0 0 379022 0 0 454826 0 0 545791 0 0 654949 0 0 785939 1 0 943127 0 0 1131752 0 0 1358102 0 0 1629722 0 0 1955666 0 0 2346799 0 0 2816159 1 0 3379391 0 0 4055269 0 0 4866323 0 0 5839588 0 0 7007506 0 0 8409007 0 0 10090808 0 0 12108970 2 0 14530764 1 0 17436917 0 0 20924300 0 0 25109160 0 0 30130992 0 0 36157190 0 0 43388628 0 0 52066354 0 0 62479625 0 0 74975550 0 0 89970660 0 0 107964792 0 0 129557750 0 0 155469300 0 0 186563160 0 0 223875792 0 0 268650950 0 0 322381140 0 0 386857368 0 0 464228842 0 0 557074610 0 0 668489532 0 0 802187438 0 0 962624926 0 0 1155149911 0 0 1386179893 0 0 1663415872 0 0 1996099046 0 0 2395318855 0 0 2874382626 0 3449259151 0 4139110981 0 4966933177 0 5960319812 0 7152383774 0 8582860529 0 10299432635 0 12359319162 0 14831182994 0 17797419593 0 21356903512 0 25628284214 0 30753941057 0 36904729268 0 44285675122 0 53142810146 0 63771372175 0 76525646610 0 91830775932 0 110196931118 0 132236317342 0 158683580810 0 190420296972 0 228504356366 0 274205227639 0 329046273167 0 394855527800 0 473826633360 0 568591960032 0 682310352038 0 818772422446 0 982526906935 0 1179032288322 0 1414838745986 0 Estimated cardinality: 6 EncodingStats minTTL: 0 EncodingStats minLocalDeletionTime: 1453373205 EncodingStats minTimestamp: 1453373205271670 KeyType: org.apache.cassandra.db.marshal.CompositeType(org.apache.cassandra.db.marshal.TimeUUIDType,org.apache.cassandra.db.marshal.TimeUUIDType) ClusteringTypes: [org.apache.cassandra.db.marshal.TimeUUIDType] StaticColumns: {} RegularColumns: {gross:org.apache.cassandra.db.marshal.DecimalType, assetid:org.apache.cassandra.db.marshal.UTF8Type, raw:org.apache.cassandra.db.marshal.BytesType, lineid:org.apache.cassandra.db.marshal.UTF8Type, owners:org.apache.cassandra.db.marshal.MapType(org.apache.cassandra.db.marshal.TimeUUIDType,org.apache.cassandra.db.marshal.DecimalType), unassigned:org.apache.cassandra.db.marshal.BooleanType, type:org.apache.cassandra.db.marshal.UTF8Type, title:org.apache.cassandra.db.marshal.UTF8Type, views:org.apache.cassandra.db.marshal.LongType, platform:org.apache.cassandra.db.marshal.UTF8Type} SSTable: /var/lib/cassandra/data/cmpayments/section_lines-89867550737011e5938129463d90c3f0/ma-4807-big Partitioner: org.apache.cassandra.dht.Murmur3Partitioner Bloom Filter FP chance: 0.010000 Minimum timestamp: 1453836535120830 Maximum timestamp: 1453837141280024 SSTable min local deletion time: 1453836535 SSTable max local deletion time: 2147483647 Compressor: org.apache.cassandra.io.compress.LZ4Compressor Compression ratio: 0.43034129029425816 TTL min: 0 TTL max: 0 minClustringValues: [09b3439f-c463-11e5-b601-0242ac110002] maxClustringValues: [732d0fe4-c464-11e5-aec7-0242ac110002] Estimated droppable tombstones: 0.09901818215780964 SSTable Level: 0 Repaired at: 0 ReplayPosition(segmentId=1453796065504, position=17493268) totalColumnsSet: 1004859 totalRows: 112832 Estimated tombstone drop times: 1453836537: 768 1453836542: 960 1453836547: 1376 1453836552: 864 1453836557: 1376 1453836564: 1728 1453836571: 1440 1453836577: 1408 1453836584: 1632 1453836591: 1600 1453836599: 1056 1453836604: 896 1453836609: 1248 1453836614: 864 1453836619: 1280 1453836626: 1856 1453836633: 1088 1453836639: 1280 1453836646: 1696 1453836653: 1312 1453836660: 1792 1453836666: 832 1453836672: 1664 1453836680: 1856 1453836687: 1440 1453836692: 800 1453836697: 1216 1453836702: 960 1453836710: 1504 1453836716: 768 1453836722: 1504 1453836728: 736 1453836733: 1344 1453836738: 640 1453836744: 2016 1453836751: 1344 1453836757: 1472 1453836763: 1152 1453836767: 544 1453836772: 1312 1453836776: 608 1453836781: 1248 1453836787: 1216 1453836794: 1952 1453836802: 1376 1453836810: 1440 1453836817: 1280 1453836823: 1440 1453836831: 1344 1453836838: 1536 1453836843: 896 1453836850: 1344 1453836856: 1216 1453836861: 864 1453836867: 1632 1453836874: 1088 1453836878: 1024 1453836883: 832 1453836889: 992 1453836894: 512 1453836899: 416 1453836905: 288 1453836909: 416 1453836914: 352 1453836918: 384 1453836923: 384 1453836929: 512 1453836935: 448 1453836941: 480 1453836946: 416 1453836952: 448 1453836958: 704 1453836963: 704 1453836969: 1248 1453836974: 1216 1453836979: 896 1453836985: 1216 1453836991: 1120 1453836997: 1184 1453837003: 1120 1453837009: 1184 1453837016: 1472 1453837022: 1184 1453837029: 1600 1453837036: 1504 1453837042: 1280 1453837050: 1024 1453837054: 288 1453837062: 1280 1453837069: 1280 1453837076: 1248 1453837082: 1280 1453837090: 1408 1453837098: 1312 1453837104: 1248 1453837112: 1536 1453837120: 1536 1453837128: 1216 1453837136: 1536 2147483647: 1105140 Count Row Size Cell Count 1 0 0 2 0 0 3 0 0 4 0 0 5 0 0 6 0 0 7 0 0 8 0 0 10 0 0 12 0 0 14 0 0 17 0 0 20 0 0 24 0 0 29 0 0 35 0 0 42 0 0 50 0 0 60 0 0 72 0 0 86 0 0 103 0 0 124 0 0 149 0 0 179 0 0 215 0 0 258 0 0 310 0 0 372 0 0 446 0 0 535 0 0 642 0 0 770 0 0 924 0 0 1109 0 0 1331 0 0 1597 0 0 1916 0 0 2299 0 0 2759 0 0 3311 0 0 3973 0 0 4768 0 0 5722 0 0 6866 0 0 8239 0 0 9887 0 0 11864 0 0 14237 0 0 17084 0 0 20501 0 0 24601 0 0 29521 0 0 35425 0 0 42510 0 0 51012 0 0 61214 0 0 73457 0 0 88148 0 0 105778 0 0 126934 0 0 152321 0 0 182785 0 0 219342 0 0 263210 0 0 315852 0 0 379022 0 0 454826 0 0 545791 0 0 654949 0 0 785939 0 0 943127 0 0 1131752 0 1 1358102 0 0 1629722 0 0 1955666 0 0 2346799 0 0 2816159 0 0 3379391 0 0 4055269 0 0 4866323 0 0 5839588 0 0 7007506 0 0 8409007 0 0 10090808 0 0 12108970 0 0 14530764 0 0 17436917 0 0 20924300 0 0 25109160 0 0 30130992 0 0 36157190 0 0 43388628 0 0 52066354 1 0 62479625 0 0 74975550 0 0 89970660 0 0 107964792 0 0 129557750 0 0 155469300 0 0 186563160 0 0 223875792 0 0 268650950 0 0 322381140 0 0 386857368 0 0 464228842 0 0 557074610 0 0 668489532 0 0 802187438 0 0 962624926 0 0 1155149911 0 0 1386179893 0 0 1663415872 0 0 1996099046 0 0 2395318855 0 0 2874382626 0 3449259151 0 4139110981 0 4966933177 0 5960319812 0 7152383774 0 8582860529 0 10299432635 0 12359319162 0 14831182994 0 17797419593 0 21356903512 0 25628284214 0 30753941057 0 36904729268 0 44285675122 0 53142810146 0 63771372175 0 76525646610 0 91830775932 0 110196931118 0 132236317342 0 158683580810 0 190420296972 0 228504356366 0 274205227639 0 329046273167 0 394855527800 0 473826633360 0 568591960032 0 682310352038 0 818772422446 0 982526906935 0 1179032288322 0 1414838745986 0 Estimated cardinality: 1 EncodingStats minTTL: 0 EncodingStats minLocalDeletionTime: 1453836535 EncodingStats minTimestamp: 1453836535120830 KeyType: org.apache.cassandra.db.marshal.CompositeType(org.apache.cassandra.db.marshal.TimeUUIDType,org.apache.cassandra.db.marshal.TimeUUIDType) ClusteringTypes: [org.apache.cassandra.db.marshal.TimeUUIDType] StaticColumns: {} RegularColumns: {gross:org.apache.cassandra.db.marshal.DecimalType, assetid:org.apache.cassandra.db.marshal.UTF8Type, raw:org.apache.cassandra.db.marshal.BytesType, lineid:org.apache.cassandra.db.marshal.UTF8Type, owners:org.apache.cassandra.db.marshal.MapType(org.apache.cassandra.db.marshal.TimeUUIDType,org.apache.cassandra.db.marshal.DecimalType), unassigned:org.apache.cassandra.db.marshal.BooleanType, type:org.apache.cassandra.db.marshal.UTF8Type, title:org.apache.cassandra.db.marshal.UTF8Type, views:org.apache.cassandra.db.marshal.LongType, platform:org.apache.cassandra.db.marshal.UTF8Type} SSTable: /var/lib/cassandra/data/cmpayments/section_lines-89867550737011e5938129463d90c3f0/ma-4808-big Partitioner: org.apache.cassandra.dht.Murmur3Partitioner Bloom Filter FP chance: 0.010000 Minimum timestamp: 1453837141407997 Maximum timestamp: 1453841224171173 SSTable min local deletion time: 1453837141 SSTable max local deletion time: 2147483647 Compressor: org.apache.cassandra.io.compress.LZ4Compressor Compression ratio: 0.425758015440811 TTL min: 0 TTL max: 0 minClustringValues: [732fb62a-c464-11e5-aec8-0242ac110002] maxClustringValues: [f3e2c183-c46d-11e5-a670-0242ac110002] Estimated droppable tombstones: 0.09267497083869117 SSTable Level: 0 Repaired at: 0 ReplayPosition(segmentId=1453796065599, position=18408) totalColumnsSet: 1245993 totalRows: 139982 Estimated tombstone drop times: 1453837141: 320 1453837144: 576 1453837147: 416 1453837149: 480 1453837152: 832 1453837155: 448 1453837157: 448 1453837160: 800 1453837164: 864 1453837168: 800 1453837172: 736 1453837175: 416 1453837178: 736 1453837181: 384 1453837184: 544 1453837188: 704 1453837191: 512 1453837194: 864 1453837198: 800 1453837201: 416 1453837204: 736 1453837208: 736 1453837215: 7 1453841000: 2176 1453841003: 1152 1453841006: 2336 1453841009: 1216 1453841011: 1248 1453841014: 2432 1453841018: 2432 1453841021: 1248 1453841023: 1216 1453841026: 2144 1453841029: 1216 1453841031: 1216 1453841033: 1248 1453841036: 2528 1453841039: 1248 1453841043: 2240 1453841046: 1184 1453841049: 2304 1453841053: 2404 1453841056: 1212 1453841059: 2272 1453841062: 1216 1453841064: 1216 1453841066: 1216 1453841069: 1792 1453841074: 2240 1453841077: 1216 1453841079: 1216 1453841082: 2400 1453841085: 1216 1453841087: 1216 1453841089: 1216 1453841093: 1952 1453841097: 2432 1453841100: 1216 1453841102: 1216 1453841104: 1216 1453841106: 1216 1453841109: 1824 1453841113: 2304 1453841117: 2400 1453841120: 1120 1453841123: 2368 1453841126: 1248 1453841128: 1792 1453841132: 1472 1453841134: 1610 1453841137: 1846 1453841140: 576 1453841143: 1472 1453841145: 1632 1453841149: 1760 1453841151: 1664 1453841154: 1696 1453841158: 1408 1453841161: 1632 1453841163: 1792 1453841166: 1792 1453841169: 1856 1453841173: 1856 1453841176: 1824 1453841179: 1856 1453841182: 1728 1453841184: 1792 1453841187: 1824 1453841190: 1760 1453841193: 1824 1453841196: 1824 1453841199: 1184 1453841204: 2240 1453841207: 1760 1453841210: 1824 1453841213: 1824 1453841217: 2304 1453841220: 1632 1453841224: 7 2147483647: 1369162 Count Row Size Cell Count 1 0 0 2 0 0 3 0 0 4 0 0 5 0 0 6 0 0 7 0 0 8 0 0 10 0 0 12 0 0 14 0 0 17 0 0 20 0 0 24 0 0 29 0 0 35 0 0 42 0 0 50 0 0 60 0 0 72 0 0 86 0 0 103 0 0 124 0 0 149 0 0 179 0 0 215 0 0 258 0 0 310 0 0 372 0 0 446 0 0 535 0 0 642 0 0 770 0 0 924 0 0 1109 0 0 1331 0 0 1597 0 0 1916 0 0 2299 0 0 2759 0 0 3311 0 0 3973 0 0 4768 0 0 5722 0 0 6866 0 0 8239 0 0 9887 0 0 11864 0 0 14237 0 0 17084 0 0 20501 0 0 24601 0 0 29521 0 0 35425 0 0 42510 0 0 51012 0 0 61214 0 0 73457 0 0 88148 0 0 105778 0 0 126934 0 0 152321 0 1 182785 0 0 219342 0 0 263210 0 0 315852 0 0 379022 0 0 454826 0 0 545791 0 0 654949 0 0 785939 0 0 943127 0 0 1131752 0 0 1358102 0 1 1629722 0 0 1955666 0 0 2346799 0 0 2816159 0 0 3379391 0 0 4055269 0 0 4866323 0 0 5839588 1 0 7007506 0 0 8409007 0 0 10090808 0 0 12108970 0 0 14530764 0 0 17436917 0 0 20924300 0 0 25109160 0 0 30130992 0 0 36157190 0 0 43388628 0 0 52066354 0 0 62479625 1 0 74975550 0 0 89970660 0 0 107964792 0 0 129557750 0 0 155469300 0 0 186563160 0 0 223875792 0 0 268650950 0 0 322381140 0 0 386857368 0 0 464228842 0 0 557074610 0 0 668489532 0 0 802187438 0 0 962624926 0 0 1155149911 0 0 1386179893 0 0 1663415872 0 0 1996099046 0 0 2395318855 0 0 2874382626 0 3449259151 0 4139110981 0 4966933177 0 5960319812 0 7152383774 0 8582860529 0 10299432635 0 12359319162 0 14831182994 0 17797419593 0 21356903512 0 25628284214 0 30753941057 0 36904729268 0 44285675122 0 53142810146 0 63771372175 0 76525646610 0 91830775932 0 110196931118 0 132236317342 0 158683580810 0 190420296972 0 228504356366 0 274205227639 0 329046273167 0 394855527800 0 473826633360 0 568591960032 0 682310352038 0 818772422446 0 982526906935 0 1179032288322 0 1414838745986 0 Estimated cardinality: 2 EncodingStats minTTL: 0 EncodingStats minLocalDeletionTime: 1453837141 EncodingStats minTimestamp: 1453837141407997 KeyType: org.apache.cassandra.db.marshal.CompositeType(org.apache.cassandra.db.marshal.TimeUUIDType,org.apache.cassandra.db.marshal.TimeUUIDType) ClusteringTypes: [org.apache.cassandra.db.marshal.TimeUUIDType] StaticColumns: {} RegularColumns: {gross:org.apache.cassandra.db.marshal.DecimalType, assetid:org.apache.cassandra.db.marshal.UTF8Type, raw:org.apache.cassandra.db.marshal.BytesType, lineid:org.apache.cassandra.db.marshal.UTF8Type, owners:org.apache.cassandra.db.marshal.MapType(org.apache.cassandra.db.marshal.TimeUUIDType,org.apache.cassandra.db.marshal.DecimalType), unassigned:org.apache.cassandra.db.marshal.BooleanType, type:org.apache.cassandra.db.marshal.UTF8Type, title:org.apache.cassandra.db.marshal.UTF8Type, views:org.apache.cassandra.db.marshal.LongType, platform:org.apache.cassandra.db.marshal.UTF8Type} root@cass1:/tmp/apache-cassandra-3.4-SNAPSHOT/tools/bin# {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332)