I have experienced excessive performance issues while using collections as well. Mostly my issue was due to the excessive number of cells per partition that having a modest map size requires.
Since you are reading and writing the entire map, you can probably gain some performance the same way I did. Convert you map to be a frozen map. This essentially puts you in the same place as folks who migrate to a blob of json, but it puts the onus on Cassandra to manage serializing and deserializing the map. It does have limitations over a regular map.. You cant append values, you can't selectively ttl, reading single keys requires deserializing the whole collection. Basically anything besides reading and writing the whole collection becomes a little harder. But it is considerably faster due to the lower cell count and management overhead. Clint On Feb 8, 2016 5:11 PM, "Agrawal, Pratik" <paagr...@amazon.com> wrote: > Hello all, > > Recently we added one of the table fields from as Map<text, text> in > *Cassandra > 2.1.11*. Currently we read every field from Map and overwrite map values. > Map is of size 3. We saw that writes are 30-40% slower while reads are > 70-80% slower. Please find below some metrics that can help. > > My question is, Are there any known issues in Cassandra map performance? > As I understand it each of the CQL3 Map entry, maps to a column in > cassandra, with that assumption we are just creating 3 columns right? Any > insight on this issue would be helpful. > > Datastax Java Driver 2.1.6. > Machine: Amazon C3 2x large > CPU – pretty much same as before (around 30%) > Memory – max around 4.8 GB > > CFSTATS: > > Keyspace: Keyspace > Read Count: 28359044 > Read Latency: 2.847392469259542 ms. > Write Count: 1152765 > Write Latency: 0.14778018590085576 ms. > Pending Flushes: 0 > Table: table1 > SSTable count: 1 > SSTables in each level: [1, 0, 0, 0, 0, 0, 0, 0, 0] > Space used (live): 4119699 > Space used (total): 4119699 > Space used by snapshots (total): 90323640 > Off heap memory used (total): 2278 > SSTable Compression Ratio: 0.23172161124142604 > Number of keys (estimate): 14 > Memtable cell count: 6437 > Memtable data size: 872912 > Memtable off heap memory used: 0 > Memtable switch count: 7626 > Local read count: 27754634 > Local read latency: 1.921 ms > Local write count: 1113668 > Local write latency: 0.142 ms > Pending flushes: 0 > Bloom filter false positives: 0 > Bloom filter false ratio: 0.00000 > Bloom filter space used: 96 > Bloom filter off heap memory used: 88 > Index summary off heap memory used: 46 > Compression metadata off heap memory used: 2144 > Compacted partition minimum bytes: 315853 > Compacted partition maximum bytes: 4055269 > Compacted partition mean bytes: 2444011 > Average live cells per slice (last five minutes): 17.536775249005437 > Maximum live cells per slice (last five minutes): 1225.0 > Average tombstones per slice (last five minutes): 34.99979575985972 > Maximum tombstones per slice (last five minutes): 3430.0 > > Table: table2 > SSTable count: 1 > SSTables in each level: [1, 0, 0, 0, 0, 0, 0, 0, 0] > Space used (live): 869900 > Space used (total): 869900 > Space used by snapshots (total): 17279824 > Off heap memory used (total): 387 > SSTable Compression Ratio: 0.3999013540551859 > Number of keys (estimate): 2 > Memtable cell count: 1958 > Memtable data size: 8 > Memtable off heap memory used: 0 > Memtable switch count: 7484 > Local read count: 604412 > Local read latency: 45.421 ms > Local write count: 39097 > Local write latency: 0.337 ms > Pending flushes: 0 > Bloom filter false positives: 0 > Bloom filter false ratio: 0.00000 > Bloom filter space used: 96 > Bloom filter off heap memory used: 88 > Index summary off heap memory used: 35 > Compression metadata off heap memory used: 264 > Compacted partition minimum bytes: 1955667 > Compacted partition maximum bytes: 2346799 > Compacted partition mean bytes: 2346799 > Average live cells per slice (last five minutes): 1963.0632242863855 > Maximum live cells per slice (last five minutes): 5001.0 > Average tombstones per slice (last five minutes): 0.0 > Maximum tombstones per slice (last five minutes): 0.0 > > *NETSTATS:* > Mode: NORMAL > Not sending any streams. > Read Repair Statistics: > Attempted: 2853996 > Mismatch (Blocking): 67386 > Mismatch (Background): 9233 > Pool Name Active Pending Completed > Commands n/a 0 33953165 > Responses n/a 0 370301 > > *IOSTAT* > avg-cpu: %user %nice %system %iowait %steal %idle > 15.20 0.83 0.56 0.10 0.04 83.27 > > Device: tps Blk_read/s Blk_wrtn/s Blk_read Blk_wrtn > xvda 2.79 0.47 69.86 553719 82619304 > xvdb 14.49 3.39 775.56 4009600 917227536 > xvdc 15.13 2.98 819.93 3522250 969708944 > dm-0 49.67 6.36 1595.49 7525858 1886936320 > > *TPSTAT:* > Pool Name Active Pending Completed Blocked > All time blocked > MutationStage 0 0 1199683 0 > 0 > ReadStage 0 0 28449207 0 > 0 > RequestResponseStage 0 0 33983356 0 > 0 > ReadRepairStage 0 0 2865749 0 > 0 > CounterMutationStage 0 0 0 0 > 0 > MiscStage 0 0 0 0 > 0 > HintedHandoff 0 0 2 0 > 0 > GossipStage 0 0 270364 0 > 0 > CacheCleanupExecutor 0 0 0 0 > 0 > InternalResponseStage 0 0 0 0 > 0 > CommitLogArchiver 0 0 0 0 > 0 > CompactionExecutor 0 0 111234 0 > 0 > ValidationExecutor 0 0 0 0 > 0 > MigrationStage 0 0 7 0 > 0 > AntiEntropyStage 0 0 0 0 > 0 > PendingRangeCalculator 0 0 3 0 > 0 > Sampler 0 0 0 0 > 0 > MemtableFlushWriter 0 0 45884 0 > 0 > MemtablePostFlush 0 0 47461 0 > 0 > MemtableReclaimMemory 0 0 45884 0 > 0 > > Message type Dropped > READ 0 > RANGE_SLICE 0 > _TRACE 0 > MUTATION 0 > COUNTER_MUTATION 0 > BINARY 0 > REQUEST_RESPONSE 0 > PAGED_RANGE 0 > READ_REPAIR 0 > > Thanks, > Pratik >
