[ https://issues.apache.org/jira/browse/HBASE-9775?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13799681#comment-13799681 ]
Elliott Clark commented on HBASE-9775: -------------------------------------- Here's what I'm seeing in the logs. {code} 13-10-18 17:14:02,641 WARN [RpcServer.responder] ipc.RpcServer: Exception while changing ops : java.nio.channels.CancelledKeyException 2013-10-18 17:14:02,642 WARN [RpcServer.handler=3,port=60020] ipc.RpcServer: (responseTooSlow): {"processingtimems":13904,"call":"Multi(org.apache.hadoop.hbase.protobuf.generated.ClientProtos$MultiRequest)","client":"10.20.130.107:39862","starttimems":1382141628563,"queuetimems":30966,"class":"HRegionServer","responsesize":3330303,"method":"Multi"} 2013-10-18 17:14:03,576 INFO [Thread-14] regionserver.HStore: Added hdfs://e1101.halxg.cloudera.com:8020/hbase/data/default/IntegrationTestBigLinkedList/663b4cb3fa350cd61c9f3317eb6c577a/meta/dcec7a6ef36e437da110aef12b507d72, entries=1271646, sequenceid=2223534, filesize=94.8 M 2013-10-18 17:14:03,693 INFO [Thread-14] regionserver.HRegion: Finished memstore flush of ~262.0 M/274675536, currentsize=0/0 for region IntegrationTestBigLinkedList,\xB6\x1B\xE0|0+!\xA1\xC0\xB5\x0F\xAB\x85\x9C\xA6\xB0,1382141290187.663b4cb3fa350cd61c9f3317eb6c577a. in 198100ms, sequenceid=2223534, compaction requested=false 2013-10-18 17:14:03,746 DEBUG [Thread-14] regionserver.HStore: Cannot split region due to reference files being there 2013-10-18 17:14:03,789 DEBUG [Thread-14] regionserver.HRegion: Started memstore flush for IntegrationTestBigLinkedList,\xB4\x1D\xFF\x1A,1382141290187.2845b99ca6c5c8c86f150b1cb17b7063., current region memstore size 264.7 M 2013-10-18 17:14:10,220 WARN [RpcServer.handler=20,port=60020] ipc.RpcServer: (responseTooSlow): {"processingtimems":10663,"call":"Multi(org.apache.hadoop.hbase.protobuf.generated.ClientProtos$MultiRequest)","client":"10.20.132.117:35031","starttimems":1382141639521,"queuetimems":31152,"class":"HRegionServer","responsesize":1596044,"method":"Multi"} 2013-10-18 17:14:11,777 INFO [JvmPauseMonitor] util.JvmPauseMonitor: Detected pause in JVM or host machine (eg GC): pause of approximately 1023ms GC pool 'ParNew' had collection(s): count=1 time=1102ms 2013-10-18 17:14:14,620 INFO [JvmPauseMonitor] util.JvmPauseMonitor: Detected pause in JVM or host machine (eg GC): pause of approximately 1300ms GC pool 'ParNew' had collection(s): count=1 time=1383ms {code} > Client write path perf issues > ----------------------------- > > Key: HBASE-9775 > URL: https://issues.apache.org/jira/browse/HBASE-9775 > Project: HBase > Issue Type: Bug > Components: Client > Affects Versions: 0.96.0 > Reporter: Elliott Clark > Priority: Critical > Attachments: Charts Search Cloudera Manager - ITBLL.png, Charts > Search Cloudera Manager.png, hbase-9775.patch, job_run.log, short_ycsb.png, > ycsb_insert_94_vs_96.png, ycsb.png > > > Testing on larger clusters has not had the desired throughput increases. -- This message was sent by Atlassian JIRA (v6.1#6144)