I ran PE.randomRead against 0.94.7 and 0.94.9. 0.94.7: 589036ms 0.94.9: 583656ms
This is repeatable. So I am not sure how you arrive at your numbers where 0.94.8 (and 0.94.9) are supposedly 20% slower than 0.94.7 in this test. -- Lars ----- Original Message ----- From: Jean-Marc Spaggiari <[email protected]> To: [email protected]; lars hofhansl <[email protected]> Cc: Sent: Friday, June 28, 2013 8:06 PM Subject: Re: [ANNOUNCE] The 1st hbase 0.94.9 release candidate is available for download Sorry for the delay team. The new PDF with the 0.94.8 and 0.94.9 performance graphs is now online. http://www.spaggiari.org/media/blogs/hbase/pdf/performances_20130628.pdf JM 2013/6/25 Jean-Marc Spaggiari <[email protected]>: > Here is my feedback. > > - Downloaded the files, checked the signature, uncompressed, checked > changes.txt => Fine > - Checked the documentation => One page not found under "Ref Guide > (multi-page)" link on the index page. Should it be there? > - Ran the test suite on 2 different servers, > TestRegionObserverScannerOpenHook failed 10 times in a row with 8 > thread, 4, 2 and 1. > - IntegrationTestLoadAndVerify => REFERENCES_CHECKED=985565 with > 0.94.9 and REFERENCES_CHECKED=9855611 with 0.94.8. Don't know what to > deduct. > - integrationTestBigLinkedList => Fine > - HBCK still reporting fine after all those tests. > > However, I have this exception many times on the logs, every 3 seconds: > > 2013-06-24 22:24:01,159 ERROR > org.apache.hadoop.hbase.regionserver.HRegionServer: Closing scanner > for > IntegrationTestLoadAndVerify,\xA8k\xCA\x1A\xF2\x86\xBC\x8A,1372126832536.31c325cb8ae32b1ed0f4c4667d787165. > org.apache.hadoop.hbase.NotServingRegionException: Region is not > online: > IntegrationTestLoadAndVerify,\xA8k\xCA\x1A\xF2\x86\xBC\x8A,1372126832536.31c325cb8ae32b1ed0f4c4667d787165. > at >org.apache.hadoop.hbase.regionserver.HRegionServer.getRegion(HRegionServer.java:3509) > at >org.apache.hadoop.hbase.regionserver.HRegionServer$ScannerListener.leaseExpired(HRegionServer.java:2740) > at org.apache.hadoop.hbase.regionserver.Leases.run(Leases.java:100) > at java.lang.Thread.run(Thread.java:662) > > - Performed a rolling restart for the RS. 2500 regions to move > around... => Fine. Took 74 minutes. Changed to regoin_mover to > multhread, took 16 minutes. > - Looked at the master and region logs, everything seems to be ok. > - Merged all the regions from a table to a single region. > - Checked HBCK, one issue on one table which is not the one merged. > Issue was not there before the restarts, etc. Few hbck repairs > repaired everything. > - Logs still good, table merged correctly. > - Triggered a major compaction, got the table splitted back to 36 regions. > - Ran the balalancer, region got spreaded over the nodes. => Fine, but > had to run it twice. First time 50% of the regions stayed assigned to > one RS. > - Stopped, restored my customed balancer, restart, rebalanced => Fine. > - Create table, put, scan, get, disable, drop, balance, list, > truncate_preserve, truncate from shell => Fine > - Ran one MR job (416 mappers), went well, nothing in the logs. > > So to summarize: +1 ;) Even is there is 2 things to verify > (TestRegionObserverScannerOpenHook and NotServingRegionException). I > don't think they related to this release. > > I will post the performances comparison later tonight or tomorrow. > (Need to add 0.94.8 also since I'm late), > > JM > > 2013/6/24 Jean-Marc Spaggiari <[email protected]>: >> Oh! Cool! A new toy for the week! ;) >> >> Thanks Lars! I will start right now to test it! (And I will do the >> remaining perf testing just after or in parallel...) >> >> 2013/6/24 lars hofhansl <[email protected]>: >>> The 1st 0.94.9 RC is available for download at >>> http://people.apache.org/~larsh/hbase-0.94.9-rc0/ >>> Signed with my code signing key: C7CFE328 >>> >>> Like the previous point releases, 0.94.9 is a primarily a bug fix release. >>> This RC is comparatively small with 26 issues resolved against it: >>> [HBASE-8453] - TestImportExport failing again due to configuration >>>issues >>> [HBASE-8494] - TestRemoteAdmin#testClusterStatus should not assume >>>'requests' does not change >>> [HBASE-8522] - Archived hfiles and old hlogs may be deleted immediately >>>by HFileCleaner, LogCleaner in HMaster >>> [HBASE-8555] - FilterList correctness may be affected by random >>>ordering of sub-filter(list) >>> [HBASE-8590] - [0.94] BlockingMetaScannerVisitor should check for >>>parent meta entry while waiting for split daughter >>> [HBASE-8639] - Poor performance of htable#getscanner in multithreaded >>>environment due to DNS.getDefaultHost() being called in >>>ScannerCallable#prepare() >>> [HBASE-8640] - ServerName in master may not initialize with the >>>configured ipc address of hbase.master.ipc.address >>> [HBASE-8655] - Backport to 94 - HBASE-8346(Prefetching .META. rows in >>>case only when useCache is set to true) >>> [HBASE-8656] - Rpc call may not be notified in SecureClient >>> [HBASE-8671] - Per-region WAL breaks CP backwards compatibility in 0.94 >>>for non-enabled case >>> [HBASE-8684] - Table Coprocessor can't access external HTable by default >>> [HBASE-8700] - IntegrationTestBigLinkedList can fail due to random >>>number collision >>> [HBASE-8724] - [0.94] ExportSnapshot should not use hbase.tmp.dir as a >>>staging dir on hdfs >>> [HBASE-8742] - HTableDescriptor Properties not preserved when cloning >>> [HBASE-8743] - upgrade hadoop-23 version to 0.23.7 >>> [HBASE-8749] - Potential race condition between >>>FSUtils.renameAndSetModifyTime() and HFile/LogCleaner >>> [HBASE-8762] - Performance/operational penalty when calling HTable.get >>>with a list of one Get >>> [HBASE-8783] - RSSnapshotManager.ZKProcedureMemberRpcs may be >>>initialized with the wrong server name >>> [HBASE-5083] - Backup HMaster should have http infoport open with link >>>to the active master >>> [HBASE-8609] - Make the CopyTable support startRow, stopRow options >>> [HBASE-8636] - Backport KeyValue Codec to 0.94 (HBASE-7413) >>> [HBASE-8683] - Add major compaction support in CompactionTool >>> [HBASE-8692] - [AccessController] Restrict HTableDescriptor enumeration >>> [HBASE-8702] - Make WALEditCodec pluggable >>> [HBASE-8504] - HTable.getRegionsInRange() should provide a non-cached >>>API >>> [HBASE-8603] - Backport HBASE-6921 to 0.94 >>> >>> The full list of changes is also available here: >>> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12310753&version=12324431 >>> >>> Please try out the RC, check out the doc, take it for a spin, etc, and vote >>> +1/-1 by July 2nd on whether we should release this as 0.94.9. >>> >>> The release testing spreadsheet is available here: >>> https://docs.google.com/spreadsheet/ccc?key=0AvLqcVIqNtlTdGFHaktDR0FqLWdtQWZwdncyVzE3Z2c#gid=0 >>> (if you test this release, please add your test to this spreadsheet, so >>> that we can gauge the coverage) >>> >>> Thanks. >>> >>> -- Lars >>> >>> ps. I will likely be without access to the Internet for the next five days
