Nice analysis Andy. St.Ack On Mon, Dec 18, 2017 at 11:54 AM, Andrew Purtell <apurt...@apache.org> wrote:
> And here are comparative profiling results between 1.2.6 and 1.4.0 RC1. I > collected Flight Recorder traces for the randomRead, randomWrite, > sequentialRead, sequentialWrite, and increment PerformanceEvaluation > workloads. (Also, randomSeekScan but I think I made an error during test > execution, so the data is included but not analyzed.) Traces were collected > from the regionserver on a single host dev cluster running separate > processes for namenode, datanode, zookeeper, master, regionserver, and PE > client. To prime the system I loaded 10 million rows using './bin/hbase pe > --nomapred --blockEncoding=FAST_DIFF --rows=10000000 randomWrite 1' and > took a snapshot. For every workload I did the following: > > 1. Disable the table, restore from snapshot, enable the table > 2. Shut down the regionserver > 3. Start the regionserver > 4. Start continuous profiling > 5. Run the workload > 6. Dump the JFR data > 7. Repeat for next workload > > Because the lifetime of the regionserver process, therefore the JFR event > trace, is the same as the workload you can compare all counts directly. > > Find the data here: https://bigdatasf.s3.amazonaws.com/assets/pe_1.2.6_ > to_1.4.0RC1_jfr.tar.xz (17 MB) > > Some observations: > > - GC times are improved with 1.4.0 RC1 versus 1.2.6: fewer > collections, shorter pause times, especially for read workloads > - Object allocation counts are fewer with 1.4.0 RC1 versus 1.2.6 > - Object allocation rate is significantly decreased with 1.4.0 RC1 > versus 1.2.6, especially for read workloads. This can be explained I think > by reduced allocation of byte[], which dominates out-of-TLAB counts (99% of > all such allocations) > - Allocation rate 3.5x less for sequentialRead workload > - Allocation rate 1.9x less for randomRead workload > - Allocation rate 2.2x less for increment workload > - Typically, lower rates measured for allocations in TLABs too > - Method profiles look pretty similar > - On CPU time in ConcurrentSkipListMap in the 1.2.6 read traces is > missing from the 1.4.0 RC1 read traces, except increment, where it is very > similar. This might be an efficiency win. > > > On Wed, Dec 13, 2017 at 7:24 PM, Andrew Purtell <apurt...@apache.org> > wrote: > >> And here are results for PE. >> >> https://bigdatasf.s3.amazonaws.com/assets/pe_1.2.6_1.4.0RC1.png >> https://bigdatasf.s3.amazonaws.com/assets/pe_1.2.6_1.4.0RC1.xlsx >> >> [image: Inline image 1] >> >> Similar to the YCSB results, 1.2.6 and 1.4.0 RC1 are close, but 1.4.0 RC1 >> shows superior results. >> >> Cluster: 1 master, 5 regionservers, 1 client >> AWS Instance Type: c3.4xlarge >> Hadoop: 2.7.4 >> OS: Amazon Linux AMI 2017.09, kernel 4.4.51-40.58.amzn1.x86_64 >> >> On Tue, Dec 12, 2017 at 5:50 PM, Andrew Purtell <apurt...@apache.org> >> wrote: >> >>> Here are results from a YCSB comparison between 1.2.6 and 1.4.0 RC1: >>> >>> https://bigdatasf.s3.amazonaws.com/assets/ycsb_1.2.6_to_1.4.0RC1.png >>> https://bigdatasf.s3.amazonaws.com/assets/ycsb_1.2.6_to_1.4.0RC1.xlsx >>> >>> [image: Inline image 1] >>> >>> They are pretty closely matched but generally 1.4.0 RC1 shows superior >>> results. >>> >>> [R] and [W] mean different things depending on YCSB workload. They will >>> be obvious except for Workload F, where [R] is READ and [W] is UPDATE. I >>> didn't include numbers for READ-MODIFY-WRITE. >>> >>> Cluster: 1 master, 5 regionservers, 1 client >>> AWS Instance Type: c3.4xlarge >>> Hadoop: 2.7.4 >>> OS: Amazon Linux AMI 2017.09, kernel 4.4.51-40.58.amzn1.x86_64 >>> >>> >>> On Fri, Dec 8, 2017 at 5:50 PM, Andrew Purtell <apurt...@apache.org> >>> wrote: >>> >>>> The second HBase 1.4.0 release candidate (RC1) is available for >>>> download at https://dist.apache.org/repos/ >>>> dist/dev/hbase/hbase-1.4.0RC1/ and Maven artifacts are available in >>>> the temporary repository https://repository.apache.org/ >>>> content/repositories/orgapachehbase-1186/ . >>>> >>>> The git tag corresponding to the candidate is '1.4.0RC1' (10b9b9fae6). >>>> >>>> A detailed source and binary compatibility report for this release is >>>> available for your review at https://dist.apache.org/repos/ >>>> dist/dev/hbase/hbase-1.4.0RC1/hbase-1.3.1-1.4.0RC1_compatibi >>>> lity_report.html . All reported compatibility issues should comply >>>> with policy but please review them carefully. >>>> >>>> A list of the 660 issues resolved in this release can be found at >>>> https://s.apache.org/OErT . >>>> >>>> The changes since RC0 are: >>>> >>>> * [HBASE-19180] - Remove unused imports from AlwaysPasses >>>> * [HBASE-19373] - Fix Checkstyle error in hbase-annotations >>>> * [HBASE-19435] - Reopen Files for ClosedChannelException in >>>> BucketCache >>>> * [HBASE-19465] - Required httpcore and httpclient jars not >>>> included in binary distribution >>>> * [HBASE-19467] - rsgroups shell commands don't properly display >>>> elapsed time >>>> >>>> Please try out the candidate and vote +1/0/-1. >>>> >>>> This vote will be open for at least 72 hours. Unless objection I will >>>> try to close it Monday December 18, 2017 if we have sufficient votes. >>>> >>>> Prior to making this announcement I made the following preflight checks >>>> to 1.4.0RC0 (3d571827cb): >>>> >>>> RAT check passes (7u80) >>>> Unit test suite passes (8u131) >>>> LTT load 1M rows with 100% verification and 20% updates (8u131) >>>> PE randomWrite, randomRead, scanRange100 (8u131) >>>> 100M rows ITBLL (8u131) >>>> >>>> and the following preflight checks to 1.4.0RC1 (10b9b9fae6): >>>> >>>> RAT check passes (7u80) >>>> Unit test suite passes (8u131) >>>> >>>> Between now and when I want to close the vote I'll write up human >>>> readable release notes for the release announcement as promised. >>>> >>>> I also have agreed to run a scale ITBLL test, a performance comparison >>>> with 1.2 using YCSB, a performance comparison with 1.2 using PE, and an >>>> analysis of code and allocation hot spot changes from 1.2, all of which I >>>> will publish when available and factor in to my vote. >>>> >>>> -- >>>> Best regards, >>>> Andrew >>>> >>>> >>> >>> >>> -- >>> Best regards, >>> Andrew >>> >>> Words like orphans lost among the crosstalk, meaning torn from truth's >>> decrepit hands >>> - A23, Crosstalk >>> >> >> >> >> -- >> Best regards, >> Andrew >> >> Words like orphans lost among the crosstalk, meaning torn from truth's >> decrepit hands >> - A23, Crosstalk >> > > > > -- > Best regards, > Andrew > > Words like orphans lost among the crosstalk, meaning torn from truth's > decrepit hands > - A23, Crosstalk >