Here is my RC5 testing so far: - checked checksums, sigs - checked the bin and src artifacts - checked layouts - checked java files in src tarball, and jar files in bin tarball - checked the book and the site (they are new style) - checked javadocs for both devapi and userapi - checked reported version, build time, revision - run some smoke tests using shell - started local mode - run LTT local mode - checked the webUIs of master and region servers - checked JMX dump and debug dump - Build src with hadoop versions 2.2.0 2.3.0 2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.0 - Deployed at a 6 node cluster with Hadoop-2.6.0 - Run LTT over tables with NONE, DIFF, FAST_DIFF, and PREFIX encoding - Run LTT over tables with NONE, GZ, LZO, LZ4 and SNAPPY compression - Tested with Bucket cache with 500M off heap
I am still running larger scale tests, and some CM tests on the cluster. Will report back tomorrow and cast my vote. Enis On Wed, Feb 18, 2015 at 4:47 PM, 张铎 <palomino...@gmail.com> wrote: > TestCacheOnWrite itself has some problems. It > uses TestHFileWriterV2.randomOrderedKey to generate a random byte array, > then use first 32 bytes as row and other parts as family and qualifier, but > TestHFileWriterV2.randomOrderedKey may return a byte array only contains 32 > bytes, so there will be family and qualifier with zero length. > > I do not know if this is the reason why this test is flaky since it use a > Random with pre-defined seed so the random sequence should be stable. I can > modify the KeyValue generation part to see if it helps. > > BTW, the name 'randomOrderedKey' is ambiguous, may change to ' > randomOrderedRow'? > > 2015-02-19 7:11 GMT+08:00 Andrew Purtell <apurt...@apache.org>: > > > I'm not able to get a clean unit test run when building from source using > > 7u67. TestCacheOnWrite and TestSplitLogManager fail for me, maybe more > but > > the build doesn't get past hbase-server. Maybe these are known issues? If > > not I'll dig in when I get some time. > > > > > > On Sat, Feb 14, 2015 at 9:55 PM, Enis Söztutar <e...@apache.org> wrote: > > > > > It gives me great pleasure to announce that the sixth release candidate > > for > > > the release > > > 1.0.0 (HBase-1.0.0RC5), is available for download at > > > https://dist.apache.org/repos/dist/dev/hbase/hbase-1.0.0RC5/ > > > > > > Maven artifacts are also available in the temporary repository > > > https://repository.apache.org/content/repositories/orgapachehbase-1065 > > > > > > Signed with my code signing key E964B5FF. Can be found here: > > > https://people.apache.org/keys/committer/enis.asc > > > > > > Signed tag in the repository can be found here: > > > > > > > > > https://git-wip-us.apache.org/repos/asf?p=hbase.git;a=tag;h=c4660912e9b46c917a9aba2106be4bf74182a764 > > > > > > HBase 1.0.0 is the next stable release, and the start of "semantic > > > versioned" > > > releases (See [1]). > > > > > > The theme of 1.0.0 release is to become a stable base for future 1.x > > series > > > of releases. We aim to achieve at least the same level of stability of > > 0.98 > > > releases. > > > > > > 1.0.0 release contains 202 fixes on top of 0.99.2 release. Together > with > > > the > > > previous 0.99.x releases, major changes in 1.0.0 are listed (but not > > > limited to) > > > below. Note that all previous 0.99.x releases are developer preview > > > releases, and will > > > NOT be supported in any form. > > > > > > API Cleanup and changes > > > 1.0.0 introduces new APIs, and deprecates some of commonly-used > > > client side APIs (HTableInterface, HTable and HBaseAdmin). > > > We advise to update your application to use the new style of APIs, > > since > > > deprecated APIs might be removed in future releases (2.x). See [2] > and > > > [3] > > > for an overview of changes. All Client side API's are marked with > > > InterfaceAudience.Public class, indicating that the class/method is > an > > > official "client API" for HBase. All 1.x releases are planned to be > API > > > compatible for these classes. See [1] for an overview. > > > > > > Master runs a Region Server as well > > > Starting with 1.0.0, the HBase master server and backup master > servers > > > will > > > also act as a region server. RPC port and info port for web UI is > > shared > > > for > > > the master and region server roles. Active master can host regions of > > > defined tables if configured (disabled by default). Backup masters > will > > > not > > > host regions. > > > > > > Read availability using timeline consistent region replicas > > > This release contains Phase 1 items for experimental "Read > availability > > > using > > > timeline consistent region replicas" feature. A region can be hosted > in > > > multiple region servers in read-only mode. One of the replicas for > the > > > region > > > will be primary, accepting writes, and other replicas will be sharing > > the > > > same > > > data files. Read requests can be done against any replica for the > > region > > > with > > > backup RPCs for high availability with timeline consistency > guarantees. > > > More > > > information can be found at HBASE-10070. > > > > > > Online config change and other forward ports from 0.89-fb branch > > > HBASE-12147 forward ported online config change which enables some of > > the > > > configuration from the server to be reloaded without restarting the > > > region > > > servers. > > > > > > Other notable improvements in 1.0.0 (including previous 0.99.x) are > > > - A new web skin in time for 1.0 (http://hbase.apache.org) > > > - Automatic tuning of global memstore and block cache sizes > > > - Various security, tags and visibility labels improvements > > > - Bucket cache improvements (usability and compressed data blocks) > > > - A new pluggable replication endpoint to plug in to HBase's > > inter-cluster > > > replication to replicate to a custom data store > > > - A Dockerfile to easily build and run HBase from source > > > - Truncate table command > > > - Region assignment to use hbase:meta table instead of zookeeper for > > > faster > > > region assignment (disabled by default) > > > - Extensive documentation improvements > > > - [HBASE-12511] - namespace permissions - add support from table > > creation > > > privilege in a namespace 'C' > > > - [HBASE-12568] - Adopt Semantic Versioning and document it in the > book > > > - [HBASE-12640] - Add Thrift-over-HTTPS and doAs support for Thrift > > Server > > > - [HBASE-12651] - Backport HBASE-12559 'Provide LoadBalancer with > online > > > configuration capability' to branch-1 > > > - [HBASE-10560] - Per cell TTLs > > > - [HBASE-11997] - CopyTable with bulkload > > > - [HBASE-11990] - Make setting the start and stop row for a specific > > > prefix easier > > > - [HBASE-12220] - Add hedgedReads and hedgedReadWins metrics > > > - [HBASE-12090] - Bytes: more Unsafe, more Faster > > > - [HBASE-12032] - Script to stop regionservers via RPC > > > - [HBASE-11907] - Use the joni byte[] regex engine in place of > j.u.regex > > > in RegexStringComparator > > > - [HBASE-11796] - Add client support for atomic checkAndMutate > > > - [HBASE-11804] - Raise default heap size if unspecified > > > - [HBASE-11890] - HBase REST Client is hard coded to http protocol > > > - [HBASE-12126] - Region server coprocessor endpoint > > > - [HBASE-12183] - FuzzyRowFilter doesn't support reverse scans > > > - [HBASE-12075] - Preemptive Fast Fail > > > - [HBASE-12354] - Update dependencies in time for 1.0 release > > > - [HBASE-12363] - Improve how KEEP_DELETED_CELLS works with > MIN_VERSIONS > > > - [HBASE-12434] - Add a command to compact all the regions in a > > > regionserver > > > - [HBASE-8707] - Add LongComparator for filter > > > - [HBASE-12286] - [shell] Add server/cluster online load of > > configuration > > > changes > > > - [HBASE-12361] - Show data locality of region in table page > > > - [HBASE-12496] - A blockedRequestsCount metric > > > - [HBASE-12730] - Backport HBASE-5162 (Basic client pushback > mechanism) > > to > > > branch-1 > > > - [HBASE-12731] - Heap occupancy based client pushback > > > - [HBASE-12728] - buffered writes substantially less useful after > > removal > > > of HTablePool > > > - [HBASE-5699] - Run with > 1 WAL in HRegionServer > > > - Various fixes to REST server > > > - Internal refactoring for abstracting away zookeeper usage > > > - Better support for Cell interface internally in read and write paths > > for > > > better performance and flexibility > > > - Combining internal mvcc and seqId infrastructure > > > - Ring buffer based WAL improvements > > > - Switch to using new style of client APIs internally (in a lot of > > places) > > > - Improvements in visibility labels > > > - Perf improvements > > > - Numerous improvements in other areas and bug fixes. > > > > > > 1.0.0 release has these changes in default behavior (see [4]) > > > - Zookeeper 3.4.x is required > > > - Hadoop-2.x is required > > > - JDK-1.7 is required > > > - Default ports have changed from 600XX range into out of ephemeral > > 160XX > > > range. > > > - Hfile version 3 has been enabled by default. > > > - Slab cache has been removed (use bucket cache instead) > > > - Default heap for region servers (and master) is changed from 1GB to > > > the jdk's default. On most platforms, it will default to 1/4 of > > physical > > > memory > > > - Some client facing APIs (HTableInterface, etc) has been deprecated > and > > > replaced. > > > - Custom Filter implementations should switch to using the new > > recommended > > > APIs. > > > - Some authorization permissions (especially regarding namespaces) > have > > > been > > > changed (HBASE-12511, HBASE-8015) > > > - Mapred package and classes under there has been un-deprecated. > > > - Two new modules (jar files): hbase-annotations and hbase-rest. > > > - HTrace dependency is moved to the Apache artifacts ( > > > https://htrace.incubator.apache.org/) > > > - Various updates to dependency versions > > > > > > Full list of the issues can be found at: > > > - 1.0.0 issues : > > > > > > > > > https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12310753&version=12325852 > > > - 0.99.2 issues : > > > > > > > > > https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12310753&version=12325675 > > > - 0.99.1 issues : > > > > > > > > > https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12310753&version=12328551 > > > - 0.99.0 issues : > > > > > > > > > https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12310753&version=12328822 > > > > > > Compatibility > > > ------------- > > > Source Compatibility: > > > Client side code in HBase-1.0.0 is (mostly) source compatible with > > earlier > > > versions. Some minor API changes might be needed from the client side. > > > > > > > > > Wire Compatibility: > > > HBase-1.0.0 release is wire compatible with 0.98.x releases. Clients > and > > > servers running in different versions as long as new features are not > > used > > > should be possible. > > > A rolling upgrade from 0.98.x clusters to 1.0.0 is supported as well. > > 1.0.0 > > > introduces a new file format (hfile v3) that is enabled by default that > > > 0.96.x code cannot read. Thus, rolling upgrade from 0.96 directly to > > 1.0.0 > > > is > > > not supported. > > > 1.0.0 is NOT wire compatible with earlier releases (0.94, etc). > > > > > > Binary Compatibility: > > > Binary compatibility at the Java API layer with earlier versions > (0.98.x, > > > 0.96.x and 0.94.x) is not supported. You may have to recompile your > > client > > > code and any server side code (coprocessors, filters etc) referring to > > > hbase jars. > > > > > > Upgrading > > > --------- > > > See [4] for upgrade instructions and extended discussion on the > changes. > > > > > > From 0.98.x : Upgrade from 0.98.x in regular upgrade or rolling upgrade > > > fashion > > > is supported. > > > > > > From 0.96.x : Upgrade from 0.96.x is supported with a shutdown and > > restart > > > of > > > the cluster. > > > > > > From 0.94.x : Upgrade from 0.94.x is supported similar to upgrade from > > > 0.94 -> 0.96. The upgrade script should be run to rewrite cluster level > > > metadata. > > > See [5] for details. > > > > > > > > > Supported Hadoop versions > > > ------------------------- > > > 1.0.0 release drops support for Hadoop-1.x releases. Only Hadoop-2.x > > > releases are supported. Hadoop-2.4.x, Hadoop-2.5.x and Hadoop-2.6.x > > > releases > > > are the most tested hadoop releases and we recommend running with those > > > versions > > > (or later versions). Earlier Hadoop-2 based releases (hadoop-2.2.x and > > > 2.3.x) > > > are not tested to the full extend. More information can be found here: > > > https://hbase.apache.org/book/configuration.html#hadoop > > > > > > > > > Supported Java versions > > > ------------------------- > > > 1.0.0 release drops support for JDK6. Only JDK7 is supported. JDK8 > > support > > > is experimental. More information can be found here: > > > https://hbase.apache.org/book/configuration.html#java > > > > > > > > > Voting > > > ------ > > > Please try to test and vote on this release by Feb 19 2015 11:59PM PDT. > > > We will have a 5 day voting period because this is the sixth RC. Please > > try > > > to > > > spend some time for testing this important release. > > > > > > [] +1 Release the artifacts as 1.0.0 > > > [] -1 DO NOT release the artifacts as 1.0.0, because... > > > > > > Gratitute > > > --------- > > > Last but not least, 1.0.0 release has been in a long time making with > > > contributions from a very large group of awesome people and hard work > > from > > > committers and contributors. We would like to extend our thanks to > > > everybody > > > who worked on this release or contributed to HBase over the years. > > > > > > > > > References > > > ---------- > > > [1] https://hbase.apache.org/book/upgrading.html#hbase.versioning > > > [2] http://www.slideshare.net/xefyr/apache-hbase-10-release > > > [3] http://s.apache.org/hbase-1.0-api > > > [4] https://hbase.apache.org/book/upgrade1.0.html#upgrade1.0.changes > > > [5] https://hbase.apache.org/book/upgrade1.0.html#upgrade1.0.from.0.94 > > > > > > Enis > > > > > > > > > > > -- > > Best regards, > > > > - Andy > > > > Problems worthy of attack prove their worth by hitting back. - Piet Hein > > (via Tom White) > > >
