Thanks for tests and votes so far. Here is my official +1 (see above for my tests).
I was inspecting a similar issue today in the cluster with IntegrationTestLoadAndVerify with destructive CM, and the log splitting did not finish in 3 attempts. I think the issue is a not that critical though, I think it is due to a race between timeout monitor and some slowness in task heartbeat due to a loaded master (will open a jira if I nail down the exact reason). We need at least one more vote. I'll extend the voting for this RC (RC5) until 20th midnight PST. Please let me know if you intend to spend some more time on testing. We can extend the deadline. Cheers, Enis On Thu, Feb 19, 2015 at 6:12 PM, Andrew Purtell <apurt...@apache.org> wrote: > Maybe its just me > > > On Thu, Feb 19, 2015 at 11:10 AM, Enis Söztutar <enis....@gmail.com> > wrote: > > > I did not run the unit tests locally. I was relying on the following > build > > which is 100% pass: > > > > https://builds.apache.org/view/All/job/HBase-1.0/746/ > > > > It has the exact RC5 bits (6c98bff7b719efdb16f71606f3b7d8229445eb81). Let > > me kick a local UT run just in case. > > > > Enis > > > > On Thu, Feb 19, 2015 at 9:05 AM, Andrew Purtell <apurt...@apache.org> > > wrote: > > > > > That list is missing unit testing. Did you happen to do that when > > building > > > but not mention it Enis? If so did they all pass for you? > > > > > > On Wed, Feb 18, 2015 at 7:48 PM, Enis Söztutar <enis....@gmail.com> > > wrote: > > > > > > > Here is my RC5 testing so far: > > > > > > > > - checked checksums, sigs > > > > - checked the bin and src artifacts > > > > - checked layouts > > > > - checked java files in src tarball, and jar files in bin tarball > > > > - checked the book and the site (they are new style) > > > > - checked javadocs for both devapi and userapi > > > > - checked reported version, build time, revision > > > > - run some smoke tests using shell > > > > - started local mode > > > > - run LTT local mode > > > > - checked the webUIs of master and region servers > > > > - checked JMX dump and debug dump > > > > - Build src with hadoop versions 2.2.0 2.3.0 2.4.0 2.4.1 2.5.0 > 2.5.1 > > > > 2.5.2 2.6.0 > > > > - Deployed at a 6 node cluster with Hadoop-2.6.0 > > > > - Run LTT over tables with NONE, DIFF, FAST_DIFF, and PREFIX encoding > > > > - Run LTT over tables with NONE, GZ, LZO, LZ4 and SNAPPY compression > > > > - Tested with Bucket cache with 500M off heap > > > > > > > > I am still running larger scale tests, and some CM tests on the > > cluster. > > > > Will report back tomorrow and cast my vote. > > > > > > > > Enis > > > > > > > > > > > > On Wed, Feb 18, 2015 at 4:47 PM, 张铎 <palomino...@gmail.com> wrote: > > > > > > > > > TestCacheOnWrite itself has some problems. It > > > > > uses TestHFileWriterV2.randomOrderedKey to generate a random byte > > > array, > > > > > then use first 32 bytes as row and other parts as family and > > qualifier, > > > > but > > > > > TestHFileWriterV2.randomOrderedKey may return a byte array only > > > contains > > > > 32 > > > > > bytes, so there will be family and qualifier with zero length. > > > > > > > > > > I do not know if this is the reason why this test is flaky since it > > > use a > > > > > Random with pre-defined seed so the random sequence should be > > stable. I > > > > can > > > > > modify the KeyValue generation part to see if it helps. > > > > > > > > > > BTW, the name 'randomOrderedKey' is ambiguous, may change to ' > > > > > randomOrderedRow'? > > > > > > > > > > 2015-02-19 7:11 GMT+08:00 Andrew Purtell <apurt...@apache.org>: > > > > > > > > > > > I'm not able to get a clean unit test run when building from > source > > > > using > > > > > > 7u67. TestCacheOnWrite and TestSplitLogManager fail for me, maybe > > > more > > > > > but > > > > > > the build doesn't get past hbase-server. Maybe these are known > > > issues? > > > > If > > > > > > not I'll dig in when I get some time. > > > > > > > > > > > > > > > > > > On Sat, Feb 14, 2015 at 9:55 PM, Enis Söztutar <e...@apache.org> > > > > wrote: > > > > > > > > > > > > > It gives me great pleasure to announce that the sixth release > > > > candidate > > > > > > for > > > > > > > the release > > > > > > > 1.0.0 (HBase-1.0.0RC5), is available for download at > > > > > > > https://dist.apache.org/repos/dist/dev/hbase/hbase-1.0.0RC5/ > > > > > > > > > > > > > > Maven artifacts are also available in the temporary repository > > > > > > > > > > > > https://repository.apache.org/content/repositories/orgapachehbase-1065 > > > > > > > > > > > > > > Signed with my code signing key E964B5FF. Can be found here: > > > > > > > https://people.apache.org/keys/committer/enis.asc > > > > > > > > > > > > > > Signed tag in the repository can be found here: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > https://git-wip-us.apache.org/repos/asf?p=hbase.git;a=tag;h=c4660912e9b46c917a9aba2106be4bf74182a764 > > > > > > > > > > > > > > HBase 1.0.0 is the next stable release, and the start of > > "semantic > > > > > > > versioned" > > > > > > > releases (See [1]). > > > > > > > > > > > > > > The theme of 1.0.0 release is to become a stable base for > future > > > 1.x > > > > > > series > > > > > > > of releases. We aim to achieve at least the same level of > > stability > > > > of > > > > > > 0.98 > > > > > > > releases. > > > > > > > > > > > > > > 1.0.0 release contains 202 fixes on top of 0.99.2 release. > > Together > > > > > with > > > > > > > the > > > > > > > previous 0.99.x releases, major changes in 1.0.0 are listed > (but > > > not > > > > > > > limited to) > > > > > > > below. Note that all previous 0.99.x releases are developer > > preview > > > > > > > releases, and will > > > > > > > NOT be supported in any form. > > > > > > > > > > > > > > API Cleanup and changes > > > > > > > 1.0.0 introduces new APIs, and deprecates some of > commonly-used > > > > > > > client side APIs (HTableInterface, HTable and HBaseAdmin). > > > > > > > We advise to update your application to use the new style of > > > APIs, > > > > > > since > > > > > > > deprecated APIs might be removed in future releases (2.x). > See > > > [2] > > > > > and > > > > > > > [3] > > > > > > > for an overview of changes. All Client side API's are marked > > with > > > > > > > InterfaceAudience.Public class, indicating that the > > class/method > > > is > > > > > an > > > > > > > official "client API" for HBase. All 1.x releases are planned > > to > > > be > > > > > API > > > > > > > compatible for these classes. See [1] for an overview. > > > > > > > > > > > > > > Master runs a Region Server as well > > > > > > > Starting with 1.0.0, the HBase master server and backup > master > > > > > servers > > > > > > > will > > > > > > > also act as a region server. RPC port and info port for web > UI > > is > > > > > > shared > > > > > > > for > > > > > > > the master and region server roles. Active master can host > > > regions > > > > of > > > > > > > defined tables if configured (disabled by default). Backup > > > masters > > > > > will > > > > > > > not > > > > > > > host regions. > > > > > > > > > > > > > > Read availability using timeline consistent region replicas > > > > > > > This release contains Phase 1 items for experimental "Read > > > > > availability > > > > > > > using > > > > > > > timeline consistent region replicas" feature. A region can be > > > > hosted > > > > > in > > > > > > > multiple region servers in read-only mode. One of the > replicas > > > for > > > > > the > > > > > > > region > > > > > > > will be primary, accepting writes, and other replicas will be > > > > sharing > > > > > > the > > > > > > > same > > > > > > > data files. Read requests can be done against any replica for > > the > > > > > > region > > > > > > > with > > > > > > > backup RPCs for high availability with timeline consistency > > > > > guarantees. > > > > > > > More > > > > > > > information can be found at HBASE-10070. > > > > > > > > > > > > > > Online config change and other forward ports from 0.89-fb > branch > > > > > > > HBASE-12147 forward ported online config change which enables > > > some > > > > of > > > > > > the > > > > > > > configuration from the server to be reloaded without > restarting > > > the > > > > > > > region > > > > > > > servers. > > > > > > > > > > > > > > Other notable improvements in 1.0.0 (including previous 0.99.x) > > are > > > > > > > - A new web skin in time for 1.0 (http://hbase.apache.org) > > > > > > > - Automatic tuning of global memstore and block cache sizes > > > > > > > - Various security, tags and visibility labels improvements > > > > > > > - Bucket cache improvements (usability and compressed data > > blocks) > > > > > > > - A new pluggable replication endpoint to plug in to HBase's > > > > > > inter-cluster > > > > > > > replication to replicate to a custom data store > > > > > > > - A Dockerfile to easily build and run HBase from source > > > > > > > - Truncate table command > > > > > > > - Region assignment to use hbase:meta table instead of > zookeeper > > > for > > > > > > > faster > > > > > > > region assignment (disabled by default) > > > > > > > - Extensive documentation improvements > > > > > > > - [HBASE-12511] - namespace permissions - add support from > table > > > > > > creation > > > > > > > privilege in a namespace 'C' > > > > > > > - [HBASE-12568] - Adopt Semantic Versioning and document it in > > the > > > > > book > > > > > > > - [HBASE-12640] - Add Thrift-over-HTTPS and doAs support for > > > Thrift > > > > > > Server > > > > > > > - [HBASE-12651] - Backport HBASE-12559 'Provide LoadBalancer > > with > > > > > online > > > > > > > configuration capability' to branch-1 > > > > > > > - [HBASE-10560] - Per cell TTLs > > > > > > > - [HBASE-11997] - CopyTable with bulkload > > > > > > > - [HBASE-11990] - Make setting the start and stop row for a > > > specific > > > > > > > prefix easier > > > > > > > - [HBASE-12220] - Add hedgedReads and hedgedReadWins metrics > > > > > > > - [HBASE-12090] - Bytes: more Unsafe, more Faster > > > > > > > - [HBASE-12032] - Script to stop regionservers via RPC > > > > > > > - [HBASE-11907] - Use the joni byte[] regex engine in place of > > > > > j.u.regex > > > > > > > in RegexStringComparator > > > > > > > - [HBASE-11796] - Add client support for atomic checkAndMutate > > > > > > > - [HBASE-11804] - Raise default heap size if unspecified > > > > > > > - [HBASE-11890] - HBase REST Client is hard coded to http > > protocol > > > > > > > - [HBASE-12126] - Region server coprocessor endpoint > > > > > > > - [HBASE-12183] - FuzzyRowFilter doesn't support reverse scans > > > > > > > - [HBASE-12075] - Preemptive Fast Fail > > > > > > > - [HBASE-12354] - Update dependencies in time for 1.0 release > > > > > > > - [HBASE-12363] - Improve how KEEP_DELETED_CELLS works with > > > > > MIN_VERSIONS > > > > > > > - [HBASE-12434] - Add a command to compact all the regions in > a > > > > > > > regionserver > > > > > > > - [HBASE-8707] - Add LongComparator for filter > > > > > > > - [HBASE-12286] - [shell] Add server/cluster online load of > > > > > > configuration > > > > > > > changes > > > > > > > - [HBASE-12361] - Show data locality of region in table page > > > > > > > - [HBASE-12496] - A blockedRequestsCount metric > > > > > > > - [HBASE-12730] - Backport HBASE-5162 (Basic client pushback > > > > > mechanism) > > > > > > to > > > > > > > branch-1 > > > > > > > - [HBASE-12731] - Heap occupancy based client pushback > > > > > > > - [HBASE-12728] - buffered writes substantially less useful > > after > > > > > > removal > > > > > > > of HTablePool > > > > > > > - [HBASE-5699] - Run with > 1 WAL in HRegionServer > > > > > > > - Various fixes to REST server > > > > > > > - Internal refactoring for abstracting away zookeeper usage > > > > > > > - Better support for Cell interface internally in read and > write > > > > paths > > > > > > for > > > > > > > better performance and flexibility > > > > > > > - Combining internal mvcc and seqId infrastructure > > > > > > > - Ring buffer based WAL improvements > > > > > > > - Switch to using new style of client APIs internally (in a > lot > > of > > > > > > places) > > > > > > > - Improvements in visibility labels > > > > > > > - Perf improvements > > > > > > > - Numerous improvements in other areas and bug fixes. > > > > > > > > > > > > > > 1.0.0 release has these changes in default behavior (see [4]) > > > > > > > - Zookeeper 3.4.x is required > > > > > > > - Hadoop-2.x is required > > > > > > > - JDK-1.7 is required > > > > > > > - Default ports have changed from 600XX range into out of > > > ephemeral > > > > > > 160XX > > > > > > > range. > > > > > > > - Hfile version 3 has been enabled by default. > > > > > > > - Slab cache has been removed (use bucket cache instead) > > > > > > > - Default heap for region servers (and master) is changed from > > 1GB > > > > to > > > > > > > the jdk's default. On most platforms, it will default to 1/4 > > of > > > > > > physical > > > > > > > memory > > > > > > > - Some client facing APIs (HTableInterface, etc) has been > > > deprecated > > > > > and > > > > > > > replaced. > > > > > > > - Custom Filter implementations should switch to using the new > > > > > > recommended > > > > > > > APIs. > > > > > > > - Some authorization permissions (especially regarding > > namespaces) > > > > > have > > > > > > > been > > > > > > > changed (HBASE-12511, HBASE-8015) > > > > > > > - Mapred package and classes under there has been > un-deprecated. > > > > > > > - Two new modules (jar files): hbase-annotations and > > hbase-rest. > > > > > > > - HTrace dependency is moved to the Apache artifacts ( > > > > > > > https://htrace.incubator.apache.org/) > > > > > > > - Various updates to dependency versions > > > > > > > > > > > > > > Full list of the issues can be found at: > > > > > > > - 1.0.0 issues : > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12310753&version=12325852 > > > > > > > - 0.99.2 issues : > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12310753&version=12325675 > > > > > > > - 0.99.1 issues : > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12310753&version=12328551 > > > > > > > - 0.99.0 issues : > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12310753&version=12328822 > > > > > > > > > > > > > > Compatibility > > > > > > > ------------- > > > > > > > Source Compatibility: > > > > > > > Client side code in HBase-1.0.0 is (mostly) source compatible > > with > > > > > > earlier > > > > > > > versions. Some minor API changes might be needed from the > client > > > > side. > > > > > > > > > > > > > > > > > > > > > Wire Compatibility: > > > > > > > HBase-1.0.0 release is wire compatible with 0.98.x releases. > > > Clients > > > > > and > > > > > > > servers running in different versions as long as new features > are > > > not > > > > > > used > > > > > > > should be possible. > > > > > > > A rolling upgrade from 0.98.x clusters to 1.0.0 is supported as > > > well. > > > > > > 1.0.0 > > > > > > > introduces a new file format (hfile v3) that is enabled by > > default > > > > that > > > > > > > 0.96.x code cannot read. Thus, rolling upgrade from 0.96 > directly > > > to > > > > > > 1.0.0 > > > > > > > is > > > > > > > not supported. > > > > > > > 1.0.0 is NOT wire compatible with earlier releases (0.94, etc). > > > > > > > > > > > > > > Binary Compatibility: > > > > > > > Binary compatibility at the Java API layer with earlier > versions > > > > > (0.98.x, > > > > > > > 0.96.x and 0.94.x) is not supported. You may have to recompile > > your > > > > > > client > > > > > > > code and any server side code (coprocessors, filters etc) > > referring > > > > to > > > > > > > hbase jars. > > > > > > > > > > > > > > Upgrading > > > > > > > --------- > > > > > > > See [4] for upgrade instructions and extended discussion on the > > > > > changes. > > > > > > > > > > > > > > From 0.98.x : Upgrade from 0.98.x in regular upgrade or rolling > > > > upgrade > > > > > > > fashion > > > > > > > is supported. > > > > > > > > > > > > > > From 0.96.x : Upgrade from 0.96.x is supported with a shutdown > > and > > > > > > restart > > > > > > > of > > > > > > > the cluster. > > > > > > > > > > > > > > From 0.94.x : Upgrade from 0.94.x is supported similar to > upgrade > > > > from > > > > > > > 0.94 -> 0.96. The upgrade script should be run to rewrite > cluster > > > > level > > > > > > > metadata. > > > > > > > See [5] for details. > > > > > > > > > > > > > > > > > > > > > Supported Hadoop versions > > > > > > > ------------------------- > > > > > > > 1.0.0 release drops support for Hadoop-1.x releases. Only > > > Hadoop-2.x > > > > > > > releases are supported. Hadoop-2.4.x, Hadoop-2.5.x and > > Hadoop-2.6.x > > > > > > > releases > > > > > > > are the most tested hadoop releases and we recommend running > with > > > > those > > > > > > > versions > > > > > > > (or later versions). Earlier Hadoop-2 based releases > > (hadoop-2.2.x > > > > and > > > > > > > 2.3.x) > > > > > > > are not tested to the full extend. More information can be > found > > > > here: > > > > > > > https://hbase.apache.org/book/configuration.html#hadoop > > > > > > > > > > > > > > > > > > > > > Supported Java versions > > > > > > > ------------------------- > > > > > > > 1.0.0 release drops support for JDK6. Only JDK7 is supported. > > JDK8 > > > > > > support > > > > > > > is experimental. More information can be found here: > > > > > > > https://hbase.apache.org/book/configuration.html#java > > > > > > > > > > > > > > > > > > > > > Voting > > > > > > > ------ > > > > > > > Please try to test and vote on this release by Feb 19 2015 > > 11:59PM > > > > PDT. > > > > > > > We will have a 5 day voting period because this is the sixth > RC. > > > > Please > > > > > > try > > > > > > > to > > > > > > > spend some time for testing this important release. > > > > > > > > > > > > > > [] +1 Release the artifacts as 1.0.0 > > > > > > > [] -1 DO NOT release the artifacts as 1.0.0, because... > > > > > > > > > > > > > > Gratitute > > > > > > > --------- > > > > > > > Last but not least, 1.0.0 release has been in a long time > making > > > with > > > > > > > contributions from a very large group of awesome people and > hard > > > work > > > > > > from > > > > > > > committers and contributors. We would like to extend our thanks > > to > > > > > > > everybody > > > > > > > who worked on this release or contributed to HBase over the > > years. > > > > > > > > > > > > > > > > > > > > > References > > > > > > > ---------- > > > > > > > [1] > > https://hbase.apache.org/book/upgrading.html#hbase.versioning > > > > > > > [2] http://www.slideshare.net/xefyr/apache-hbase-10-release > > > > > > > [3] http://s.apache.org/hbase-1.0-api > > > > > > > [4] > > > https://hbase.apache.org/book/upgrade1.0.html#upgrade1.0.changes > > > > > > > [5] > > > > https://hbase.apache.org/book/upgrade1.0.html#upgrade1.0.from.0.94 > > > > > > > > > > > > > > Enis > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > Best regards, > > > > > > > > > > > > - Andy > > > > > > > > > > > > Problems worthy of attack prove their worth by hitting back. - > Piet > > > > Hein > > > > > > (via Tom White) > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > Best regards, > > > > > > - Andy > > > > > > Problems worthy of attack prove their worth by hitting back. - Piet > Hein > > > (via Tom White) > > > > > > > > > -- > Best regards, > > - Andy > > Problems worthy of attack prove their worth by hitting back. - Piet Hein > (via Tom White) >
