Re: [VOTE] First release candidate for HBase 1.1.13 (RC0) is available
+1 - verified tarballs vs public key on people.apache.org. - extracted bin tgz: - inspect structure. look good. - with jdk1.8.0_65: - run LoadTestTool against standalone bin tgz with FAST_DIFF block encoder and ROWCOL blooms. No issues, logs look good. - poked around webUI. looks good. - load the site, browsed book. - extracted src tgz: - inspect structure. look good. - run LoadTestTool against standalone built from src tgz with FAST_DIFF block encoder and ROWCOL blooms. No issues, logs look good. - poked around webUI. looks good. - ran the hbase-downstreamer project vs. the staged maven repository. tests pass. On Thu, Dec 7, 2017 at 1:44 PM, Ted Yu wrote: > +1 > > Checked sums and signatures: ok > Ran unit tests: passed > Started standalone cluster and did some basic operations > > On Thu, Dec 7, 2017 at 1:14 PM, Andrew Purtell > wrote: > > > +1 > > > > Checked sums and signatures: ok > > Checked compat report: ok > > RAT check passed: ok (7u80) > > Built from source: ok (7u80) > > Unit tests pass: ok (8u131) > > 1M row LTT: ok (8u131) > > > > > > On Thu, Dec 7, 2017 at 8:40 AM, Nick Dimiduk > wrote: > > > > > No one has voted a binding -1 with actionable changes, so as far as I'm > > > concerned this RC remains valid. If people need more time, we can > extend > > > this vote. > > > > > > Thanks, > > > Nick > > > > > > On Thu, Dec 7, 2017 at 8:07 AM, Ted Yu wrote: > > > > > > > Nick: > > > > Originally you set tomorrow as deadline. > > > > > > > > Is there a new RC coming out (w.r.t. Mike's comment) ? > > > > > > > > Cheers > > > > > > > > On Mon, Dec 4, 2017 at 8:37 PM, Nick Dimiduk > > > wrote: > > > > > > > > > Mike: > > > > > > > > > > > Do you plan to make a human-readable set of release notes in > > addition > > > > to > > > > > the list of JIRA issues resolved? > > > > > > > > > > Not as such. For all branch-1.1 releases, I've written up a little > > > > > human-friendly summary in the ANNOUNCE email. Basically, expanding > on > > > the > > > > > list of JIRA tickets I highlight in the RC notes to include their > > full > > > > > ticket summaries. I haven't followed the details of the branch-1.4 > > > > release > > > > > line, so I'm not sure what additional information you might be > hoping > > > > for. > > > > > > > > > > > tar missing hbase-native-client (present in tag) > > > > > > > > > > That's been the case since rel/1.1.0. We as a community have never > > > > shipped > > > > > a binary native client in this release line and we've never claimed > > > that > > > > > the native sources packaged herein are ready for production > > > consumption. > > > > > They probably should have been dropped from the branch before > initial > > > > > release, but that was not done. I have no objection to dropping > them > > > > from a > > > > > branch-1.1 release; from the git log, I see no commit activity to > > that > > > > > module since Jan 2014. I don't see any of this as a blocker for > this > > > RC. > > > > > > > > > > > WARNING! HBase file layout needs to be upgraded ... > > > > > > > > > > When I test these RC's on a Mac, I explicitly set hbase.tmp.dir to > a > > > > > location specific to the candidate I've unpacked. This has the > > benefit > > > > > avoiding cross-version conflicts and other weirdness of Mac tmp > > > directory > > > > > management. For instance, > > > > > > > > > > > > > > > > > > > > hbase.tmp.dir/tmp/hbase-1.1. > > > > > 13/tmp > > > > > > > > > > > > > > > Peter: > > > > > > > > > > > In the logs I saw this line. Source code repository URL looks > > > > incorrect. > > > > > > 2017-12-04 10:13:27,028 INFO [main] util.VersionInfo: Source > code > > > > > repository *git://diocles.local/Volumes/hbase-1.1.13/hbase* > > > > > revision=c64bf8a9f35352cd504f2b8f4b02f9148cf45ab6 > > > > > > > > > > Looking through the log, HBASE-16538 / > > > > > 851c89af6ef9a78e2e3bc9ad3153367e85731c81 looks suspicious. It > looks > > > like > > > > > that change first shipped in 1.1.7. Indeed, I see the equivelant > line > > > in > > > > > the binary release of 1.1.12. > > > > > > > > > > On Mon, Dec 4, 2017 at 4:12 PM, Stack wrote: > > > > > > > > > > > On Mon, Dec 4, 2017 at 11:52 AM, Andrew Purtell < > > apurt...@apache.org > > > > > > > > > > wrote: > > > > > > > > > > > > > > I think this can be fixed by using `git archive` to generate > > the > > > > src > > > > > > tar > > > > > > > instead of the src assembly. > > > > > > > > > > > > > > We could update the make_rc.sh script to create the source > > tarball > > > in > > > > > > this > > > > > > > way. Would you be willing to make a patch for that? > > > > > > > > > > > > > > > > > > > > (I think you fellows already figured this going by JIRA > > movement) > > > > > > > > > > > > HBASE-19152 "Update refguide 'how to build an RC' and the > > make_rc.sh > > > > > > script" changed make_rc.sh to use git archive. Wasn't backported > > > > though. > > > > > > S > > > > > > > > > > > > > > > > > > > > > > > > > F
[jira] [Created] (HBASE-19470) Compaction state in Table web UI is not right when table is disabled
Guanghao Zhang created HBASE-19470: -- Summary: Compaction state in Table web UI is not right when table is disabled Key: HBASE-19470 URL: https://issues.apache.org/jira/browse/HBASE-19470 Project: HBase Issue Type: Bug Affects Versions: 1.4.0 Reporter: Guanghao Zhang Priority: Trivial Table Attributes Attribute Name Value Description Enabled false Is the table enabled Compaction sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)java.lang.reflect.Constructor.newInstance(Constructor.java:423)org.apache.hadoop.hbase.ipc.RemoteWithExtrasException.instantiateException(RemoteWithExtrasException.java:95)org.apache.hadoop.hbase.ipc.RemoteWithExtrasException.unwrapRemoteException(RemoteWithExtrasException.java:85)org.apache.hadoop.hbase.protobuf.ProtobufUtil.makeIOExceptionOfException(ProtobufUtil.java:371)org.apache.hadoop.hbase.protobuf.ProtobufUtil.getRemoteException(ProtobufUtil.java:330)org.apache.hadoop.hbase.client.HBaseAdmin.getCompactionState(HBaseAdmin.java:3455)org.apache.hadoop.hbase.generated.master.table_jsp._jspService(table_jsp.java:283)org.apache.jasper.runtime.HttpJspBase.service(HttpJspBase.java:109)javax.servlet.http.HttpServlet.service(HttpServlet.java:820)org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511)org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1221)org.apache.hadoop.hbase.http.lib.StaticUserWebFilter$StaticUserFilter.doFilter(StaticUserWebFilter.java:113)org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)org.apache.hadoop.hbase.http.ClickjackingPreventionFilter.doFilter(ClickjackingPreventionFilter.java:48)org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)org.apache.hadoop.hbase.http.HttpServer$QuotingInputFilter.doFilter(HttpServer.java:1432)org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)org.apache.hadoop.hbase.http.NoCacheFilter.doFilter(NoCacheFilter.java:49)org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)org.apache.hadoop.hbase.http.NoCacheFilter.doFilter(NoCacheFilter.java:49)org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:399)org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182)org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:766)org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:450)org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:230)org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)org.mortbay.jetty.Server.handle(Server.java:326)org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:928)org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:549)org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:212)org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:410)org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582) Unknown -- This message was sent by Atlassian JIRA (v6.4.14#64029)
Re: [VOTE] The second HBase 1.4.0 release candidate (RC1) is available
+1 (non-binding) hbase-1.4.0-bin.tar.gz (jdk8u144) - Verified md5sum: ok - Start HBase in standalone mode: ok - Verified with shell, create/disable/enable/drop/get/put/scan/delete: ok - Checked master/regionserver/region Web UI: ok hbase-1.4.0-src.tar.gz (jdk8u144) - Verified md5sum: ok - Build tarball: ok - Start HBase in standalone mode: ok - Verified with shell, create/disable/enable/drop/get/put/scan/delete: ok - Checked master/regionserver/region Web UI: ok One trivial bug for Table ui: the compaction state will show a exception message when the table is disabled. Will fill a issue for this. Thanks. 2017-12-09 9:50 GMT+08:00 Andrew Purtell : > The second HBase 1.4.0 release candidate (RC1) is available for download > at https://dist.apache.org/repos/dist/dev/hbase/hbase-1.4.0RC1/ and Maven > artifacts are available in the temporary repository > https://repository.apache.org/content/repositories/orgapachehbase-1186/ . > > The git tag corresponding to the candidate is '1.4.0RC1' (10b9b9fae6). > > A detailed source and binary compatibility report for this release is > available for your review at > https://dist.apache.org/repos/dist/dev/hbase/hbase-1.4.0RC1/ > hbase-1.3.1-1.4.0RC1_compatibility_report.html > . All reported compatibility issues should comply with policy but please > review them carefully. > > A list of the 660 issues resolved in this release can be found at > https://s.apache.org/OErT . > > The changes since RC0 are: > > * [HBASE-19180] - Remove unused imports from AlwaysPasses > * [HBASE-19373] - Fix Checkstyle error in hbase-annotations > * [HBASE-19435] - Reopen Files for ClosedChannelException in > BucketCache > * [HBASE-19465] - Required httpcore and httpclient jars not included in > binary distribution > * [HBASE-19467] - rsgroups shell commands don't properly display > elapsed time > > Please try out the candidate and vote +1/0/-1. > > This vote will be open for at least 72 hours. Unless objection I will try > to close it Monday December 18, 2017 if we have sufficient votes. > > Prior to making this announcement I made the following preflight checks to > 1.4.0RC0 (3d571827cb): > > RAT check passes (7u80) > Unit test suite passes (8u131) > LTT load 1M rows with 100% verification and 20% updates (8u131) > PE randomWrite, randomRead, scanRange100 (8u131) > 100M rows ITBLL (8u131) > > and the following preflight checks to 1.4.0RC1 (10b9b9fae6): > > RAT check passes (7u80) > Unit test suite passes (8u131) > > Between now and when I want to close the vote I'll write up human readable > release notes for the release announcement as promised. > > I also have agreed to run a scale ITBLL test, a performance comparison with > 1.2 using YCSB, a performance comparison with 1.2 using PE, and an > analysis of code and allocation hot spot changes from 1.2, all of which I > will publish when available and factor in to my vote. > > -- > Best regards, > Andrew >
[VOTE] The second HBase 1.4.0 release candidate (RC1) is available
The second HBase 1.4.0 release candidate (RC1) is available for download at https://dist.apache.org/repos/dist/dev/hbase/hbase-1.4.0RC1/ and Maven artifacts are available in the temporary repository https://repository.apache.org/content/repositories/orgapachehbase-1186/ . The git tag corresponding to the candidate is '1.4.0RC1' (10b9b9fae6). A detailed source and binary compatibility report for this release is available for your review at https://dist.apache.org/repos/dist/dev/hbase/hbase-1.4.0RC1/hbase-1.3.1-1.4.0RC1_compatibility_report.html . All reported compatibility issues should comply with policy but please review them carefully. A list of the 660 issues resolved in this release can be found at https://s.apache.org/OErT . The changes since RC0 are: * [HBASE-19180] - Remove unused imports from AlwaysPasses * [HBASE-19373] - Fix Checkstyle error in hbase-annotations * [HBASE-19435] - Reopen Files for ClosedChannelException in BucketCache * [HBASE-19465] - Required httpcore and httpclient jars not included in binary distribution * [HBASE-19467] - rsgroups shell commands don't properly display elapsed time Please try out the candidate and vote +1/0/-1. This vote will be open for at least 72 hours. Unless objection I will try to close it Monday December 18, 2017 if we have sufficient votes. Prior to making this announcement I made the following preflight checks to 1.4.0RC0 (3d571827cb): RAT check passes (7u80) Unit test suite passes (8u131) LTT load 1M rows with 100% verification and 20% updates (8u131) PE randomWrite, randomRead, scanRange100 (8u131) 100M rows ITBLL (8u131) and the following preflight checks to 1.4.0RC1 (10b9b9fae6): RAT check passes (7u80) Unit test suite passes (8u131) Between now and when I want to close the vote I'll write up human readable release notes for the release announcement as promised. I also have agreed to run a scale ITBLL test, a performance comparison with 1.2 using YCSB, a performance comparison with 1.2 using PE, and an analysis of code and allocation hot spot changes from 1.2, all of which I will publish when available and factor in to my vote. -- Best regards, Andrew
[jira] [Created] (HBASE-19469) Review Of BackupSystemTable
BELUGA BEHR created HBASE-19469: --- Summary: Review Of BackupSystemTable Key: HBASE-19469 URL: https://issues.apache.org/jira/browse/HBASE-19469 Project: HBase Issue Type: Improvement Components: hbase Affects Versions: 3.0.0 Reporter: BELUGA BEHR Priority: Trivial * Remove superfluous logging guards * Fix spelling mistake in log message * Fixed some DEBUG log messages that were guarded by an _isTraceEnabled_ * Use more Java auto-close functionality * Some code simplifications * Use better data structures for merge/disjoin -- This message was sent by Atlassian JIRA (v6.4.14#64029)
Re: HBase Outage - Drop table operation stuck in "DELETE_TABLE_PRE_OPERATION"
>From the line number of ProcedureSyncWait.java, it seems you are using 1.2.x release. Can you check master log prior to 2017-12-08 18:59 ? Pastebin relevant master log snippet (after necessary redaction). Once we see the master log, we can see what might cause the DeleteTableProcedure to be stuck. bq. Why rebalancing or other rest of operations are stuck? If there is region in transition, the balancer wouldn't run. "hbck –repair" combines many fixes. Normally admin is supposed to analyze the particular inconsistencies before issuing proper fix. Cheers On Fri, Dec 8, 2017 at 1:50 PM, Murthy boddu wrote: > Hi, > > > > We recently ran into a production issue, here is the summary of events that > we went through, in timeline order: > > > >1. One of the region servers went down (it became inaccessible) >2. Region transition initiated, some regions of multiple tables were >stuck in transition status. Most of them are in status “OPEN_FAIILED” or >“OPENING” or “PENDING”, “CLOSE_FAILED” >3. Client requests to those tables are still being diverted to lost >server causing failures/time outs. (Which can we do about it ?) >4. After waiting for many hours, we ran hbck –repair per table which >resolved issues with some of them. >5. One table, whose data can get stale in hours, we planned to recreate >it to avoid any corruption. Disabling of table went through fine but >dropping the table stuck at state “DELETE_TABLE_PRE_OPERATION”, it is >waiting for regions in transition to finish. The regions it is > complaining >is in “OPENING” status. > > Here is the exception: > > > > 2017-12-08 18:59:17,975 WARN [ProcedureExecutor-10] > procedure.DeleteTableProcedure: Retriable error trying to delete > table=Queue-SCKAD state=DELETE_TABLE_PRE_OPERATION > > org.apache.hadoop.hbase.exceptions.TimeoutIOException: Timed out while > waiting on regions > Queue-SCKAD,B19,1502479054304.15a44cf47634d7d2264eaf00d61f6036. in > transition > > at > org.apache.hadoop.hbase.master.procedure.ProcedureSyncWait.waitFor( > ProcedureSyncWait.java:123) > > at > org.apache.hadoop.hbase.master.procedure.ProcedureSyncWait.waitFor( > ProcedureSyncWait.java:103) > > > >1. This operation has been running for more than 24 hours and doesn’t >time out (isn't there a 2 hour timeout for client operations at HBase > level >? ). Enabling the table back also queues up with no progress. >2. Because the table is in disable status, running hbck isn’t helping as >it says regions = 0. >3. We added new node to the cluster to replace the old one, we see that >HBase balancer doesn’t kick in at all. So, basically, region movement is >totally stuck. >4. No missing data on HDFS, 100% consistent. Hbck detail report on whole >cluster also returns OK. > > > > I can provide additional logs if you request, but can you suggest how we > can resolve this problem with the cluster? Does restarting hbase master > process would help? We can’t afford another outage on the cluster making > the situation tricky. > > > > My questions: > > > >1. Why drop operation need to wait for regions in transition to finish? >Is there a way we can abort the on-going region movement or even the > drop >operation? >2. Why rebalancing or other rest of operations are stuck? >3. Can you please suggest what action can be taken to resolve this? > > > > Thank you for your time and help. > > > > Regards >
[jira] [Resolved] (HBASE-19467) rsgroups shell commands don't properly display elapsed time
[ https://issues.apache.org/jira/browse/HBASE-19467?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell resolved HBASE-19467. Resolution: Fixed Pushed trivial changes to branch-1.4 and branch-1 after manual verification. > rsgroups shell commands don't properly display elapsed time > --- > > Key: HBASE-19467 > URL: https://issues.apache.org/jira/browse/HBASE-19467 > Project: HBase > Issue Type: Bug >Affects Versions: 1.4.0 >Reporter: Andrew Purtell >Assignee: Andrew Purtell >Priority: Trivial > Fix For: 1.4.0 > > Attachments: HBASE-19467-branch-1.patch > > > hbase(main):001:0> list_rsgroups > GROUPS > default > 1 row(s) in 1512748253.9920 seconds > hbase(main):002:0> add_rsgroup 'my_group' > hbase(main):003:0> list_rsgroups > GROUPS > default > my_group > 2 row(s) in 1512748276.9400 seconds -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HBASE-19468) FNFE during scans and flushes
Thiruvel Thirumoolan created HBASE-19468: Summary: FNFE during scans and flushes Key: HBASE-19468 URL: https://issues.apache.org/jira/browse/HBASE-19468 Project: HBase Issue Type: Sub-task Affects Versions: 1.3.1 Reporter: Thiruvel Thirumoolan Priority: Minor We see FNFE exceptions on our 1.3 clusters when scans and flushes happen at the same time. This causes regionserver to throw a UnknownScannerException and client retries. This happens during the following sequence: 1. Scanner open, client fetched some rows from regionserver and working on it 2. Flush happens and storeScanner is updated with flushed files (StoreScanner.updateReaders()) 3. Compaction discharger runs and cleans up the newly flushed file as we don't have new scanners on it yet. 4. Client issues scan.next and during StoreScanner.resetScannerStack(), we get a FNFE. RegionServer throws a UnknownScannerThe client retries in 1.3. With branch-1.4, the scan fails with a DoNotRetryIOException. [~ram_krish], My proposal is to increment the reader count during updateReaders() and decrement it during resetScannerStack(), so discharger doesn't clean it up. Scan lease expiries also have to be taken care of. Am I missing anything? Is there a better approach? -- This message was sent by Atlassian JIRA (v6.4.14#64029)
HBase Outage - Drop table operation stuck in "DELETE_TABLE_PRE_OPERATION"
Hi, We recently ran into a production issue, here is the summary of events that we went through, in timeline order: 1. One of the region servers went down (it became inaccessible) 2. Region transition initiated, some regions of multiple tables were stuck in transition status. Most of them are in status “OPEN_FAIILED” or “OPENING” or “PENDING”, “CLOSE_FAILED” 3. Client requests to those tables are still being diverted to lost server causing failures/time outs. (Which can we do about it ?) 4. After waiting for many hours, we ran hbck –repair per table which resolved issues with some of them. 5. One table, whose data can get stale in hours, we planned to recreate it to avoid any corruption. Disabling of table went through fine but dropping the table stuck at state “DELETE_TABLE_PRE_OPERATION”, it is waiting for regions in transition to finish. The regions it is complaining is in “OPENING” status. Here is the exception: 2017-12-08 18:59:17,975 WARN [ProcedureExecutor-10] procedure.DeleteTableProcedure: Retriable error trying to delete table=Queue-SCKAD state=DELETE_TABLE_PRE_OPERATION org.apache.hadoop.hbase.exceptions.TimeoutIOException: Timed out while waiting on regions Queue-SCKAD,B19,1502479054304.15a44cf47634d7d2264eaf00d61f6036. in transition at org.apache.hadoop.hbase.master.procedure.ProcedureSyncWait.waitFor(ProcedureSyncWait.java:123) at org.apache.hadoop.hbase.master.procedure.ProcedureSyncWait.waitFor(ProcedureSyncWait.java:103) 1. This operation has been running for more than 24 hours and doesn’t time out (isn't there a 2 hour timeout for client operations at HBase level ? ). Enabling the table back also queues up with no progress. 2. Because the table is in disable status, running hbck isn’t helping as it says regions = 0. 3. We added new node to the cluster to replace the old one, we see that HBase balancer doesn’t kick in at all. So, basically, region movement is totally stuck. 4. No missing data on HDFS, 100% consistent. Hbck detail report on whole cluster also returns OK. I can provide additional logs if you request, but can you suggest how we can resolve this problem with the cluster? Does restarting hbase master process would help? We can’t afford another outage on the cluster making the situation tricky. My questions: 1. Why drop operation need to wait for regions in transition to finish? Is there a way we can abort the on-going region movement or even the drop operation? 2. Why rebalancing or other rest of operations are stuck? 3. Can you please suggest what action can be taken to resolve this? Thank you for your time and help. Regards
[jira] [Resolved] (HBASE-19465) Required httpcore and httpclient jars not included in binary distribution
[ https://issues.apache.org/jira/browse/HBASE-19465?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell resolved HBASE-19465. Resolution: Fixed Pushed trivial change to branch-1 and branch-1.4 > Required httpcore and httpclient jars not included in binary distribution > - > > Key: HBASE-19465 > URL: https://issues.apache.org/jira/browse/HBASE-19465 > Project: HBase > Issue Type: Bug >Affects Versions: 1.4.0 >Reporter: Andrew Purtell >Assignee: Andrew Purtell >Priority: Blocker > Fix For: 1.4.0 > > Attachments: HBASE-19465-branch-1.patch > > > The httpcore and httpclient dependency jars are not included into the binary > distribution. This means MR/YARN jobs will fail if launched from the binary > tarball as distributed, including those running in "local mode", e.g. ITBLL: > {noformat} > 2017-12-08 22:40:36,134 INFO [LocalJobRunner Map Task Executor #0] > mapred.LocalJobRunner: Finishing task: attempt_local936234315_0001_m_03_0 > 2017-12-08 22:40:36,134 INFO [Thread-23] mapred.LocalJobRunner: map task > executor complete. > 2017-12-08 22:40:36,147 WARN [Thread-23] mapred.LocalJobRunner: > job_local936234315_0001 > java.lang.NoClassDefFoundError: org/apache/http/client/methods/HttpUriRequest > at > org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:546) > Caused by: java.lang.ClassNotFoundException: > org.apache.http.client.methods.HttpUriRequest > at java.net.URLClassLoader.findClass(URLClassLoader.java:381) > at java.lang.ClassLoader.loadClass(ClassLoader.java:424) > at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:335) > at java.lang.ClassLoader.loadClass(ClassLoader.java:357) > ... 1 more > Exception in thread "Thread-23" java.lang.NoClassDefFoundError: > org/apache/http/client/methods/HttpUriRequest > at > org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:562) > Caused by: java.lang.ClassNotFoundException: > org.apache.http.client.methods.HttpUriRequest > at java.net.URLClassLoader.findClass(URLClassLoader.java:381) > at java.lang.ClassLoader.loadClass(ClassLoader.java:424) > at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:335) > at java.lang.ClassLoader.loadClass(ClassLoader.java:357) > ... 1 more > 2017-12-08 22:40:37,078 INFO [main] mapreduce.Job: Job > job_local936234315_0001 failed with state FAILED due to: NA > {noformat} -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[CANCELLED] [VOTE] The first HBase 1.4.0 release candidate (RC0) is available
Based on other feedback I filed HBASE-19467 and will fix it too. On Fri, Dec 8, 2017 at 2:59 PM, Andrew Purtell wrote: > -1, see HBASE-19465 > > RC1 coming as soon as I fix this. > > On Wed, Dec 6, 2017 at 8:40 PM, Andrew Purtell > wrote: > >> The first HBase 1.4.0 release candidate (RC0) is available for download >> at https://dist.apache.org/repos/dist/dev/hbase/hbase-1.4.0RC0/ and >> Maven artifacts are available in the temporary repository >> https://repository.apache.org/content/repositories/orgapachehbase-1185/ >> . >> >> The git tag corresponding to the candidate is '1.4.0RC0' (3d571827cb). >> >> A detailed source and binary compatibility report for this release is >> available for your review at https://dist.apache.org/repos/ >> dist/dev/hbase/hbase-1.4.0RC0/hbase-1.3.1-1.4.0RC0_compatibi >> lity_report.html . All reported compatibility issues should comply with >> policy but please review them carefully. >> >> A list of the 652 issues resolved in this release can be found at >> https://s.apache.org/OErT . >> >> Please try out the candidate and vote +1/0/-1. >> >> This vote will be open for at least 72 hours. Unless objection I will try >> to close it Monday December 18, 2017 if we have sufficient votes. >> >> Prior to making this announcement I made the following preflight checks >> to 1.4.0RC0 (3d571827cb): >> >>- RAT check passes (7u80) >>- Unit test suite passes (8u131) >>- LTT load 1M rows with 100% verification and 20% updates (8u131) >>- PE randomWrite, randomRead, scanRange100 (8u131) >>- 100M rows ITBLL (8u131) >> >> Between now and when I want to close the vote I'll write up human >> readable release notes for the release announcement as promised. I also >> have agreed to run a scale ITBLL test, a performance comparison with 1.2 >> using YCSB, a performance comparison with 1.2 using PE, and an analysis of >> code and allocation hot spot changes from 1.2, all of which I will publish >> when available and factor in to my vote. >> >> >> -- >> Best regards, >> Andrew >> >> > > > -- > Best regards, > Andrew > > Words like orphans lost among the crosstalk, meaning torn from truth's > decrepit hands >- A23, Crosstalk > -- Best regards, Andrew Words like orphans lost among the crosstalk, meaning torn from truth's decrepit hands - A23, Crosstalk
[jira] [Created] (HBASE-19467) rsgroups shell commands don't properly display elapsed time
Andrew Purtell created HBASE-19467: -- Summary: rsgroups shell commands don't properly display elapsed time Key: HBASE-19467 URL: https://issues.apache.org/jira/browse/HBASE-19467 Project: HBase Issue Type: Bug Affects Versions: 1.4.0 Reporter: Andrew Purtell Assignee: Andrew Purtell Priority: Minor Fix For: 1.4.0 hbase(main):001:0> list_rsgroups GROUPS default 1 row(s) in 1512748253.9920 seconds hbase(main):002:0> add_rsgroup 'my_group' hbase(main):003:0> list_rsgroups GROUPS default my_group 2 row(s) in 1512748276.9400 seconds -- This message was sent by Atlassian JIRA (v6.4.14#64029)
Re: [VOTE] The first HBase 1.4.0 release candidate (RC0) is available
-1, see HBASE-19465 RC1 coming as soon as I fix this. On Wed, Dec 6, 2017 at 8:40 PM, Andrew Purtell wrote: > The first HBase 1.4.0 release candidate (RC0) is available for download > at https://dist.apache.org/repos/dist/dev/hbase/hbase-1.4.0RC0/ and Maven > artifacts are available in the temporary repository > https://repository.apache.org/content/repositories/orgapachehbase-1185/ . > > The git tag corresponding to the candidate is '1.4.0RC0' (3d571827cb). > > A detailed source and binary compatibility report for this release is > available for your review at https://dist.apache.org/repos/ > dist/dev/hbase/hbase-1.4.0RC0/hbase-1.3.1-1.4.0RC0_ > compatibility_report.html . All reported compatibility issues should > comply with policy but please review them carefully. > > A list of the 652 issues resolved in this release can be found at > https://s.apache.org/OErT . > > Please try out the candidate and vote +1/0/-1. > > This vote will be open for at least 72 hours. Unless objection I will try > to close it Monday December 18, 2017 if we have sufficient votes. > > Prior to making this announcement I made the following preflight checks to > 1.4.0RC0 (3d571827cb): > >- RAT check passes (7u80) >- Unit test suite passes (8u131) >- LTT load 1M rows with 100% verification and 20% updates (8u131) >- PE randomWrite, randomRead, scanRange100 (8u131) >- 100M rows ITBLL (8u131) > > Between now and when I want to close the vote I'll write up human readable > release notes for the release announcement as promised. I also have agreed > to run a scale ITBLL test, a performance comparison with 1.2 using YCSB, a > performance comparison with 1.2 using PE, and an analysis of code and > allocation hot spot changes from 1.2, all of which I will publish when > available and factor in to my vote. > > > -- > Best regards, > Andrew > > -- Best regards, Andrew Words like orphans lost among the crosstalk, meaning torn from truth's decrepit hands - A23, Crosstalk
[jira] [Created] (HBASE-19466) Rare failure in TestScannerCursor
Andrew Purtell created HBASE-19466: -- Summary: Rare failure in TestScannerCursor Key: HBASE-19466 URL: https://issues.apache.org/jira/browse/HBASE-19466 Project: HBase Issue Type: Bug Affects Versions: 1.4.0 Reporter: Andrew Purtell Priority: Minor Fix For: 1.4.1, 1.5.0 I think we just need to increase the timeout interval to deal with occasional slowdowns on test executors. 1998 ms is a pretty short timeout. By the way "rpcTimetout" in the exception message is a misspelling. [ERROR] Tests run: 2, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 37.412 s <<< FAILURE! - in org.apache.hadoop.hbase.regionserver.TestScannerCursor [ERROR] testHeartbeatWithSparseFilter(org.apache.hadoop.hbase.regionserver.TestScannerCursor) Time elapsed: 35.604 s <<< ERROR! org.apache.hadoop.hbase.client.RetriesExhaustedException: Failed after attempts=36, exceptions: Thu Dec 07 22:27:16 UTC 2017, null, java.net.SocketTimeoutException: callTimeout=4000, callDuration=4108: Call to ip-172-31-47-35.us-west-2.compute.internal/172.31.47.35:35690 failed on local exception: org.apache.hadoop.hbase.ipc.CallTimeoutException: Call id=52, waitTime=2002, rpcTimetout=1998 row '' on table 'TestScannerCursor' at region=TestScannerCursor,,1512685598567.1d4e59215a881d6ccbd0b5b5bdec5587., hostname=ip-172-31-47-35.us-west-2.compute.internal,35690,1512685593244, seqNum=2 at org.apache.hadoop.hbase.regionserver.TestScannerCursor.testHeartbeatWithSparseFilter(TestScannerCursor.java:154) Caused by: java.net.SocketTimeoutException: callTimeout=4000, callDuration=4108: Call to ip-172-31-47-35.us-west-2.compute.internal/172.31.47.35:35690 failed on local exception: org.apache.hadoop.hbase.ipc.CallTimeoutException: Call id=52, waitTime=2002, rpcTimetout=1998 row '' on table 'TestScannerCursor' at region=TestScannerCursor,,1512685598567.1d4e59215a881d6ccbd0b5b5bdec5587., hostname=ip-172-31-47-35.us-west-2.compute.internal,35690,1512685593244, seqNum=2 Caused by: java.io.IOException: Call to ip-172-31-47-35.us-west-2.compute.internal/172.31.47.35:35690 failed on local exception: org.apache.hadoop.hbase.ipc.CallTimeoutException: Call id=52, waitTime=2002, rpcTimetout=1998 Caused by: org.apache.hadoop.hbase.ipc.CallTimeoutException: Call id=52, waitTime=2002, rpcTimetout=1998 -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HBASE-19465) Required httpcore and httpclient jars not included in binary distribution
Andrew Purtell created HBASE-19465: -- Summary: Required httpcore and httpclient jars not included in binary distribution Key: HBASE-19465 URL: https://issues.apache.org/jira/browse/HBASE-19465 Project: HBase Issue Type: Bug Affects Versions: 1.4.0 Reporter: Andrew Purtell Assignee: Andrew Purtell Fix For: 1.4.0 The httpcore and httpclient dependency jars are not included into the binary distribution. This means MR/YARN jobs will fail if launched from the binary tarball as distributed, including those running in "local mode", e.g. ITBLL: {noformat} 2017-12-08 22:40:36,134 INFO [LocalJobRunner Map Task Executor #0] mapred.LocalJobRunner: Finishing task: attempt_local936234315_0001_m_03_0 2017-12-08 22:40:36,134 INFO [Thread-23] mapred.LocalJobRunner: map task executor complete. 2017-12-08 22:40:36,147 WARN [Thread-23] mapred.LocalJobRunner: job_local936234315_0001 java.lang.NoClassDefFoundError: org/apache/http/client/methods/HttpUriRequest at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:546) Caused by: java.lang.ClassNotFoundException: org.apache.http.client.methods.HttpUriRequest at java.net.URLClassLoader.findClass(URLClassLoader.java:381) at java.lang.ClassLoader.loadClass(ClassLoader.java:424) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:335) at java.lang.ClassLoader.loadClass(ClassLoader.java:357) ... 1 more Exception in thread "Thread-23" java.lang.NoClassDefFoundError: org/apache/http/client/methods/HttpUriRequest at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:562) Caused by: java.lang.ClassNotFoundException: org.apache.http.client.methods.HttpUriRequest at java.net.URLClassLoader.findClass(URLClassLoader.java:381) at java.lang.ClassLoader.loadClass(ClassLoader.java:424) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:335) at java.lang.ClassLoader.loadClass(ClassLoader.java:357) ... 1 more 2017-12-08 22:40:37,078 INFO [main] mapreduce.Job: Job job_local936234315_0001 failed with state FAILED due to: NA {noformat} -- This message was sent by Atlassian JIRA (v6.4.14#64029)
Moving To SLF4J and Log4J2
Hello Team, Maybe it's time to finally realize the dream of HBASE-10092 Once the logging infrastructure is upgraded, I would be happy to assist in reviewing and improving the log messages. I've done a bit of similar work for the Hive project. It's tedious but doesn't require deep knowledge of the overall project and can be done piecemeal. Commons Logging states that it requires one to use code guards for logging debug/trace statements. However, SLF4J recommends using parameters for logging which is faster than guards, and I personally think it's a cleaner approach. The logging parameters are only resolved if the log level is appropriate and therefore guards are often unnecessary. It removes code (the guards) and complexity from the application code. This seems like a straight-forward path for the project for eking out some performance. https://www.slf4j.org/faq.html#logging_performance https://commons.apache.org/proper/commons-logging/guide.html#Code_Guards Thanks.
[jira] [Created] (HBASE-19464) Replace StringBuffer with StringBuilder for hbase-common
BELUGA BEHR created HBASE-19464: --- Summary: Replace StringBuffer with StringBuilder for hbase-common Key: HBASE-19464 URL: https://issues.apache.org/jira/browse/HBASE-19464 Project: HBase Issue Type: Improvement Components: hbase Affects Versions: 2.0.0 Reporter: BELUGA BEHR Priority: Trivial Replace {{StringBuffer}} with non-synchronized version {{StringBuilder}} -- This message was sent by Atlassian JIRA (v6.4.14#64029)
Re: Testing and CI -- Apache Jenkins Builds (WAS -> Re: Testing)
Thanks Andrew. I disabled the job. Use the nightly going forward. The jdk7 builds seem to run fine. The jdk8 has some timeout going on. Need to dig in. You can see here: https://builds.apache.org/view/H-L/view/HBase/job/HBase%20Nightly/job/branch-1.4/ Thanks, M On Fri, Dec 8, 2017 at 11:29 AM, Andrew Purtell wrote: > Ok with me, Stack. Thanks for asking. > > > On Thu, Nov 30, 2017 at 5:33 PM, Stack wrote: > > > On the move over to nightly test runs: > > > > 1.2 nightly had a successful build last night after the branch-1 > > stabilization effort (HBASE-19204) and fixing a few unit test failures. > See > > build 150 > > https://builds.apache.org/view/H-L/view/HBase/job/HBase% > > 20Nightly/job/branch-1.2/ > > It then failed, 151, because of timed out test. Need to dig in. Clean up > a > > few more unit tests and branch-1.2 is probably ready for a > release-cutting. > > > > 1.3 has a few flakies. The last build failed because of: > > > > Test Result (1 failure / ±0) > > org.apache.hadoop.hbase.regionserver.TestEncryptionKeyRotation. > > testCFKeyRotation > > > > Just a little effort should turn 1.3 green. > > > > I was going to disable the 1.4 job, > > https://builds.apache.org/view/H-L/view/HBase/job/HBase-1.4/, in favor > of > > the 1.4 nightly, > > https://builds.apache.org/view/H-L/view/HBase/job/HBase% > > 20Nightly/job/branch-1.4/, > > if ok w/ you Andrew Purtell... And move over the branch-1, branch-2, and > > master too. > > > > Thanks, > > S > > > > > > > > On Wed, Nov 29, 2017 at 8:06 AM, Stack wrote: > > > > > Example of the new nice reporting: vhttps://builds.apache.org/ > > > view/H-L/view/HBase/job/HBase%20Nightly/job/branch-1.2/ > > > S > > > > > > On Wed, Nov 29, 2017 at 8:06 AM, Stack wrote: > > > > > >> Note that I have disabled the HBase-1.2-JDK7, HBase-1.2-JDK8, > > >> HBase-1.3-JDK7, and HBase-1.3-JDK8 jobs. They have been broken for a > > good > > >> while now. In their place, refer to an ongoing Sean "Nightly" project, > > an > > >> effort he has been at for a while. It does more checking with pretty > > >> reports that will help figuring general stability over time. See under > > >> https://builds.apache.org/view/H-L/view/HBase/job/HBase%20Nightly/ > > >> See the nightly builds for 1.2 and 1.3. They have some teething issues > > >> still but are almost there. See the 1.2 build from last night. In > recent > > >> days, the 1.2 branch went from trash-can fire to stable. See how all > > tests > > >> passed in the last build but then we failed generating the src bundle > on > > >> the end (this is what I mean by 'teething' issue). Will work on fixing > > this > > >> last step and moving over 1.4, etc., in the next few days. > > >> > > >> FYI, > > >> St.Ack > > >> > > >> > > >> On Tue, Nov 7, 2017 at 7:45 AM, Stack wrote: > > >> > > >>> On Tue, Nov 7, 2017 at 6:10 AM, Sean Busbey > wrote: > > >>> > > > Should I be able to see the machine dir when I look at nightlies > > output? > > > (Was trying to see what else is running). > > > > Ah. we don't have the same machine sampling on nightly as we do in > > precommit. I am 80% on a patch for HBASE-19189 (run test ad-hoc > > repeatedly) that includes pulling that information gathering into a > > place where we could also use it in nightly. > > > > > > >>> Sweet. > > >>> > > >>> > > >>> > > Did we ever figure out how many cores we expect our tests to need? > It > > looks like the Hadoop nodes have 8 cores. (with 2 executors that > means > > 4 is our fair share) > > > > > > >>> At the end of the thread inquiry I suggested that we don't use enough > > >>> cores, that we could up our fork counts and tests would complete in > > less > > >>> time. I wanted to experiment some w/ high fork counts -- 16 or so -- > > to see > > >>> if concurrent running brought on more failure. > > >>> > > >>> St.Ack > > >>> > > >>> > > >>> > > >>> > > On Tue, Nov 7, 2017 at 8:05 AM, Sean Busbey > > wrote: > > > surefire results get zipped up (we were filling the jenkins hosts > > with > > > old test logs previously) and stored in a file called > > "test_logs.zip" > > > for each jvm run. So if that happend in the jdk7 run for > branch-1.2, > > > it'd be in artifacts -> output-jdk7 -> test_logs.zip. > > > > > > I don't know if the archival process grabs things from surefire > that > > > aren't the surefire XML files, but we can update it to do so if it > > > doesn't. > > > > > > On Mon, Nov 6, 2017 at 11:39 PM, Stack wrote: > > >> I see this in the 1.2 nightly just when it gives up the ghost > > >> > > >> [WARNING] Corrupted STDOUT by directly writing to native stream > in > > >> forked JVM 2. See FAQ web page and the dump file > > >> /testptch/hbase/hbase-server/target/surefire-reports/2017-11 > > -06T20-11-30_219-jvmRun2.dumpstream > > >> > > >> .. but the pointed to dumpstream doesn
Re: Testing and CI -- Apache Jenkins Builds (WAS -> Re: Testing)
Ok with me, Stack. Thanks for asking. On Thu, Nov 30, 2017 at 5:33 PM, Stack wrote: > On the move over to nightly test runs: > > 1.2 nightly had a successful build last night after the branch-1 > stabilization effort (HBASE-19204) and fixing a few unit test failures. See > build 150 > https://builds.apache.org/view/H-L/view/HBase/job/HBase% > 20Nightly/job/branch-1.2/ > It then failed, 151, because of timed out test. Need to dig in. Clean up a > few more unit tests and branch-1.2 is probably ready for a release-cutting. > > 1.3 has a few flakies. The last build failed because of: > > Test Result (1 failure / ±0) > org.apache.hadoop.hbase.regionserver.TestEncryptionKeyRotation. > testCFKeyRotation > > Just a little effort should turn 1.3 green. > > I was going to disable the 1.4 job, > https://builds.apache.org/view/H-L/view/HBase/job/HBase-1.4/, in favor of > the 1.4 nightly, > https://builds.apache.org/view/H-L/view/HBase/job/HBase% > 20Nightly/job/branch-1.4/, > if ok w/ you Andrew Purtell... And move over the branch-1, branch-2, and > master too. > > Thanks, > S > > > > On Wed, Nov 29, 2017 at 8:06 AM, Stack wrote: > > > Example of the new nice reporting: vhttps://builds.apache.org/ > > view/H-L/view/HBase/job/HBase%20Nightly/job/branch-1.2/ > > S > > > > On Wed, Nov 29, 2017 at 8:06 AM, Stack wrote: > > > >> Note that I have disabled the HBase-1.2-JDK7, HBase-1.2-JDK8, > >> HBase-1.3-JDK7, and HBase-1.3-JDK8 jobs. They have been broken for a > good > >> while now. In their place, refer to an ongoing Sean "Nightly" project, > an > >> effort he has been at for a while. It does more checking with pretty > >> reports that will help figuring general stability over time. See under > >> https://builds.apache.org/view/H-L/view/HBase/job/HBase%20Nightly/ > >> See the nightly builds for 1.2 and 1.3. They have some teething issues > >> still but are almost there. See the 1.2 build from last night. In recent > >> days, the 1.2 branch went from trash-can fire to stable. See how all > tests > >> passed in the last build but then we failed generating the src bundle on > >> the end (this is what I mean by 'teething' issue). Will work on fixing > this > >> last step and moving over 1.4, etc., in the next few days. > >> > >> FYI, > >> St.Ack > >> > >> > >> On Tue, Nov 7, 2017 at 7:45 AM, Stack wrote: > >> > >>> On Tue, Nov 7, 2017 at 6:10 AM, Sean Busbey wrote: > >>> > > Should I be able to see the machine dir when I look at nightlies > output? > > (Was trying to see what else is running). > > Ah. we don't have the same machine sampling on nightly as we do in > precommit. I am 80% on a patch for HBASE-19189 (run test ad-hoc > repeatedly) that includes pulling that information gathering into a > place where we could also use it in nightly. > > > >>> Sweet. > >>> > >>> > >>> > Did we ever figure out how many cores we expect our tests to need? It > looks like the Hadoop nodes have 8 cores. (with 2 executors that means > 4 is our fair share) > > > >>> At the end of the thread inquiry I suggested that we don't use enough > >>> cores, that we could up our fork counts and tests would complete in > less > >>> time. I wanted to experiment some w/ high fork counts -- 16 or so -- > to see > >>> if concurrent running brought on more failure. > >>> > >>> St.Ack > >>> > >>> > >>> > >>> > On Tue, Nov 7, 2017 at 8:05 AM, Sean Busbey > wrote: > > surefire results get zipped up (we were filling the jenkins hosts > with > > old test logs previously) and stored in a file called > "test_logs.zip" > > for each jvm run. So if that happend in the jdk7 run for branch-1.2, > > it'd be in artifacts -> output-jdk7 -> test_logs.zip. > > > > I don't know if the archival process grabs things from surefire that > > aren't the surefire XML files, but we can update it to do so if it > > doesn't. > > > > On Mon, Nov 6, 2017 at 11:39 PM, Stack wrote: > >> I see this in the 1.2 nightly just when it gives up the ghost > >> > >> [WARNING] Corrupted STDOUT by directly writing to native stream in > >> forked JVM 2. See FAQ web page and the dump file > >> /testptch/hbase/hbase-server/target/surefire-reports/2017-11 > -06T20-11-30_219-jvmRun2.dumpstream > >> > >> .. but the pointed to dumpstream doesn't seem to be around post > build. > >> I am looking in wrong place? > >> > >> > >> Thanks, > >> > >> S > >> > >> > >> On Mon, Nov 6, 2017 at 8:20 PM, Stack wrote: > >> > >>> On Mon, Nov 6, 2017 at 8:35 AM, Sean Busbey < > sean.bus...@gmail.com> > wrote: > >>> > Given that all of the old post-commit tests have been posting > that > they're failing to JIRAs for what looks like a month, is there > any > reason not to switch to the new tests that also say they're > failing? > >>>
[jira] [Created] (HBASE-19463) Make CPEnv#getConnection return a facade that throws Unsupported if CP calls #close
stack created HBASE-19463: - Summary: Make CPEnv#getConnection return a facade that throws Unsupported if CP calls #close Key: HBASE-19463 URL: https://issues.apache.org/jira/browse/HBASE-19463 Project: HBase Issue Type: Improvement Components: Coprocessors Reporter: stack Fix For: 2.0.0-beta-2 Follows from HBASE-19301, a suggestion by [~zghaobac]. To prevent a CP accidentally closing the connection returned by CpEnv#getConnection -- which returns the hosting Servers Connection -- we should throw UnsupportedException if the CP calls #close Do it. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
Re: [VOTE] The first HBase 1.4.0 release candidate (RC0) is available
Thanks for taking a look Balazs. Please file a JIRA for that. It may be a cosmetic problem but if we have to spin another RC I want to be sure to fix it. > On Dec 8, 2017, at 3:09 AM, Balazs Meszaros > wrote: > > Hi, > > +1, because: > > Signatures: ok > Unit tests: ok (8u112) > RSGroups: worked for me through the shell > > -1, because: > > hbase(main):005:0> list_rsgroups > GROUPS > > test > > default > > 2 row(s) in *1512730813.6580* seconds > > Trust me, it was faster than 47 years :) Other listings (e.g. > list_namespace) displayed the elapsed time correctly. > > Best regards, > Balazs > >> On Fri, Dec 8, 2017 at 4:51 AM, Ted Yu wrote: >> >> +1 >> >> Checked signatures: good >> Ran test suite: pass >> 1M row LTT: pass >> >> Cheers >> >> On Wed, Dec 6, 2017 at 8:40 PM, Andrew Purtell >> wrote: >> >>> The first HBase 1.4.0 release candidate (RC0) is available for download >> at >>> https://dist.apache.org/repos/dist/dev/hbase/hbase-1.4.0RC0/ and Maven >>> artifacts are available in the temporary repository >>> https://repository.apache.org/content/repositories/orgapachehbase-1185/ >> . >>> >>> The git tag corresponding to the candidate is '1.4.0RC0' (3d571827cb). >>> >>> A detailed source and binary compatibility report for this release is >>> available for your review at >>> https://dist.apache.org/repos/dist/dev/hbase/hbase-1.4.0RC0/ >>> hbase-1.3.1-1.4.0RC0_compatibility_report.html >>> . All reported compatibility issues should comply with policy but please >>> review them carefully. >>> >>> A list of the 652 issues resolved in this release can be found at >>> https://s.apache.org/OErT . >>> >>> Please try out the candidate and vote +1/0/-1. >>> >>> This vote will be open for at least 72 hours. Unless objection I will try >>> to close it Monday December 18, 2017 if we have sufficient votes. >>> >>> Prior to making this announcement I made the following preflight checks >> to >>> 1.4.0RC0 (3d571827cb): >>> >>> - RAT check passes (7u80) >>> - Unit test suite passes (8u131) >>> - LTT load 1M rows with 100% verification and 20% updates (8u131) >>> - PE randomWrite, randomRead, scanRange100 (8u131) >>> - 100M rows ITBLL (8u131) >>> >>> Between now and when I want to close the vote I'll write up human >> readable >>> release notes for the release announcement as promised. I also have >> agreed >>> to run a scale ITBLL test, a performance comparison with 1.2 using YCSB, >> a >>> performance comparison with 1.2 using PE, and an analysis of code and >>> allocation hot spot changes from 1.2, all of which I will publish when >>> available and factor in to my vote. >>> >>> >>> -- >>> Best regards, >>> Andrew >>> >>
Re: [VOTE] The first HBase 1.4.0 release candidate (RC0) is available
I tried a few RS group commands: -- hbase(main):001:0> list_rsgroups GROUPS default 1 row(s) in 1512748253.9920 seconds hbase(main):002:0> add_rsgroup 'my_group' hbase(main):003:0> list_rsgroups GROUPS default my_group 2 row(s) in 1512748276.9400 seconds -- The displayed duration was off but the commands actually returned fast. I would say this is minor defect. Cheers On Fri, Dec 8, 2017 at 3:09 AM, Balazs Meszaros < balazs.mesza...@cloudera.com> wrote: > Hi, > > +1, because: > > Signatures: ok > Unit tests: ok (8u112) > RSGroups: worked for me through the shell > > -1, because: > > hbase(main):005:0> list_rsgroups > GROUPS > > test > > default > > 2 row(s) in *1512730813.6580* seconds > > Trust me, it was faster than 47 years :) Other listings (e.g. > list_namespace) displayed the elapsed time correctly. > > Best regards, > Balazs > > On Fri, Dec 8, 2017 at 4:51 AM, Ted Yu wrote: > > > +1 > > > > Checked signatures: good > > Ran test suite: pass > > 1M row LTT: pass > > > > Cheers > > > > On Wed, Dec 6, 2017 at 8:40 PM, Andrew Purtell > > wrote: > > > > > The first HBase 1.4.0 release candidate (RC0) is available for > download > > at > > > https://dist.apache.org/repos/dist/dev/hbase/hbase-1.4.0RC0/ and Maven > > > artifacts are available in the temporary repository > > > https://repository.apache.org/content/repositories/ > orgapachehbase-1185/ > > . > > > > > > The git tag corresponding to the candidate is '1.4.0RC0' (3d571827cb). > > > > > > A detailed source and binary compatibility report for this release is > > > available for your review at > > > https://dist.apache.org/repos/dist/dev/hbase/hbase-1.4.0RC0/ > > > hbase-1.3.1-1.4.0RC0_compatibility_report.html > > > . All reported compatibility issues should comply with policy but > please > > > review them carefully. > > > > > > A list of the 652 issues resolved in this release can be found at > > > https://s.apache.org/OErT . > > > > > > Please try out the candidate and vote +1/0/-1. > > > > > > This vote will be open for at least 72 hours. Unless objection I will > try > > > to close it Monday December 18, 2017 if we have sufficient votes. > > > > > > Prior to making this announcement I made the following preflight checks > > to > > > 1.4.0RC0 (3d571827cb): > > > > > >- RAT check passes (7u80) > > >- Unit test suite passes (8u131) > > >- LTT load 1M rows with 100% verification and 20% updates (8u131) > > >- PE randomWrite, randomRead, scanRange100 (8u131) > > >- 100M rows ITBLL (8u131) > > > > > > Between now and when I want to close the vote I'll write up human > > readable > > > release notes for the release announcement as promised. I also have > > agreed > > > to run a scale ITBLL test, a performance comparison with 1.2 using > YCSB, > > a > > > performance comparison with 1.2 using PE, and an analysis of code and > > > allocation hot spot changes from 1.2, all of which I will publish when > > > available and factor in to my vote. > > > > > > > > > -- > > > Best regards, > > > Andrew > > > > > >
Re: [VOTE] The first HBase 1.4.0 release candidate (RC0) is available
+1 checksum, signature: ok tests: ok LTT with 1M rows: ok basic operations from shell: ok rat check: ok On Thu, Dec 7, 2017 at 5:40 AM, Andrew Purtell wrote: > The first HBase 1.4.0 release candidate (RC0) is available for download at > https://dist.apache.org/repos/dist/dev/hbase/hbase-1.4.0RC0/ and Maven > artifacts are available in the temporary repository > https://repository.apache.org/content/repositories/orgapachehbase-1185/ . > > The git tag corresponding to the candidate is '1.4.0RC0' (3d571827cb). > > A detailed source and binary compatibility report for this release is > available for your review at > https://dist.apache.org/repos/dist/dev/hbase/hbase-1.4.0RC0/ > hbase-1.3.1-1.4.0RC0_compatibility_report.html > . All reported compatibility issues should comply with policy but please > review them carefully. > > A list of the 652 issues resolved in this release can be found at > https://s.apache.org/OErT . > > Please try out the candidate and vote +1/0/-1. > > This vote will be open for at least 72 hours. Unless objection I will try > to close it Monday December 18, 2017 if we have sufficient votes. > > Prior to making this announcement I made the following preflight checks to > 1.4.0RC0 (3d571827cb): > >- RAT check passes (7u80) >- Unit test suite passes (8u131) >- LTT load 1M rows with 100% verification and 20% updates (8u131) >- PE randomWrite, randomRead, scanRange100 (8u131) >- 100M rows ITBLL (8u131) > > Between now and when I want to close the vote I'll write up human readable > release notes for the release announcement as promised. I also have agreed > to run a scale ITBLL test, a performance comparison with 1.2 using YCSB, a > performance comparison with 1.2 using PE, and an analysis of code and > allocation hot spot changes from 1.2, all of which I will publish when > available and factor in to my vote. > > > -- > Best regards, > Andrew >
Re: [VOTE] The first HBase 1.4.0 release candidate (RC0) is available
+1 Unit test suite passes (8u131) hadoop-2.7.4 pass hadoop-2.7.3 pass hadoop-2.7.2 pass hadoop-2.7.1 pass hadoop-2.6.4 pass hadoop-2.6.3 pass hadoop-2.6.2 pass hadoop-2.6.1 pass On 2017-12-07 12:40, Andrew Purtell wrote: > The first HBase 1.4.0â release candidate (RC0) is available for download at > https://dist.apache.org/repos/dist/dev/hbase/hbase-1.4.0RC0/ and Maven > artifacts are available in the temporary repository > https://repository.apache.org/content/repositories/orgapachehbase-1185/ . > > The git tag corresponding to the candidate is '1.4.0RC0' (3d571827cb). > > A detailed source and binary compatibility report for this release is > available for your review at > https://dist.apache.org/repos/dist/dev/hbase/hbase-1.4.0RC0/hbase-1.3.1-1.4.0RC0_compatibility_report.html > â. All reported compatibility issues should comply with policy but please > review them carefully. > > A list of the â652 issues resolved in this release can be found at > https://s.apache.org/OErT . > > Please try out the candidate and vote +1/0/-1. > > This vote will be open for at least 72 hours. Unless objection I will try > to close it âMonday December 18, 2017 if we have sufficient votes. > > Prior to making this announcement I made the following preflight checks to > 1.4.0RC0 (3d571827cb): > >- RAT check passes (7u80) >- Unit test suite passes (8u131) >- LTT load 1M rows with 100% verification and 20% updates (8u131) >- PE randomWrite, randomRead, scanRange100 (8u131) >- 100M rows ITBLL (8u131) > > Between now and when I want to close the vote I'll write up human readable > release notes for the release announcement as promised. I also have agreed > to run a scale ITBLL test, a performance comparison with 1.2 using YCSB, a > performance comparison with 1.2 using PE, and an analysis of code and > allocation hot spot changes from 1.2, all of which I will publish when > available and factor in to my vote. > > > -- > Best regards, > Andrew >
Re: [VOTE] The first HBase 1.4.0 release candidate (RC0) is available
Hi, +1, because: Signatures: ok Unit tests: ok (8u112) RSGroups: worked for me through the shell -1, because: hbase(main):005:0> list_rsgroups GROUPS test default 2 row(s) in *1512730813.6580* seconds Trust me, it was faster than 47 years :) Other listings (e.g. list_namespace) displayed the elapsed time correctly. Best regards, Balazs On Fri, Dec 8, 2017 at 4:51 AM, Ted Yu wrote: > +1 > > Checked signatures: good > Ran test suite: pass > 1M row LTT: pass > > Cheers > > On Wed, Dec 6, 2017 at 8:40 PM, Andrew Purtell > wrote: > > > The first HBase 1.4.0 release candidate (RC0) is available for download > at > > https://dist.apache.org/repos/dist/dev/hbase/hbase-1.4.0RC0/ and Maven > > artifacts are available in the temporary repository > > https://repository.apache.org/content/repositories/orgapachehbase-1185/ > . > > > > The git tag corresponding to the candidate is '1.4.0RC0' (3d571827cb). > > > > A detailed source and binary compatibility report for this release is > > available for your review at > > https://dist.apache.org/repos/dist/dev/hbase/hbase-1.4.0RC0/ > > hbase-1.3.1-1.4.0RC0_compatibility_report.html > > . All reported compatibility issues should comply with policy but please > > review them carefully. > > > > A list of the 652 issues resolved in this release can be found at > > https://s.apache.org/OErT . > > > > Please try out the candidate and vote +1/0/-1. > > > > This vote will be open for at least 72 hours. Unless objection I will try > > to close it Monday December 18, 2017 if we have sufficient votes. > > > > Prior to making this announcement I made the following preflight checks > to > > 1.4.0RC0 (3d571827cb): > > > >- RAT check passes (7u80) > >- Unit test suite passes (8u131) > >- LTT load 1M rows with 100% verification and 20% updates (8u131) > >- PE randomWrite, randomRead, scanRange100 (8u131) > >- 100M rows ITBLL (8u131) > > > > Between now and when I want to close the vote I'll write up human > readable > > release notes for the release announcement as promised. I also have > agreed > > to run a scale ITBLL test, a performance comparison with 1.2 using YCSB, > a > > performance comparison with 1.2 using PE, and an analysis of code and > > allocation hot spot changes from 1.2, all of which I will publish when > > available and factor in to my vote. > > > > > > -- > > Best regards, > > Andrew > > >
[jira] [Created] (HBASE-19462) Deprecate all addImmutable in Put
Chia-Ping Tsai created HBASE-19462: -- Summary: Deprecate all addImmutable in Put Key: HBASE-19462 URL: https://issues.apache.org/jira/browse/HBASE-19462 Project: HBase Issue Type: Sub-task Reporter: Chia-Ping Tsai Assignee: Chia-Ping Tsai Users are able to use {{CellBuilder}} to build the cell without array copy for Put/Delete/Increment/Append, and we always do the copy if user pass the {{ByteBuffer}}. Hence, the {{addImmutable}} is unnecessary. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HBASE-19461) TestRSGroups is broke
stack created HBASE-19461: - Summary: TestRSGroups is broke Key: HBASE-19461 URL: https://issues.apache.org/jira/browse/HBASE-19461 Project: HBase Issue Type: Bug Components: test Reporter: stack Fix For: 2.0.0-beta-1 I noticed this. Tried bisect'ing but nought definitive at mo. I see testDefaultNamespaceCreateAndAssign failing most of the time. The crazy getTable parse in RegionInfo gets a negative offset because the passed in region name is nonsense. -- This message was sent by Atlassian JIRA (v6.4.14#64029)