[jira] [Commented] (HBASE-5782) Edits can be appended out of seqid order since HBASE-4487

[Hadoop QA (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-5782?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13255381#comment-13255381
 ] 

Hadoop QA commented on HBASE-5782:
--

-1 overall.  Here are the results of testing the latest attachment 
  http://issues.apache.org/jira/secure/attachment/12522913/hbase-5782.txt
  against trunk revision .

+1 @author.  The patch does not contain any @author tags.

+1 tests included.  The patch appears to include 3 new or modified tests.

+1 javadoc.  The javadoc tool did not generate any warning messages.

+1 javac.  The applied patch does not increase the total number of javac 
compiler warnings.

-1 findbugs.  The patch appears to introduce 4 new Findbugs (version 1.3.9) 
warnings.

+1 release audit.  The applied patch does not increase the total number of 
release audit warnings.

 -1 core tests.  The patch failed these unit tests:
 

Test results: 
https://builds.apache.org/job/PreCommit-HBASE-Build/1549//testReport/
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/1549//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
Console output: 
https://builds.apache.org/job/PreCommit-HBASE-Build/1549//console

This message is automatically generated.

> Edits can be appended out of seqid order since HBASE-4487
> -
>
> Key: HBASE-5782
> URL: https://issues.apache.org/jira/browse/HBASE-5782
> Project: HBase
>  Issue Type: Bug
>  Components: wal
>Affects Versions: 0.94.0
>Reporter: Gopinathan A
>Assignee: ramkrishna.s.vasudevan
>Priority: Blocker
> Fix For: 0.94.0
>
> Attachments: 5782-sketch.txt, 5782.txt, 5782.unfinished-stack.txt, 
> HBASE-5782.patch, hbase-5782.txt
>
>
> Create a table with 1000 splits, after the region assignemnt, kill the 
> regionserver wich contains META table.
> Here few regions are missing after the log splitting and region assigment. 
> HBCK report shows multiple region holes are got created.
> Same scenario was verified mulitple times in 0.92.1, no issues.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5782) Edits can be appended out of seqid order since HBASE-4487

[Todd Lipcon (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-5782?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13255395#comment-13255395
 ] 

Todd Lipcon commented on HBASE-5782:


Looks like some test timeouts... will investigate tomorrow.

> Edits can be appended out of seqid order since HBASE-4487
> -
>
> Key: HBASE-5782
> URL: https://issues.apache.org/jira/browse/HBASE-5782
> Project: HBase
>  Issue Type: Bug
>  Components: wal
>Affects Versions: 0.94.0
>Reporter: Gopinathan A
>Assignee: ramkrishna.s.vasudevan
>Priority: Blocker
> Fix For: 0.94.0
>
> Attachments: 5782-sketch.txt, 5782.txt, 5782.unfinished-stack.txt, 
> HBASE-5782.patch, hbase-5782.txt
>
>
> Create a table with 1000 splits, after the region assignemnt, kill the 
> regionserver wich contains META table.
> Here few regions are missing after the log splitting and region assigment. 
> HBCK report shows multiple region holes are got created.
> Same scenario was verified mulitple times in 0.92.1, no issues.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5732) Remove the SecureRPCEngine and merge the security-related logic in the core engine

[Andrew Purtell (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-5732?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13255398#comment-13255398
 ] 

Andrew Purtell commented on HBASE-5732:
---

bq. Andrew, this patch hasn't been tested yet. But if it is made to run with 
hadoop-1.0, I am pretty sure that it will run on all the versions post 1.0. The 
UGI class of Hadoop hasn't changed incompatibly since 1.0.

I'm all for removing shims and hacks wherever possible, but if we end up here 
again then this just churns our API along with Hadoop core.

> Remove the SecureRPCEngine and merge the security-related logic in the core 
> engine
> --
>
> Key: HBASE-5732
> URL: https://issues.apache.org/jira/browse/HBASE-5732
> Project: HBase
>  Issue Type: Improvement
>Reporter: Devaraj Das
> Attachments: rpcengine-merge.patch
>
>
> Remove the SecureRPCEngine and merge the security-related logic in the core 
> engine. Follow up to HBASE-5727.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5545) region can't be opened for a long time. Because the creating File failed.

[ramkrishna.s.vasudevan (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-5545?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13255441#comment-13255441
 ] 

ramkrishna.s.vasudevan commented on HBASE-5545:
---

@Gao
Are you working on this patch? 

> region can't be opened for a long time. Because the creating File failed.
> -
>
> Key: HBASE-5545
> URL: https://issues.apache.org/jira/browse/HBASE-5545
> Project: HBase
>  Issue Type: Bug
>  Components: regionserver
>Affects Versions: 0.90.6
>Reporter: gaojinchao
>Assignee: gaojinchao
> Fix For: 0.90.7, 0.92.2
>
>
> Scenario:
> 
> 1. File is created 
> 2. But while writing data, all datanodes might have crashed. So writing data 
> will fail.
> 3. Now even if close is called in finally block, close also will fail and 
> throw the Exception because writing data failed.
> 4. After this if RS try to create the same file again, then 
> AlreadyBeingCreatedException will come.
> Suggestion to handle this scenario.
> ---
> 1. Check for the existence of the file, if exists delete the file and create 
> new file. 
> Here delete call for the file will not check whether the file is open or 
> closed.
> Overwrite Option:
> 
> 1. Overwrite option will be applicable if you are trying to overwrite a 
> closed file.
> 2. If the file is not closed, then even with overwrite option Same 
> AlreadyBeingCreatedException will be thrown.
> This is the expected behaviour to avoid the Multiple clients writing to same 
> file.
> Region server logs:
> org.apache.hadoop.ipc.RemoteException: 
> org.apache.hadoop.hdfs.protocol.AlreadyBeingCreatedException: failed to 
> create file /hbase/test1/12c01902324218d14b17a5880f24f64b/.tmp/.regioninfo 
> for 
> DFSClient_hb_rs_158-1-131-48,20020,1331107668635_1331107669061_-252463556_25 
> on client 158.1.132.19 because current leaseholder is trying to recreate file.
> at 
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.recoverLeaseInternal(FSNamesystem.java:1570)
> at 
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.startFileInternal(FSNamesystem.java:1440)
> at 
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.startFile(FSNamesystem.java:1382)
> at org.apache.hadoop.hdfs.server.namenode.NameNode.create(NameNode.java:658)
> at sun.reflect.GeneratedMethodAccessor9.invoke(Unknown Source)
> at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:547)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1137)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1133)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:396)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1131)
> at org.apache.hadoop.ipc.Client.call(Client.java:961)
> at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:245)
> at $Proxy6.create(Unknown Source)
> at sun.reflect.GeneratedMethodAccessor14.invoke(Unknown Source)
> at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at $Proxy6.create(Unknown Source)
> at 
> org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.(DFSClient.java:3643)
> at org.apache.hadoop.hdfs.DFSClient.create(DFSClient.java:778)
> at 
> org.apache.hadoop.hdfs.DistributedFileSystem.create(DistributedFileSystem.java:364)
> at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:630)
> at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:611)
> at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:518)
> at 
> org.apache.hadoop.hbase.regionserver.HRegion.checkRegioninfoOnFilesystem(HRegion.java:424)
> at org.apache.hadoop.hbase.regionserver.HRegion.initialize(HRegion.java:340)
> at org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:2672)
> at org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:2658)
> at 
> org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.openRegion(OpenRegionHandler.java:330)
> at 
> org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.process(OpenRegionHandler.java:116)
> at org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:158)
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:662)
> [2012-03-07 20:51:45,858] [WARN ] 
> [RS_OPEN_REGION-158-1-131-48,20020,1331107668635-23] 
> [com.huawei.isap.ump.ha.client.RPCRetryAndSwitchInvoker 131] Retrying the 
> method call

[jira] [Commented] (HBASE-5546) Master assigns region in the original region server when opening region failed

[ramkrishna.s.vasudevan (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-5546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13255442#comment-13255442
 ] 

ramkrishna.s.vasudevan commented on HBASE-5546:
---

@Gao
Are you working on this patch?

> Master assigns region in the original region server when opening region 
> failed  
> 
>
> Key: HBASE-5546
> URL: https://issues.apache.org/jira/browse/HBASE-5546
> Project: HBase
>  Issue Type: Bug
>  Components: master
>Affects Versions: 0.92.0
>Reporter: gaojinchao
>Priority: Minor
> Fix For: 0.92.2
>
>
> Master assigns region in the original region server when 
> RS_ZK_REGION_FAILED_OPEN envent was coming.
> Maybe we should choose other region server.
> [2012-03-07 10:14:21,750] [DEBUG] [main-EventThread] 
> [org.apache.hadoop.hbase.master.AssignmentManager 553] Handling 
> transition=RS_ZK_REGION_FAILED_OPEN, server=158-1-130-11,20020,1331108408232, 
> region=c70e98bdca98a0657a56436741523053
> [2012-03-07 10:14:31,826] [DEBUG] [main-EventThread] 
> [org.apache.hadoop.hbase.master.AssignmentManager 553] Handling 
> transition=RS_ZK_REGION_FAILED_OPEN, server=158-1-130-11,20020,1331108408232, 
> region=c70e98bdca98a0657a56436741523053
> [2012-03-07 10:14:41,903] [DEBUG] [main-EventThread] 
> [org.apache.hadoop.hbase.master.AssignmentManager 553] Handling 
> transition=RS_ZK_REGION_FAILED_OPEN, server=158-1-130-11,20020,1331108408232, 
> region=c70e98bdca98a0657a56436741523053
> [2012-03-07 10:14:51,975] [DEBUG] [main-EventThread] 
> [org.apache.hadoop.hbase.master.AssignmentManager 553] Handling 
> transition=RS_ZK_REGION_FAILED_OPEN, server=158-1-130-11,20020,1331108408232, 
> region=c70e98bdca98a0657a56436741523053
> [2012-03-07 10:15:02,056] [DEBUG] [main-EventThread] 
> [org.apache.hadoop.hbase.master.AssignmentManager 553] Handling 
> transition=RS_ZK_REGION_FAILED_OPEN, server=158-1-130-11,20020,1331108408232, 
> region=c70e98bdca98a0657a56436741523053
> [2012-03-07 10:15:12,167] [DEBUG] [main-EventThread] 
> [org.apache.hadoop.hbase.master.AssignmentManager 553] Handling 
> transition=RS_ZK_REGION_FAILED_OPEN, server=158-1-130-11,20020,1331108408232, 
> region=c70e98bdca98a0657a56436741523053
> [2012-03-07 10:15:22,231] [DEBUG] [main-EventThread] 
> [org.apache.hadoop.hbase.master.AssignmentManager 553] Handling 
> transition=RS_ZK_REGION_FAILED_OPEN, server=158-1-130-11,20020,1331108408232, 
> region=c70e98bdca98a0657a56436741523053
> [2012-03-07 10:15:32,303] [DEBUG] [main-EventThread] 
> [org.apache.hadoop.hbase.master.AssignmentManager 553] Handling 
> transition=RS_ZK_REGION_FAILED_OPEN, server=158-1-130-11,20020,1331108408232, 
> region=c70e98bdca98a0657a56436741523053
> [2012-03-07 10:15:42,375] [DEBUG] [main-EventThread] 
> [org.apache.hadoop.hbase.master.AssignmentManager 553] Handling 
> transition=RS_ZK_REGION_FAILED_OPEN, server=158-1-130-11,20020,1331108408232, 
> region=c70e98bdca98a0657a56436741523053
> [2012-03-07 10:15:52,447] [DEBUG] [main-EventThread] 
> [org.apache.hadoop.hbase.master.AssignmentManager 553] Handling 
> transition=RS_ZK_REGION_FAILED_OPEN, server=158-1-130-11,20020,1331108408232, 
> region=c70e98bdca98a0657a56436741523053
> [2012-03-07 10:16:02,528] [DEBUG] [main-EventThread] 
> [org.apache.hadoop.hbase.master.AssignmentManager 553] Handling 
> transition=RS_ZK_REGION_FAILED_OPEN, server=158-1-130-11,20020,1331108408232, 
> region=c70e98bdca98a0657a56436741523053
> [2012-03-07 10:16:12,600] [DEBUG] [main-EventThread] 
> [org.apache.hadoop.hbase.master.AssignmentManager 553] Handling 
> transition=RS_ZK_REGION_FAILED_OPEN, server=158-1-130-11,20020,1331108408232, 
> region=c70e98bdca98a0657a56436741523053
> [2012-03-07 10:16:22,676] [DEBUG] [main-EventThread] 
> [org.apache.hadoop.hbase.master.AssignmentManager 553] Handling 
> transition=RS_ZK_REGION_FAILED_OPEN, server=158-1-130-11,20020,1331108408232, 
> region=c70e98bdca98a0657a56436741523053

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5654) [findbugs] Address dodgy bugs

[Ashutosh Jindal (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-5654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13255475#comment-13255475
 ] 

Ashutosh Jindal commented on HBASE-5654:


@Jon
bq.add bounds checks? Result of integer multiplication cast to long in 
org.apache.hadoop.hbase.io.hfile.LruBlockCache.calculateOverhead(long, long, 
int)
bq.Result of integer multiplication cast to long in 
org.apache.hadoop.hbase.regionserver.IncreasingToUpperBoundRegionSplitPolicy.getSizeToCheck(int)
I am not getting what kind of bound checks can be added here. Can u please take 
a look. 

> [findbugs] Address dodgy bugs
> -
>
> Key: HBASE-5654
> URL: https://issues.apache.org/jira/browse/HBASE-5654
> Project: HBase
>  Issue Type: Sub-task
>  Components: scripts
>Affects Versions: 0.96.0
>Reporter: Jonathan Hsieh
>Assignee: Ashutosh Jindal
>  Labels: patch
> Fix For: 0.96.0
>
> Attachments: Hbase-5654.patch
>
>
> See 
> https://builds.apache.org/job/PreCommit-HBASE-Build/1313//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html#Warnings_STYLE
> This may be broken down further.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (HBASE-5806) Handle split region related failures on master restart and RS restart

[ramkrishna.s.vasudevan (Created) (JIRA)]

Handle split region related failures on master restart and RS restart
-

 Key: HBASE-5806
 URL: https://issues.apache.org/jira/browse/HBASE-5806
 Project: HBase
  Issue Type: Bug
Affects Versions: 0.92.1
Reporter: ramkrishna.s.vasudevan
 Fix For: 0.92.2, 0.96.0, 0.94.1


This issue is raised to solve issues that comes out of partial region split 
happened and the region node in the ZK which is in RS_ZK_REGION_SPLITTING and 
RS_ZK_REGION_SPLIT is not yet processed.
This also tries to address HBASE-5615.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5806) Handle split region related failures on master restart and RS restart

[ramkrishna.s.vasudevan (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-5806?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13255487#comment-13255487
 ] 

ramkrishna.s.vasudevan commented on HBASE-5806:
---

We will come up with a patch sooner. 

> Handle split region related failures on master restart and RS restart
> -
>
> Key: HBASE-5806
> URL: https://issues.apache.org/jira/browse/HBASE-5806
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 0.92.1
>Reporter: ramkrishna.s.vasudevan
> Fix For: 0.92.2, 0.96.0, 0.94.1
>
>
> This issue is raised to solve issues that comes out of partial region split 
> happened and the region node in the ZK which is in RS_ZK_REGION_SPLITTING and 
> RS_ZK_REGION_SPLIT is not yet processed.
> This also tries to address HBASE-5615.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-4435) Add Group By functionality using Coprocessors

[lifeng (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-4435?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13255546#comment-13255546
 ] 

lifeng commented on HBASE-4435:
---

when can this patch be put into hbase?

> Add Group By functionality using Coprocessors
> -
>
> Key: HBASE-4435
> URL: https://issues.apache.org/jira/browse/HBASE-4435
> Project: HBase
>  Issue Type: Improvement
>  Components: coprocessors
>Reporter: Nichole Treadway
>Priority: Minor
> Attachments: HBase-4435.patch
>
>
> Adds in a Group By -like functionality to HBase, using the Coprocessor 
> framework. 
> It provides the ability to group the result set on one or more columns 
> (groupBy families). It computes statistics (max, min, sum, count, sum of 
> squares, number missing) for a second column, called the stats column. 
> To use, I've provided two implementations.
> 1. In the first, you specify a single group-by column and a stats field:
>   statsMap = gbc.getStats(tableName, scan, groupByFamily, 
> groupByQualifier, statsFamily, statsQualifier, statsFieldColumnInterpreter);
> The result is a map with the Group By column value (as a String) to a 
> GroupByStatsValues object. The GroupByStatsValues object has max,min,sum etc. 
> of the stats column for that group.
> 2. The second implementation allows you to specify a list of group-by columns 
> and a stats field. The List of group-by columns is expected to contain lists 
> of {column family, qualifier} pairs. 
>   statsMap = gbc.getStats(tableName, scan, listOfGroupByColumns, 
> statsFamily, statsQualifier, statsFieldColumnInterpreter);
> The GroupByStatsValues code is adapted from the Solr Stats component.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5620) Convert the client protocol of HRegionInterface to PB

[Zhihong Yu (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-5620?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=1324#comment-1324
 ] 

Zhihong Yu commented on HBASE-5620:
---

@Stack:
Do you think the addendum is ready to be integrated ?

> Convert the client protocol of HRegionInterface to PB
> -
>
> Key: HBASE-5620
> URL: https://issues.apache.org/jira/browse/HBASE-5620
> Project: HBase
>  Issue Type: Sub-task
>  Components: ipc, master, migration, regionserver
>Reporter: Jimmy Xiang
>Assignee: Jimmy Xiang
> Fix For: 0.96.0
>
> Attachments: hbase-5620-sec.patch, hbase-5620_v3.patch, 
> hbase-5620_v4.patch, hbase-5620_v4.patch
>
>


--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-4435) Add Group By functionality using Coprocessors

[Zhihong Yu (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-4435?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=1327#comment-1327
 ] 

Zhihong Yu commented on HBASE-4435:
---

@Nicole:
The attached patch is half year old. Do you have a newer version ?

> Add Group By functionality using Coprocessors
> -
>
> Key: HBASE-4435
> URL: https://issues.apache.org/jira/browse/HBASE-4435
> Project: HBase
>  Issue Type: Improvement
>  Components: coprocessors
>Reporter: Nichole Treadway
>Priority: Minor
> Attachments: HBase-4435.patch
>
>
> Adds in a Group By -like functionality to HBase, using the Coprocessor 
> framework. 
> It provides the ability to group the result set on one or more columns 
> (groupBy families). It computes statistics (max, min, sum, count, sum of 
> squares, number missing) for a second column, called the stats column. 
> To use, I've provided two implementations.
> 1. In the first, you specify a single group-by column and a stats field:
>   statsMap = gbc.getStats(tableName, scan, groupByFamily, 
> groupByQualifier, statsFamily, statsQualifier, statsFieldColumnInterpreter);
> The result is a map with the Group By column value (as a String) to a 
> GroupByStatsValues object. The GroupByStatsValues object has max,min,sum etc. 
> of the stats column for that group.
> 2. The second implementation allows you to specify a list of group-by columns 
> and a stats field. The List of group-by columns is expected to contain lists 
> of {column family, qualifier} pairs. 
>   statsMap = gbc.getStats(tableName, scan, listOfGroupByColumns, 
> statsFamily, statsQualifier, statsFieldColumnInterpreter);
> The GroupByStatsValues code is adapted from the Solr Stats component.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (HBASE-5807) TestHLogRecordReader does not compile against Hadoop 2

[Andrew Purtell (Created) (JIRA)]

TestHLogRecordReader does not compile against Hadoop 2
--

 Key: HBASE-5807
 URL: https://issues.apache.org/jira/browse/HBASE-5807
 Project: HBase
  Issue Type: Bug
Affects Versions: 0.94.0, 0.96.0
Reporter: Andrew Purtell


See https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-23/144/console

I see this trying to compile against branch 2 also.

{noformat}
[ERROR] Failed to execute goal 
org.apache.maven.plugins:maven-compiler-plugin:2.0.2:testCompile 
(default-testCompile) on project hbase: Compilation failure: Compilation 
failure:
[ERROR] 
/home/jenkins/jenkins-slave/workspace/HBase-TRUNK-on-Hadoop-23/trunk/src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java:[1333,35]
 cannot find symbol
[ERROR] symbol  : method getJobTracker()
[ERROR] location: class org.apache.hadoop.mapred.MiniMRCluster.JobTrackerRunner
[ERROR] 
[ERROR] 
/home/jenkins/jenkins-slave/workspace/HBase-TRUNK-on-Hadoop-23/trunk/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHLogRecordReader.java:[147,46]
 org.apache.hadoop.mapreduce.JobContext is abstract; cannot be instantiated
[ERROR] 
[ERROR] 
/home/jenkins/jenkins-slave/workspace/HBase-TRUNK-on-Hadoop-23/trunk/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHLogRecordReader.java:[153,29]
 org.apache.hadoop.mapreduce.JobContext is abstract; cannot be instantiated
[ERROR] 
[ERROR] 
/home/jenkins/jenkins-slave/workspace/HBase-TRUNK-on-Hadoop-23/trunk/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHLogRecordReader.java:[194,46]
 org.apache.hadoop.mapreduce.JobContext is abstract; cannot be instantiated
[ERROR] 
[ERROR] 
/home/jenkins/jenkins-slave/workspace/HBase-TRUNK-on-Hadoop-23/trunk/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHLogRecordReader.java:[206,29]
 org.apache.hadoop.mapreduce.JobContext is abstract; cannot be instantiated
[ERROR] 
[ERROR] 
/home/jenkins/jenkins-slave/workspace/HBase-TRUNK-on-Hadoop-23/trunk/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHLogRecordReader.java:[213,29]
 org.apache.hadoop.mapreduce.JobContext is abstract; cannot be instantiated
[ERROR] 
[ERROR] 
/home/jenkins/jenkins-slave/workspace/HBase-TRUNK-on-Hadoop-23/trunk/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHLogRecordReader.java:[226,29]
 org.apache.hadoop.mapreduce.TaskAttemptContext is abstract; cannot be 
instantiated
{noformat}


--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5788) Move Dynamic Metrics storage off of HRegion.

[Elliott Clark (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-5788?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13255635#comment-13255635
 ] 

Elliott Clark commented on HBASE-5788:
--

RegionMetricsStorage could be a better name this class holds and mutates the 
numbers that will be exposed as dynamic region metrics.

I'll add the comments, and check the line length.  Not sure how I messed up 
eclipse.

The data members can be package private now that they are in the 
regionserver.metrics package.  I'll check to make sure.

Everything else is accessed in the regionserver package so they need to be 
public :-/

TestRegionServerMetrics covers most of the functionality of the new class but I 
can create a new set of more explicit tests if you think that is needed.

> Move Dynamic Metrics storage off of HRegion.
> 
>
> Key: HBASE-5788
> URL: https://issues.apache.org/jira/browse/HBASE-5788
> Project: HBase
>  Issue Type: Improvement
>  Components: regionserver
>Reporter: Elliott Clark
>Assignee: Elliott Clark
>Priority: Minor
> Attachments: HBASE-5788-0.patch, HBASE-5788-1.patch, 
> HBASE-5788-2.patch
>
>
> HRegion right now has the responsibility of storing static counts and latency 
> numbers for use by the metrics package.  Since these maps are incremented and 
> set from lots of places it makes adding functionality hard.
>  
> So move the metrics functionality into SchemaMetrics making it more than just 
> a class for naming.  The next step will be to simplify the api exposed so 
> that using it will be easier.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (HBASE-5808) Remove TestHLogBench, HLogPerformanceEvaluation is more comprehensive

[stack (Created) (JIRA)]

Remove TestHLogBench, HLogPerformanceEvaluation is more comprehensive
-

 Key: HBASE-5808
 URL: https://issues.apache.org/jira/browse/HBASE-5808
 Project: HBase
  Issue Type: Task
Reporter: stack
 Attachments: rm.txt

We added HLogPerformanceEvaluation recently though we had TestHLogBench which 
does much the same thing (Didn't notice).  Remove TestHLogBench rather than 
HLogPerformanceEvaluation because the latter does more (has a verify step and 
can take more options; also TestHLogBench does not actually write a log)

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5808) Remove TestHLogBench, HLogPerformanceEvaluation is more comprehensive

[stack (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-5808?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stack updated HBASE-5808:
-

Attachment: rm.txt

Just remove the file.

> Remove TestHLogBench, HLogPerformanceEvaluation is more comprehensive
> -
>
> Key: HBASE-5808
> URL: https://issues.apache.org/jira/browse/HBASE-5808
> Project: HBase
>  Issue Type: Task
>Reporter: stack
> Attachments: rm.txt
>
>
> We added HLogPerformanceEvaluation recently though we had TestHLogBench which 
> does much the same thing (Didn't notice).  Remove TestHLogBench rather than 
> HLogPerformanceEvaluation because the latter does more (has a verify step and 
> can take more options; also TestHLogBench does not actually write a log)

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Resolved] (HBASE-5808) Remove TestHLogBench, HLogPerformanceEvaluation is more comprehensive

[stack (Resolved) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-5808?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stack resolved HBASE-5808.
--

   Resolution: Fixed
Fix Version/s: 0.96.0
 Assignee: stack

Committed to trunk

> Remove TestHLogBench, HLogPerformanceEvaluation is more comprehensive
> -
>
> Key: HBASE-5808
> URL: https://issues.apache.org/jira/browse/HBASE-5808
> Project: HBase
>  Issue Type: Task
>Reporter: stack
>Assignee: stack
> Fix For: 0.96.0
>
> Attachments: rm.txt
>
>
> We added HLogPerformanceEvaluation recently though we had TestHLogBench which 
> does much the same thing (Didn't notice).  Remove TestHLogBench rather than 
> HLogPerformanceEvaluation because the latter does more (has a verify step and 
> can take more options; also TestHLogBench does not actually write a log)

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5545) region can't be opened for a long time. Because the creating File failed.

[ramkrishna.s.vasudevan (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-5545?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

ramkrishna.s.vasudevan updated HBASE-5545:
--

Fix Version/s: 0.94.1

> region can't be opened for a long time. Because the creating File failed.
> -
>
> Key: HBASE-5545
> URL: https://issues.apache.org/jira/browse/HBASE-5545
> Project: HBase
>  Issue Type: Bug
>  Components: regionserver
>Affects Versions: 0.90.6
>Reporter: gaojinchao
>Assignee: gaojinchao
> Fix For: 0.90.7, 0.92.2, 0.94.1
>
>
> Scenario:
> 
> 1. File is created 
> 2. But while writing data, all datanodes might have crashed. So writing data 
> will fail.
> 3. Now even if close is called in finally block, close also will fail and 
> throw the Exception because writing data failed.
> 4. After this if RS try to create the same file again, then 
> AlreadyBeingCreatedException will come.
> Suggestion to handle this scenario.
> ---
> 1. Check for the existence of the file, if exists delete the file and create 
> new file. 
> Here delete call for the file will not check whether the file is open or 
> closed.
> Overwrite Option:
> 
> 1. Overwrite option will be applicable if you are trying to overwrite a 
> closed file.
> 2. If the file is not closed, then even with overwrite option Same 
> AlreadyBeingCreatedException will be thrown.
> This is the expected behaviour to avoid the Multiple clients writing to same 
> file.
> Region server logs:
> org.apache.hadoop.ipc.RemoteException: 
> org.apache.hadoop.hdfs.protocol.AlreadyBeingCreatedException: failed to 
> create file /hbase/test1/12c01902324218d14b17a5880f24f64b/.tmp/.regioninfo 
> for 
> DFSClient_hb_rs_158-1-131-48,20020,1331107668635_1331107669061_-252463556_25 
> on client 158.1.132.19 because current leaseholder is trying to recreate file.
> at 
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.recoverLeaseInternal(FSNamesystem.java:1570)
> at 
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.startFileInternal(FSNamesystem.java:1440)
> at 
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.startFile(FSNamesystem.java:1382)
> at org.apache.hadoop.hdfs.server.namenode.NameNode.create(NameNode.java:658)
> at sun.reflect.GeneratedMethodAccessor9.invoke(Unknown Source)
> at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:547)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1137)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1133)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:396)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1131)
> at org.apache.hadoop.ipc.Client.call(Client.java:961)
> at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:245)
> at $Proxy6.create(Unknown Source)
> at sun.reflect.GeneratedMethodAccessor14.invoke(Unknown Source)
> at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at $Proxy6.create(Unknown Source)
> at 
> org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.(DFSClient.java:3643)
> at org.apache.hadoop.hdfs.DFSClient.create(DFSClient.java:778)
> at 
> org.apache.hadoop.hdfs.DistributedFileSystem.create(DistributedFileSystem.java:364)
> at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:630)
> at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:611)
> at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:518)
> at 
> org.apache.hadoop.hbase.regionserver.HRegion.checkRegioninfoOnFilesystem(HRegion.java:424)
> at org.apache.hadoop.hbase.regionserver.HRegion.initialize(HRegion.java:340)
> at org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:2672)
> at org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:2658)
> at 
> org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.openRegion(OpenRegionHandler.java:330)
> at 
> org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.process(OpenRegionHandler.java:116)
> at org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:158)
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:662)
> [2012-03-07 20:51:45,858] [WARN ] 
> [RS_OPEN_REGION-158-1-131-48,20020,1331107668635-23] 
> [com.huawei.isap.ump.ha.client.RPCRetryAndSwitchInvoker 131] Retrying the 
> method call: public abstract void 
> org.apache.hadoop.hdfs.protocol.ClientProtocol

[jira] [Commented] (HBASE-5792) HLog Performance Evaluation Tool

[stack (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-5792?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13255644#comment-13255644
 ] 

stack commented on HBASE-5792:
--

@Todd Thanks.  I removed TestHLogBench over in HBASE-5808.  The new test does 
verify and actually writes a log which TestHLogBench does not.

> HLog Performance Evaluation Tool
> 
>
> Key: HBASE-5792
> URL: https://issues.apache.org/jira/browse/HBASE-5792
> Project: HBase
>  Issue Type: Test
>  Components: wal
>Reporter: Matteo Bertozzi
>Assignee: Matteo Bertozzi
>Priority: Minor
>  Labels: performance, wal
> Fix For: 0.96.0
>
> Attachments: HBASE-5792-v0.patch, HBASE-5792-v1.patch, 
> HBASE-5792-v2.patch, verify.txt, verify.txt
>
>
> Related to HDFS-3280 and the HBase WAL slowdown on 0.23+
> It would be nice to have a simple tool like HFilePerformanceEvaluation, ...
> to be able to check easily the HLog performance.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5545) region can't be opened for a long time. Because the creating File failed.

[ramkrishna.s.vasudevan (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-5545?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13255642#comment-13255642
 ] 

ramkrishna.s.vasudevan commented on HBASE-5545:
---

@Lars
Do you mind taking this into 0.94.0? Though it exists in previous versions!!

> region can't be opened for a long time. Because the creating File failed.
> -
>
> Key: HBASE-5545
> URL: https://issues.apache.org/jira/browse/HBASE-5545
> Project: HBase
>  Issue Type: Bug
>  Components: regionserver
>Affects Versions: 0.90.6
>Reporter: gaojinchao
>Assignee: gaojinchao
> Fix For: 0.90.7, 0.92.2, 0.94.1
>
>
> Scenario:
> 
> 1. File is created 
> 2. But while writing data, all datanodes might have crashed. So writing data 
> will fail.
> 3. Now even if close is called in finally block, close also will fail and 
> throw the Exception because writing data failed.
> 4. After this if RS try to create the same file again, then 
> AlreadyBeingCreatedException will come.
> Suggestion to handle this scenario.
> ---
> 1. Check for the existence of the file, if exists delete the file and create 
> new file. 
> Here delete call for the file will not check whether the file is open or 
> closed.
> Overwrite Option:
> 
> 1. Overwrite option will be applicable if you are trying to overwrite a 
> closed file.
> 2. If the file is not closed, then even with overwrite option Same 
> AlreadyBeingCreatedException will be thrown.
> This is the expected behaviour to avoid the Multiple clients writing to same 
> file.
> Region server logs:
> org.apache.hadoop.ipc.RemoteException: 
> org.apache.hadoop.hdfs.protocol.AlreadyBeingCreatedException: failed to 
> create file /hbase/test1/12c01902324218d14b17a5880f24f64b/.tmp/.regioninfo 
> for 
> DFSClient_hb_rs_158-1-131-48,20020,1331107668635_1331107669061_-252463556_25 
> on client 158.1.132.19 because current leaseholder is trying to recreate file.
> at 
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.recoverLeaseInternal(FSNamesystem.java:1570)
> at 
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.startFileInternal(FSNamesystem.java:1440)
> at 
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.startFile(FSNamesystem.java:1382)
> at org.apache.hadoop.hdfs.server.namenode.NameNode.create(NameNode.java:658)
> at sun.reflect.GeneratedMethodAccessor9.invoke(Unknown Source)
> at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:547)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1137)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1133)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:396)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1131)
> at org.apache.hadoop.ipc.Client.call(Client.java:961)
> at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:245)
> at $Proxy6.create(Unknown Source)
> at sun.reflect.GeneratedMethodAccessor14.invoke(Unknown Source)
> at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at $Proxy6.create(Unknown Source)
> at 
> org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.(DFSClient.java:3643)
> at org.apache.hadoop.hdfs.DFSClient.create(DFSClient.java:778)
> at 
> org.apache.hadoop.hdfs.DistributedFileSystem.create(DistributedFileSystem.java:364)
> at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:630)
> at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:611)
> at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:518)
> at 
> org.apache.hadoop.hbase.regionserver.HRegion.checkRegioninfoOnFilesystem(HRegion.java:424)
> at org.apache.hadoop.hbase.regionserver.HRegion.initialize(HRegion.java:340)
> at org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:2672)
> at org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:2658)
> at 
> org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.openRegion(OpenRegionHandler.java:330)
> at 
> org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.process(OpenRegionHandler.java:116)
> at org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:158)
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:662)
> [2012-03-07 20:51:45,858] [WARN ] 
> [RS_OPEN_REGION-158-1-131-48,20020,1331107668635-23] 
> [com.huawei.isap.ump.ha.client.RPC

[jira] [Commented] (HBASE-5788) Move Dynamic Metrics storage off of HRegion.

[stack (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-5788?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13255645#comment-13255645
 ] 

stack commented on HBASE-5788:
--

bq. TestRegionServerMetrics covers most of the functionality of the new class 
but I can create a new set of more explicit tests if you think that is needed.

Probably no need if we have some coverage already.  Just want to make sure the 
class does its basic contract.  Easier figuring this stuff in a unit test than 
up on a cluster, yadda, yadda, you know what I'm at.

> Move Dynamic Metrics storage off of HRegion.
> 
>
> Key: HBASE-5788
> URL: https://issues.apache.org/jira/browse/HBASE-5788
> Project: HBase
>  Issue Type: Improvement
>  Components: regionserver
>Reporter: Elliott Clark
>Assignee: Elliott Clark
>Priority: Minor
> Attachments: HBASE-5788-0.patch, HBASE-5788-1.patch, 
> HBASE-5788-2.patch
>
>
> HRegion right now has the responsibility of storing static counts and latency 
> numbers for use by the metrics package.  Since these maps are incremented and 
> set from lots of places it makes adding functionality hard.
>  
> So move the metrics functionality into SchemaMetrics making it more than just 
> a class for naming.  The next step will be to simplify the api exposed so 
> that using it will be easier.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (HBASE-5809) Avoid move api to take the destination server same as the source server.

[ramkrishna.s.vasudevan (Created) (JIRA)]

Avoid move api to take the destination server same as the source server.


 Key: HBASE-5809
 URL: https://issues.apache.org/jira/browse/HBASE-5809
 Project: HBase
  Issue Type: Bug
Reporter: ramkrishna.s.vasudevan
Priority: Minor


In Move currently we take any destination specified and if the destination is 
same as the source we still do unassign and assign.  Here we can have problems 
due to RegionAlreadyInTransitionException and thus hanging the region in RIT 
for long time.  We can avoid this scenario by not allowing the move to happen 
in this scenario.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5620) Convert the client protocol of HRegionInterface to PB

[stack (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-5620?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13255667#comment-13255667
 ] 

stack commented on HBASE-5620:
--

@Jimmy So every invocation will throw an exception?

{code}
+// For protobuf protocols, ServiceException is expected
{code}

Whats the Set in Invocation doing?   You add it but don't seem to access it?

I like the removal of a call method down through the rpc stack

> Convert the client protocol of HRegionInterface to PB
> -
>
> Key: HBASE-5620
> URL: https://issues.apache.org/jira/browse/HBASE-5620
> Project: HBase
>  Issue Type: Sub-task
>  Components: ipc, master, migration, regionserver
>Reporter: Jimmy Xiang
>Assignee: Jimmy Xiang
> Fix For: 0.96.0
>
> Attachments: hbase-5620-sec.patch, hbase-5620_v3.patch, 
> hbase-5620_v4.patch, hbase-5620_v4.patch
>
>


--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5733) AssignmentManager#processDeadServersAndRegionsInTransition can fail with NPE.

[Uma Maheswara Rao G (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-5733?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Uma Maheswara Rao G updated HBASE-5733:
---

Attachment: HBASE-5733.patch

Attached the same patch as previous, with removal of FATAL log.

> AssignmentManager#processDeadServersAndRegionsInTransition can fail with NPE.
> -
>
> Key: HBASE-5733
> URL: https://issues.apache.org/jira/browse/HBASE-5733
> Project: HBase
>  Issue Type: Bug
>  Components: master
>Affects Versions: 0.96.0
>Reporter: Uma Maheswara Rao G
>Assignee: Uma Maheswara Rao G
> Attachments: HBASE-5733.patch, HBASE-5733.patch, HBASE-5733.patch
>
>
> Found while going through the code...
> AssignmentManager#processDeadServersAndRegionsInTransition can fail with NPE 
> as this is directly iterating the nodes from 
> listChildrenAndWatchForNewChildren with-out checking for null.
> Here also we need to handle with  null  check like other places.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5620) Convert the client protocol of HRegionInterface to PB

[Jimmy Xiang (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-5620?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13255704#comment-13255704
 ] 

Jimmy Xiang commented on HBASE-5620:


@Stack, not every invocation will throw an exception.  In case it throws an 
exception, it should be a ServiceException for pb.  It used to be IOException.  
Without the change, for pb calls, it won't get a ServiceException in case 
something goes wrong.  It gets an undeclared exception
with the cause to be an IOE, and the upper layer doesn't know how to handle it.

The Set in Invocation is used to decide if a protocol a pb one, so 
ServiceException should be used.  I put it there because it is
used for both WritableRpcEngine and SecureRpcEngine.


> Convert the client protocol of HRegionInterface to PB
> -
>
> Key: HBASE-5620
> URL: https://issues.apache.org/jira/browse/HBASE-5620
> Project: HBase
>  Issue Type: Sub-task
>  Components: ipc, master, migration, regionserver
>Reporter: Jimmy Xiang
>Assignee: Jimmy Xiang
> Fix For: 0.96.0
>
> Attachments: hbase-5620-sec.patch, hbase-5620_v3.patch, 
> hbase-5620_v4.patch, hbase-5620_v4.patch
>
>


--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5782) Edits can be appended out of seqid order since HBASE-4487

[Lars Hofhansl (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-5782?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lars Hofhansl updated HBASE-5782:
-

Attachment: 5782-lars-v2.txt

In the interest of keeping this simple, here's another simple patch.

This patch assumes that:
# It is only the appends that need to happen in order.
# The sync is the time consuming operation.

So a special lock is only held for enforce the ordering of the appends (appends 
are single threaded) and syncs can happen in parallel without this lock held.

This should give us the maximum concurrency possible, while keeping the change 
small and palletable for 0.94.

Please let me know whether I am smoking something.

> Edits can be appended out of seqid order since HBASE-4487
> -
>
> Key: HBASE-5782
> URL: https://issues.apache.org/jira/browse/HBASE-5782
> Project: HBase
>  Issue Type: Bug
>  Components: wal
>Affects Versions: 0.94.0
>Reporter: Gopinathan A
>Assignee: ramkrishna.s.vasudevan
>Priority: Blocker
> Fix For: 0.94.0
>
> Attachments: 5782-lars-v2.txt, 5782-sketch.txt, 5782.txt, 
> 5782.unfinished-stack.txt, HBASE-5782.patch, hbase-5782.txt
>
>
> Create a table with 1000 splits, after the region assignemnt, kill the 
> regionserver wich contains META table.
> Here few regions are missing after the log splitting and region assigment. 
> HBCK report shows multiple region holes are got created.
> Same scenario was verified mulitple times in 0.92.1, no issues.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5808) Remove TestHLogBench, HLogPerformanceEvaluation is more comprehensive

[Hudson (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-5808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13255715#comment-13255715
 ] 

Hudson commented on HBASE-5808:
---

Integrated in HBase-TRUNK #2774 (See 
[https://builds.apache.org/job/HBase-TRUNK/2774/])
HBASE-5808 Remove TestHLogBench, HLogPerformanceEvaluation is more 
comprehensive (Revision 1327143)

 Result = FAILURE
stack : 
Files : 
* 
/hbase/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/wal/TestHLogBench.java


> Remove TestHLogBench, HLogPerformanceEvaluation is more comprehensive
> -
>
> Key: HBASE-5808
> URL: https://issues.apache.org/jira/browse/HBASE-5808
> Project: HBase
>  Issue Type: Task
>Reporter: stack
>Assignee: stack
> Fix For: 0.96.0
>
> Attachments: rm.txt
>
>
> We added HLogPerformanceEvaluation recently though we had TestHLogBench which 
> does much the same thing (Didn't notice).  Remove TestHLogBench rather than 
> HLogPerformanceEvaluation because the latter does more (has a verify step and 
> can take more options; also TestHLogBench does not actually write a log)

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5545) region can't be opened for a long time. Because the creating File failed.

[Lars Hofhansl (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-5545?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13255714#comment-13255714
 ] 

Lars Hofhansl commented on HBASE-5545:
--

If this gets done before HBASE-5782 I'll pull it in, otherwise I don't think I 
should hold up the release for this. Sounds good?


> region can't be opened for a long time. Because the creating File failed.
> -
>
> Key: HBASE-5545
> URL: https://issues.apache.org/jira/browse/HBASE-5545
> Project: HBase
>  Issue Type: Bug
>  Components: regionserver
>Affects Versions: 0.90.6
>Reporter: gaojinchao
>Assignee: gaojinchao
> Fix For: 0.90.7, 0.92.2, 0.94.1
>
>
> Scenario:
> 
> 1. File is created 
> 2. But while writing data, all datanodes might have crashed. So writing data 
> will fail.
> 3. Now even if close is called in finally block, close also will fail and 
> throw the Exception because writing data failed.
> 4. After this if RS try to create the same file again, then 
> AlreadyBeingCreatedException will come.
> Suggestion to handle this scenario.
> ---
> 1. Check for the existence of the file, if exists delete the file and create 
> new file. 
> Here delete call for the file will not check whether the file is open or 
> closed.
> Overwrite Option:
> 
> 1. Overwrite option will be applicable if you are trying to overwrite a 
> closed file.
> 2. If the file is not closed, then even with overwrite option Same 
> AlreadyBeingCreatedException will be thrown.
> This is the expected behaviour to avoid the Multiple clients writing to same 
> file.
> Region server logs:
> org.apache.hadoop.ipc.RemoteException: 
> org.apache.hadoop.hdfs.protocol.AlreadyBeingCreatedException: failed to 
> create file /hbase/test1/12c01902324218d14b17a5880f24f64b/.tmp/.regioninfo 
> for 
> DFSClient_hb_rs_158-1-131-48,20020,1331107668635_1331107669061_-252463556_25 
> on client 158.1.132.19 because current leaseholder is trying to recreate file.
> at 
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.recoverLeaseInternal(FSNamesystem.java:1570)
> at 
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.startFileInternal(FSNamesystem.java:1440)
> at 
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.startFile(FSNamesystem.java:1382)
> at org.apache.hadoop.hdfs.server.namenode.NameNode.create(NameNode.java:658)
> at sun.reflect.GeneratedMethodAccessor9.invoke(Unknown Source)
> at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:547)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1137)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1133)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:396)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1131)
> at org.apache.hadoop.ipc.Client.call(Client.java:961)
> at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:245)
> at $Proxy6.create(Unknown Source)
> at sun.reflect.GeneratedMethodAccessor14.invoke(Unknown Source)
> at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at $Proxy6.create(Unknown Source)
> at 
> org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.(DFSClient.java:3643)
> at org.apache.hadoop.hdfs.DFSClient.create(DFSClient.java:778)
> at 
> org.apache.hadoop.hdfs.DistributedFileSystem.create(DistributedFileSystem.java:364)
> at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:630)
> at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:611)
> at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:518)
> at 
> org.apache.hadoop.hbase.regionserver.HRegion.checkRegioninfoOnFilesystem(HRegion.java:424)
> at org.apache.hadoop.hbase.regionserver.HRegion.initialize(HRegion.java:340)
> at org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:2672)
> at org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:2658)
> at 
> org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.openRegion(OpenRegionHandler.java:330)
> at 
> org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.process(OpenRegionHandler.java:116)
> at org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:158)
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:662)
> [2012-03-07 20:51:45,858] [WARN ] 
> [RS_OPEN_REGION-158-1-131-48,20020,1331107668635-23] 
> [com

[jira] [Commented] (HBASE-5654) [findbugs] Address dodgy bugs

[Jonathan Hsieh (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-5654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13255719#comment-13255719
 ] 

Jonathan Hsieh commented on HBASE-5654:
---

There are a few match related entries:

Here's how one could likely be fixed.

{code}
  public static long calculateOverhead(long maxSize, long blockSize, int 
concurrency){
// FindBugs ICAST_INTEGER_MULTIPLY_CAST_TO_LONG
return CACHE_FIXED_OVERHEAD + ClassSize.CONCURRENT_HASHMAP +
((long)Math.ceil(maxSize*1.2/blockSize)
* ClassSize.CONCURRENT_HASHMAP_ENTRY) +
  //  (concurrency * ClassSize.CONCURRENT_HASHMAP_SEGMENT);
  ((long)concurrency * (long)ClassSize.CONCURRENT_HASHMAP_SEGMENT);
  }
{code}

For bounds checks, I think there was another where a double gets converted to a 
long (what happens is double is larger than max long?)

> [findbugs] Address dodgy bugs
> -
>
> Key: HBASE-5654
> URL: https://issues.apache.org/jira/browse/HBASE-5654
> Project: HBase
>  Issue Type: Sub-task
>  Components: scripts
>Affects Versions: 0.96.0
>Reporter: Jonathan Hsieh
>Assignee: Ashutosh Jindal
>  Labels: patch
> Fix For: 0.96.0
>
> Attachments: Hbase-5654.patch
>
>
> See 
> https://builds.apache.org/job/PreCommit-HBASE-Build/1313//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html#Warnings_STYLE
> This may be broken down further.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5782) Edits can be appended out of seqid order since HBASE-4487

[Todd Lipcon (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-5782?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13255718#comment-13255718
 ] 

Todd Lipcon commented on HBASE-5782:


Don't think you're smoking something. It looks reasonable, though I think there 
are still races around the update of doneUpTo, syncedTilHere, etc. These races 
may be "safe" in that they only result in too-low values (and thus extra 
needless syncs) but I'm not 100% sure of it. Maybe we can extend the 
verification test case to run as a stress test, including rolls, and run it 
overnight or something?

> Edits can be appended out of seqid order since HBASE-4487
> -
>
> Key: HBASE-5782
> URL: https://issues.apache.org/jira/browse/HBASE-5782
> Project: HBase
>  Issue Type: Bug
>  Components: wal
>Affects Versions: 0.94.0
>Reporter: Gopinathan A
>Assignee: ramkrishna.s.vasudevan
>Priority: Blocker
> Fix For: 0.94.0
>
> Attachments: 5782-lars-v2.txt, 5782-sketch.txt, 5782.txt, 
> 5782.unfinished-stack.txt, HBASE-5782.patch, hbase-5782.txt
>
>
> Create a table with 1000 splits, after the region assignemnt, kill the 
> regionserver wich contains META table.
> Here few regions are missing after the log splitting and region assigment. 
> HBCK report shows multiple region holes are got created.
> Same scenario was verified mulitple times in 0.92.1, no issues.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5733) AssignmentManager#processDeadServersAndRegionsInTransition can fail with NPE.

[Hadoop QA (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-5733?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13255732#comment-13255732
 ] 

Hadoop QA commented on HBASE-5733:
--

-1 overall.  Here are the results of testing the latest attachment 
  http://issues.apache.org/jira/secure/attachment/12522970/HBASE-5733.patch
  against trunk revision .

+1 @author.  The patch does not contain any @author tags.

+1 tests included.  The patch appears to include 3 new or modified tests.

+1 javadoc.  The javadoc tool did not generate any warning messages.

+1 javac.  The applied patch does not increase the total number of javac 
compiler warnings.

-1 findbugs.  The patch appears to introduce 4 new Findbugs (version 1.3.9) 
warnings.

+1 release audit.  The applied patch does not increase the total number of 
release audit warnings.

 -1 core tests.  The patch failed these unit tests:
 

Test results: 
https://builds.apache.org/job/PreCommit-HBASE-Build/1550//testReport/
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/1550//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
Console output: 
https://builds.apache.org/job/PreCommit-HBASE-Build/1550//console

This message is automatically generated.

> AssignmentManager#processDeadServersAndRegionsInTransition can fail with NPE.
> -
>
> Key: HBASE-5733
> URL: https://issues.apache.org/jira/browse/HBASE-5733
> Project: HBase
>  Issue Type: Bug
>  Components: master
>Affects Versions: 0.96.0
>Reporter: Uma Maheswara Rao G
>Assignee: Uma Maheswara Rao G
> Attachments: HBASE-5733.patch, HBASE-5733.patch, HBASE-5733.patch
>
>
> Found while going through the code...
> AssignmentManager#processDeadServersAndRegionsInTransition can fail with NPE 
> as this is directly iterating the nodes from 
> listChildrenAndWatchForNewChildren with-out checking for null.
> Here also we need to handle with  null  check like other places.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5798) NPE running hbck on 0.94 out of reportTablesInFlux

[Anoop Sam John (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-5798?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Anoop Sam John updated HBASE-5798:
--

Attachment: HBASE-5798_trunk.patch
HBASE-5798_94.patch

Patch for trunk and 0.94

> NPE running hbck on 0.94 out of reportTablesInFlux
> --
>
> Key: HBASE-5798
> URL: https://issues.apache.org/jira/browse/HBASE-5798
> Project: HBase
>  Issue Type: Bug
>  Components: hbck
>Affects Versions: 0.94.0, 0.96.0
>Reporter: stack
>Assignee: Anoop Sam John
> Attachments: HBASE-5798_94.patch, HBASE-5798_trunk.patch
>
>
> Got this playing w/ hbck going against the 0.94RC:
> {code}
> 12/04/16 17:03:14 INFO util.HBaseFsck: getHTableDescriptors == tableNames => 
> []
> Exception in thread "main" java.lang.NullPointerException
> at 
> org.apache.hadoop.hbase.util.HBaseFsck.reportTablesInFlux(HBaseFsck.java:553)
> at 
> org.apache.hadoop.hbase.util.HBaseFsck.onlineConsistencyRepair(HBaseFsck.java:344)
> at 
> org.apache.hadoop.hbase.util.HBaseFsck.onlineHbck(HBaseFsck.java:380)
> at org.apache.hadoop.hbase.util.HBaseFsck.main(HBaseFsck.java:3033)
> {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-3585) isLegalFamilyName() can throw ArrayOutOfBoundException

[Uma Maheswara Rao G (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-3585?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Uma Maheswara Rao G updated HBASE-3585:
---

Attachment: HBASE-3585.patch

> isLegalFamilyName() can throw ArrayOutOfBoundException
> --
>
> Key: HBASE-3585
> URL: https://issues.apache.org/jira/browse/HBASE-3585
> Project: HBase
>  Issue Type: Bug
>  Components: client
>Affects Versions: 0.90.1
>Reporter: Prakash Khemani
>Assignee: Uma Maheswara Rao G
>Priority: Minor
> Attachments: HBASE-3585.patch
>
>
> org.apache.hadoop.hbase.HColumnDescriptor.isLegalFamilyName(byte[]) accesses 
> byte[0] w/o first checking the array length.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5782) Edits can be appended out of seqid order since HBASE-4487

[Hadoop QA (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-5782?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13255760#comment-13255760
 ] 

Hadoop QA commented on HBASE-5782:
--

-1 overall.  Here are the results of testing the latest attachment 
  http://issues.apache.org/jira/secure/attachment/12522975/5782-lars-v2.txt
  against trunk revision .

+1 @author.  The patch does not contain any @author tags.

-1 tests included.  The patch doesn't appear to include any new or modified 
tests.
Please justify why no new tests are needed for this 
patch.
Also please list what manual steps were performed to 
verify this patch.

+1 javadoc.  The javadoc tool did not generate any warning messages.

+1 javac.  The applied patch does not increase the total number of javac 
compiler warnings.

-1 findbugs.  The patch appears to introduce 4 new Findbugs (version 1.3.9) 
warnings.

+1 release audit.  The applied patch does not increase the total number of 
release audit warnings.

 -1 core tests.  The patch failed these unit tests:
 

Test results: 
https://builds.apache.org/job/PreCommit-HBASE-Build/1551//testReport/
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/1551//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
Console output: 
https://builds.apache.org/job/PreCommit-HBASE-Build/1551//console

This message is automatically generated.

> Edits can be appended out of seqid order since HBASE-4487
> -
>
> Key: HBASE-5782
> URL: https://issues.apache.org/jira/browse/HBASE-5782
> Project: HBase
>  Issue Type: Bug
>  Components: wal
>Affects Versions: 0.94.0
>Reporter: Gopinathan A
>Assignee: ramkrishna.s.vasudevan
>Priority: Blocker
> Fix For: 0.94.0
>
> Attachments: 5782-lars-v2.txt, 5782-sketch.txt, 5782.txt, 
> 5782.unfinished-stack.txt, HBASE-5782.patch, hbase-5782.txt
>
>
> Create a table with 1000 splits, after the region assignemnt, kill the 
> regionserver wich contains META table.
> Here few regions are missing after the log splitting and region assigment. 
> HBCK report shows multiple region holes are got created.
> Same scenario was verified mulitple times in 0.92.1, no issues.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-3585) isLegalFamilyName() can throw ArrayOutOfBoundException

[Uma Maheswara Rao G (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-3585?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Uma Maheswara Rao G updated HBASE-3585:
---

Status: Patch Available  (was: Open)

Still the ctor wraps it to empty array only. SO, it can throw AIOE in 
isLegalFamilyName.

Added the preconditions check for empty FamilyName. It will throw 
IllegalArgumentException if the familyName empty.

> isLegalFamilyName() can throw ArrayOutOfBoundException
> --
>
> Key: HBASE-3585
> URL: https://issues.apache.org/jira/browse/HBASE-3585
> Project: HBase
>  Issue Type: Bug
>  Components: client
>Affects Versions: 0.90.1
>Reporter: Prakash Khemani
>Assignee: Uma Maheswara Rao G
>Priority: Minor
> Attachments: HBASE-3585.patch
>
>
> org.apache.hadoop.hbase.HColumnDescriptor.isLegalFamilyName(byte[]) accesses 
> byte[0] w/o first checking the array length.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5732) Remove the SecureRPCEngine and merge the security-related logic in the core engine

[Devaraj Das (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-5732?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13255765#comment-13255765
 ] 

Devaraj Das commented on HBASE-5732:


bq. I'm all for removing shims and hacks wherever possible, but if we end up 
here again then this just churns our API along with Hadoop core.

Andrew, over in Hadoop, there is a discussion around making some of the APIs in 
UGI public (and bound to contracts :-) ) but maybe you are right - things could 
change even then.

I'll revert User.java and have it only support hadoop-1.0++. Sounds reasonable?

> Remove the SecureRPCEngine and merge the security-related logic in the core 
> engine
> --
>
> Key: HBASE-5732
> URL: https://issues.apache.org/jira/browse/HBASE-5732
> Project: HBase
>  Issue Type: Improvement
>Reporter: Devaraj Das
> Attachments: rpcengine-merge.patch
>
>
> Remove the SecureRPCEngine and merge the security-related logic in the core 
> engine. Follow up to HBASE-5727.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5732) Remove the SecureRPCEngine and merge the security-related logic in the core engine

[Devaraj Das (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-5732?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13255767#comment-13255767
 ] 

Devaraj Das commented on HBASE-5732:


bq. over in Hadoop, there is a discussion around making some of the APIs in UGI 
public
The Hadoop jira is HADOOP-8152

> Remove the SecureRPCEngine and merge the security-related logic in the core 
> engine
> --
>
> Key: HBASE-5732
> URL: https://issues.apache.org/jira/browse/HBASE-5732
> Project: HBase
>  Issue Type: Improvement
>Reporter: Devaraj Das
> Attachments: rpcengine-merge.patch
>
>
> Remove the SecureRPCEngine and merge the security-related logic in the core 
> engine. Follow up to HBASE-5727.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5545) region can't be opened for a long time. Because the creating File failed.

[ramkrishna.s.vasudevan (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-5545?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

ramkrishna.s.vasudevan updated HBASE-5545:
--

Status: Patch Available  (was: Open)

> region can't be opened for a long time. Because the creating File failed.
> -
>
> Key: HBASE-5545
> URL: https://issues.apache.org/jira/browse/HBASE-5545
> Project: HBase
>  Issue Type: Bug
>  Components: regionserver
>Affects Versions: 0.90.6
>Reporter: gaojinchao
>Assignee: gaojinchao
> Fix For: 0.90.7, 0.92.2, 0.94.1
>
> Attachments: HBASE-5545.patch
>
>
> Scenario:
> 
> 1. File is created 
> 2. But while writing data, all datanodes might have crashed. So writing data 
> will fail.
> 3. Now even if close is called in finally block, close also will fail and 
> throw the Exception because writing data failed.
> 4. After this if RS try to create the same file again, then 
> AlreadyBeingCreatedException will come.
> Suggestion to handle this scenario.
> ---
> 1. Check for the existence of the file, if exists delete the file and create 
> new file. 
> Here delete call for the file will not check whether the file is open or 
> closed.
> Overwrite Option:
> 
> 1. Overwrite option will be applicable if you are trying to overwrite a 
> closed file.
> 2. If the file is not closed, then even with overwrite option Same 
> AlreadyBeingCreatedException will be thrown.
> This is the expected behaviour to avoid the Multiple clients writing to same 
> file.
> Region server logs:
> org.apache.hadoop.ipc.RemoteException: 
> org.apache.hadoop.hdfs.protocol.AlreadyBeingCreatedException: failed to 
> create file /hbase/test1/12c01902324218d14b17a5880f24f64b/.tmp/.regioninfo 
> for 
> DFSClient_hb_rs_158-1-131-48,20020,1331107668635_1331107669061_-252463556_25 
> on client 158.1.132.19 because current leaseholder is trying to recreate file.
> at 
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.recoverLeaseInternal(FSNamesystem.java:1570)
> at 
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.startFileInternal(FSNamesystem.java:1440)
> at 
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.startFile(FSNamesystem.java:1382)
> at org.apache.hadoop.hdfs.server.namenode.NameNode.create(NameNode.java:658)
> at sun.reflect.GeneratedMethodAccessor9.invoke(Unknown Source)
> at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:547)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1137)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1133)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:396)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1131)
> at org.apache.hadoop.ipc.Client.call(Client.java:961)
> at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:245)
> at $Proxy6.create(Unknown Source)
> at sun.reflect.GeneratedMethodAccessor14.invoke(Unknown Source)
> at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at $Proxy6.create(Unknown Source)
> at 
> org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.(DFSClient.java:3643)
> at org.apache.hadoop.hdfs.DFSClient.create(DFSClient.java:778)
> at 
> org.apache.hadoop.hdfs.DistributedFileSystem.create(DistributedFileSystem.java:364)
> at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:630)
> at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:611)
> at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:518)
> at 
> org.apache.hadoop.hbase.regionserver.HRegion.checkRegioninfoOnFilesystem(HRegion.java:424)
> at org.apache.hadoop.hbase.regionserver.HRegion.initialize(HRegion.java:340)
> at org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:2672)
> at org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:2658)
> at 
> org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.openRegion(OpenRegionHandler.java:330)
> at 
> org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.process(OpenRegionHandler.java:116)
> at org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:158)
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:662)
> [2012-03-07 20:51:45,858] [WARN ] 
> [RS_OPEN_REGION-158-1-131-48,20020,1331107668635-23] 
> [com.huawei.isap.ump.ha.client.RPCRetryAndSwitchInvoker 131] Retrying the 
> method call: public abstra

[jira] [Updated] (HBASE-5545) region can't be opened for a long time. Because the creating File failed.

[ramkrishna.s.vasudevan (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-5545?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

ramkrishna.s.vasudevan updated HBASE-5545:
--

Attachment: HBASE-5545.patch

Patch for 0.94.

> region can't be opened for a long time. Because the creating File failed.
> -
>
> Key: HBASE-5545
> URL: https://issues.apache.org/jira/browse/HBASE-5545
> Project: HBase
>  Issue Type: Bug
>  Components: regionserver
>Affects Versions: 0.90.6
>Reporter: gaojinchao
>Assignee: gaojinchao
> Fix For: 0.90.7, 0.92.2, 0.94.1
>
> Attachments: HBASE-5545.patch
>
>
> Scenario:
> 
> 1. File is created 
> 2. But while writing data, all datanodes might have crashed. So writing data 
> will fail.
> 3. Now even if close is called in finally block, close also will fail and 
> throw the Exception because writing data failed.
> 4. After this if RS try to create the same file again, then 
> AlreadyBeingCreatedException will come.
> Suggestion to handle this scenario.
> ---
> 1. Check for the existence of the file, if exists delete the file and create 
> new file. 
> Here delete call for the file will not check whether the file is open or 
> closed.
> Overwrite Option:
> 
> 1. Overwrite option will be applicable if you are trying to overwrite a 
> closed file.
> 2. If the file is not closed, then even with overwrite option Same 
> AlreadyBeingCreatedException will be thrown.
> This is the expected behaviour to avoid the Multiple clients writing to same 
> file.
> Region server logs:
> org.apache.hadoop.ipc.RemoteException: 
> org.apache.hadoop.hdfs.protocol.AlreadyBeingCreatedException: failed to 
> create file /hbase/test1/12c01902324218d14b17a5880f24f64b/.tmp/.regioninfo 
> for 
> DFSClient_hb_rs_158-1-131-48,20020,1331107668635_1331107669061_-252463556_25 
> on client 158.1.132.19 because current leaseholder is trying to recreate file.
> at 
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.recoverLeaseInternal(FSNamesystem.java:1570)
> at 
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.startFileInternal(FSNamesystem.java:1440)
> at 
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.startFile(FSNamesystem.java:1382)
> at org.apache.hadoop.hdfs.server.namenode.NameNode.create(NameNode.java:658)
> at sun.reflect.GeneratedMethodAccessor9.invoke(Unknown Source)
> at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:547)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1137)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1133)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:396)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1131)
> at org.apache.hadoop.ipc.Client.call(Client.java:961)
> at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:245)
> at $Proxy6.create(Unknown Source)
> at sun.reflect.GeneratedMethodAccessor14.invoke(Unknown Source)
> at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at $Proxy6.create(Unknown Source)
> at 
> org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.(DFSClient.java:3643)
> at org.apache.hadoop.hdfs.DFSClient.create(DFSClient.java:778)
> at 
> org.apache.hadoop.hdfs.DistributedFileSystem.create(DistributedFileSystem.java:364)
> at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:630)
> at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:611)
> at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:518)
> at 
> org.apache.hadoop.hbase.regionserver.HRegion.checkRegioninfoOnFilesystem(HRegion.java:424)
> at org.apache.hadoop.hbase.regionserver.HRegion.initialize(HRegion.java:340)
> at org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:2672)
> at org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:2658)
> at 
> org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.openRegion(OpenRegionHandler.java:330)
> at 
> org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.process(OpenRegionHandler.java:116)
> at org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:158)
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:662)
> [2012-03-07 20:51:45,858] [WARN ] 
> [RS_OPEN_REGION-158-1-131-48,20020,1331107668635-23] 
> [com.huawei.isap.ump.ha.client.RPCRetryAndSwitchInvoker 131] Retrying the 
> metho

[jira] [Commented] (HBASE-5782) Edits can be appended out of seqid order since HBASE-4487

[Lars Hofhansl (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-5782?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13255770#comment-13255770
 ] 

Lars Hofhansl commented on HBASE-5782:
--

Yep. It's crucial that syncTillHere is updated last.
The pendingWrites are appended strictly in order, so there is a very short race 
that might lead to sync be issued multiple time when only one was necessary (it 
seems the same race condition existed before).
So I think it is safe and low risk.

The question now is: Do this for 0.94 and then a more elaborate rewrite in 
trunk, or do a more risky rewrite for 0.94?


> Edits can be appended out of seqid order since HBASE-4487
> -
>
> Key: HBASE-5782
> URL: https://issues.apache.org/jira/browse/HBASE-5782
> Project: HBase
>  Issue Type: Bug
>  Components: wal
>Affects Versions: 0.94.0
>Reporter: Gopinathan A
>Assignee: ramkrishna.s.vasudevan
>Priority: Blocker
> Fix For: 0.94.0
>
> Attachments: 5782-lars-v2.txt, 5782-sketch.txt, 5782.txt, 
> 5782.unfinished-stack.txt, HBASE-5782.patch, hbase-5782.txt
>
>
> Create a table with 1000 splits, after the region assignemnt, kill the 
> regionserver wich contains META table.
> Here few regions are missing after the log splitting and region assigment. 
> HBCK report shows multiple region holes are got created.
> Same scenario was verified mulitple times in 0.92.1, no issues.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (HBASE-5810) HBASE-5620 Convert the client protocol of HRegionInterface to PB addendum

[stack (Created) (JIRA)]

HBASE-5620 Convert the client protocol of HRegionInterface to PB addendum
-

 Key: HBASE-5810
 URL: https://issues.apache.org/jira/browse/HBASE-5810
 Project: HBase
  Issue Type: Task
Reporter: stack
 Attachments: hbase-5620-sec.patch

Apply an addendum that Jimmy made here rather than over in hbase-5620, a week 
or so after it went in.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5810) HBASE-5620 Convert the client protocol of HRegionInterface to PB addendum

[stack (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-5810?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stack updated HBASE-5810:
-

Attachment: hbase-5620-sec.patch

Copied over from hbase-5620

> HBASE-5620 Convert the client protocol of HRegionInterface to PB addendum
> -
>
> Key: HBASE-5810
> URL: https://issues.apache.org/jira/browse/HBASE-5810
> Project: HBase
>  Issue Type: Task
>Reporter: stack
> Attachments: hbase-5620-sec.patch
>
>
> Apply an addendum that Jimmy made here rather than over in hbase-5620, a week 
> or so after it went in.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5620) Convert the client protocol of HRegionInterface to PB

[stack (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-5620?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13255778#comment-13255778
 ] 

stack commented on HBASE-5620:
--

I made HBASE-5810 to apply this Jimmy.   Good stuff.

> Convert the client protocol of HRegionInterface to PB
> -
>
> Key: HBASE-5620
> URL: https://issues.apache.org/jira/browse/HBASE-5620
> Project: HBase
>  Issue Type: Sub-task
>  Components: ipc, master, migration, regionserver
>Reporter: Jimmy Xiang
>Assignee: Jimmy Xiang
> Fix For: 0.96.0
>
> Attachments: hbase-5620-sec.patch, hbase-5620_v3.patch, 
> hbase-5620_v4.patch, hbase-5620_v4.patch
>
>


--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5810) HBASE-5620 Convert the client protocol of HRegionInterface to PB addendum

[stack (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-5810?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stack updated HBASE-5810:
-

Fix Version/s: 0.96.0
 Assignee: Jimmy Xiang
 Hadoop Flags: Reviewed
   Status: Patch Available  (was: Open)

> HBASE-5620 Convert the client protocol of HRegionInterface to PB addendum
> -
>
> Key: HBASE-5810
> URL: https://issues.apache.org/jira/browse/HBASE-5810
> Project: HBase
>  Issue Type: Task
>Reporter: stack
>Assignee: Jimmy Xiang
> Fix For: 0.96.0
>
> Attachments: hbase-5620-sec.patch
>
>
> Apply an addendum that Jimmy made here rather than over in hbase-5620, a week 
> or so after it went in.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5545) region can't be opened for a long time. Because the creating File failed.

[Lars Hofhansl (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-5545?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13255779#comment-13255779
 ] 

Lars Hofhansl commented on HBASE-5545:
--

+1 on patch. I assume there no strange race conditions in this part of the code.

> region can't be opened for a long time. Because the creating File failed.
> -
>
> Key: HBASE-5545
> URL: https://issues.apache.org/jira/browse/HBASE-5545
> Project: HBase
>  Issue Type: Bug
>  Components: regionserver
>Affects Versions: 0.90.6
>Reporter: gaojinchao
>Assignee: gaojinchao
> Fix For: 0.90.7, 0.92.2, 0.94.0
>
> Attachments: HBASE-5545.patch
>
>
> Scenario:
> 
> 1. File is created 
> 2. But while writing data, all datanodes might have crashed. So writing data 
> will fail.
> 3. Now even if close is called in finally block, close also will fail and 
> throw the Exception because writing data failed.
> 4. After this if RS try to create the same file again, then 
> AlreadyBeingCreatedException will come.
> Suggestion to handle this scenario.
> ---
> 1. Check for the existence of the file, if exists delete the file and create 
> new file. 
> Here delete call for the file will not check whether the file is open or 
> closed.
> Overwrite Option:
> 
> 1. Overwrite option will be applicable if you are trying to overwrite a 
> closed file.
> 2. If the file is not closed, then even with overwrite option Same 
> AlreadyBeingCreatedException will be thrown.
> This is the expected behaviour to avoid the Multiple clients writing to same 
> file.
> Region server logs:
> org.apache.hadoop.ipc.RemoteException: 
> org.apache.hadoop.hdfs.protocol.AlreadyBeingCreatedException: failed to 
> create file /hbase/test1/12c01902324218d14b17a5880f24f64b/.tmp/.regioninfo 
> for 
> DFSClient_hb_rs_158-1-131-48,20020,1331107668635_1331107669061_-252463556_25 
> on client 158.1.132.19 because current leaseholder is trying to recreate file.
> at 
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.recoverLeaseInternal(FSNamesystem.java:1570)
> at 
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.startFileInternal(FSNamesystem.java:1440)
> at 
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.startFile(FSNamesystem.java:1382)
> at org.apache.hadoop.hdfs.server.namenode.NameNode.create(NameNode.java:658)
> at sun.reflect.GeneratedMethodAccessor9.invoke(Unknown Source)
> at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:547)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1137)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1133)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:396)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1131)
> at org.apache.hadoop.ipc.Client.call(Client.java:961)
> at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:245)
> at $Proxy6.create(Unknown Source)
> at sun.reflect.GeneratedMethodAccessor14.invoke(Unknown Source)
> at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at $Proxy6.create(Unknown Source)
> at 
> org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.(DFSClient.java:3643)
> at org.apache.hadoop.hdfs.DFSClient.create(DFSClient.java:778)
> at 
> org.apache.hadoop.hdfs.DistributedFileSystem.create(DistributedFileSystem.java:364)
> at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:630)
> at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:611)
> at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:518)
> at 
> org.apache.hadoop.hbase.regionserver.HRegion.checkRegioninfoOnFilesystem(HRegion.java:424)
> at org.apache.hadoop.hbase.regionserver.HRegion.initialize(HRegion.java:340)
> at org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:2672)
> at org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:2658)
> at 
> org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.openRegion(OpenRegionHandler.java:330)
> at 
> org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.process(OpenRegionHandler.java:116)
> at org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:158)
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:662)
> [2012-03-07 20:51:45,858] [WARN ] 
> [RS_OPEN_REGION-158-1-131-48,20020,1331107668635-23] 
> [com.huawei.

[jira] [Updated] (HBASE-5545) region can't be opened for a long time. Because the creating File failed.

[Lars Hofhansl (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-5545?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lars Hofhansl updated HBASE-5545:
-

Fix Version/s: (was: 0.94.1)
   0.94.0

> region can't be opened for a long time. Because the creating File failed.
> -
>
> Key: HBASE-5545
> URL: https://issues.apache.org/jira/browse/HBASE-5545
> Project: HBase
>  Issue Type: Bug
>  Components: regionserver
>Affects Versions: 0.90.6
>Reporter: gaojinchao
>Assignee: gaojinchao
> Fix For: 0.90.7, 0.92.2, 0.94.0
>
> Attachments: HBASE-5545.patch
>
>
> Scenario:
> 
> 1. File is created 
> 2. But while writing data, all datanodes might have crashed. So writing data 
> will fail.
> 3. Now even if close is called in finally block, close also will fail and 
> throw the Exception because writing data failed.
> 4. After this if RS try to create the same file again, then 
> AlreadyBeingCreatedException will come.
> Suggestion to handle this scenario.
> ---
> 1. Check for the existence of the file, if exists delete the file and create 
> new file. 
> Here delete call for the file will not check whether the file is open or 
> closed.
> Overwrite Option:
> 
> 1. Overwrite option will be applicable if you are trying to overwrite a 
> closed file.
> 2. If the file is not closed, then even with overwrite option Same 
> AlreadyBeingCreatedException will be thrown.
> This is the expected behaviour to avoid the Multiple clients writing to same 
> file.
> Region server logs:
> org.apache.hadoop.ipc.RemoteException: 
> org.apache.hadoop.hdfs.protocol.AlreadyBeingCreatedException: failed to 
> create file /hbase/test1/12c01902324218d14b17a5880f24f64b/.tmp/.regioninfo 
> for 
> DFSClient_hb_rs_158-1-131-48,20020,1331107668635_1331107669061_-252463556_25 
> on client 158.1.132.19 because current leaseholder is trying to recreate file.
> at 
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.recoverLeaseInternal(FSNamesystem.java:1570)
> at 
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.startFileInternal(FSNamesystem.java:1440)
> at 
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.startFile(FSNamesystem.java:1382)
> at org.apache.hadoop.hdfs.server.namenode.NameNode.create(NameNode.java:658)
> at sun.reflect.GeneratedMethodAccessor9.invoke(Unknown Source)
> at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:547)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1137)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1133)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:396)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1131)
> at org.apache.hadoop.ipc.Client.call(Client.java:961)
> at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:245)
> at $Proxy6.create(Unknown Source)
> at sun.reflect.GeneratedMethodAccessor14.invoke(Unknown Source)
> at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at $Proxy6.create(Unknown Source)
> at 
> org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.(DFSClient.java:3643)
> at org.apache.hadoop.hdfs.DFSClient.create(DFSClient.java:778)
> at 
> org.apache.hadoop.hdfs.DistributedFileSystem.create(DistributedFileSystem.java:364)
> at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:630)
> at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:611)
> at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:518)
> at 
> org.apache.hadoop.hbase.regionserver.HRegion.checkRegioninfoOnFilesystem(HRegion.java:424)
> at org.apache.hadoop.hbase.regionserver.HRegion.initialize(HRegion.java:340)
> at org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:2672)
> at org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:2658)
> at 
> org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.openRegion(OpenRegionHandler.java:330)
> at 
> org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.process(OpenRegionHandler.java:116)
> at org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:158)
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:662)
> [2012-03-07 20:51:45,858] [WARN ] 
> [RS_OPEN_REGION-158-1-131-48,20020,1331107668635-23] 
> [com.huawei.isap.ump.ha.client.RPCRetryAndSwitchInvoker 131] Retrying the 
> method call: public ab

[jira] [Commented] (HBASE-5545) region can't be opened for a long time. Because the creating File failed.

[Zhihong Yu (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-5545?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13255780#comment-13255780
 ] 

Zhihong Yu commented on HBASE-5545:
---

{code}
+// in the .tmp directory then on next time creation we will be getting
{code}
Should read 'then the next creation ...'
{code}
+  fs.delete(tmpPath, true);
{code}
Please check the return value from fs.delete().

> region can't be opened for a long time. Because the creating File failed.
> -
>
> Key: HBASE-5545
> URL: https://issues.apache.org/jira/browse/HBASE-5545
> Project: HBase
>  Issue Type: Bug
>  Components: regionserver
>Affects Versions: 0.90.6
>Reporter: gaojinchao
>Assignee: gaojinchao
> Fix For: 0.90.7, 0.92.2, 0.94.0
>
> Attachments: HBASE-5545.patch
>
>
> Scenario:
> 
> 1. File is created 
> 2. But while writing data, all datanodes might have crashed. So writing data 
> will fail.
> 3. Now even if close is called in finally block, close also will fail and 
> throw the Exception because writing data failed.
> 4. After this if RS try to create the same file again, then 
> AlreadyBeingCreatedException will come.
> Suggestion to handle this scenario.
> ---
> 1. Check for the existence of the file, if exists delete the file and create 
> new file. 
> Here delete call for the file will not check whether the file is open or 
> closed.
> Overwrite Option:
> 
> 1. Overwrite option will be applicable if you are trying to overwrite a 
> closed file.
> 2. If the file is not closed, then even with overwrite option Same 
> AlreadyBeingCreatedException will be thrown.
> This is the expected behaviour to avoid the Multiple clients writing to same 
> file.
> Region server logs:
> org.apache.hadoop.ipc.RemoteException: 
> org.apache.hadoop.hdfs.protocol.AlreadyBeingCreatedException: failed to 
> create file /hbase/test1/12c01902324218d14b17a5880f24f64b/.tmp/.regioninfo 
> for 
> DFSClient_hb_rs_158-1-131-48,20020,1331107668635_1331107669061_-252463556_25 
> on client 158.1.132.19 because current leaseholder is trying to recreate file.
> at 
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.recoverLeaseInternal(FSNamesystem.java:1570)
> at 
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.startFileInternal(FSNamesystem.java:1440)
> at 
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.startFile(FSNamesystem.java:1382)
> at org.apache.hadoop.hdfs.server.namenode.NameNode.create(NameNode.java:658)
> at sun.reflect.GeneratedMethodAccessor9.invoke(Unknown Source)
> at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:547)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1137)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1133)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:396)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1131)
> at org.apache.hadoop.ipc.Client.call(Client.java:961)
> at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:245)
> at $Proxy6.create(Unknown Source)
> at sun.reflect.GeneratedMethodAccessor14.invoke(Unknown Source)
> at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at $Proxy6.create(Unknown Source)
> at 
> org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.(DFSClient.java:3643)
> at org.apache.hadoop.hdfs.DFSClient.create(DFSClient.java:778)
> at 
> org.apache.hadoop.hdfs.DistributedFileSystem.create(DistributedFileSystem.java:364)
> at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:630)
> at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:611)
> at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:518)
> at 
> org.apache.hadoop.hbase.regionserver.HRegion.checkRegioninfoOnFilesystem(HRegion.java:424)
> at org.apache.hadoop.hbase.regionserver.HRegion.initialize(HRegion.java:340)
> at org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:2672)
> at org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:2658)
> at 
> org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.openRegion(OpenRegionHandler.java:330)
> at 
> org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.process(OpenRegionHandler.java:116)
> at org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:158)
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.l

[jira] [Commented] (HBASE-5545) region can't be opened for a long time. Because the creating File failed.

[Uma Maheswara Rao G (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-5545?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13255781#comment-13255781
 ] 

Uma Maheswara Rao G commented on HBASE-5545:


If that is the case, why can't we create the file with overwrite flag as true?. 
It will automatically deletes the file if exists right.

> region can't be opened for a long time. Because the creating File failed.
> -
>
> Key: HBASE-5545
> URL: https://issues.apache.org/jira/browse/HBASE-5545
> Project: HBase
>  Issue Type: Bug
>  Components: regionserver
>Affects Versions: 0.90.6
>Reporter: gaojinchao
>Assignee: gaojinchao
> Fix For: 0.90.7, 0.92.2, 0.94.0
>
> Attachments: HBASE-5545.patch
>
>
> Scenario:
> 
> 1. File is created 
> 2. But while writing data, all datanodes might have crashed. So writing data 
> will fail.
> 3. Now even if close is called in finally block, close also will fail and 
> throw the Exception because writing data failed.
> 4. After this if RS try to create the same file again, then 
> AlreadyBeingCreatedException will come.
> Suggestion to handle this scenario.
> ---
> 1. Check for the existence of the file, if exists delete the file and create 
> new file. 
> Here delete call for the file will not check whether the file is open or 
> closed.
> Overwrite Option:
> 
> 1. Overwrite option will be applicable if you are trying to overwrite a 
> closed file.
> 2. If the file is not closed, then even with overwrite option Same 
> AlreadyBeingCreatedException will be thrown.
> This is the expected behaviour to avoid the Multiple clients writing to same 
> file.
> Region server logs:
> org.apache.hadoop.ipc.RemoteException: 
> org.apache.hadoop.hdfs.protocol.AlreadyBeingCreatedException: failed to 
> create file /hbase/test1/12c01902324218d14b17a5880f24f64b/.tmp/.regioninfo 
> for 
> DFSClient_hb_rs_158-1-131-48,20020,1331107668635_1331107669061_-252463556_25 
> on client 158.1.132.19 because current leaseholder is trying to recreate file.
> at 
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.recoverLeaseInternal(FSNamesystem.java:1570)
> at 
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.startFileInternal(FSNamesystem.java:1440)
> at 
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.startFile(FSNamesystem.java:1382)
> at org.apache.hadoop.hdfs.server.namenode.NameNode.create(NameNode.java:658)
> at sun.reflect.GeneratedMethodAccessor9.invoke(Unknown Source)
> at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:547)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1137)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1133)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:396)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1131)
> at org.apache.hadoop.ipc.Client.call(Client.java:961)
> at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:245)
> at $Proxy6.create(Unknown Source)
> at sun.reflect.GeneratedMethodAccessor14.invoke(Unknown Source)
> at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at $Proxy6.create(Unknown Source)
> at 
> org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.(DFSClient.java:3643)
> at org.apache.hadoop.hdfs.DFSClient.create(DFSClient.java:778)
> at 
> org.apache.hadoop.hdfs.DistributedFileSystem.create(DistributedFileSystem.java:364)
> at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:630)
> at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:611)
> at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:518)
> at 
> org.apache.hadoop.hbase.regionserver.HRegion.checkRegioninfoOnFilesystem(HRegion.java:424)
> at org.apache.hadoop.hbase.regionserver.HRegion.initialize(HRegion.java:340)
> at org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:2672)
> at org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:2658)
> at 
> org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.openRegion(OpenRegionHandler.java:330)
> at 
> org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.process(OpenRegionHandler.java:116)
> at org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:158)
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:662)
> [2012-03-07 20:51:45,858] [WARN ] 
> 

[jira] [Commented] (HBASE-5782) Edits can be appended out of seqid order since HBASE-4487

[Jean-Daniel Cryans (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-5782?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13255784#comment-13255784
 ] 

Jean-Daniel Cryans commented on HBASE-5782:
---

bq. The pendingWrites are appended strictly in order, so there is a very short 
race that might lead to sync be issued multiple time when only one was 
necessary (it seems the same race condition existed before).

A short race is better than what we currently do in 0.92 (and before) where 
everything syncs everything.

> Edits can be appended out of seqid order since HBASE-4487
> -
>
> Key: HBASE-5782
> URL: https://issues.apache.org/jira/browse/HBASE-5782
> Project: HBase
>  Issue Type: Bug
>  Components: wal
>Affects Versions: 0.94.0
>Reporter: Gopinathan A
>Assignee: ramkrishna.s.vasudevan
>Priority: Blocker
> Fix For: 0.94.0
>
> Attachments: 5782-lars-v2.txt, 5782-sketch.txt, 5782.txt, 
> 5782.unfinished-stack.txt, HBASE-5782.patch, hbase-5782.txt
>
>
> Create a table with 1000 splits, after the region assignemnt, kill the 
> regionserver wich contains META table.
> Here few regions are missing after the log splitting and region assigment. 
> HBCK report shows multiple region holes are got created.
> Same scenario was verified mulitple times in 0.92.1, no issues.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5788) Move Dynamic Metrics storage off of HRegion.

[Elliott Clark (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-5788?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Elliott Clark updated HBASE-5788:
-

Attachment: HBASE-5788-3.patch

Lines should all be < 100.

Added comments on RegionMetricsStorage.

Made data in RegionMetricsStorage private with public getters (tests require it 
to be public).


> Move Dynamic Metrics storage off of HRegion.
> 
>
> Key: HBASE-5788
> URL: https://issues.apache.org/jira/browse/HBASE-5788
> Project: HBase
>  Issue Type: Improvement
>  Components: regionserver
>Reporter: Elliott Clark
>Assignee: Elliott Clark
>Priority: Minor
> Attachments: HBASE-5788-0.patch, HBASE-5788-1.patch, 
> HBASE-5788-2.patch, HBASE-5788-3.patch
>
>
> HRegion right now has the responsibility of storing static counts and latency 
> numbers for use by the metrics package.  Since these maps are incremented and 
> set from lots of places it makes adding functionality hard.
>  
> So move the metrics functionality into SchemaMetrics making it more than just 
> a class for naming.  The next step will be to simplify the api exposed so 
> that using it will be easier.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-3585) isLegalFamilyName() can throw ArrayOutOfBoundException

[Hadoop QA (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-3585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13255820#comment-13255820
 ] 

Hadoop QA commented on HBASE-3585:
--

-1 overall.  Here are the results of testing the latest attachment 
  http://issues.apache.org/jira/secure/attachment/12522987/HBASE-3585.patch
  against trunk revision .

+1 @author.  The patch does not contain any @author tags.

+1 tests included.  The patch appears to include 3 new or modified tests.

+1 javadoc.  The javadoc tool did not generate any warning messages.

+1 javac.  The applied patch does not increase the total number of javac 
compiler warnings.

-1 findbugs.  The patch appears to introduce 4 new Findbugs (version 1.3.9) 
warnings.

+1 release audit.  The applied patch does not increase the total number of 
release audit warnings.

 -1 core tests.  The patch failed these unit tests:
   
org.apache.hadoop.hbase.regionserver.TestServerCustomProtocol

Test results: 
https://builds.apache.org/job/PreCommit-HBASE-Build/1552//testReport/
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/1552//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
Console output: 
https://builds.apache.org/job/PreCommit-HBASE-Build/1552//console

This message is automatically generated.

> isLegalFamilyName() can throw ArrayOutOfBoundException
> --
>
> Key: HBASE-3585
> URL: https://issues.apache.org/jira/browse/HBASE-3585
> Project: HBase
>  Issue Type: Bug
>  Components: client
>Affects Versions: 0.90.1
>Reporter: Prakash Khemani
>Assignee: Uma Maheswara Rao G
>Priority: Minor
> Attachments: HBASE-3585.patch
>
>
> org.apache.hadoop.hbase.HColumnDescriptor.isLegalFamilyName(byte[]) accesses 
> byte[0] w/o first checking the array length.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5545) region can't be opened for a long time. Because the creating File failed.

[Hadoop QA (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-5545?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13255824#comment-13255824
 ] 

Hadoop QA commented on HBASE-5545:
--

-1 overall.  Here are the results of testing the latest attachment 
  http://issues.apache.org/jira/secure/attachment/12522989/HBASE-5545.patch
  against trunk revision .

+1 @author.  The patch does not contain any @author tags.

-1 tests included.  The patch doesn't appear to include any new or modified 
tests.
Please justify why no new tests are needed for this 
patch.
Also please list what manual steps were performed to 
verify this patch.

+1 javadoc.  The javadoc tool did not generate any warning messages.

+1 javac.  The applied patch does not increase the total number of javac 
compiler warnings.

-1 findbugs.  The patch appears to introduce 4 new Findbugs (version 1.3.9) 
warnings.

+1 release audit.  The applied patch does not increase the total number of 
release audit warnings.

 -1 core tests.  The patch failed these unit tests:
 

Test results: 
https://builds.apache.org/job/PreCommit-HBASE-Build/1553//testReport/
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/1553//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
Console output: 
https://builds.apache.org/job/PreCommit-HBASE-Build/1553//console

This message is automatically generated.

> region can't be opened for a long time. Because the creating File failed.
> -
>
> Key: HBASE-5545
> URL: https://issues.apache.org/jira/browse/HBASE-5545
> Project: HBase
>  Issue Type: Bug
>  Components: regionserver
>Affects Versions: 0.90.6
>Reporter: gaojinchao
>Assignee: gaojinchao
> Fix For: 0.90.7, 0.92.2, 0.94.0
>
> Attachments: HBASE-5545.patch
>
>
> Scenario:
> 
> 1. File is created 
> 2. But while writing data, all datanodes might have crashed. So writing data 
> will fail.
> 3. Now even if close is called in finally block, close also will fail and 
> throw the Exception because writing data failed.
> 4. After this if RS try to create the same file again, then 
> AlreadyBeingCreatedException will come.
> Suggestion to handle this scenario.
> ---
> 1. Check for the existence of the file, if exists delete the file and create 
> new file. 
> Here delete call for the file will not check whether the file is open or 
> closed.
> Overwrite Option:
> 
> 1. Overwrite option will be applicable if you are trying to overwrite a 
> closed file.
> 2. If the file is not closed, then even with overwrite option Same 
> AlreadyBeingCreatedException will be thrown.
> This is the expected behaviour to avoid the Multiple clients writing to same 
> file.
> Region server logs:
> org.apache.hadoop.ipc.RemoteException: 
> org.apache.hadoop.hdfs.protocol.AlreadyBeingCreatedException: failed to 
> create file /hbase/test1/12c01902324218d14b17a5880f24f64b/.tmp/.regioninfo 
> for 
> DFSClient_hb_rs_158-1-131-48,20020,1331107668635_1331107669061_-252463556_25 
> on client 158.1.132.19 because current leaseholder is trying to recreate file.
> at 
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.recoverLeaseInternal(FSNamesystem.java:1570)
> at 
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.startFileInternal(FSNamesystem.java:1440)
> at 
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.startFile(FSNamesystem.java:1382)
> at org.apache.hadoop.hdfs.server.namenode.NameNode.create(NameNode.java:658)
> at sun.reflect.GeneratedMethodAccessor9.invoke(Unknown Source)
> at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:547)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1137)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1133)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:396)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1131)
> at org.apache.hadoop.ipc.Client.call(Client.java:961)
> at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:245)
> at $Proxy6.create(Unknown Source)
> at sun.reflect.GeneratedMethodAccessor14.invoke(Unknown Source)
> at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at $Proxy6.create(Unknown Source)
> at 
> org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.(DFSClient.java:3643)
> at org.apache.hadoop.hdfs.DFSClient.create(DFSClient.java:778)
> at 
> org.apache.hadoop.hdfs.DistributedFileSystem.creat

[jira] [Commented] (HBASE-5782) Edits can be appended out of seqid order since HBASE-4487

[Lars Hofhansl (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-5782?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13255828#comment-13255828
 ] 

Lars Hofhansl commented on HBASE-5782:
--

Anybody opposed to do my patch for 0.94 and do a rewrite in trunk? Todd? Stack? 
Ted?


> Edits can be appended out of seqid order since HBASE-4487
> -
>
> Key: HBASE-5782
> URL: https://issues.apache.org/jira/browse/HBASE-5782
> Project: HBase
>  Issue Type: Bug
>  Components: wal
>Affects Versions: 0.94.0
>Reporter: Gopinathan A
>Assignee: ramkrishna.s.vasudevan
>Priority: Blocker
> Fix For: 0.94.0
>
> Attachments: 5782-lars-v2.txt, 5782-sketch.txt, 5782.txt, 
> 5782.unfinished-stack.txt, HBASE-5782.patch, hbase-5782.txt
>
>
> Create a table with 1000 splits, after the region assignemnt, kill the 
> regionserver wich contains META table.
> Here few regions are missing after the log splitting and region assigment. 
> HBCK report shows multiple region holes are got created.
> Same scenario was verified mulitple times in 0.92.1, no issues.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5782) Edits can be appended out of seqid order since HBASE-4487

[Todd Lipcon (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-5782?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13255832#comment-13255832
 ] 

Todd Lipcon commented on HBASE-5782:


Seems reasonable to me. Did yours pass tests, etc?

> Edits can be appended out of seqid order since HBASE-4487
> -
>
> Key: HBASE-5782
> URL: https://issues.apache.org/jira/browse/HBASE-5782
> Project: HBase
>  Issue Type: Bug
>  Components: wal
>Affects Versions: 0.94.0
>Reporter: Gopinathan A
>Assignee: ramkrishna.s.vasudevan
>Priority: Blocker
> Fix For: 0.94.0
>
> Attachments: 5782-lars-v2.txt, 5782-sketch.txt, 5782.txt, 
> 5782.unfinished-stack.txt, HBASE-5782.patch, hbase-5782.txt
>
>
> Create a table with 1000 splits, after the region assignemnt, kill the 
> regionserver wich contains META table.
> Here few regions are missing after the log splitting and region assigment. 
> HBCK report shows multiple region holes are got created.
> Same scenario was verified mulitple times in 0.92.1, no issues.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5782) Edits can be appended out of seqid order since HBASE-4487

[Zhihong Yu (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-5782?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13255834#comment-13255834
 ] 

Zhihong Yu commented on HBASE-5782:
---

HLogPerformanceEvaluation verification is needed as well.

> Edits can be appended out of seqid order since HBASE-4487
> -
>
> Key: HBASE-5782
> URL: https://issues.apache.org/jira/browse/HBASE-5782
> Project: HBase
>  Issue Type: Bug
>  Components: wal
>Affects Versions: 0.94.0
>Reporter: Gopinathan A
>Assignee: ramkrishna.s.vasudevan
>Priority: Blocker
> Fix For: 0.94.0
>
> Attachments: 5782-lars-v2.txt, 5782-sketch.txt, 5782.txt, 
> 5782.unfinished-stack.txt, HBASE-5782.patch, hbase-5782.txt
>
>
> Create a table with 1000 splits, after the region assignemnt, kill the 
> regionserver wich contains META table.
> Here few regions are missing after the log splitting and region assigment. 
> HBCK report shows multiple region holes are got created.
> Same scenario was verified mulitple times in 0.92.1, no issues.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5810) HBASE-5620 Convert the client protocol of HRegionInterface to PB addendum

[Hadoop QA (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-5810?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13255847#comment-13255847
 ] 

Hadoop QA commented on HBASE-5810:
--

-1 overall.  Here are the results of testing the latest attachment 
  http://issues.apache.org/jira/secure/attachment/12522990/hbase-5620-sec.patch
  against trunk revision .

+1 @author.  The patch does not contain any @author tags.

-1 tests included.  The patch doesn't appear to include any new or modified 
tests.
Please justify why no new tests are needed for this 
patch.
Also please list what manual steps were performed to 
verify this patch.

+1 javadoc.  The javadoc tool did not generate any warning messages.

+1 javac.  The applied patch does not increase the total number of javac 
compiler warnings.

-1 findbugs.  The patch appears to introduce 4 new Findbugs (version 1.3.9) 
warnings.

+1 release audit.  The applied patch does not increase the total number of 
release audit warnings.

 -1 core tests.  The patch failed these unit tests:
 

Test results: 
https://builds.apache.org/job/PreCommit-HBASE-Build/1554//testReport/
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/1554//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
Console output: 
https://builds.apache.org/job/PreCommit-HBASE-Build/1554//console

This message is automatically generated.

> HBASE-5620 Convert the client protocol of HRegionInterface to PB addendum
> -
>
> Key: HBASE-5810
> URL: https://issues.apache.org/jira/browse/HBASE-5810
> Project: HBase
>  Issue Type: Task
>Reporter: stack
>Assignee: Jimmy Xiang
> Fix For: 0.96.0
>
> Attachments: hbase-5620-sec.patch
>
>
> Apply an addendum that Jimmy made here rather than over in hbase-5620, a week 
> or so after it went in.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-2214) Do HBASE-1996 -- setting size to return in scan rather than count of rows -- properly

[jirapos...@reviews.apache.org (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-2214?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13255866#comment-13255866
 ] 

jirapos...@reviews.apache.org commented on HBASE-2214:
--



bq.  On 2012-04-17 05:46:43, Michael Stack wrote:
bq.  > I think this an important addition.  Any chance of a test Ferdy to 
demonstrate it works.  Its kinda crazy you only have to do this much to make it 
work.

It was indeed to good to be true. It does not yet work. There are still some 
other classes that need modification. (In order to make it work by piggybacking 
on the existing maxScannerResultSize mechanism).

Will get back at this with new version of patch.

Thanks.


bq.  On 2012-04-17 05:46:43, Michael Stack wrote:
bq.  > /src/main/java/org/apache/hadoop/hbase/regionserver/RegionScanner.java, 
line 26
bq.  > 
bq.  >
bq.  > This is not used?

It is, namely in the javadoc comment. Removing it makes the link unclickable.


bq.  On 2012-04-17 05:46:43, Michael Stack wrote:
bq.  > /src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java, line 
3376
bq.  > 

bq.  >
bq.  > Needs to be public?

I cannot reduce the visibilty because it needs to in the interface. (Yeah it's 
a bit of a hack I guess)


bq.  On 2012-04-17 05:46:43, Michael Stack wrote:
bq.  > /src/main/java/org/apache/hadoop/hbase/regionserver/RegionScanner.java, 
line 59
bq.  > 
bq.  >
bq.  > Is this a good name for this attribute?  Its the result size per 
next?  getMaxResultSize?  Or something like that?

maxResultSize seems better indeed. I will rename this next patch.


- ferdy


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/4726/#review6968
---


On 2012-04-15 13:38:31, ferdy wrote:
bq.  
bq.  ---
bq.  This is an automatically generated e-mail. To reply, visit:
bq.  https://reviews.apache.org/r/4726/
bq.  ---
bq.  
bq.  (Updated 2012-04-15 13:38:31)
bq.  
bq.  
bq.  Review request for hbase and Ted Yu.
bq.  
bq.  
bq.  Summary
bq.  ---
bq.  
bq.  HBASE-2214 per scan max buffersize.
bq.  
bq.  
bq.  This addresses bug HBASE-2214.
bq.  https://issues.apache.org/jira/browse/HBASE-2214
bq.  
bq.  
bq.  Diffs
bq.  -
bq.  
bq./src/main/java/org/apache/hadoop/hbase/client/Scan.java 1326084 
bq./src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java 1326084 
bq./src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java 
1326084 
bq./src/main/java/org/apache/hadoop/hbase/regionserver/RegionScanner.java 
1326084 
bq.
/src/test/java/org/apache/hadoop/hbase/coprocessor/TestCoprocessorInterface.java
 1326084 
bq.  
bq.  Diff: https://reviews.apache.org/r/4726/diff
bq.  
bq.  
bq.  Testing
bq.  ---
bq.  
bq.  
bq.  Thanks,
bq.  
bq.  ferdy
bq.  
bq.



> Do HBASE-1996 -- setting size to return in scan rather than count of rows -- 
> properly
> -
>
> Key: HBASE-2214
> URL: https://issues.apache.org/jira/browse/HBASE-2214
> Project: HBase
>  Issue Type: New Feature
>Reporter: stack
>Assignee: Ferdy Galema
> Fix For: 0.94.1
>
> Attachments: HBASE-2214-0.94.txt, HBASE-2214_with_broken_TestShell.txt
>
>
> The notion that you set size rather than row count specifying how many rows a 
> scanner should return in each cycle was raised over in hbase-1966.  Its a 
> good one making hbase "regular" though the data under it may vary.  
> HBase-1966 was committed but the patch was constrained by the fact that it 
> needed to not change RPC interface.  This issue is about doing hbase-1966 for 
> 0.21 in a clean, unconstrained way.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5788) Move Dynamic Metrics storage off of HRegion.

[Elliott Clark (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-5788?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Elliott Clark updated HBASE-5788:
-

Attachment: HBASE-5788-4.patch

Add @InterfaceAudience.Private to the new class.

> Move Dynamic Metrics storage off of HRegion.
> 
>
> Key: HBASE-5788
> URL: https://issues.apache.org/jira/browse/HBASE-5788
> Project: HBase
>  Issue Type: Improvement
>  Components: regionserver
>Reporter: Elliott Clark
>Assignee: Elliott Clark
>Priority: Minor
> Attachments: HBASE-5788-0.patch, HBASE-5788-1.patch, 
> HBASE-5788-2.patch, HBASE-5788-3.patch, HBASE-5788-4.patch
>
>
> HRegion right now has the responsibility of storing static counts and latency 
> numbers for use by the metrics package.  Since these maps are incremented and 
> set from lots of places it makes adding functionality hard.
>  
> So move the metrics functionality into SchemaMetrics making it more than just 
> a class for naming.  The next step will be to simplify the api exposed so 
> that using it will be easier.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-3585) isLegalFamilyName() can throw ArrayOutOfBoundException

[Uma Maheswara Rao G (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-3585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13255873#comment-13255873
 ] 

Uma Maheswara Rao G commented on HBASE-3585:


Test failure and findbugs are unrelated to this change.

> isLegalFamilyName() can throw ArrayOutOfBoundException
> --
>
> Key: HBASE-3585
> URL: https://issues.apache.org/jira/browse/HBASE-3585
> Project: HBase
>  Issue Type: Bug
>  Components: client
>Affects Versions: 0.90.1
>Reporter: Prakash Khemani
>Assignee: Uma Maheswara Rao G
>Priority: Minor
> Attachments: HBASE-3585.patch
>
>
> org.apache.hadoop.hbase.HColumnDescriptor.isLegalFamilyName(byte[]) accesses 
> byte[0] w/o first checking the array length.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5782) Edits can be appended out of seqid order since HBASE-4487

[stack (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-5782?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13255877#comment-13255877
 ] 

stack commented on HBASE-5782:
--

Looking at Lars patch.

On you 1. and 2. above, apparently the append is also expensive according to 
Dhruba.  Just saying.

Also on "...might lead to sync be issued multiple time when only one was 
necessary (it seems the same race condition existed before)."

Yes, this we have always had.

I'd say kill this stuff... it looks like rubbish to me:

{code}
+  syncBatchSize.addAndGet(doneUpto - this.syncedTillHere);
{code}

Its not read by anyone, looks like the math can go wonky, and when it is read, 
its set back to zero which is probably unexpected.  Kill it I'd say.

I think this is ok:

{code}
+  this.syncedTillHere = Math.max(this.syncedTillHere, doneUpto);
{code}

but this is racy

{code}
   long doneUpto = this.unflushedEntries.get();
{code}

It could be low in number; i.e. we could be putting into hdfs more edits than 
the current value of unflushedEntries if we read after an edit has been added 
to the queue but before the above is updated.  Is that ok?  Its ok if this is a 
little sloppy especially if it under reports?

On tactic for 0.94, sure on doing this for 0.94 though I like Todds fix better. 
 The verification tool will help you figure if this slows stuff much and if we 
are writing out of order.  Let me know if you want me to run it for you.  Let 
me add in log rolling too as per Todd suggestion.

> Edits can be appended out of seqid order since HBASE-4487
> -
>
> Key: HBASE-5782
> URL: https://issues.apache.org/jira/browse/HBASE-5782
> Project: HBase
>  Issue Type: Bug
>  Components: wal
>Affects Versions: 0.94.0
>Reporter: Gopinathan A
>Assignee: ramkrishna.s.vasudevan
>Priority: Blocker
> Fix For: 0.94.0
>
> Attachments: 5782-lars-v2.txt, 5782-sketch.txt, 5782.txt, 
> 5782.unfinished-stack.txt, HBASE-5782.patch, hbase-5782.txt
>
>
> Create a table with 1000 splits, after the region assignemnt, kill the 
> regionserver wich contains META table.
> Here few regions are missing after the log splitting and region assigment. 
> HBCK report shows multiple region holes are got created.
> Same scenario was verified mulitple times in 0.92.1, no issues.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5780) Fix race in HBase regionserver startup vs ZK SASL authentication

[Eugene Koontz (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-5780?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13255887#comment-13255887
 ] 

Eugene Koontz commented on HBASE-5780:
--

I tried:

{code}
mvn clean ; while [ "$?" -eq "0" ] ; do mvn -PrunDevTests,localTests,security 
test -Dtest=TestReplicationPeer ; done
{code}

and the test consistently passes for me on both trunk and 0.94 branches.

> Fix race in HBase regionserver startup vs ZK SASL authentication
> 
>
> Key: HBASE-5780
> URL: https://issues.apache.org/jira/browse/HBASE-5780
> Project: HBase
>  Issue Type: Bug
>  Components: security
>Affects Versions: 0.92.1, 0.94.0
>Reporter: Shaneal Manek
>Assignee: Shaneal Manek
> Fix For: 0.92.2, 0.94.0, 0.96.0
>
> Attachments: HBASE-5780-v2.patch, HBASE-5780.patch, 
> TestReplicationPeer-Security-output.log, TestReplicationPeer-output.log, 
> testoutput.tar.gz
>
>
> Secure RegionServers sometimes fail to start with the following backtrace:
> 2012-03-22 17:20:16,737 FATAL 
> org.apache.hadoop.hbase.regionserver.HRegionServer: ABORTING region server 
> centos60-20.ent.cloudera.com,60020,1332462015929: Unexpected exception during 
> initialization, aborting
> org.apache.zookeeper.KeeperException$NoAuthException: KeeperErrorCode = 
> NoAuth for /hbase/shutdown
> at org.apache.zookeeper.KeeperException.create(KeeperException.java:113)
> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
> at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1131)
> at 
> org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.getData(RecoverableZooKeeper.java:295)
> at org.apache.hadoop.hbase.zookeeper.ZKUtil.getDataInternal(ZKUtil.java:518)
> at org.apache.hadoop.hbase.zookeeper.ZKUtil.getDataAndWatch(ZKUtil.java:494)
> at 
> org.apache.hadoop.hbase.zookeeper.ZooKeeperNodeTracker.start(ZooKeeperNodeTracker.java:77)
> at 
> org.apache.hadoop.hbase.regionserver.HRegionServer.initializeZooKeeper(HRegionServer.java:569)
> at 
> org.apache.hadoop.hbase.regionserver.HRegionServer.preRegistrationInitialization(HRegionServer.java:532)
> at 
> org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:634)
> at java.lang.Thread.run(Thread.java:662)

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5782) Edits can be appended out of seqid order since HBASE-4487

[stack (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-5782?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13255885#comment-13255885
 ] 

stack commented on HBASE-5782:
--

Can we try and make Todd's work?  It does some nice cleanup.

> Edits can be appended out of seqid order since HBASE-4487
> -
>
> Key: HBASE-5782
> URL: https://issues.apache.org/jira/browse/HBASE-5782
> Project: HBase
>  Issue Type: Bug
>  Components: wal
>Affects Versions: 0.94.0
>Reporter: Gopinathan A
>Assignee: ramkrishna.s.vasudevan
>Priority: Blocker
> Fix For: 0.94.0
>
> Attachments: 5782-lars-v2.txt, 5782-sketch.txt, 5782.txt, 
> 5782.unfinished-stack.txt, HBASE-5782.patch, hbase-5782.txt
>
>
> Create a table with 1000 splits, after the region assignemnt, kill the 
> regionserver wich contains META table.
> Here few regions are missing after the log splitting and region assigment. 
> HBCK report shows multiple region holes are got created.
> Same scenario was verified mulitple times in 0.92.1, no issues.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (HBASE-5811) TestLoadAndSwitchEncodeOnDisk fails sometimes

[stack (Created) (JIRA)]

TestLoadAndSwitchEncodeOnDisk fails sometimes
-

 Key: HBASE-5811
 URL: https://issues.apache.org/jira/browse/HBASE-5811
 Project: HBase
  Issue Type: Bug
Reporter: stack


Looks like its dependent on isTableEnabled actually returning true when the 
table is enabled only, isTableEnabled looks like its set whenever any region 
from a table is enabled which is not the semantic I remember it always having.  
This needs fixing.  Meantime the test TestLoadAndSwitchEncodeOnDisk will fail 
for me sometimes.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5811) TestLoadAndSwitchEncodeOnDisk fails sometimes

[stack (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-5811?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stack updated HBASE-5811:
-

Status: Patch Available  (was: Open)

> TestLoadAndSwitchEncodeOnDisk fails sometimes
> -
>
> Key: HBASE-5811
> URL: https://issues.apache.org/jira/browse/HBASE-5811
> Project: HBase
>  Issue Type: Bug
>Reporter: stack
> Attachments: 5811.txt
>
>
> Looks like its dependent on isTableEnabled actually returning true when the 
> table is enabled only, isTableEnabled looks like its set whenever any region 
> from a table is enabled which is not the semantic I remember it always 
> having.  This needs fixing.  Meantime the test TestLoadAndSwitchEncodeOnDisk 
> will fail for me sometimes.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5811) TestLoadAndSwitchEncodeOnDisk fails sometimes

[stack (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-5811?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stack updated HBASE-5811:
-

Attachment: 5811.txt

Adds verify that all regions are online before we move to the compaction that 
would fail if table not completely online.

> TestLoadAndSwitchEncodeOnDisk fails sometimes
> -
>
> Key: HBASE-5811
> URL: https://issues.apache.org/jira/browse/HBASE-5811
> Project: HBase
>  Issue Type: Bug
>Reporter: stack
> Attachments: 5811.txt
>
>
> Looks like its dependent on isTableEnabled actually returning true when the 
> table is enabled only, isTableEnabled looks like its set whenever any region 
> from a table is enabled which is not the semantic I remember it always 
> having.  This needs fixing.  Meantime the test TestLoadAndSwitchEncodeOnDisk 
> will fail for me sometimes.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5782) Edits can be appended out of seqid order since HBASE-4487

[Lars Hofhansl (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-5782?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13255911#comment-13255911
 ] 

Lars Hofhansl commented on HBASE-5782:
--

@Stack: the doneUpTo is fine, because it is only used to set syncedTillHere. We 
won't write more into the log (once we take the pendingWrites they are gone), 
but we might sync too much unnecessarily.
Will do the (performance) testing now. I don't have a cluster at my disposal 
atm, so I'll do it with a local HDFS instance.

If we'd rather finish Todd's for 0.94 that'd be nice.


> Edits can be appended out of seqid order since HBASE-4487
> -
>
> Key: HBASE-5782
> URL: https://issues.apache.org/jira/browse/HBASE-5782
> Project: HBase
>  Issue Type: Bug
>  Components: wal
>Affects Versions: 0.94.0
>Reporter: Gopinathan A
>Assignee: ramkrishna.s.vasudevan
>Priority: Blocker
> Fix For: 0.94.0
>
> Attachments: 5782-lars-v2.txt, 5782-sketch.txt, 5782.txt, 
> 5782.unfinished-stack.txt, HBASE-5782.patch, hbase-5782.txt
>
>
> Create a table with 1000 splits, after the region assignemnt, kill the 
> regionserver wich contains META table.
> Here few regions are missing after the log splitting and region assigment. 
> HBCK report shows multiple region holes are got created.
> Same scenario was verified mulitple times in 0.92.1, no issues.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5803) [89-fb] Upgrade hbase 0.89-fb to Thrift 0.8.0 and bring Thrift server enhancements from trunk

[Phabricator (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-5803?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Phabricator updated HBASE-5803:
---

Attachment: D2811.2.patch

mbautin updated the revision "[jira] [HBASE-5803] [89-fb] Upgrade hbase 0.89-fb 
to Thrift 0.8.0 and bring Thrift server enhancements from trunk".
Reviewers: sc, nspiegelberg, Kannan, Liyin, JIRA

  - Addressing Scott's comment.
  - Breaking TestThriftServerCmdLine into multiple parts because its running 
time became too long (will be ported to trunk).
  - Not waiting before a retry in a multiput if all errors were 
NotServingRegionExceptions (this will also be a separate JIRA but is included 
here for testing).


REVISION DETAIL
  https://reviews.facebook.net/D2811

AFFECTED FILES
  pom.xml
  src/main/java/org/apache/hadoop/hbase/HConstants.java
  src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java
  src/main/java/org/apache/hadoop/hbase/filter/ParseFilter.java
  src/main/java/org/apache/hadoop/hbase/regionserver/HRegionThriftServer.java
  src/main/java/org/apache/hadoop/hbase/thrift/HThreadedSelectorServerArgs.java
  src/main/java/org/apache/hadoop/hbase/thrift/TBoundedThreadPoolServer.java
  src/main/java/org/apache/hadoop/hbase/thrift/ThriftMetrics.java
  src/main/java/org/apache/hadoop/hbase/thrift/ThriftServer.java
  src/main/java/org/apache/hadoop/hbase/thrift/ThriftServerRunner.java
  src/main/java/org/apache/hadoop/hbase/thrift/ThriftUtilities.java
  src/main/java/org/apache/hadoop/hbase/thrift/generated/AlreadyExists.java
  src/main/java/org/apache/hadoop/hbase/thrift/generated/BatchMutation.java
  src/main/java/org/apache/hadoop/hbase/thrift/generated/ColumnDescriptor.java
  src/main/java/org/apache/hadoop/hbase/thrift/generated/Hbase.java
  src/main/java/org/apache/hadoop/hbase/thrift/generated/IOError.java
  src/main/java/org/apache/hadoop/hbase/thrift/generated/IllegalArgument.java
  src/main/java/org/apache/hadoop/hbase/thrift/generated/Mutation.java
  src/main/java/org/apache/hadoop/hbase/thrift/generated/TCell.java
  src/main/java/org/apache/hadoop/hbase/thrift/generated/TRegionInfo.java
  src/main/java/org/apache/hadoop/hbase/thrift/generated/TRowResult.java
  src/main/java/org/apache/hadoop/hbase/thrift/generated/TScan.java
  src/main/java/org/apache/hadoop/hbase/util/Bytes.java
  src/test/java/org/apache/hadoop/hbase/thrift/TestCallQueue.java
  src/test/java/org/apache/hadoop/hbase/thrift/TestHsHaServerCmdLine.java
  src/test/java/org/apache/hadoop/hbase/thrift/TestNonblockingServerCmdLine.java
  src/test/java/org/apache/hadoop/hbase/thrift/TestThreadPoolServerCmdLine.java
  
src/test/java/org/apache/hadoop/hbase/thrift/TestThreadedSelectorServerCmdLine.java
  src/test/java/org/apache/hadoop/hbase/thrift/TestThriftServer.java
  src/test/java/org/apache/hadoop/hbase/thrift/ThriftServerCmdLineTestBase.java


> [89-fb] Upgrade hbase 0.89-fb to Thrift 0.8.0 and bring Thrift server 
> enhancements from trunk
> -
>
> Key: HBASE-5803
> URL: https://issues.apache.org/jira/browse/HBASE-5803
> Project: HBase
>  Issue Type: Improvement
>Reporter: Mikhail Bautin
>Assignee: Mikhail Bautin
> Attachments: D2811.1.patch, D2811.2.patch
>
>
> TBoundedThreadPoolServer has been a problem for us when there is a large 
> number of clients. We need to migrate to 0.8.0. in 89-fb and bring the 
> relevant improvements from trunk, including supporting 
> TThreadedSelectorServer.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5782) Edits can be appended out of seqid order since HBASE-4487

[stack (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-5782?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13255937#comment-13255937
 ] 

stack commented on HBASE-5782:
--

bq. We won't write more into the log (once we take the pendingWrites they are 
gone

Is that so?  We don't get the pendingWrites until we are under the flush lock 
but we've taken doneUpTo before we go under the lock.

> Edits can be appended out of seqid order since HBASE-4487
> -
>
> Key: HBASE-5782
> URL: https://issues.apache.org/jira/browse/HBASE-5782
> Project: HBase
>  Issue Type: Bug
>  Components: wal
>Affects Versions: 0.94.0
>Reporter: Gopinathan A
>Assignee: ramkrishna.s.vasudevan
>Priority: Blocker
> Fix For: 0.94.0
>
> Attachments: 5782-lars-v2.txt, 5782-sketch.txt, 5782.txt, 
> 5782.unfinished-stack.txt, HBASE-5782.patch, hbase-5782.txt
>
>
> Create a table with 1000 splits, after the region assignemnt, kill the 
> regionserver wich contains META table.
> Here few regions are missing after the log splitting and region assigment. 
> HBCK report shows multiple region holes are got created.
> Same scenario was verified mulitple times in 0.92.1, no issues.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5782) Edits can be appended out of seqid order since HBASE-4487

[Lars Hofhansl (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-5782?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13255939#comment-13255939
 ] 

Lars Hofhansl commented on HBASE-5782:
--

Ah, I see. Yes, doneUpto line should be pulled into the synchronized block.

> Edits can be appended out of seqid order since HBASE-4487
> -
>
> Key: HBASE-5782
> URL: https://issues.apache.org/jira/browse/HBASE-5782
> Project: HBase
>  Issue Type: Bug
>  Components: wal
>Affects Versions: 0.94.0
>Reporter: Gopinathan A
>Assignee: ramkrishna.s.vasudevan
>Priority: Blocker
> Fix For: 0.94.0
>
> Attachments: 5782-lars-v2.txt, 5782-sketch.txt, 5782.txt, 
> 5782.unfinished-stack.txt, HBASE-5782.patch, hbase-5782.txt
>
>
> Create a table with 1000 splits, after the region assignemnt, kill the 
> regionserver wich contains META table.
> Here few regions are missing after the log splitting and region assigment. 
> HBCK report shows multiple region holes are got created.
> Same scenario was verified mulitple times in 0.92.1, no issues.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5811) TestLoadAndSwitchEncodeOnDisk fails sometimes

[Hadoop QA (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-5811?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13255943#comment-13255943
 ] 

Hadoop QA commented on HBASE-5811:
--

-1 overall.  Here are the results of testing the latest attachment 
  http://issues.apache.org/jira/secure/attachment/12523012/5811.txt
  against trunk revision .

+1 @author.  The patch does not contain any @author tags.

+1 tests included.  The patch appears to include 3 new or modified tests.

+1 javadoc.  The javadoc tool did not generate any warning messages.

+1 javac.  The applied patch does not increase the total number of javac 
compiler warnings.

-1 findbugs.  The patch appears to introduce 4 new Findbugs (version 1.3.9) 
warnings.

+1 release audit.  The applied patch does not increase the total number of 
release audit warnings.

 -1 core tests.  The patch failed these unit tests:
 

Test results: 
https://builds.apache.org/job/PreCommit-HBASE-Build/1555//testReport/
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/1555//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
Console output: 
https://builds.apache.org/job/PreCommit-HBASE-Build/1555//console

This message is automatically generated.

> TestLoadAndSwitchEncodeOnDisk fails sometimes
> -
>
> Key: HBASE-5811
> URL: https://issues.apache.org/jira/browse/HBASE-5811
> Project: HBase
>  Issue Type: Bug
>Reporter: stack
> Attachments: 5811.txt
>
>
> Looks like its dependent on isTableEnabled actually returning true when the 
> table is enabled only, isTableEnabled looks like its set whenever any region 
> from a table is enabled which is not the semantic I remember it always 
> having.  This needs fixing.  Meantime the test TestLoadAndSwitchEncodeOnDisk 
> will fail for me sometimes.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5812) Add log rolling to HLogPerformanceEvaluation

[stack (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-5812?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13255946#comment-13255946
 ] 

stack commented on HBASE-5812:
--

Verify can deal w/ multiple logs and verify all logs were written in sequence 
id order.

> Add log rolling to HLogPerformanceEvaluation
> 
>
> Key: HBASE-5812
> URL: https://issues.apache.org/jira/browse/HBASE-5812
> Project: HBase
>  Issue Type: Task
>Reporter: stack
> Attachments: 5812.txt
>
>
> Add being able to ask that HLogPerformanceEvaluation rolls logs when its 
> running.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (HBASE-5812) Add log rolling to HLogPerformanceEvaluation

[stack (Created) (JIRA)]

Add log rolling to HLogPerformanceEvaluation


 Key: HBASE-5812
 URL: https://issues.apache.org/jira/browse/HBASE-5812
 Project: HBase
  Issue Type: Task
Reporter: stack
 Attachments: 5812.txt

Add being able to ask that HLogPerformanceEvaluation rolls logs when its 
running.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5812) Add log rolling to HLogPerformanceEvaluation

[stack (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-5812?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stack updated HBASE-5812:
-

Attachment: 5812.txt

Adds rolling logs at N edits.

> Add log rolling to HLogPerformanceEvaluation
> 
>
> Key: HBASE-5812
> URL: https://issues.apache.org/jira/browse/HBASE-5812
> Project: HBase
>  Issue Type: Task
>Reporter: stack
> Attachments: 5812.txt
>
>
> Add being able to ask that HLogPerformanceEvaluation rolls logs when its 
> running.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Resolved] (HBASE-5812) Add log rolling to HLogPerformanceEvaluation

[stack (Resolved) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-5812?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stack resolved HBASE-5812.
--

   Resolution: Fixed
Fix Version/s: 0.96.0
 Assignee: stack

Tried it w/ Lars's patch (You have to have a patch else verify won't 
complete... with three threads only we write out of order)

> Add log rolling to HLogPerformanceEvaluation
> 
>
> Key: HBASE-5812
> URL: https://issues.apache.org/jira/browse/HBASE-5812
> Project: HBase
>  Issue Type: Task
>Reporter: stack
>Assignee: stack
> Fix For: 0.96.0
>
> Attachments: 5812.txt
>
>
> Add being able to ask that HLogPerformanceEvaluation rolls logs when its 
> running.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5811) TestLoadAndSwitchEncodeOnDisk fails sometimes

[stack (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-5811?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stack updated HBASE-5811:
-

Attachment: 5812.txt

Retry.  I see an OOME in unrelated test in console output.  Retrying for kicks.

> TestLoadAndSwitchEncodeOnDisk fails sometimes
> -
>
> Key: HBASE-5811
> URL: https://issues.apache.org/jira/browse/HBASE-5811
> Project: HBase
>  Issue Type: Bug
>Reporter: stack
> Attachments: 5811.txt, 5812.txt
>
>
> Looks like its dependent on isTableEnabled actually returning true when the 
> table is enabled only, isTableEnabled looks like its set whenever any region 
> from a table is enabled which is not the semantic I remember it always 
> having.  This needs fixing.  Meantime the test TestLoadAndSwitchEncodeOnDisk 
> will fail for me sometimes.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5811) TestLoadAndSwitchEncodeOnDisk fails sometimes

[Hadoop QA (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-5811?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13255958#comment-13255958
 ] 

Hadoop QA commented on HBASE-5811:
--

-1 overall.  Here are the results of testing the latest attachment 
  http://issues.apache.org/jira/secure/attachment/12523023/5812.txt
  against trunk revision .

+1 @author.  The patch does not contain any @author tags.

+1 tests included.  The patch appears to include 3 new or modified tests.

-1 patch.  The patch command could not apply the patch.

Console output: 
https://builds.apache.org/job/PreCommit-HBASE-Build/1556//console

This message is automatically generated.

> TestLoadAndSwitchEncodeOnDisk fails sometimes
> -
>
> Key: HBASE-5811
> URL: https://issues.apache.org/jira/browse/HBASE-5811
> Project: HBase
>  Issue Type: Bug
>Reporter: stack
> Attachments: 5811.txt, 5812.txt
>
>
> Looks like its dependent on isTableEnabled actually returning true when the 
> table is enabled only, isTableEnabled looks like its set whenever any region 
> from a table is enabled which is not the semantic I remember it always 
> having.  This needs fixing.  Meantime the test TestLoadAndSwitchEncodeOnDisk 
> will fail for me sometimes.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5782) Edits can be appended out of seqid order since HBASE-4487

[stack (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-5782?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13255959#comment-13255959
 ] 

stack commented on HBASE-5782:
--

@Lars But it own't matter right since the map we are getting from is not under 
our new flush lock?  I think its harmless.  We will undercount whats been 
flushed I believe; we'll not overcount (and so possible lose data)?

I added log rolling and tested your patch using HLogPerformanceEvaluation.  It 
'works' at least.  If you want me to compare before and after, just say.

> Edits can be appended out of seqid order since HBASE-4487
> -
>
> Key: HBASE-5782
> URL: https://issues.apache.org/jira/browse/HBASE-5782
> Project: HBase
>  Issue Type: Bug
>  Components: wal
>Affects Versions: 0.94.0
>Reporter: Gopinathan A
>Assignee: ramkrishna.s.vasudevan
>Priority: Blocker
> Fix For: 0.94.0
>
> Attachments: 5782-lars-v2.txt, 5782-sketch.txt, 5782.txt, 
> 5782.unfinished-stack.txt, HBASE-5782.patch, hbase-5782.txt
>
>
> Create a table with 1000 splits, after the region assignemnt, kill the 
> regionserver wich contains META table.
> Here few regions are missing after the log splitting and region assigment. 
> HBCK report shows multiple region holes are got created.
> Same scenario was verified mulitple times in 0.92.1, no issues.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (HBASE-5813) Retry immediately after a NotServingRegionException in a multiput

[Mikhail Bautin (Created) (JIRA)]

Retry immediately after a NotServingRegionException in a multiput
-

 Key: HBASE-5813
 URL: https://issues.apache.org/jira/browse/HBASE-5813
 Project: HBase
  Issue Type: Improvement
Reporter: Mikhail Bautin
Assignee: Mikhail Bautin


After we get some errors in a multiput we invalidate the region location cache 
and wait for the configured time interval according to the backoff policy. 
However, if all "errors" in multiput processing were 
NotServingRegionExceptions, we don't really need to wait. We can retry 
immediately.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5813) Retry immediately after a NotServingRegionException in a multiput

[Phabricator (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-5813?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Phabricator updated HBASE-5813:
---

Attachment: D2847.1.patch

mbautin requested code review of "[jira] [HBASE-5813] [89-fb] Retry immediately 
after a NotServingRegionException in a multiput".
Reviewers: Liyin, Kannan, khemani, todd, tedyu, stack, JIRA

  After we get some errors in a multiput we invalidate the region location 
cache and wait for the configured time interval according to the backoff 
policy. However, if all "errors" in multiput processing were 
NotServingRegionExceptions, we don't really need to wait. We can retry 
immediately.


TEST PLAN
  Run unit tests

REVISION DETAIL
  https://reviews.facebook.net/D2847

AFFECTED FILES
  src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java

MANAGE HERALD DIFFERENTIAL RULES
  https://reviews.facebook.net/herald/view/differential/

WHY DID I GET THIS EMAIL?
  https://reviews.facebook.net/herald/transcript/6453/

Tip: use the X-Herald-Rules header to filter Herald messages in your client.


> Retry immediately after a NotServingRegionException in a multiput
> -
>
> Key: HBASE-5813
> URL: https://issues.apache.org/jira/browse/HBASE-5813
> Project: HBase
>  Issue Type: Improvement
>Reporter: Mikhail Bautin
>Assignee: Mikhail Bautin
> Attachments: D2847.1.patch
>
>
> After we get some errors in a multiput we invalidate the region location 
> cache and wait for the configured time interval according to the backoff 
> policy. However, if all "errors" in multiput processing were 
> NotServingRegionExceptions, we don't really need to wait. We can retry 
> immediately.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5788) Move Dynamic Metrics storage off of HRegion.

[stack (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-5788?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stack updated HBASE-5788:
-

Hadoop Flags: Reviewed
  Status: Patch Available  (was: Open)

lgtm

Trying against hadoopqa

> Move Dynamic Metrics storage off of HRegion.
> 
>
> Key: HBASE-5788
> URL: https://issues.apache.org/jira/browse/HBASE-5788
> Project: HBase
>  Issue Type: Improvement
>  Components: regionserver
>Reporter: Elliott Clark
>Assignee: Elliott Clark
>Priority: Minor
> Attachments: HBASE-5788-0.patch, HBASE-5788-1.patch, 
> HBASE-5788-2.patch, HBASE-5788-3.patch, HBASE-5788-4.patch
>
>
> HRegion right now has the responsibility of storing static counts and latency 
> numbers for use by the metrics package.  Since these maps are incremented and 
> set from lots of places it makes adding functionality hard.
>  
> So move the metrics functionality into SchemaMetrics making it more than just 
> a class for naming.  The next step will be to simplify the api exposed so 
> that using it will be easier.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5813) Retry immediately after a NotServingRegionException in a multiput

[Phabricator (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-5813?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Phabricator updated HBASE-5813:
---

Attachment: D2847.2.patch

mbautin updated the revision "[jira] [HBASE-5813] [89-fb] Retry immediately 
after a NotServingRegionException in a multiput".
Reviewers: Liyin, Kannan, khemani, todd, tedyu, stack, JIRA

  Addressing Liyin's offline feedback: not waiting at the last retry either.

REVISION DETAIL
  https://reviews.facebook.net/D2847

AFFECTED FILES
  src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java


> Retry immediately after a NotServingRegionException in a multiput
> -
>
> Key: HBASE-5813
> URL: https://issues.apache.org/jira/browse/HBASE-5813
> Project: HBase
>  Issue Type: Improvement
>Reporter: Mikhail Bautin
>Assignee: Mikhail Bautin
> Attachments: D2847.1.patch, D2847.2.patch
>
>
> After we get some errors in a multiput we invalidate the region location 
> cache and wait for the configured time interval according to the backoff 
> policy. However, if all "errors" in multiput processing were 
> NotServingRegionExceptions, we don't really need to wait. We can retry 
> immediately.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5782) Edits can be appended out of seqid order since HBASE-4487

[Lars Hofhansl (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-5782?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13255974#comment-13255974
 ] 

Lars Hofhansl commented on HBASE-5782:
--

@Stack: I ran it too. It works fine. The "interesting" thing I find that it is 
faster *with* the patch! And that scares me.
I ran with 100 threads, 1 iterations. With the patch it took 29s, without 
it took 43s. This is on a 6 core machine with hyper threading.

Now, the HLogPerformanceEvaluation does not work with -nosync and -verify, 
right? (presumably because no final sync is issued).

Wouldn't mind to get some other numbers from you as well if you had some time.

> Edits can be appended out of seqid order since HBASE-4487
> -
>
> Key: HBASE-5782
> URL: https://issues.apache.org/jira/browse/HBASE-5782
> Project: HBase
>  Issue Type: Bug
>  Components: wal
>Affects Versions: 0.94.0
>Reporter: Gopinathan A
>Assignee: ramkrishna.s.vasudevan
>Priority: Blocker
> Fix For: 0.94.0
>
> Attachments: 5782-lars-v2.txt, 5782-sketch.txt, 5782.txt, 
> 5782.unfinished-stack.txt, HBASE-5782.patch, hbase-5782.txt
>
>
> Create a table with 1000 splits, after the region assignemnt, kill the 
> regionserver wich contains META table.
> Here few regions are missing after the log splitting and region assigment. 
> HBCK report shows multiple region holes are got created.
> Same scenario was verified mulitple times in 0.92.1, no issues.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-3614) Expose per-region request rate metrics

[Elliott Clark (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-3614?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Elliott Clark updated HBASE-3614:
-

Attachment: Screen Shot 2012-04-17 at 2.41.27 PM.png

Screen shot of jmx stats after some of Stack's comments.

> Expose per-region request rate metrics
> --
>
> Key: HBASE-3614
> URL: https://issues.apache.org/jira/browse/HBASE-3614
> Project: HBase
>  Issue Type: Improvement
>  Components: metrics, regionserver
>Reporter: Gary Helmling
>Assignee: Elliott Clark
>Priority: Minor
> Attachments: HBASE-3614-0.patch, HBASE-3614-1.patch, Screen Shot 
> 2012-04-17 at 2.41.27 PM.png
>
>
> We currently export metrics on request rates for each region server, and this 
> can help with identifying uneven load at a high level. But once you see a 
> given server under high load, you're forced to extrapolate based on your 
> application patterns and the data it's serving what the likely culprit is.  
> This can and should be much easier if we just exported request rate metrics 
> per-region on each server.
> Dynamically updating the metrics keys based on assigned regions may pose some 
> minor challenges, but this seems a very valuable diagnostic tool to have 
> available.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5813) Retry immediately after a NotServingRegionException in a multiput

[Phabricator (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-5813?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13255978#comment-13255978
 ] 

Phabricator commented on HBASE-5813:


tedyu has accepted the revision "[jira] [HBASE-5813] [89-fb] Retry immediately 
after a NotServingRegionException in a multiput".

INLINE COMMENTS
  src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java:1629 
Please remove 'after' or change it to 'after failure'.

REVISION DETAIL
  https://reviews.facebook.net/D2847

BRANCH
  retry_immediately_after_a_HBASE-5813_v2


> Retry immediately after a NotServingRegionException in a multiput
> -
>
> Key: HBASE-5813
> URL: https://issues.apache.org/jira/browse/HBASE-5813
> Project: HBase
>  Issue Type: Improvement
>Reporter: Mikhail Bautin
>Assignee: Mikhail Bautin
> Attachments: D2847.1.patch, D2847.2.patch
>
>
> After we get some errors in a multiput we invalidate the region location 
> cache and wait for the configured time interval according to the backoff 
> policy. However, if all "errors" in multiput processing were 
> NotServingRegionExceptions, we don't really need to wait. We can retry 
> immediately.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5803) [89-fb] Upgrade hbase 0.89-fb to Thrift 0.8.0 and bring Thrift server enhancements from trunk

[Phabricator (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-5803?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Phabricator updated HBASE-5803:
---

Attachment: D2811.3.patch

mbautin updated the revision "[jira] [HBASE-5803] [89-fb] Upgrade hbase 0.89-fb 
to Thrift 0.8.0 and bring Thrift server enhancements from trunk".
Reviewers: sc, nspiegelberg, Kannan, Liyin, JIRA

  Moved HConnectionManager changes to D2847.

REVISION DETAIL
  https://reviews.facebook.net/D2811

AFFECTED FILES
  pom.xml
  src/main/java/org/apache/hadoop/hbase/HConstants.java
  src/main/java/org/apache/hadoop/hbase/filter/ParseFilter.java
  src/main/java/org/apache/hadoop/hbase/regionserver/HRegionThriftServer.java
  src/main/java/org/apache/hadoop/hbase/thrift/HThreadedSelectorServerArgs.java
  src/main/java/org/apache/hadoop/hbase/thrift/TBoundedThreadPoolServer.java
  src/main/java/org/apache/hadoop/hbase/thrift/ThriftMetrics.java
  src/main/java/org/apache/hadoop/hbase/thrift/ThriftServer.java
  src/main/java/org/apache/hadoop/hbase/thrift/ThriftServerRunner.java
  src/main/java/org/apache/hadoop/hbase/thrift/ThriftUtilities.java
  src/main/java/org/apache/hadoop/hbase/thrift/generated/AlreadyExists.java
  src/main/java/org/apache/hadoop/hbase/thrift/generated/BatchMutation.java
  src/main/java/org/apache/hadoop/hbase/thrift/generated/ColumnDescriptor.java
  src/main/java/org/apache/hadoop/hbase/thrift/generated/Hbase.java
  src/main/java/org/apache/hadoop/hbase/thrift/generated/IOError.java
  src/main/java/org/apache/hadoop/hbase/thrift/generated/IllegalArgument.java
  src/main/java/org/apache/hadoop/hbase/thrift/generated/Mutation.java
  src/main/java/org/apache/hadoop/hbase/thrift/generated/TCell.java
  src/main/java/org/apache/hadoop/hbase/thrift/generated/TRegionInfo.java
  src/main/java/org/apache/hadoop/hbase/thrift/generated/TRowResult.java
  src/main/java/org/apache/hadoop/hbase/thrift/generated/TScan.java
  src/main/java/org/apache/hadoop/hbase/util/Bytes.java
  src/test/java/org/apache/hadoop/hbase/thrift/TestCallQueue.java
  src/test/java/org/apache/hadoop/hbase/thrift/TestHsHaServerCmdLine.java
  src/test/java/org/apache/hadoop/hbase/thrift/TestNonblockingServerCmdLine.java
  src/test/java/org/apache/hadoop/hbase/thrift/TestThreadPoolServerCmdLine.java
  
src/test/java/org/apache/hadoop/hbase/thrift/TestThreadedSelectorServerCmdLine.java
  src/test/java/org/apache/hadoop/hbase/thrift/TestThriftServer.java
  src/test/java/org/apache/hadoop/hbase/thrift/ThriftServerCmdLineTestBase.java


> [89-fb] Upgrade hbase 0.89-fb to Thrift 0.8.0 and bring Thrift server 
> enhancements from trunk
> -
>
> Key: HBASE-5803
> URL: https://issues.apache.org/jira/browse/HBASE-5803
> Project: HBase
>  Issue Type: Improvement
>Reporter: Mikhail Bautin
>Assignee: Mikhail Bautin
> Attachments: D2811.1.patch, D2811.2.patch, D2811.3.patch
>
>
> TBoundedThreadPoolServer has been a problem for us when there is a large 
> number of clients. We need to migrate to 0.8.0. in 89-fb and bring the 
> relevant improvements from trunk, including supporting 
> TThreadedSelectorServer.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5763) Fix random failures in TestFSErrorsExposed

[Phabricator (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-5763?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Phabricator updated HBASE-5763:
---

Attachment: D2793.3.patch

mbautin updated the revision "[jira] [HBASE-5763] Fix random failures in 
TestFSErrorsExposed".
Reviewers: stack, tedyu, jdcryans, lhofhansl, jmhsieh, JIRA

  Addressing Ted's feedback.

REVISION DETAIL
  https://reviews.facebook.net/D2793

AFFECTED FILES
  src/test/java/org/apache/hadoop/hbase/MiniHBaseCluster.java
  src/test/java/org/apache/hadoop/hbase/regionserver/TestFSErrorsExposed.java


> Fix random failures in TestFSErrorsExposed
> --
>
> Key: HBASE-5763
> URL: https://issues.apache.org/jira/browse/HBASE-5763
> Project: HBase
>  Issue Type: Bug
>Reporter: Mikhail Bautin
>Assignee: Mikhail Bautin
>Priority: Minor
> Attachments: D2739.1.patch, D2739.2.patch, D2739.3.patch, 
> D2739.4.patch, D2793.1.patch, D2793.2.patch, D2793.3.patch, 
> Fix-TestFSErrorsExposed-2012-04-13_18_59_36.patch, 
> Fix-TestFSErrorsExposed-2012-04-16_15_41_24.patch
>
>


--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-3508) LruBlockCache statistics thread should have a name

[Phabricator (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-3508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Phabricator updated HBASE-3508:
---

Attachment: D2853.1.patch

mbautin requested code review of "[jira] [HBASE-3508] [89-fb] LruBlockCache 
statistics thread should have a name".
Reviewers: Liyin, Kannan, aaiyer, todd, JIRA

  Pulling in the small patch from trunk: HBASE-3508.
  Original description below.

  Currently it's just an unnamed threadpool. It's annoying.


TEST PLAN
  Run a minicluster unit test in a debugger. Ensure that the cache stats thread 
has a name.

REVISION DETAIL
  https://reviews.facebook.net/D2853

AFFECTED FILES
  src/main/java/org/apache/hadoop/hbase/io/hfile/LruBlockCache.java

MANAGE HERALD DIFFERENTIAL RULES
  https://reviews.facebook.net/herald/view/differential/

WHY DID I GET THIS EMAIL?
  https://reviews.facebook.net/herald/transcript/6477/

Tip: use the X-Herald-Rules header to filter Herald messages in your client.


> LruBlockCache statistics thread should have a name
> --
>
> Key: HBASE-3508
> URL: https://issues.apache.org/jira/browse/HBASE-3508
> Project: HBase
>  Issue Type: Improvement
>Reporter: Todd Lipcon
>Assignee: Todd Lipcon
>Priority: Trivial
> Fix For: 0.90.1
>
> Attachments: D2853.1.patch, hbase-3508.txt
>
>
> Currently it's just an unnamed threadpool. It's annoying.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5741) ImportTsv does not check for table existence

[Clint Heath (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-5741?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13255995#comment-13255995
 ] 

Clint Heath commented on HBASE-5741:


Lars, et al, thank you for your questions and comments.  I may be missing 
something, so please correct me if I'm wrong, but here's how I see the 
situation:

1) importtsv will auto-split the input data in region-server-sized Hfiles, 
based on hbase.hregion.max.filesize (because we use the HFileOutputFormat and 
it checks this parameter from the client configs).

2) completebulkload then distributes the resulting HFiles to the region 
servers, thereby populating your cluster with all the regions needed to host 
your data.  Alleviating all the memstore flushes, compactions, splitting, and 
other overhead associated with running billions of individual put()s to 
incrementally load your data.  This is why we recommend the bulk load process 
to initially populate HBase tables.

3) pre-splitting your table (based on rowkeys, as is the only mechanism) is 
useless in this scenario, because it will not take into account the size of 
data associated with each key.  The customer would literally have to walk 
through their data and determine how many Gigs of data are associated with each 
rowkey in order to preemptively split their tables in a bulk load 
scenario...and even then, HBase only takes those pre-splits as "suggestions" 
and if the data ends up needing to be split differently, that will happen 
automatically. So what purpose does pre-splitting serve in a bulk load?  Think 
"initial load" of massive amounts of data.

4) Lars had a good point about compression, yes, typically a customer *should* 
set up their table with compression first, but they typically don't know this 
and they can do it after the fact without consequence.

In conclusion, we provide this handy command-line tool to help our HBase 
customers (most of whom are complete newbies and don't know what they're doing) 
get their HBase tables up and running, yet we give them conflicting information 
about how to use the tool.  If we need to tell them to pre-create the table, 
then we should tell them so...clearly.  However, I think it is just simpler and 
more intuitive if the tool did that automatically, as the javadocs indicate it 
will.  Since region splits are determined internally by the HFileOutputFormat 
anyway (if they use the bulkload option), what's the harm?  This enables people 
to get their data loaded into HBase with the least amount of education, 
ramp-up, java development, data mining, etc., etc.

> ImportTsv does not check for table existence 
> -
>
> Key: HBASE-5741
> URL: https://issues.apache.org/jira/browse/HBASE-5741
> Project: HBase
>  Issue Type: Bug
>  Components: mapreduce
>Affects Versions: 0.90.4
>Reporter: Clint Heath
>Assignee: Himanshu Vashishtha
> Fix For: 0.96.0, 0.94.1
>
> Attachments: 5741-94.txt, 5741-v3.txt, HBase-5741-v2.patch, 
> HBase-5741.patch
>
>
> The usage statement for the "importtsv" command to hbase claims this:
> "Note: if you do not use this option, then the target table must already 
> exist in HBase" (in reference to the "importtsv.bulk.output" command-line 
> option)
> The truth is, the table must exist no matter what, importtsv cannot and will 
> not create it for you.
> This is the case because the createSubmittableJob method of ImportTsv does 
> not even attempt to check if the table exists already, much less create it:
> (From org.apache.hadoop.hbase.mapreduce.ImportTsv.java)
> 305 HTable table = new HTable(conf, tableName);
> The HTable method signature in use there assumes the table exists and runs a 
> meta scan on it:
> (From org.apache.hadoop.hbase.client.HTable.java)
> 142 * Creates an object to access a HBase table.
> ...
> 151 public HTable(Configuration conf, final String tableName)
> What we should do inside of createSubmittableJob is something similar to what 
> the "completebulkloads" command would do:
> (Taken from org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles.java)
> 690 boolean tableExists = this.doesTableExist(tableName);
> 691 if (!tableExists) this.createTable(tableName,dirPath);
> Currently the docs are misleading, the table in fact must exist prior to 
> running importtsv. We should check if it exists rather than assume it's 
> already there and throw the below exception:
> 12/03/14 17:15:42 WARN client.HConnectionManager$HConnectionImplementation: 
> Encountered problems when prefetch META table: 
> org.apache.hadoop.hbase.TableNotFoundException: Cannot find row in .META. for 
> table: myTable2, row=myTable2,,99
>   at 
> org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:150)
> ...

--
This message is automatically ge

[jira] [Updated] (HBASE-5813) Retry immediately after a NotServingRegionException in a multiput

[Phabricator (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-5813?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Phabricator updated HBASE-5813:
---

Attachment: D2847.3.patch

mbautin updated the revision "[jira] [HBASE-5813] [89-fb] Retry immediately 
after a NotServingRegionException in a multiput".
Reviewers: Liyin, Kannan, khemani, todd, tedyu, stack, JIRA

  Addressing Ted's comment.

REVISION DETAIL
  https://reviews.facebook.net/D2847

AFFECTED FILES
  src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java


> Retry immediately after a NotServingRegionException in a multiput
> -
>
> Key: HBASE-5813
> URL: https://issues.apache.org/jira/browse/HBASE-5813
> Project: HBase
>  Issue Type: Improvement
>Reporter: Mikhail Bautin
>Assignee: Mikhail Bautin
> Attachments: D2847.1.patch, D2847.2.patch, D2847.3.patch
>
>
> After we get some errors in a multiput we invalidate the region location 
> cache and wait for the configured time interval according to the backoff 
> policy. However, if all "errors" in multiput processing were 
> NotServingRegionExceptions, we don't really need to wait. We can retry 
> immediately.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-3508) LruBlockCache statistics thread should have a name

[Phabricator (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-3508?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13256004#comment-13256004
 ] 

Phabricator commented on HBASE-3508:


Liyin has accepted the revision "[jira] [HBASE-3508] [89-fb] LruBlockCache 
statistics thread should have a name".

  LGTM:) One minor comment.

INLINE COMMENTS
  src/main/java/org/apache/hadoop/hbase/io/hfile/LruBlockCache.java:129 May set 
it as a daemon thread here

REVISION DETAIL
  https://reviews.facebook.net/D2853

BRANCH
  lrublockcache_statistics_thread_should_have_a_HBASE-3508


> LruBlockCache statistics thread should have a name
> --
>
> Key: HBASE-3508
> URL: https://issues.apache.org/jira/browse/HBASE-3508
> Project: HBase
>  Issue Type: Improvement
>Reporter: Todd Lipcon
>Assignee: Todd Lipcon
>Priority: Trivial
> Fix For: 0.90.1
>
> Attachments: D2853.1.patch, hbase-3508.txt
>
>
> Currently it's just an unnamed threadpool. It's annoying.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5788) Move Dynamic Metrics storage off of HRegion.

[Hadoop QA (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-5788?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13256005#comment-13256005
 ] 

Hadoop QA commented on HBASE-5788:
--

-1 overall.  Here are the results of testing the latest attachment 
  http://issues.apache.org/jira/secure/attachment/12523008/HBASE-5788-4.patch
  against trunk revision .

+1 @author.  The patch does not contain any @author tags.

+1 tests included.  The patch appears to include 3 new or modified tests.

+1 javadoc.  The javadoc tool did not generate any warning messages.

+1 javac.  The applied patch does not increase the total number of javac 
compiler warnings.

-1 findbugs.  The patch appears to introduce 4 new Findbugs (version 1.3.9) 
warnings.

+1 release audit.  The applied patch does not increase the total number of 
release audit warnings.

 -1 core tests.  The patch failed these unit tests:
   org.apache.hadoop.hbase.replication.TestReplication

Test results: 
https://builds.apache.org/job/PreCommit-HBASE-Build/1557//testReport/
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/1557//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
Console output: 
https://builds.apache.org/job/PreCommit-HBASE-Build/1557//console

This message is automatically generated.

> Move Dynamic Metrics storage off of HRegion.
> 
>
> Key: HBASE-5788
> URL: https://issues.apache.org/jira/browse/HBASE-5788
> Project: HBase
>  Issue Type: Improvement
>  Components: regionserver
>Reporter: Elliott Clark
>Assignee: Elliott Clark
>Priority: Minor
> Attachments: HBASE-5788-0.patch, HBASE-5788-1.patch, 
> HBASE-5788-2.patch, HBASE-5788-3.patch, HBASE-5788-4.patch
>
>
> HRegion right now has the responsibility of storing static counts and latency 
> numbers for use by the metrics package.  Since these maps are incremented and 
> set from lots of places it makes adding functionality hard.
>  
> So move the metrics functionality into SchemaMetrics making it more than just 
> a class for naming.  The next step will be to simplify the api exposed so 
> that using it will be easier.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5813) Retry immediately after a NotServingRegionException in a multiput

[Phabricator (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-5813?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Phabricator updated HBASE-5813:
---

Attachment: D2847.4.patch

mbautin updated the revision "[jira] [HBASE-5813] [89-fb] Retry immediately 
after a NotServingRegionException in a multiput".
Reviewers: Liyin, Kannan, khemani, todd, tedyu, stack, JIRA

  Using the previous logic (enabling the wait) in case of InterruptedException.

REVISION DETAIL
  https://reviews.facebook.net/D2847

AFFECTED FILES
  src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java


> Retry immediately after a NotServingRegionException in a multiput
> -
>
> Key: HBASE-5813
> URL: https://issues.apache.org/jira/browse/HBASE-5813
> Project: HBase
>  Issue Type: Improvement
>Reporter: Mikhail Bautin
>Assignee: Mikhail Bautin
> Attachments: D2847.1.patch, D2847.2.patch, D2847.3.patch, 
> D2847.4.patch
>
>
> After we get some errors in a multiput we invalidate the region location 
> cache and wait for the configured time interval according to the backoff 
> policy. However, if all "errors" in multiput processing were 
> NotServingRegionExceptions, we don't really need to wait. We can retry 
> immediately.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5763) Fix random failures in TestFSErrorsExposed

[Hadoop QA (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-5763?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13256028#comment-13256028
 ] 

Hadoop QA commented on HBASE-5763:
--

-1 overall.  Here are the results of testing the latest attachment 
  http://issues.apache.org/jira/secure/attachment/12523031/D2793.3.patch
  against trunk revision .

+1 @author.  The patch does not contain any @author tags.

+1 tests included.  The patch appears to include 6 new or modified tests.

+1 javadoc.  The javadoc tool did not generate any warning messages.

+1 javac.  The applied patch does not increase the total number of javac 
compiler warnings.

-1 findbugs.  The patch appears to introduce 4 new Findbugs (version 1.3.9) 
warnings.

+1 release audit.  The applied patch does not increase the total number of 
release audit warnings.

 -1 core tests.  The patch failed these unit tests:
 

Test results: 
https://builds.apache.org/job/PreCommit-HBASE-Build/1558//testReport/
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/1558//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
Console output: 
https://builds.apache.org/job/PreCommit-HBASE-Build/1558//console

This message is automatically generated.

> Fix random failures in TestFSErrorsExposed
> --
>
> Key: HBASE-5763
> URL: https://issues.apache.org/jira/browse/HBASE-5763
> Project: HBase
>  Issue Type: Bug
>Reporter: Mikhail Bautin
>Assignee: Mikhail Bautin
>Priority: Minor
> Attachments: D2739.1.patch, D2739.2.patch, D2739.3.patch, 
> D2739.4.patch, D2793.1.patch, D2793.2.patch, D2793.3.patch, 
> Fix-TestFSErrorsExposed-2012-04-13_18_59_36.patch, 
> Fix-TestFSErrorsExposed-2012-04-16_15_41_24.patch
>
>


--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5782) Edits can be appended out of seqid order since HBASE-4487

[Todd Lipcon (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-5782?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13256030#comment-13256030
 ] 

Todd Lipcon commented on HBASE-5782:


I think for 0.94.0 we should commit Lars's, and file another JIRA to clean up / 
rewrite this stuff to be less circuitous. Could definitely do with a cleanup.

> Edits can be appended out of seqid order since HBASE-4487
> -
>
> Key: HBASE-5782
> URL: https://issues.apache.org/jira/browse/HBASE-5782
> Project: HBase
>  Issue Type: Bug
>  Components: wal
>Affects Versions: 0.94.0
>Reporter: Gopinathan A
>Assignee: ramkrishna.s.vasudevan
>Priority: Blocker
> Fix For: 0.94.0
>
> Attachments: 5782-lars-v2.txt, 5782-sketch.txt, 5782.txt, 
> 5782.unfinished-stack.txt, HBASE-5782.patch, hbase-5782.txt
>
>
> Create a table with 1000 splits, after the region assignemnt, kill the 
> regionserver wich contains META table.
> Here few regions are missing after the log splitting and region assigment. 
> HBCK report shows multiple region holes are got created.
> Same scenario was verified mulitple times in 0.92.1, no issues.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5788) Move Dynamic Metrics storage off of HRegion.

[stack (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-5788?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stack updated HBASE-5788:
-

   Resolution: Fixed
Fix Version/s: 0.96.0
   Status: Resolved  (was: Patch Available)

Applied to Trunk (Just TestReplication failed... which seems unrelated).  
Thanks for the patch Elliott.

> Move Dynamic Metrics storage off of HRegion.
> 
>
> Key: HBASE-5788
> URL: https://issues.apache.org/jira/browse/HBASE-5788
> Project: HBase
>  Issue Type: Improvement
>  Components: regionserver
>Reporter: Elliott Clark
>Assignee: Elliott Clark
>Priority: Minor
> Fix For: 0.96.0
>
> Attachments: HBASE-5788-0.patch, HBASE-5788-1.patch, 
> HBASE-5788-2.patch, HBASE-5788-3.patch, HBASE-5788-4.patch
>
>
> HRegion right now has the responsibility of storing static counts and latency 
> numbers for use by the metrics package.  Since these maps are incremented and 
> set from lots of places it makes adding functionality hard.
>  
> So move the metrics functionality into SchemaMetrics making it more than just 
> a class for naming.  The next step will be to simplify the api exposed so 
> that using it will be easier.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5803) [89-fb] Upgrade hbase 0.89-fb to Thrift 0.8.0 and bring Thrift server enhancements from trunk

[Phabricator (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-5803?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13256040#comment-13256040
 ] 

Phabricator commented on HBASE-5803:


sc has accepted the revision "[jira] [HBASE-5803] [89-fb] Upgrade hbase 0.89-fb 
to Thrift 0.8.0 and bring Thrift server enhancements from trunk".

  Looks good to me. Good work!

REVISION DETAIL
  https://reviews.facebook.net/D2811

BRANCH
  upgrade_hbase_0_89_fb_to_thrift_0_8_0_and_HBASE-5803_v9


> [89-fb] Upgrade hbase 0.89-fb to Thrift 0.8.0 and bring Thrift server 
> enhancements from trunk
> -
>
> Key: HBASE-5803
> URL: https://issues.apache.org/jira/browse/HBASE-5803
> Project: HBase
>  Issue Type: Improvement
>Reporter: Mikhail Bautin
>Assignee: Mikhail Bautin
> Attachments: D2811.1.patch, D2811.2.patch, D2811.3.patch
>
>
> TBoundedThreadPoolServer has been a problem for us when there is a large 
> number of clients. We need to migrate to 0.8.0. in 89-fb and bring the 
> relevant improvements from trunk, including supporting 
> TThreadedSelectorServer.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5782) Edits can be appended out of seqid order since HBASE-4487

[stack (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-5782?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13256043#comment-13256043
 ] 

stack commented on HBASE-5782:
--

I made the hlog perf tool work on hdfs and ran some basic tests.  Both Todd an 
Lars' patches seem faster than what we have currently.

Running w/o a fix on hdfs w/ current trunk I have to disable verify because it 
fails (verify happens after we print out test timings).

$ ./bin/hbase 
org.apache.hadoop.hbase.regionserver.wal.HLogPerformanceEvaluation -conf 
/home/stack/hadoop-conf/core-site.xml -path hdfs://sv4r11s38:7000/tmp -threads 
100 -roll 1

12/04/17 22:58:28 INFO wal.HLogPerformanceEvaluation: Summary: threads=100, 
iterations=1 took 100.630s 9937.395ops/s
12/04/17 23:00:33 INFO wal.HLogPerformanceEvaluation: Summary: threads=100, 
iterations=1 took 94.945s 10532.413ops/s

Todd patch on hdfs:

$ ./bin/hbase 
org.apache.hadoop.hbase.regionserver.wal.HLogPerformanceEvaluation -conf 
/home/stack/hadoop-conf/core-site.xml -path hdfs://sv4r11s38:7000/tmp -threads 
100 -roll 1 -verify

12/04/17 22:53:35 INFO wal.HLogPerformanceEvaluation: Summary: threads=100, 
iterations=1 took 81.202s 12314.967ops/s

Lars patch:

12/04/17 23:07:08 INFO wal.HLogPerformanceEvaluation: Summary: threads=100, 
iterations=1 took 76.800s 13020.833ops/s

For Todd and Lars, both pass verify which checks that seqids are ordered and 
that we wrote as much as we think we did.

> Edits can be appended out of seqid order since HBASE-4487
> -
>
> Key: HBASE-5782
> URL: https://issues.apache.org/jira/browse/HBASE-5782
> Project: HBase
>  Issue Type: Bug
>  Components: wal
>Affects Versions: 0.94.0
>Reporter: Gopinathan A
>Assignee: ramkrishna.s.vasudevan
>Priority: Blocker
> Fix For: 0.94.0
>
> Attachments: 5782-lars-v2.txt, 5782-sketch.txt, 5782.txt, 
> 5782.unfinished-stack.txt, HBASE-5782.patch, hbase-5782.txt
>
>
> Create a table with 1000 splits, after the region assignemnt, kill the 
> regionserver wich contains META table.
> Here few regions are missing after the log splitting and region assigment. 
> HBCK report shows multiple region holes are got created.
> Same scenario was verified mulitple times in 0.92.1, no issues.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5782) Edits can be appended out of seqid order since HBASE-4487

[stack (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-5782?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13256044#comment-13256044
 ] 

stack commented on HBASE-5782:
--

Ok on lars patch into 0.94.

> Edits can be appended out of seqid order since HBASE-4487
> -
>
> Key: HBASE-5782
> URL: https://issues.apache.org/jira/browse/HBASE-5782
> Project: HBase
>  Issue Type: Bug
>  Components: wal
>Affects Versions: 0.94.0
>Reporter: Gopinathan A
>Assignee: ramkrishna.s.vasudevan
>Priority: Blocker
> Fix For: 0.94.0
>
> Attachments: 5782-lars-v2.txt, 5782-sketch.txt, 5782.txt, 
> 5782.unfinished-stack.txt, HBASE-5782.patch, hbase-5782.txt
>
>
> Create a table with 1000 splits, after the region assignemnt, kill the 
> regionserver wich contains META table.
> Here few regions are missing after the log splitting and region assigment. 
> HBCK report shows multiple region holes are got created.
> Same scenario was verified mulitple times in 0.92.1, no issues.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (HBASE-5814) Make HLogPerformanceEvaluation work on top of hdfs

[stack (Created) (JIRA)]

Make HLogPerformanceEvaluation work on top of hdfs
--

 Key: HBASE-5814
 URL: https://issues.apache.org/jira/browse/HBASE-5814
 Project: HBase
  Issue Type: Improvement
Reporter: stack




--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5812) Add log rolling to HLogPerformanceEvaluation

[Hudson (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-5812?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13256050#comment-13256050
 ] 

Hudson commented on HBASE-5812:
---

Integrated in HBase-TRUNK #2775 (See 
[https://builds.apache.org/job/HBase-TRUNK/2775/])
HBASE-5812 Add log rolling to HLogPerformanceEvaluation (Revision 1327282)

 Result = FAILURE
stack : 
Files : 
* 
/hbase/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/wal/HLogPerformanceEvaluation.java


> Add log rolling to HLogPerformanceEvaluation
> 
>
> Key: HBASE-5812
> URL: https://issues.apache.org/jira/browse/HBASE-5812
> Project: HBase
>  Issue Type: Task
>Reporter: stack
>Assignee: stack
> Fix For: 0.96.0
>
> Attachments: 5812.txt
>
>
> Add being able to ask that HLogPerformanceEvaluation rolls logs when its 
> running.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5814) Make HLogPerformanceEvaluation work on top of hdfs

[stack (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-5814?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stack updated HBASE-5814:
-

Attachment: 5814.txt

What I actually committed.

> Make HLogPerformanceEvaluation work on top of hdfs
> --
>
> Key: HBASE-5814
> URL: https://issues.apache.org/jira/browse/HBASE-5814
> Project: HBase
>  Issue Type: Improvement
>Reporter: stack
> Attachments: 5814.txt, diff.txt
>
>


--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira