[jira] [Commented] (HBASE-6917) Trunk jdk7 build broke because we moved to zk 3.4.4

[Andrew Purtell (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-6917?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13467555#comment-13467555
 ] 

Andrew Purtell commented on HBASE-6917:
---

Yep that's ZOOKEEPER-1550.

> Trunk jdk7 build broke because we moved to zk 3.4.4
> ---
>
> Key: HBASE-6917
> URL: https://issues.apache.org/jira/browse/HBASE-6917
> Project: HBase
>  Issue Type: Bug
>  Components: test
>Reporter: stack
>Assignee: stack
> Fix For: 0.96.0
>
> Attachments: 6917.txt
>
>
> Chatted w/ Mahadev and he confirmed issues running 3.4.4 w/ jdk7.  Will be 
> fixed in zk3.4.5.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-6918) Debugging to help figure what is different up on jenkins when TestHeapSize runs

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-6918?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13467554#comment-13467554
 ] 

Hudson commented on HBASE-6918:
---

Integrated in HBase-TRUNK #3404 (See 
[https://builds.apache.org/job/HBase-TRUNK/3404/])
HBASE-6918 Debugging to help figure what is different up on jenkins when 
TestHeapSize runs (Revision 1392753)

 Result = FAILURE
stack : 
Files : 
* 
/hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/io/TestHeapSize.java


> Debugging to help figure what is different up on jenkins when TestHeapSize 
> runs
> ---
>
> Key: HBASE-6918
> URL: https://issues.apache.org/jira/browse/HBASE-6918
> Project: HBase
>  Issue Type: Bug
>  Components: test
>Affects Versions: 0.96.0
>Reporter: stack
>Assignee: stack
> Fix For: 0.96.0
>
> Attachments: 6918.txt
>
>
> TestHeapSize will pass locally then fail up on jenkins.  Add debugging to the 
> test that prints detail on environment and jvm in particular to can figure 
> what is different between two environments (it used to be that the d32 flag 
> was needed to mimic jenkins but now we are jdk7, the d32 flag doesn't work w/ 
> my oracle jvm... let me see what jenkins has).

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-6914) Scans/Gets/Mutations don't give a good error if the table is disabled.

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-6914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13467567#comment-13467567
 ] 

Hadoop QA commented on HBASE-6914:
--

{color:red}-1 overall{color}.  Here are the results of testing the latest 
attachment 
  http://issues.apache.org/jira/secure/attachment/12547348/HBASE-6914-3.patch
  against trunk revision .

{color:green}+1 @author{color}.  The patch does not contain any @author 
tags.

{color:green}+1 tests included{color}.  The patch appears to include 3 new 
or modified tests.

{color:green}+1 hadoop2.0{color}.  The patch compiles against the hadoop 
2.0 profile.

{color:red}-1 javadoc{color}.  The javadoc tool appears to have generated 
149 warning messages.

{color:green}+1 javac{color}.  The applied patch does not increase the 
total number of javac compiler warnings.

{color:red}-1 findbugs{color}.  The patch appears to introduce 7 new 
Findbugs (version 1.3.9) warnings.

{color:green}+1 release audit{color}.  The applied patch does not increase 
the total number of release audit warnings.

{color:green}+1 core tests{color}.  The patch passed unit tests in .

Test results: 
https://builds.apache.org/job/PreCommit-HBASE-Build/2983//testReport/
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/2983//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/2983//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/2983//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/2983//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/2983//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output: 
https://builds.apache.org/job/PreCommit-HBASE-Build/2983//console

This message is automatically generated.

> Scans/Gets/Mutations don't give a good error if the table is disabled.
> --
>
> Key: HBASE-6914
> URL: https://issues.apache.org/jira/browse/HBASE-6914
> Project: HBase
>  Issue Type: Improvement
>Reporter: Elliott Clark
>Assignee: Elliott Clark
> Fix For: 0.92.3, 0.94.3, 0.96.0
>
> Attachments: HBASE-6914-0.patch, HBASE-6914-1.patch, 
> HBASE-6914-2.patch, HBASE-6914-3.patch
>
>
> Scan a table that is disabled will have the client retry multiple times and 
> then will error out with NotServingRegionException.  If the table is disabled 
> there's no need to re-try and the message should be more explicit.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5937) Refactor HLog into an interface.

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-5937?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13467575#comment-13467575
 ] 

Hadoop QA commented on HBASE-5937:
--

{color:red}-1 overall{color}.  Here are the results of testing the latest 
attachment 
  http://issues.apache.org/jira/secure/attachment/12547291/HBASE-5937.v2.patch
  against trunk revision .

{color:green}+1 @author{color}.  The patch does not contain any @author 
tags.

{color:green}+1 tests included{color}.  The patch appears to include 72 new 
or modified tests.

{color:green}+1 hadoop2.0{color}.  The patch compiles against the hadoop 
2.0 profile.

{color:red}-1 javadoc{color}.  The javadoc tool appears to have generated 
157 warning messages.

{color:green}+1 javac{color}.  The applied patch does not increase the 
total number of javac compiler warnings.

{color:red}-1 findbugs{color}.  The patch appears to introduce 6 new 
Findbugs (version 1.3.9) warnings.

{color:green}+1 release audit{color}.  The applied patch does not increase 
the total number of release audit warnings.

 {color:red}-1 core tests{color}.  The patch failed these unit tests:
   org.apache.hadoop.hbase.client.TestHCM

Test results: 
https://builds.apache.org/job/PreCommit-HBASE-Build/2984//testReport/
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/2984//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/2984//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/2984//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/2984//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/2984//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Console output: 
https://builds.apache.org/job/PreCommit-HBASE-Build/2984//console

This message is automatically generated.

> Refactor HLog into an interface.
> 
>
> Key: HBASE-5937
> URL: https://issues.apache.org/jira/browse/HBASE-5937
> Project: HBase
>  Issue Type: Sub-task
>Reporter: Li Pi
>Assignee: Flavio Junqueira
>Priority: Minor
> Attachments: 5937-hlog-with-javadoc.txt, HBASE-5937.patch, 
> HBASE-5937.patch, HBASE-5937.patch, HBASE-5937.patch, HBASE-5937.patch, 
> HBASE-5937.v2.patch, 
> org.apache.hadoop.hbase.client.TestMultiParallel-output.txt
>
>
> What the summary says. Create HLog interface. Make current implementation use 
> it.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-6805) Extend co-processor framework to provide observers for filter operations

[Jason Dai (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-6805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13467577#comment-13467577
 ] 

Jason Dai commented on HBASE-6805:
--

[~apurtell] The example in the updated patch file shows a possible example: the 
value of each cell stored in the table is automatically encrypted by the CP, 
which then needs to decrypt the cell before applying filter operations 
(filterKeyValue, transform, etc.). By implementing the filter CP, the 
encryption can be transparent to the user code. Similarly, for DOT, multiple 
fields are encoded in a single cell by the CP, and each field needs to be 
extracted before applying filter operations so that it can be transparent to 
the user.

bq. If extending the CP hook model to internal filter methods, we must be 
deeply concerned about the costs of iterating CP hook lists during 
filtering/scanning. CPs extend the code path, first of all. Then, if hooks are 
registered, there will be method invocation and object allocation costs for 
_every_ filter operation, twice.

While there are two method invocations for each filter operation, these method 
invocations are actually only called for the topmost filter (which 
FilterWrapper wraps), not for each filter contained in the chained FilterList 
or other composite filters. In our DOT benchmarking, these CP operations are 
never the hotspot in scanning.

Having said that, CP operations could become a potential performance issue if 
we have a long list of CPs loaded. For instance, database trigger like CPs only 
execute upon data mutation (i.e., Put), but are still invoked for 
Get/Scan/Filter. One way to address this issue is that, instead of iterating 
the global _coprocessor_ set in these pre* & post* operations, the 
RegionCoprocessorHost can maintain several CP set, and iterate a different set 
in each different CP operation: one for region operations 
(preOpen/postOpen/preClose/...), one for update (prePut & postPut), one for 
read (preGet/postGet/preScannerOpen/...), and one for filter 
(preFilterKeyvalue/postFilterKeyvalue/); when loading each CP, it can be 
registered in appropriate sets (just as endpoints are registered in 
_Region.protocolHandlers_).

> Extend co-processor framework to provide observers for filter operations
> 
>
> Key: HBASE-6805
> URL: https://issues.apache.org/jira/browse/HBASE-6805
> Project: HBase
>  Issue Type: Sub-task
>  Components: Coprocessors
>Affects Versions: 0.96.0
>Reporter: Jason Dai
> Attachments: extend_coprocessor.patch
>
>
> There are several filter operations (e.g., filterKeyValue, filterRow, 
> transform, etc.) at the region server side that either exclude KVs from the 
> returned results, or transform the returned KV. We need to provide observers 
> (e.g., preFilterKeyValue and postFilterKeyValue) for these operations in the 
> same way as the observers for other data access operations (e.g., preGet and 
> postGet). This extension is needed to support DOT (e.g., extracting 
> individual fields from the document in the observers before passing them to 
> the related filter operations) 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-6317) Master clean start up and Partially enabled tables make region assignment inconsistent.

[ramkrishna.s.vasudevan (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-6317?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

ramkrishna.s.vasudevan updated HBASE-6317:
--

  Resolution: Fixed
Hadoop Flags: Reviewed
  Status: Resolved  (was: Patch Available)

Thanks for the patch Rajesh
Thanks for the review Stack, Jimmy and Ted.

> Master clean start up and Partially enabled tables make region assignment 
> inconsistent.
> ---
>
> Key: HBASE-6317
> URL: https://issues.apache.org/jira/browse/HBASE-6317
> Project: HBase
>  Issue Type: Bug
>Reporter: ramkrishna.s.vasudevan
>Assignee: rajeshbabu
> Fix For: 0.92.3, 0.94.3, 0.96.0
>
> Attachments: HBASE-6317_94_3.patch, HBASE-6317_94.patch, 
> HBASE-6317_trunk_2.patch, HBASE-6317_trunk_3.patch, HBASE-6317_trunk_4.patch, 
> HBASE-6317_trunk_5.patch
>
>
> If we have a  table in partially enabled state (ENABLING) then on HMaster 
> restart we treat it as a clean cluster start up and do a bulk assign.  
> Currently in 0.94 bulk assign will not handle ALREADY_OPENED scenarios and it 
> leads to region assignment problems.  Analysing more on this we found that we 
> have better way to handle these scenarios.
> {code}
> if (false == checkIfRegionBelongsToDisabled(regionInfo)
> && false == checkIfRegionsBelongsToEnabling(regionInfo)) {
>   synchronized (this.regions) {
> regions.put(regionInfo, regionLocation);
> addToServers(regionLocation, regionInfo);
>   }
> {code}
> We dont add to regions map so that enable table handler can handle it.  But 
> as nothing is added to regions map we think it as a clean cluster start up.
> Will come up with a patch tomorrow.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5937) Refactor HLog into an interface.

[Flavio Junqueira (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-5937?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13467589#comment-13467589
 ] 

Flavio Junqueira commented on HBASE-5937:
-

I'll add an issue to the umbrella jira to revisit javadocs.

I don't see how TestHCM could possibly have failed because of this patch.

I'll fix the findbugs warnings.

> Refactor HLog into an interface.
> 
>
> Key: HBASE-5937
> URL: https://issues.apache.org/jira/browse/HBASE-5937
> Project: HBase
>  Issue Type: Sub-task
>Reporter: Li Pi
>Assignee: Flavio Junqueira
>Priority: Minor
> Attachments: 5937-hlog-with-javadoc.txt, HBASE-5937.patch, 
> HBASE-5937.patch, HBASE-5937.patch, HBASE-5937.patch, HBASE-5937.patch, 
> HBASE-5937.v2.patch, 
> org.apache.hadoop.hbase.client.TestMultiParallel-output.txt
>
>
> What the summary says. Create HLog interface. Make current implementation use 
> it.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-6900) RegionScanner.reseek() creates NPE when a flush or compaction happens before the reseek.

[ramkrishna.s.vasudevan (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-6900?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13467590#comment-13467590
 ] 

ramkrishna.s.vasudevan commented on HBASE-6900:
---

[~lhofhansl]
Can you take a look at this?  So that i can rebase the patch based on your 
suggestion.

> RegionScanner.reseek() creates NPE when a flush or compaction happens before 
> the reseek.
> 
>
> Key: HBASE-6900
> URL: https://issues.apache.org/jira/browse/HBASE-6900
> Project: HBase
>  Issue Type: Bug
>Reporter: ramkrishna.s.vasudevan
>Assignee: ramkrishna.s.vasudevan
> Attachments: HBASE-6900_1.patch, HBASE-6900.patch
>
>
> HBASE-5520 introduced reseek() on the RegionScanner.  
> Now when a scanner is created we have the StoreScanner heap.  After this if a 
> flush or compaction happens parallely all the StoreScannerObservers are 
> cleared so that whenever a new next() call happens we tend to recreate the 
> scanner based on the latest store files.
> The reseek() in StoreScanner expects the heap not to be null because always 
> reseek would be called from next()
> {code}
> public synchronized boolean reseek(KeyValue kv) throws IOException {
> //Heap cannot be null, because this is only called from next() which
> //guarantees that heap will never be null before this call.
> if (explicitColumnQuery && lazySeekEnabledGlobally) {
>   return heap.requestSeek(kv, true, useRowColBloom);
> } else {
>   return heap.reseek(kv);
> }
>   }
> {code}
> Now when we call RegionScanner.reseek() directly using CPs we tend to get a 
> NPE.  In our case it happened when a major compaction was going on.  I will 
> also attach a testcase to show the problem.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-6317) Master clean start up and Partially enabled tables make region assignment inconsistent.

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-6317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13467601#comment-13467601
 ] 

Hudson commented on HBASE-6317:
---

Integrated in HBase-TRUNK #3405 (See 
[https://builds.apache.org/job/HBase-TRUNK/3405/])
HBASE-6317 Master clean start up and Partially enabled tables make region 
assignment inconsistent (RajeshBabu) (Revision 1392802)

 Result = FAILURE
ramkrishna : 
Files : 
* 
/hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/master/handler/EnableTableHandler.java
* 
/hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestAssignmentManager.java


> Master clean start up and Partially enabled tables make region assignment 
> inconsistent.
> ---
>
> Key: HBASE-6317
> URL: https://issues.apache.org/jira/browse/HBASE-6317
> Project: HBase
>  Issue Type: Bug
>Reporter: ramkrishna.s.vasudevan
>Assignee: rajeshbabu
> Fix For: 0.92.3, 0.94.3, 0.96.0
>
> Attachments: HBASE-6317_94_3.patch, HBASE-6317_94.patch, 
> HBASE-6317_trunk_2.patch, HBASE-6317_trunk_3.patch, HBASE-6317_trunk_4.patch, 
> HBASE-6317_trunk_5.patch
>
>
> If we have a  table in partially enabled state (ENABLING) then on HMaster 
> restart we treat it as a clean cluster start up and do a bulk assign.  
> Currently in 0.94 bulk assign will not handle ALREADY_OPENED scenarios and it 
> leads to region assignment problems.  Analysing more on this we found that we 
> have better way to handle these scenarios.
> {code}
> if (false == checkIfRegionBelongsToDisabled(regionInfo)
> && false == checkIfRegionsBelongsToEnabling(regionInfo)) {
>   synchronized (this.regions) {
> regions.put(regionInfo, regionLocation);
> addToServers(regionLocation, regionInfo);
>   }
> {code}
> We dont add to regions map so that enable table handler can handle it.  But 
> as nothing is added to regions map we think it as a clean cluster start up.
> Will come up with a patch tomorrow.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-6317) Master clean start up and Partially enabled tables make region assignment inconsistent.

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-6317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13467624#comment-13467624
 ] 

Hudson commented on HBASE-6317:
---

Integrated in HBase-TRUNK-on-Hadoop-2.0.0 #203 (See 
[https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-2.0.0/203/])
HBASE-6317 Master clean start up and Partially enabled tables make region 
assignment inconsistent (RajeshBabu) (Revision 1392802)

 Result = FAILURE
ramkrishna : 
Files : 
* 
/hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/master/handler/EnableTableHandler.java
* 
/hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestAssignmentManager.java


> Master clean start up and Partially enabled tables make region assignment 
> inconsistent.
> ---
>
> Key: HBASE-6317
> URL: https://issues.apache.org/jira/browse/HBASE-6317
> Project: HBase
>  Issue Type: Bug
>Reporter: ramkrishna.s.vasudevan
>Assignee: rajeshbabu
> Fix For: 0.92.3, 0.94.3, 0.96.0
>
> Attachments: HBASE-6317_94_3.patch, HBASE-6317_94.patch, 
> HBASE-6317_trunk_2.patch, HBASE-6317_trunk_3.patch, HBASE-6317_trunk_4.patch, 
> HBASE-6317_trunk_5.patch
>
>
> If we have a  table in partially enabled state (ENABLING) then on HMaster 
> restart we treat it as a clean cluster start up and do a bulk assign.  
> Currently in 0.94 bulk assign will not handle ALREADY_OPENED scenarios and it 
> leads to region assignment problems.  Analysing more on this we found that we 
> have better way to handle these scenarios.
> {code}
> if (false == checkIfRegionBelongsToDisabled(regionInfo)
> && false == checkIfRegionsBelongsToEnabling(regionInfo)) {
>   synchronized (this.regions) {
> regions.put(regionInfo, regionLocation);
> addToServers(regionLocation, regionInfo);
>   }
> {code}
> We dont add to regions map so that enable table handler can handle it.  But 
> as nothing is added to regions map we think it as a clean cluster start up.
> Will come up with a patch tomorrow.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-6918) Debugging to help figure what is different up on jenkins when TestHeapSize runs

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-6918?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13467623#comment-13467623
 ] 

Hudson commented on HBASE-6918:
---

Integrated in HBase-TRUNK-on-Hadoop-2.0.0 #203 (See 
[https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-2.0.0/203/])
HBASE-6918 Debugging to help figure what is different up on jenkins when 
TestHeapSize runs (Revision 1392753)

 Result = FAILURE
stack : 
Files : 
* 
/hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/io/TestHeapSize.java


> Debugging to help figure what is different up on jenkins when TestHeapSize 
> runs
> ---
>
> Key: HBASE-6918
> URL: https://issues.apache.org/jira/browse/HBASE-6918
> Project: HBase
>  Issue Type: Bug
>  Components: test
>Affects Versions: 0.96.0
>Reporter: stack
>Assignee: stack
> Fix For: 0.96.0
>
> Attachments: 6918.txt
>
>
> TestHeapSize will pass locally then fail up on jenkins.  Add debugging to the 
> test that prints detail on environment and jvm in particular to can figure 
> what is different between two environments (it used to be that the d32 flag 
> was needed to mimic jenkins but now we are jdk7, the d32 flag doesn't work w/ 
> my oracle jvm... let me see what jenkins has).

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-6915) String and ConcurrentHashMap sizes change on jdk7; makes TestHeapSize fail

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-6915?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13467622#comment-13467622
 ] 

Hudson commented on HBASE-6915:
---

Integrated in HBase-TRUNK-on-Hadoop-2.0.0 #203 (See 
[https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-2.0.0/203/])
HBASE-6915 String and ConcurrentHashMap sizes change on jdk7; makes 
TestHeapSize fail (Revision 1392731)

 Result = FAILURE
stack : 
Files : 
* 
/hbase/trunk/hbase-common/src/main/java/org/apache/hadoop/hbase/util/ClassSize.java


> String and ConcurrentHashMap sizes change on jdk7; makes TestHeapSize fail
> --
>
> Key: HBASE-6915
> URL: https://issues.apache.org/jira/browse/HBASE-6915
> Project: HBase
>  Issue Type: Bug
>Reporter: stack
>Assignee: stack
> Fix For: 0.96.0
>
> Attachments: jdk7.txt
>
>


--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-6917) Trunk jdk7 build broke because we moved to zk 3.4.4

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-6917?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13467625#comment-13467625
 ] 

Hudson commented on HBASE-6917:
---

Integrated in HBase-TRUNK-on-Hadoop-2.0.0 #203 (See 
[https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-2.0.0/203/])
HBASE-6917 Trunk jdk7 build broke because we moved to zk 3.4.4 (Revision 
1392746)

 Result = FAILURE
stack : 
Files : 
* /hbase/trunk/pom.xml


> Trunk jdk7 build broke because we moved to zk 3.4.4
> ---
>
> Key: HBASE-6917
> URL: https://issues.apache.org/jira/browse/HBASE-6917
> Project: HBase
>  Issue Type: Bug
>  Components: test
>Reporter: stack
>Assignee: stack
> Fix For: 0.96.0
>
> Attachments: 6917.txt
>
>
> Chatted w/ Mahadev and he confirmed issues running 3.4.4 w/ jdk7.  Will be 
> fixed in zk3.4.5.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Reopened] (HBASE-6317) Master clean start up and Partially enabled tables make region assignment inconsistent.

[ramkrishna.s.vasudevan (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-6317?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

ramkrishna.s.vasudevan reopened HBASE-6317:
---


Not committed to other branches other than trunk.. Will resolve commiting in 
other branches also.

> Master clean start up and Partially enabled tables make region assignment 
> inconsistent.
> ---
>
> Key: HBASE-6317
> URL: https://issues.apache.org/jira/browse/HBASE-6317
> Project: HBase
>  Issue Type: Bug
>Reporter: ramkrishna.s.vasudevan
>Assignee: rajeshbabu
> Fix For: 0.92.3, 0.94.3, 0.96.0
>
> Attachments: HBASE-6317_94_3.patch, HBASE-6317_94.patch, 
> HBASE-6317_trunk_2.patch, HBASE-6317_trunk_3.patch, HBASE-6317_trunk_4.patch, 
> HBASE-6317_trunk_5.patch
>
>
> If we have a  table in partially enabled state (ENABLING) then on HMaster 
> restart we treat it as a clean cluster start up and do a bulk assign.  
> Currently in 0.94 bulk assign will not handle ALREADY_OPENED scenarios and it 
> leads to region assignment problems.  Analysing more on this we found that we 
> have better way to handle these scenarios.
> {code}
> if (false == checkIfRegionBelongsToDisabled(regionInfo)
> && false == checkIfRegionsBelongsToEnabling(regionInfo)) {
>   synchronized (this.regions) {
> regions.put(regionInfo, regionLocation);
> addToServers(regionLocation, regionInfo);
>   }
> {code}
> We dont add to regions map so that enable table handler can handle it.  But 
> as nothing is added to regions map we think it as a clean cluster start up.
> Will come up with a patch tomorrow.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5937) Refactor HLog into an interface.

[Flavio Junqueira (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-5937?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Flavio Junqueira updated HBASE-5937:


Status: Open  (was: Patch Available)

> Refactor HLog into an interface.
> 
>
> Key: HBASE-5937
> URL: https://issues.apache.org/jira/browse/HBASE-5937
> Project: HBase
>  Issue Type: Sub-task
>Reporter: Li Pi
>Assignee: Flavio Junqueira
>Priority: Minor
> Attachments: 5937-hlog-with-javadoc.txt, HBASE-5937.patch, 
> HBASE-5937.patch, HBASE-5937.patch, HBASE-5937.patch, HBASE-5937.patch, 
> HBASE-5937.v2.patch, HBASE-5937.v3.patch, 
> org.apache.hadoop.hbase.client.TestMultiParallel-output.txt
>
>
> What the summary says. Create HLog interface. Make current implementation use 
> it.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5937) Refactor HLog into an interface.

[Flavio Junqueira (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-5937?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Flavio Junqueira updated HBASE-5937:


Attachment: HBASE-5937.v3.patch

> Refactor HLog into an interface.
> 
>
> Key: HBASE-5937
> URL: https://issues.apache.org/jira/browse/HBASE-5937
> Project: HBase
>  Issue Type: Sub-task
>Reporter: Li Pi
>Assignee: Flavio Junqueira
>Priority: Minor
> Attachments: 5937-hlog-with-javadoc.txt, HBASE-5937.patch, 
> HBASE-5937.patch, HBASE-5937.patch, HBASE-5937.patch, HBASE-5937.patch, 
> HBASE-5937.v2.patch, HBASE-5937.v3.patch, 
> org.apache.hadoop.hbase.client.TestMultiParallel-output.txt
>
>
> What the summary says. Create HLog interface. Make current implementation use 
> it.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5937) Refactor HLog into an interface.

[Flavio Junqueira (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-5937?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Flavio Junqueira updated HBASE-5937:


Status: Patch Available  (was: Open)

I have checked the new findbugs warning and they haven't been introduced by any 
of the changes I made. They appear as new because I moved code around. 

I have looked into finxing them, though. I fixed one about a volatile variable 
by making it an AtomicInteger. The warning was about incrementing a volatile 
variable. 

There are two warnings about static variables being set to a mutable array in 
HLog. This is due to the use of Bytes.toBytes(). I didn't want to change that, 
so I left as is. 

There two other warnings related to this code excerpt:

{noformat}
  synchronized (closeLogSyncer) {
closeLogSyncer.set(true);
closeLogSyncer.notifyAll();
  }

{noformat}

where closeLogSyncer is an AtomicBoolean, so the warning is saying that we 
shouldn't synchronize. It sounds right to me, but I wanted to make sure that it 
is correct. I can create a jira to fix it if I get a confirmation. It is a 
pretty simple fix.

> Refactor HLog into an interface.
> 
>
> Key: HBASE-5937
> URL: https://issues.apache.org/jira/browse/HBASE-5937
> Project: HBase
>  Issue Type: Sub-task
>Reporter: Li Pi
>Assignee: Flavio Junqueira
>Priority: Minor
> Attachments: 5937-hlog-with-javadoc.txt, HBASE-5937.patch, 
> HBASE-5937.patch, HBASE-5937.patch, HBASE-5937.patch, HBASE-5937.patch, 
> HBASE-5937.v2.patch, HBASE-5937.v3.patch, 
> org.apache.hadoop.hbase.client.TestMultiParallel-output.txt
>
>
> What the summary says. Create HLog interface. Make current implementation use 
> it.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Assigned] (HBASE-5582) "No HServerInfo found for" should be a WARNING message

[Kevin Odell (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-5582?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kevin Odell reassigned HBASE-5582:
--

Assignee: Kevin Odell

> "No HServerInfo found for" should be a WARNING message
> --
>
> Key: HBASE-5582
> URL: https://issues.apache.org/jira/browse/HBASE-5582
> Project: HBase
>  Issue Type: Improvement
>  Components: master
>Affects Versions: 0.90.4
>Reporter: Shrijeet Paliwal
>Assignee: Kevin Odell
>Priority: Trivial
>  Labels: newbie
>
> The message from RegionServerTracker "No HServerInfo found for..." is easy to 
> miss. It should not be INFO. 
> From irc chat 
> {noformat}
> jdcryans
> JohnP789: can you grep for "No HServerInfo found for" in that log?
> jdcryans
> wait I see it
> jdcryans
> ok there's your problem
> shrijeet_
> Yes it is there
> shrijeet_
> jdcryans: it should be INFO, why?
> jdcryans
> it shouldn't be INFO, it's so easy to miss
> jdcryans
> it's not the first time we have to look super closely to figure this one out
> shrijeet_
> yes , I will file a jira
> jdcryans
> in any case it's a mismatch in that machine's DNS config
> shrijeet_
> anyways JohnP789 is waiting :) go on
> JohnP789
> haha!
> JohnP789
> yes...  ???  :-)
> jdcryans
> the master is expecting a RS called 
> "localhost.localdomain,53875,1328924863478"
> 17:26 jdcryans
> but the RS calls itself "localhost,53875,1328924863478"
> {noformat}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5937) Refactor HLog into an interface.

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-5937?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13467813#comment-13467813
 ] 

Hadoop QA commented on HBASE-5937:
--

{color:red}-1 overall{color}.  Here are the results of testing the latest 
attachment 
  http://issues.apache.org/jira/secure/attachment/12547389/HBASE-5937.v3.patch
  against trunk revision .

{color:green}+1 @author{color}.  The patch does not contain any @author 
tags.

{color:green}+1 tests included{color}.  The patch appears to include 72 new 
or modified tests.

{color:green}+1 hadoop2.0{color}.  The patch compiles against the hadoop 
2.0 profile.

{color:red}-1 javadoc{color}.  The javadoc tool appears to have generated 
157 warning messages.

{color:green}+1 javac{color}.  The applied patch does not increase the 
total number of javac compiler warnings.

{color:red}-1 findbugs{color}.  The patch appears to introduce 5 new 
Findbugs (version 1.3.9) warnings.

{color:green}+1 release audit{color}.  The applied patch does not increase 
the total number of release audit warnings.

{color:green}+1 core tests{color}.  The patch passed unit tests in .

Test results: 
https://builds.apache.org/job/PreCommit-HBASE-Build/2985//testReport/
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/2985//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/2985//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/2985//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/2985//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/2985//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output: 
https://builds.apache.org/job/PreCommit-HBASE-Build/2985//console

This message is automatically generated.

> Refactor HLog into an interface.
> 
>
> Key: HBASE-5937
> URL: https://issues.apache.org/jira/browse/HBASE-5937
> Project: HBase
>  Issue Type: Sub-task
>Reporter: Li Pi
>Assignee: Flavio Junqueira
>Priority: Minor
> Attachments: 5937-hlog-with-javadoc.txt, HBASE-5937.patch, 
> HBASE-5937.patch, HBASE-5937.patch, HBASE-5937.patch, HBASE-5937.patch, 
> HBASE-5937.v2.patch, HBASE-5937.v3.patch, 
> org.apache.hadoop.hbase.client.TestMultiParallel-output.txt
>
>
> What the summary says. Create HLog interface. Make current implementation use 
> it.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5937) Refactor HLog into an interface.

[Ted Yu (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-5937?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13467829#comment-13467829
 ] 

Ted Yu commented on HBASE-5937:
---

Nice to see a green QA run.
{code}
-@InterfaceAudience.Private
-public class HLog implements Syncable {
-  static final Log LOG = LogFactory.getLog(HLog.class);
-  public static final byte [] METAFAMILY = Bytes.toBytes("METAFAMILY");
-  static final byte [] METAROW = Bytes.toBytes("METAROW");
+public interface HLog {
{code}
Please restore annotation for HLog. I suggest using Public + Evolving.


> Refactor HLog into an interface.
> 
>
> Key: HBASE-5937
> URL: https://issues.apache.org/jira/browse/HBASE-5937
> Project: HBase
>  Issue Type: Sub-task
>Reporter: Li Pi
>Assignee: Flavio Junqueira
>Priority: Minor
> Attachments: 5937-hlog-with-javadoc.txt, HBASE-5937.patch, 
> HBASE-5937.patch, HBASE-5937.patch, HBASE-5937.patch, HBASE-5937.patch, 
> HBASE-5937.v2.patch, HBASE-5937.v3.patch, 
> org.apache.hadoop.hbase.client.TestMultiParallel-output.txt
>
>
> What the summary says. Create HLog interface. Make current implementation use 
> it.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (HBASE-6919) Remove unnecessary cast from Bytes.readVLong

[James Taylor (JIRA)]

James Taylor created HBASE-6919:
---

 Summary: Remove unnecessary cast from Bytes.readVLong
 Key: HBASE-6919
 URL: https://issues.apache.org/jira/browse/HBASE-6919
 Project: HBase
  Issue Type: Bug
Reporter: James Taylor
Priority: Minor


Remove the throws IOException so that caller doesn't have to catch and ignore.

  public static long readVLong(final byte [] buffer, final int offset)
  throws IOException

Also, add

  public static int readVInt(final byte [] buffer, final int offset)
  throws IOException {
return (int)readVLong(buffer,offset);
  }

and these are useful too:

/**
 * Put long as variable length encoded number at the offset in
 * the result byte array.
 * @param vint Integer to make a vint of.
 * @param result buffer to put vint into
 * @return Vint length in bytes of vint
 */
public static int vintToBytes(byte[] result, int offset, final long vint) {
  long i = vint;
  if (i >= -112 && i <= 127) {
result[offset] = (byte) i;
return 1;
  }

  int len = -112;
  if (i < 0) {
i ^= -1L; // take one's complement'
len = -120;
  }

  long tmp = i;
  while (tmp != 0) {
tmp = tmp >> 8;
len--;
  }

  result[offset++] = (byte) len;

  len = (len < -120) ? -(len + 120) : -(len + 112);

  for (int idx = len; idx != 0; idx--) {
int shiftbits = (idx - 1) * 8;
long mask = 0xFFL << shiftbits;
result[offset++] = (byte)((i & mask) >> shiftbits);
  }
  return len + 1;
}

/**
 * Decode a vint from the buffer pointed at to by ptr and
 * increment the offset of the ptr by the length of the
 * vint.
 * @param ptr a pointer to a byte array buffer
 * @return the decoded vint value as an int
 */
public static int vintFromBytes(ImmutableBytesWritable ptr) {
return (int) vlongFromBytes(ptr);
}

/**
 * Decode a vint from the buffer pointed at to by ptr and
 * increment the offset of the ptr by the length of the
 * vint.
 * @param ptr a pointer to a byte array buffer
 * @return the decoded vint value as a long
 */
public static long vlongFromBytes(ImmutableBytesWritable ptr) {
final byte [] buffer = ptr.get();
final int offset = ptr.getOffset();
byte firstByte = buffer[offset];
int len = WritableUtils.decodeVIntSize(firstByte);
if (len == 1) {
ptr.set(buffer, offset+1, ptr.getLength());
return firstByte;
}
long i = 0;
for (int idx = 0; idx < len-1; idx++) {
byte b = buffer[offset + 1 + idx];
i = i << 8;
i = i | (b & 0xFF);
}
ptr.set(buffer, offset+len, ptr.getLength());
return (WritableUtils.isNegativeVInt(firstByte) ? ~i : i);
}




--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-6914) Scans/Gets/Mutations don't give a good error if the table is disabled.

[Elliott Clark (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-6914?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Elliott Clark updated HBASE-6914:
-

Component/s: Client

> Scans/Gets/Mutations don't give a good error if the table is disabled.
> --
>
> Key: HBASE-6914
> URL: https://issues.apache.org/jira/browse/HBASE-6914
> Project: HBase
>  Issue Type: Improvement
>  Components: Client
>Reporter: Elliott Clark
>Assignee: Elliott Clark
> Fix For: 0.92.3, 0.94.3, 0.96.0
>
> Attachments: HBASE-6914-0.patch, HBASE-6914-1.patch, 
> HBASE-6914-2.patch, HBASE-6914-3.patch
>
>
> Scan a table that is disabled will have the client retry multiple times and 
> then will error out with NotServingRegionException.  If the table is disabled 
> there's no need to re-try and the message should be more explicit.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-6914) Scans/Gets/Mutations don't give a good error if the table is disabled.

[stack (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-6914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13467849#comment-13467849
 ] 

stack commented on HBASE-6914:
--

+1 on commit again.

> Scans/Gets/Mutations don't give a good error if the table is disabled.
> --
>
> Key: HBASE-6914
> URL: https://issues.apache.org/jira/browse/HBASE-6914
> Project: HBase
>  Issue Type: Improvement
>  Components: Client
>Reporter: Elliott Clark
>Assignee: Elliott Clark
> Fix For: 0.92.3, 0.94.3, 0.96.0
>
> Attachments: HBASE-6914-0.patch, HBASE-6914-1.patch, 
> HBASE-6914-2.patch, HBASE-6914-3.patch
>
>
> Scan a table that is disabled will have the client retry multiple times and 
> then will error out with NotServingRegionException.  If the table is disabled 
> there's no need to re-try and the message should be more explicit.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-6914) Scans/Gets/Mutations don't give a good error if the table is disabled.

[stack (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-6914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13467851#comment-13467851
 ] 

stack commented on HBASE-6914:
--

Thanks for digging in on the failed test in spite of my +1

> Scans/Gets/Mutations don't give a good error if the table is disabled.
> --
>
> Key: HBASE-6914
> URL: https://issues.apache.org/jira/browse/HBASE-6914
> Project: HBase
>  Issue Type: Improvement
>  Components: Client
>Reporter: Elliott Clark
>Assignee: Elliott Clark
> Fix For: 0.92.3, 0.94.3, 0.96.0
>
> Attachments: HBASE-6914-0.patch, HBASE-6914-1.patch, 
> HBASE-6914-2.patch, HBASE-6914-3.patch
>
>
> Scan a table that is disabled will have the client retry multiple times and 
> then will error out with NotServingRegionException.  If the table is disabled 
> there's no need to re-try and the message should be more explicit.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-6919) Remove unnecessary cast from Bytes.readVLong

[stack (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-6919?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13467852#comment-13467852
 ] 

stack commented on HBASE-6919:
--

Make a patch James and we'll commit.

> Remove unnecessary cast from Bytes.readVLong
> 
>
> Key: HBASE-6919
> URL: https://issues.apache.org/jira/browse/HBASE-6919
> Project: HBase
>  Issue Type: Bug
>Reporter: James Taylor
>Priority: Minor
>
> Remove the throws IOException so that caller doesn't have to catch and ignore.
>   public static long readVLong(final byte [] buffer, final int offset)
>   throws IOException
> Also, add
>   public static int readVInt(final byte [] buffer, final int offset)
>   throws IOException {
> return (int)readVLong(buffer,offset);
>   }
> and these are useful too:
> /**
>  * Put long as variable length encoded number at the offset in
>  * the result byte array.
>  * @param vint Integer to make a vint of.
>  * @param result buffer to put vint into
>  * @return Vint length in bytes of vint
>  */
> public static int vintToBytes(byte[] result, int offset, final long vint) 
> {
>   long i = vint;
>   if (i >= -112 && i <= 127) {
> result[offset] = (byte) i;
> return 1;
>   }
>   int len = -112;
>   if (i < 0) {
> i ^= -1L; // take one's complement'
> len = -120;
>   }
>   long tmp = i;
>   while (tmp != 0) {
> tmp = tmp >> 8;
> len--;
>   }
>   result[offset++] = (byte) len;
>   len = (len < -120) ? -(len + 120) : -(len + 112);
>   for (int idx = len; idx != 0; idx--) {
> int shiftbits = (idx - 1) * 8;
> long mask = 0xFFL << shiftbits;
> result[offset++] = (byte)((i & mask) >> shiftbits);
>   }
>   return len + 1;
> }
> /**
>  * Decode a vint from the buffer pointed at to by ptr and
>  * increment the offset of the ptr by the length of the
>  * vint.
>  * @param ptr a pointer to a byte array buffer
>  * @return the decoded vint value as an int
>  */
> public static int vintFromBytes(ImmutableBytesWritable ptr) {
> return (int) vlongFromBytes(ptr);
> }
> 
> /**
>  * Decode a vint from the buffer pointed at to by ptr and
>  * increment the offset of the ptr by the length of the
>  * vint.
>  * @param ptr a pointer to a byte array buffer
>  * @return the decoded vint value as a long
>  */
> public static long vlongFromBytes(ImmutableBytesWritable ptr) {
> final byte [] buffer = ptr.get();
> final int offset = ptr.getOffset();
> byte firstByte = buffer[offset];
> int len = WritableUtils.decodeVIntSize(firstByte);
> if (len == 1) {
> ptr.set(buffer, offset+1, ptr.getLength());
> return firstByte;
> }
> long i = 0;
> for (int idx = 0; idx < len-1; idx++) {
> byte b = buffer[offset + 1 + idx];
> i = i << 8;
> i = i | (b & 0xFF);
> }
> ptr.set(buffer, offset+len, ptr.getLength());
> return (WritableUtils.isNegativeVInt(firstByte) ? ~i : i);
> }

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5937) Refactor HLog into an interface.

[stack (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-5937?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13467855#comment-13467855
 ] 

stack commented on HBASE-5937:
--

I can restore the annotation on commit.  Will leave it private for now.

> Refactor HLog into an interface.
> 
>
> Key: HBASE-5937
> URL: https://issues.apache.org/jira/browse/HBASE-5937
> Project: HBase
>  Issue Type: Sub-task
>Reporter: Li Pi
>Assignee: Flavio Junqueira
>Priority: Minor
> Attachments: 5937-hlog-with-javadoc.txt, HBASE-5937.patch, 
> HBASE-5937.patch, HBASE-5937.patch, HBASE-5937.patch, HBASE-5937.patch, 
> HBASE-5937.v2.patch, HBASE-5937.v3.patch, 
> org.apache.hadoop.hbase.client.TestMultiParallel-output.txt
>
>
> What the summary says. Create HLog interface. Make current implementation use 
> it.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (HBASE-6920) On timeout connecting to master, client can get stuck and never make progress

[Gregory Chanan (JIRA)]

Gregory Chanan created HBASE-6920:
-

 Summary: On timeout connecting to master, client can get stuck and 
never make progress
 Key: HBASE-6920
 URL: https://issues.apache.org/jira/browse/HBASE-6920
 Project: HBase
  Issue Type: Bug
Affects Versions: 0.94.2
Reporter: Gregory Chanan
Assignee: Gregory Chanan
Priority: Critical


HBASE-5058 appears to have introduced an issue where a timeout in 
HConnection.getMaster() can cause the client to never be able to connect to the 
master.  So, for example, an HBaseAdmin object can never successfully be 
initialized.

The issue is here:
{code}
if (tryMaster.isMasterRunning()) {
  this.master = tryMaster;
  this.masterLock.notifyAll();
  break;
}
{code}

If isMasterRunning times out, it throws an UndeclaredThrowableException, which 
is already not ideal, because it can be returned to the application.

 But if the first call to getMaster succeeds, it will set masterChecked = true, 
which makes us never try to reconnect; that is, we will set this.master = null 
and just throw MasterNotRunningExceptions, without even trying to connect.

I tried out a 94 client (actually a 92 client with some 94 patches) on a 
cluster with some network issues, and it would constantly get stuck as 
described above.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-6920) On timeout connecting to master, client can get stuck and never make progress

[Gregory Chanan (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-6920?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gregory Chanan updated HBASE-6920:
--

Attachment: HBASE-6920.patch

* Attached HBASE-6920.patch *

Here is a patch for this issue and a test.  The test is more complicated than I 
would like; I needed to introduce some functions / change some access controls 
to be able to use my own RPC Invoker.  Any suggestions would be welcome.

> On timeout connecting to master, client can get stuck and never make progress
> -
>
> Key: HBASE-6920
> URL: https://issues.apache.org/jira/browse/HBASE-6920
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 0.94.2
>Reporter: Gregory Chanan
>Assignee: Gregory Chanan
>Priority: Critical
> Attachments: HBASE-6920.patch
>
>
> HBASE-5058 appears to have introduced an issue where a timeout in 
> HConnection.getMaster() can cause the client to never be able to connect to 
> the master.  So, for example, an HBaseAdmin object can never successfully be 
> initialized.
> The issue is here:
> {code}
> if (tryMaster.isMasterRunning()) {
>   this.master = tryMaster;
>   this.masterLock.notifyAll();
>   break;
> }
> {code}
> If isMasterRunning times out, it throws an UndeclaredThrowableException, 
> which is already not ideal, because it can be returned to the application.
>  But if the first call to getMaster succeeds, it will set masterChecked = 
> true, which makes us never try to reconnect; that is, we will set this.master 
> = null and just throw MasterNotRunningExceptions, without even trying to 
> connect.
> I tried out a 94 client (actually a 92 client with some 94 patches) on a 
> cluster with some network issues, and it would constantly get stuck as 
> described above.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-6920) On timeout connecting to master, client can get stuck and never make progress

[Gregory Chanan (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-6920?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gregory Chanan updated HBASE-6920:
--

Status: Patch Available  (was: Open)

> On timeout connecting to master, client can get stuck and never make progress
> -
>
> Key: HBASE-6920
> URL: https://issues.apache.org/jira/browse/HBASE-6920
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 0.94.2
>Reporter: Gregory Chanan
>Assignee: Gregory Chanan
>Priority: Critical
> Attachments: HBASE-6920.patch
>
>
> HBASE-5058 appears to have introduced an issue where a timeout in 
> HConnection.getMaster() can cause the client to never be able to connect to 
> the master.  So, for example, an HBaseAdmin object can never successfully be 
> initialized.
> The issue is here:
> {code}
> if (tryMaster.isMasterRunning()) {
>   this.master = tryMaster;
>   this.masterLock.notifyAll();
>   break;
> }
> {code}
> If isMasterRunning times out, it throws an UndeclaredThrowableException, 
> which is already not ideal, because it can be returned to the application.
>  But if the first call to getMaster succeeds, it will set masterChecked = 
> true, which makes us never try to reconnect; that is, we will set this.master 
> = null and just throw MasterNotRunningExceptions, without even trying to 
> connect.
> I tried out a 94 client (actually a 92 client with some 94 patches) on a 
> cluster with some network issues, and it would constantly get stuck as 
> described above.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-6920) On timeout connecting to master, client can get stuck and never make progress

[Gregory Chanan (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-6920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13467873#comment-13467873
 ] 

Gregory Chanan commented on HBASE-6920:
---

Some other points:
* I have not tried the client app that failed in the description with a patched 
client yet.  I will do that soon.
* This seems like a pretty serious issue (I literally could not get my app to 
work with a 94 client), though it's not clear how often users will see it, 
since it only happens on errors.  It's also not new with 0.94.2 so I don't know 
if it needs to sink the RC.  Any opinions?

> On timeout connecting to master, client can get stuck and never make progress
> -
>
> Key: HBASE-6920
> URL: https://issues.apache.org/jira/browse/HBASE-6920
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 0.94.2
>Reporter: Gregory Chanan
>Assignee: Gregory Chanan
>Priority: Critical
> Attachments: HBASE-6920.patch
>
>
> HBASE-5058 appears to have introduced an issue where a timeout in 
> HConnection.getMaster() can cause the client to never be able to connect to 
> the master.  So, for example, an HBaseAdmin object can never successfully be 
> initialized.
> The issue is here:
> {code}
> if (tryMaster.isMasterRunning()) {
>   this.master = tryMaster;
>   this.masterLock.notifyAll();
>   break;
> }
> {code}
> If isMasterRunning times out, it throws an UndeclaredThrowableException, 
> which is already not ideal, because it can be returned to the application.
>  But if the first call to getMaster succeeds, it will set masterChecked = 
> true, which makes us never try to reconnect; that is, we will set this.master 
> = null and just throw MasterNotRunningExceptions, without even trying to 
> connect.
> I tried out a 94 client (actually a 92 client with some 94 patches) on a 
> cluster with some network issues, and it would constantly get stuck as 
> described above.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-6920) On timeout connecting to master, client can get stuck and never make progress

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-6920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13467874#comment-13467874
 ] 

Hadoop QA commented on HBASE-6920:
--

{color:red}-1 overall{color}.  Here are the results of testing the latest 
attachment 
  http://issues.apache.org/jira/secure/attachment/12547405/HBASE-6920.patch
  against trunk revision .

{color:green}+1 @author{color}.  The patch does not contain any @author 
tags.

{color:green}+1 tests included{color}.  The patch appears to include 4 new 
or modified tests.

{color:red}-1 patch{color}.  The patch command could not apply the patch.

Console output: 
https://builds.apache.org/job/PreCommit-HBASE-Build/2986//console

This message is automatically generated.

> On timeout connecting to master, client can get stuck and never make progress
> -
>
> Key: HBASE-6920
> URL: https://issues.apache.org/jira/browse/HBASE-6920
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 0.94.2
>Reporter: Gregory Chanan
>Assignee: Gregory Chanan
>Priority: Critical
> Attachments: HBASE-6920.patch
>
>
> HBASE-5058 appears to have introduced an issue where a timeout in 
> HConnection.getMaster() can cause the client to never be able to connect to 
> the master.  So, for example, an HBaseAdmin object can never successfully be 
> initialized.
> The issue is here:
> {code}
> if (tryMaster.isMasterRunning()) {
>   this.master = tryMaster;
>   this.masterLock.notifyAll();
>   break;
> }
> {code}
> If isMasterRunning times out, it throws an UndeclaredThrowableException, 
> which is already not ideal, because it can be returned to the application.
>  But if the first call to getMaster succeeds, it will set masterChecked = 
> true, which makes us never try to reconnect; that is, we will set this.master 
> = null and just throw MasterNotRunningExceptions, without even trying to 
> connect.
> I tried out a 94 client (actually a 92 client with some 94 patches) on a 
> cluster with some network issues, and it would constantly get stuck as 
> described above.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-6900) RegionScanner.reseek() creates NPE when a flush or compaction happens before the reseek.

[Lars Hofhansl (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-6900?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13467881#comment-13467881
 ] 

Lars Hofhansl commented on HBASE-6900:
--

[~ram_krish] Sorry for the delay.
{code}
{+if (!checkReseek() && heap != null) {}
{code}

Do we need to call checkReseek here? This is a somewhat expensive operation and 
we're now doing it for every reseek issued to a StoreScanner.
Could we just check for {{heap == null}} in the beginning of the method and 
return {{false}} if so?


> RegionScanner.reseek() creates NPE when a flush or compaction happens before 
> the reseek.
> 
>
> Key: HBASE-6900
> URL: https://issues.apache.org/jira/browse/HBASE-6900
> Project: HBase
>  Issue Type: Bug
>Reporter: ramkrishna.s.vasudevan
>Assignee: ramkrishna.s.vasudevan
> Attachments: HBASE-6900_1.patch, HBASE-6900.patch
>
>
> HBASE-5520 introduced reseek() on the RegionScanner.  
> Now when a scanner is created we have the StoreScanner heap.  After this if a 
> flush or compaction happens parallely all the StoreScannerObservers are 
> cleared so that whenever a new next() call happens we tend to recreate the 
> scanner based on the latest store files.
> The reseek() in StoreScanner expects the heap not to be null because always 
> reseek would be called from next()
> {code}
> public synchronized boolean reseek(KeyValue kv) throws IOException {
> //Heap cannot be null, because this is only called from next() which
> //guarantees that heap will never be null before this call.
> if (explicitColumnQuery && lazySeekEnabledGlobally) {
>   return heap.requestSeek(kv, true, useRowColBloom);
> } else {
>   return heap.reseek(kv);
> }
>   }
> {code}
> Now when we call RegionScanner.reseek() directly using CPs we tend to get a 
> NPE.  In our case it happened when a major compaction was going on.  I will 
> also attach a testcase to show the problem.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-6920) On timeout connecting to master, client can get stuck and never make progress

[Ted Yu (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-6920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13467885#comment-13467885
 ] 

Ted Yu commented on HBASE-6920:
---

Great finding, Gregory.
{code}
+ * @return true if should retry
+ */
+private boolean logGetMasterAttemptFailure(int tries, Exception e) {
{code}
Please rename the method to reflect the fact that it returns whether we should 
retry.
{code}
+  LOG.info("getMaster attempt " + tries + " of " + numRetries +
+" failed; retrying after sleep of " +
+ConnectionUtils.getPauseTime(this.pause, tries), e);
{code}
Should the above log be at DEBUG level ?
Comparing your code and the original, it seems that the return value of true 
actually means that we shouldn't retry.

> On timeout connecting to master, client can get stuck and never make progress
> -
>
> Key: HBASE-6920
> URL: https://issues.apache.org/jira/browse/HBASE-6920
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 0.94.2
>Reporter: Gregory Chanan
>Assignee: Gregory Chanan
>Priority: Critical
> Attachments: HBASE-6920.patch
>
>
> HBASE-5058 appears to have introduced an issue where a timeout in 
> HConnection.getMaster() can cause the client to never be able to connect to 
> the master.  So, for example, an HBaseAdmin object can never successfully be 
> initialized.
> The issue is here:
> {code}
> if (tryMaster.isMasterRunning()) {
>   this.master = tryMaster;
>   this.masterLock.notifyAll();
>   break;
> }
> {code}
> If isMasterRunning times out, it throws an UndeclaredThrowableException, 
> which is already not ideal, because it can be returned to the application.
>  But if the first call to getMaster succeeds, it will set masterChecked = 
> true, which makes us never try to reconnect; that is, we will set this.master 
> = null and just throw MasterNotRunningExceptions, without even trying to 
> connect.
> I tried out a 94 client (actually a 92 client with some 94 patches) on a 
> cluster with some network issues, and it would constantly get stuck as 
> described above.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-6920) On timeout connecting to master, client can get stuck and never make progress

[Gregory Chanan (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-6920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13467889#comment-13467889
 ] 

Gregory Chanan commented on HBASE-6920:
---

I don't have any opinion on the DEBUG level; if you think that's better I'll 
change it.

Good catch on the return value, the javadoc is incorrect.

> On timeout connecting to master, client can get stuck and never make progress
> -
>
> Key: HBASE-6920
> URL: https://issues.apache.org/jira/browse/HBASE-6920
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 0.94.2
>Reporter: Gregory Chanan
>Assignee: Gregory Chanan
>Priority: Critical
> Attachments: HBASE-6920.patch
>
>
> HBASE-5058 appears to have introduced an issue where a timeout in 
> HConnection.getMaster() can cause the client to never be able to connect to 
> the master.  So, for example, an HBaseAdmin object can never successfully be 
> initialized.
> The issue is here:
> {code}
> if (tryMaster.isMasterRunning()) {
>   this.master = tryMaster;
>   this.masterLock.notifyAll();
>   break;
> }
> {code}
> If isMasterRunning times out, it throws an UndeclaredThrowableException, 
> which is already not ideal, because it can be returned to the application.
>  But if the first call to getMaster succeeds, it will set masterChecked = 
> true, which makes us never try to reconnect; that is, we will set this.master 
> = null and just throw MasterNotRunningExceptions, without even trying to 
> connect.
> I tried out a 94 client (actually a 92 client with some 94 patches) on a 
> cluster with some network issues, and it would constantly get stuck as 
> described above.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-6920) On timeout connecting to master, client can get stuck and never make progress

[Ted Yu (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-6920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13467891#comment-13467891
 ] 

Ted Yu commented on HBASE-6920:
---

{code}
+  /** Construct a client-side proxy object, specifying an InvocationHandler 
for testing purposes */
+  VersionedProtocol getProxy(Class protocol,
{code}
Name the above method getProxyForTesting ?
{code}
+ * RpcEngine that random throws a SocketTimeoutEngine for testing.
{code}
'random' -> 'randomly'
Please correct the spelling for SocketTimeoutEngine

> On timeout connecting to master, client can get stuck and never make progress
> -
>
> Key: HBASE-6920
> URL: https://issues.apache.org/jira/browse/HBASE-6920
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 0.94.2
>Reporter: Gregory Chanan
>Assignee: Gregory Chanan
>Priority: Critical
> Attachments: HBASE-6920.patch
>
>
> HBASE-5058 appears to have introduced an issue where a timeout in 
> HConnection.getMaster() can cause the client to never be able to connect to 
> the master.  So, for example, an HBaseAdmin object can never successfully be 
> initialized.
> The issue is here:
> {code}
> if (tryMaster.isMasterRunning()) {
>   this.master = tryMaster;
>   this.masterLock.notifyAll();
>   break;
> }
> {code}
> If isMasterRunning times out, it throws an UndeclaredThrowableException, 
> which is already not ideal, because it can be returned to the application.
>  But if the first call to getMaster succeeds, it will set masterChecked = 
> true, which makes us never try to reconnect; that is, we will set this.master 
> = null and just throw MasterNotRunningExceptions, without even trying to 
> connect.
> I tried out a 94 client (actually a 92 client with some 94 patches) on a 
> cluster with some network issues, and it would constantly get stuck as 
> described above.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-6920) On timeout connecting to master, client can get stuck and never make progress

[Lars Hofhansl (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-6920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13467896#comment-13467896
 ] 

Lars Hofhansl commented on HBASE-6920:
--

Patch looks good.
I don't quite grok the changes in HBaseRPC, RpcEngine, and WritableRpcEngine. 
Are those needed for this to work?

> On timeout connecting to master, client can get stuck and never make progress
> -
>
> Key: HBASE-6920
> URL: https://issues.apache.org/jira/browse/HBASE-6920
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 0.94.2
>Reporter: Gregory Chanan
>Assignee: Gregory Chanan
>Priority: Critical
> Attachments: HBASE-6920.patch
>
>
> HBASE-5058 appears to have introduced an issue where a timeout in 
> HConnection.getMaster() can cause the client to never be able to connect to 
> the master.  So, for example, an HBaseAdmin object can never successfully be 
> initialized.
> The issue is here:
> {code}
> if (tryMaster.isMasterRunning()) {
>   this.master = tryMaster;
>   this.masterLock.notifyAll();
>   break;
> }
> {code}
> If isMasterRunning times out, it throws an UndeclaredThrowableException, 
> which is already not ideal, because it can be returned to the application.
>  But if the first call to getMaster succeeds, it will set masterChecked = 
> true, which makes us never try to reconnect; that is, we will set this.master 
> = null and just throw MasterNotRunningExceptions, without even trying to 
> connect.
> I tried out a 94 client (actually a 92 client with some 94 patches) on a 
> cluster with some network issues, and it would constantly get stuck as 
> described above.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-6920) On timeout connecting to master, client can get stuck and never make progress

[Lars Hofhansl (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-6920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13467898#comment-13467898
 ] 

Lars Hofhansl commented on HBASE-6920:
--

Also from the explanation I am not quite sure if this a problem in 0.92 as well.

> On timeout connecting to master, client can get stuck and never make progress
> -
>
> Key: HBASE-6920
> URL: https://issues.apache.org/jira/browse/HBASE-6920
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 0.94.2
>Reporter: Gregory Chanan
>Assignee: Gregory Chanan
>Priority: Critical
> Attachments: HBASE-6920.patch
>
>
> HBASE-5058 appears to have introduced an issue where a timeout in 
> HConnection.getMaster() can cause the client to never be able to connect to 
> the master.  So, for example, an HBaseAdmin object can never successfully be 
> initialized.
> The issue is here:
> {code}
> if (tryMaster.isMasterRunning()) {
>   this.master = tryMaster;
>   this.masterLock.notifyAll();
>   break;
> }
> {code}
> If isMasterRunning times out, it throws an UndeclaredThrowableException, 
> which is already not ideal, because it can be returned to the application.
>  But if the first call to getMaster succeeds, it will set masterChecked = 
> true, which makes us never try to reconnect; that is, we will set this.master 
> = null and just throw MasterNotRunningExceptions, without even trying to 
> connect.
> I tried out a 94 client (actually a 92 client with some 94 patches) on a 
> cluster with some network issues, and it would constantly get stuck as 
> described above.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (HBASE-6921) String and ConcurrentHashMap sizes change on jdk7; makes TestHeapSize fail; second attempt

[stack (JIRA)]

stack created HBASE-6921:


 Summary: String and ConcurrentHashMap sizes change on jdk7; makes 
TestHeapSize fail; second attempt
 Key: HBASE-6921
 URL: https://issues.apache.org/jira/browse/HBASE-6921
 Project: HBase
  Issue Type: Bug
  Components: test
Reporter: stack
Assignee: stack
 Fix For: 0.96.0


HBASE-6918 added debug.  Jenkins runs jdk7 23.0-b21.  Locally I have 23.6-b03.  
It looks like ConcurrentHashMap is a different size in these two jvm's (To be 
confirmed).

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-6920) On timeout connecting to master, client can get stuck and never make progress

[Gregory Chanan (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-6920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13467903#comment-13467903
 ] 

Gregory Chanan commented on HBASE-6920:
---

I think this is 0.94 only -- at least HBASE-5058 is only in 0.94.



> On timeout connecting to master, client can get stuck and never make progress
> -
>
> Key: HBASE-6920
> URL: https://issues.apache.org/jira/browse/HBASE-6920
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 0.94.2
>Reporter: Gregory Chanan
>Assignee: Gregory Chanan
>Priority: Critical
> Attachments: HBASE-6920.patch
>
>
> HBASE-5058 appears to have introduced an issue where a timeout in 
> HConnection.getMaster() can cause the client to never be able to connect to 
> the master.  So, for example, an HBaseAdmin object can never successfully be 
> initialized.
> The issue is here:
> {code}
> if (tryMaster.isMasterRunning()) {
>   this.master = tryMaster;
>   this.masterLock.notifyAll();
>   break;
> }
> {code}
> If isMasterRunning times out, it throws an UndeclaredThrowableException, 
> which is already not ideal, because it can be returned to the application.
>  But if the first call to getMaster succeeds, it will set masterChecked = 
> true, which makes us never try to reconnect; that is, we will set this.master 
> = null and just throw MasterNotRunningExceptions, without even trying to 
> connect.
> I tried out a 94 client (actually a 92 client with some 94 patches) on a 
> cluster with some network issues, and it would constantly get stuck as 
> described above.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-6920) On timeout connecting to master, client can get stuck and never make progress

[Gregory Chanan (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-6920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13467905#comment-13467905
 ] 

Gregory Chanan commented on HBASE-6920:
---

The changes in HBaseRPC, RpcEngine, and WritableRpcEngine are just for the 
test.  I wanted to simulate throwing a SocketTimeoutException on a proxy call, 
but Mockito can't mock a proxy object as far as I can tell.

So all those changes are just so I can setup my own RPC Invoker to run, which 
just randomly throws SocketTimeoutExceptions.

> On timeout connecting to master, client can get stuck and never make progress
> -
>
> Key: HBASE-6920
> URL: https://issues.apache.org/jira/browse/HBASE-6920
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 0.94.2
>Reporter: Gregory Chanan
>Assignee: Gregory Chanan
>Priority: Critical
> Attachments: HBASE-6920.patch
>
>
> HBASE-5058 appears to have introduced an issue where a timeout in 
> HConnection.getMaster() can cause the client to never be able to connect to 
> the master.  So, for example, an HBaseAdmin object can never successfully be 
> initialized.
> The issue is here:
> {code}
> if (tryMaster.isMasterRunning()) {
>   this.master = tryMaster;
>   this.masterLock.notifyAll();
>   break;
> }
> {code}
> If isMasterRunning times out, it throws an UndeclaredThrowableException, 
> which is already not ideal, because it can be returned to the application.
>  But if the first call to getMaster succeeds, it will set masterChecked = 
> true, which makes us never try to reconnect; that is, we will set this.master 
> = null and just throw MasterNotRunningExceptions, without even trying to 
> connect.
> I tried out a 94 client (actually a 92 client with some 94 patches) on a 
> cluster with some network issues, and it would constantly get stuck as 
> described above.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-6921) String and ConcurrentHashMap sizes change on jdk7; makes TestHeapSize fail; second attempt

[stack (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-6921?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stack updated HBASE-6921:
-

Attachment: 6921.txt

Remove jdk7 special handling and just use estimated sizes for String and 
ConcurrentHashMap instead.

> String and ConcurrentHashMap sizes change on jdk7; makes TestHeapSize fail; 
> second attempt
> --
>
> Key: HBASE-6921
> URL: https://issues.apache.org/jira/browse/HBASE-6921
> Project: HBase
>  Issue Type: Bug
>  Components: test
>Reporter: stack
>Assignee: stack
> Fix For: 0.96.0
>
> Attachments: 6921.txt
>
>
> HBASE-6918 added debug.  Jenkins runs jdk7 23.0-b21.  Locally I have 
> 23.6-b03.  It looks like ConcurrentHashMap is a different size in these two 
> jvm's (To be confirmed).

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-6707) TEST org.apache.hadoop.hbase.backup.example.TestZooKeeperTableArchiveClient.testMultipleTables flaps

[Jesse Yates (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-6707?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13467909#comment-13467909
 ] 

Jesse Yates commented on HBASE-6707:


[~stack] are you sure you ran v4? I can't seem to find where that output would 
even occur in the patch...

> TEST 
> org.apache.hadoop.hbase.backup.example.TestZooKeeperTableArchiveClient.testMultipleTables
>  flaps
> 
>
> Key: HBASE-6707
> URL: https://issues.apache.org/jira/browse/HBASE-6707
> Project: HBase
>  Issue Type: Bug
>  Components: test
>Reporter: Sameer Vaishampayan
>Assignee: Jesse Yates
>Priority: Critical
> Fix For: 0.96.0
>
> Attachments: hbase-6707-v0.patch, hbase-6707-v1.patch, 
> hbase-6707-v2.patch, hbase-6707-v3.patch, hbase-6707-v4.patch
>
>
> https://builds.apache.org/job/HBase-TRUNK/3293/
> Error Message
> Archived HFiles 
> (hdfs://localhost:59986/user/jenkins/hbase/.archive/otherTable/01ced3b55d7220a9c460273a4a57b198/fam)
>  should have gotten deleted, but didn't, remaining 
> files:[hdfs://localhost:59986/user/jenkins/hbase/.archive/otherTable/01ced3b55d7220a9c460273a4a57b198/fam/fc872572a1f5443eb55b6e2567cfeb1c]
> Stacktrace
> java.lang.AssertionError: Archived HFiles 
> (hdfs://localhost:59986/user/jenkins/hbase/.archive/otherTable/01ced3b55d7220a9c460273a4a57b198/fam)
>  should have gotten deleted, but didn't, remaining 
> files:[hdfs://localhost:59986/user/jenkins/hbase/.archive/otherTable/01ced3b55d7220a9c460273a4a57b198/fam/fc872572a1f5443eb55b6e2567cfeb1c]
>   at org.junit.Assert.fail(Assert.java:93)
>   at org.junit.Assert.assertTrue(Assert.java:43)
>   at org.junit.Assert.assertNull(Assert.java:551)
>   at 
> org.apache.hadoop.hbase.backup.example.TestZooKeeperTableArchiveClient.testMultipleTables(TestZooKeeperTableArchiveClient.java:291)
>   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>   at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>   at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-6707) TEST org.apache.hadoop.hbase.backup.example.TestZooKeeperTableArchiveClient.testMultipleTables flaps

[Ted Yu (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-6707?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13467921#comment-13467921
 ] 

Ted Yu commented on HBASE-6707:
---

@Jesse:
Do you mind updating review request on review board so that I can see the 
changes between the two patches ?

Thanks

> TEST 
> org.apache.hadoop.hbase.backup.example.TestZooKeeperTableArchiveClient.testMultipleTables
>  flaps
> 
>
> Key: HBASE-6707
> URL: https://issues.apache.org/jira/browse/HBASE-6707
> Project: HBase
>  Issue Type: Bug
>  Components: test
>Reporter: Sameer Vaishampayan
>Assignee: Jesse Yates
>Priority: Critical
> Fix For: 0.96.0
>
> Attachments: hbase-6707-v0.patch, hbase-6707-v1.patch, 
> hbase-6707-v2.patch, hbase-6707-v3.patch, hbase-6707-v4.patch
>
>
> https://builds.apache.org/job/HBase-TRUNK/3293/
> Error Message
> Archived HFiles 
> (hdfs://localhost:59986/user/jenkins/hbase/.archive/otherTable/01ced3b55d7220a9c460273a4a57b198/fam)
>  should have gotten deleted, but didn't, remaining 
> files:[hdfs://localhost:59986/user/jenkins/hbase/.archive/otherTable/01ced3b55d7220a9c460273a4a57b198/fam/fc872572a1f5443eb55b6e2567cfeb1c]
> Stacktrace
> java.lang.AssertionError: Archived HFiles 
> (hdfs://localhost:59986/user/jenkins/hbase/.archive/otherTable/01ced3b55d7220a9c460273a4a57b198/fam)
>  should have gotten deleted, but didn't, remaining 
> files:[hdfs://localhost:59986/user/jenkins/hbase/.archive/otherTable/01ced3b55d7220a9c460273a4a57b198/fam/fc872572a1f5443eb55b6e2567cfeb1c]
>   at org.junit.Assert.fail(Assert.java:93)
>   at org.junit.Assert.assertTrue(Assert.java:43)
>   at org.junit.Assert.assertNull(Assert.java:551)
>   at 
> org.apache.hadoop.hbase.backup.example.TestZooKeeperTableArchiveClient.testMultipleTables(TestZooKeeperTableArchiveClient.java:291)
>   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>   at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>   at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-6920) On timeout connecting to master, client can get stuck and never make progress

[Gregory Chanan (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-6920?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gregory Chanan updated HBASE-6920:
--

Attachment: HBASE-6920-v2.patch

v2, which fixes the javadoc issue (actually changes the return value of the 
code).

I left the LOG as INFO, because that's what it was before; let me know if you 
disagree.

> On timeout connecting to master, client can get stuck and never make progress
> -
>
> Key: HBASE-6920
> URL: https://issues.apache.org/jira/browse/HBASE-6920
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 0.94.2
>Reporter: Gregory Chanan
>Assignee: Gregory Chanan
>Priority: Critical
> Attachments: HBASE-6920.patch, HBASE-6920-v2.patch
>
>
> HBASE-5058 appears to have introduced an issue where a timeout in 
> HConnection.getMaster() can cause the client to never be able to connect to 
> the master.  So, for example, an HBaseAdmin object can never successfully be 
> initialized.
> The issue is here:
> {code}
> if (tryMaster.isMasterRunning()) {
>   this.master = tryMaster;
>   this.masterLock.notifyAll();
>   break;
> }
> {code}
> If isMasterRunning times out, it throws an UndeclaredThrowableException, 
> which is already not ideal, because it can be returned to the application.
>  But if the first call to getMaster succeeds, it will set masterChecked = 
> true, which makes us never try to reconnect; that is, we will set this.master 
> = null and just throw MasterNotRunningExceptions, without even trying to 
> connect.
> I tried out a 94 client (actually a 92 client with some 94 patches) on a 
> cluster with some network issues, and it would constantly get stuck as 
> described above.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-6920) On timeout connecting to master, client can get stuck and never make progress

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-6920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13467926#comment-13467926
 ] 

Hadoop QA commented on HBASE-6920:
--

{color:red}-1 overall{color}.  Here are the results of testing the latest 
attachment 
  http://issues.apache.org/jira/secure/attachment/12547415/HBASE-6920-v2.patch
  against trunk revision .

{color:green}+1 @author{color}.  The patch does not contain any @author 
tags.

{color:green}+1 tests included{color}.  The patch appears to include 4 new 
or modified tests.

{color:red}-1 patch{color}.  The patch command could not apply the patch.

Console output: 
https://builds.apache.org/job/PreCommit-HBASE-Build/2987//console

This message is automatically generated.

> On timeout connecting to master, client can get stuck and never make progress
> -
>
> Key: HBASE-6920
> URL: https://issues.apache.org/jira/browse/HBASE-6920
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 0.94.2
>Reporter: Gregory Chanan
>Assignee: Gregory Chanan
>Priority: Critical
> Attachments: HBASE-6920.patch, HBASE-6920-v2.patch
>
>
> HBASE-5058 appears to have introduced an issue where a timeout in 
> HConnection.getMaster() can cause the client to never be able to connect to 
> the master.  So, for example, an HBaseAdmin object can never successfully be 
> initialized.
> The issue is here:
> {code}
> if (tryMaster.isMasterRunning()) {
>   this.master = tryMaster;
>   this.masterLock.notifyAll();
>   break;
> }
> {code}
> If isMasterRunning times out, it throws an UndeclaredThrowableException, 
> which is already not ideal, because it can be returned to the application.
>  But if the first call to getMaster succeeds, it will set masterChecked = 
> true, which makes us never try to reconnect; that is, we will set this.master 
> = null and just throw MasterNotRunningExceptions, without even trying to 
> connect.
> I tried out a 94 client (actually a 92 client with some 94 patches) on a 
> cluster with some network issues, and it would constantly get stuck as 
> described above.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-6707) TEST org.apache.hadoop.hbase.backup.example.TestZooKeeperTableArchiveClient.testMultipleTables flaps

[Jesse Yates (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-6707?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13467927#comment-13467927
 ] 

Jesse Yates commented on HBASE-6707:


@Ted - done. review can be found here: https://reviews.apache.org/r/7344/

> TEST 
> org.apache.hadoop.hbase.backup.example.TestZooKeeperTableArchiveClient.testMultipleTables
>  flaps
> 
>
> Key: HBASE-6707
> URL: https://issues.apache.org/jira/browse/HBASE-6707
> Project: HBase
>  Issue Type: Bug
>  Components: test
>Reporter: Sameer Vaishampayan
>Assignee: Jesse Yates
>Priority: Critical
> Fix For: 0.96.0
>
> Attachments: hbase-6707-v0.patch, hbase-6707-v1.patch, 
> hbase-6707-v2.patch, hbase-6707-v3.patch, hbase-6707-v4.patch
>
>
> https://builds.apache.org/job/HBase-TRUNK/3293/
> Error Message
> Archived HFiles 
> (hdfs://localhost:59986/user/jenkins/hbase/.archive/otherTable/01ced3b55d7220a9c460273a4a57b198/fam)
>  should have gotten deleted, but didn't, remaining 
> files:[hdfs://localhost:59986/user/jenkins/hbase/.archive/otherTable/01ced3b55d7220a9c460273a4a57b198/fam/fc872572a1f5443eb55b6e2567cfeb1c]
> Stacktrace
> java.lang.AssertionError: Archived HFiles 
> (hdfs://localhost:59986/user/jenkins/hbase/.archive/otherTable/01ced3b55d7220a9c460273a4a57b198/fam)
>  should have gotten deleted, but didn't, remaining 
> files:[hdfs://localhost:59986/user/jenkins/hbase/.archive/otherTable/01ced3b55d7220a9c460273a4a57b198/fam/fc872572a1f5443eb55b6e2567cfeb1c]
>   at org.junit.Assert.fail(Assert.java:93)
>   at org.junit.Assert.assertTrue(Assert.java:43)
>   at org.junit.Assert.assertNull(Assert.java:551)
>   at 
> org.apache.hadoop.hbase.backup.example.TestZooKeeperTableArchiveClient.testMultipleTables(TestZooKeeperTableArchiveClient.java:291)
>   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>   at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>   at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-6920) On timeout connecting to master, client can get stuck and never make progress

[Lars Hofhansl (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-6920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13467933#comment-13467933
 ] 

Lars Hofhansl commented on HBASE-6920:
--

+1 on patch after addressing Ted's comments ... (although the RPC changes look 
a bit scary).

> On timeout connecting to master, client can get stuck and never make progress
> -
>
> Key: HBASE-6920
> URL: https://issues.apache.org/jira/browse/HBASE-6920
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 0.94.2
>Reporter: Gregory Chanan
>Assignee: Gregory Chanan
>Priority: Critical
> Attachments: HBASE-6920.patch, HBASE-6920-v2.patch
>
>
> HBASE-5058 appears to have introduced an issue where a timeout in 
> HConnection.getMaster() can cause the client to never be able to connect to 
> the master.  So, for example, an HBaseAdmin object can never successfully be 
> initialized.
> The issue is here:
> {code}
> if (tryMaster.isMasterRunning()) {
>   this.master = tryMaster;
>   this.masterLock.notifyAll();
>   break;
> }
> {code}
> If isMasterRunning times out, it throws an UndeclaredThrowableException, 
> which is already not ideal, because it can be returned to the application.
>  But if the first call to getMaster succeeds, it will set masterChecked = 
> true, which makes us never try to reconnect; that is, we will set this.master 
> = null and just throw MasterNotRunningExceptions, without even trying to 
> connect.
> I tried out a 94 client (actually a 92 client with some 94 patches) on a 
> cluster with some network issues, and it would constantly get stuck as 
> described above.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (HBASE-6923) Create scanner benchmark

[Karthik Ranganathan (JIRA)]

Karthik Ranganathan created HBASE-6923:
--

 Summary: Create scanner benchmark
 Key: HBASE-6923
 URL: https://issues.apache.org/jira/browse/HBASE-6923
 Project: HBase
  Issue Type: Improvement
Reporter: Karthik Ranganathan
Assignee: Karthik Ranganathan


Create a simple program to benchmark performance/throughput of scanners, and 
print some results at the end.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (HBASE-6922) HBase scanner performance improvements

[Karthik Ranganathan (JIRA)]

Karthik Ranganathan created HBASE-6922:
--

 Summary: HBase scanner performance improvements
 Key: HBASE-6922
 URL: https://issues.apache.org/jira/browse/HBASE-6922
 Project: HBase
  Issue Type: Umbrella
Reporter: Karthik Ranganathan
Assignee: Karthik Ranganathan


Umbrella task for improving through in HBase scanners.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-6923) Create scanner benchmark

[Karthik Ranganathan (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-6923?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Karthik Ranganathan updated HBASE-6923:
---

Issue Type: Sub-task  (was: Improvement)
Parent: HBASE-6922

> Create scanner benchmark
> 
>
> Key: HBASE-6923
> URL: https://issues.apache.org/jira/browse/HBASE-6923
> Project: HBase
>  Issue Type: Sub-task
>Reporter: Karthik Ranganathan
>Assignee: Karthik Ranganathan
>
> Create a simple program to benchmark performance/throughput of scanners, and 
> print some results at the end.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-6874) Implement prefetching for scanners

[Karthik Ranganathan (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-6874?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Karthik Ranganathan updated HBASE-6874:
---

Issue Type: Sub-task  (was: Improvement)
Parent: HBASE-6922

> Implement prefetching for scanners
> --
>
> Key: HBASE-6874
> URL: https://issues.apache.org/jira/browse/HBASE-6874
> Project: HBase
>  Issue Type: Sub-task
>Reporter: Karthik Ranganathan
>Assignee: Karthik Ranganathan
>
> I did some quick experiments by scanning data that should be completely in 
> memory and found that adding pre-fetching increases the throughput by about 
> 50% from 26MB/s to 39MB/s.
> The idea is to perform the next in a background thread, and keep the result 
> ready. When the scanner's next comes in, return the pre-computed result and 
> issue another background read.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-6921) String and ConcurrentHashMap sizes change on jdk7; makes TestHeapSize fail; second attempt

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-6921?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13467936#comment-13467936
 ] 

Hudson commented on HBASE-6921:
---

Integrated in HBase-TRUNK #3406 (See 
[https://builds.apache.org/job/HBase-TRUNK/3406/])
HBASE-6921 String and ConcurrentHashMap sizes change on jdk7; makes 
TestHeapSize fail; second attempt (Revision 1393062)

 Result = FAILURE
stack : 
Files : 
* 
/hbase/trunk/hbase-common/src/main/java/org/apache/hadoop/hbase/util/ClassSize.java
* 
/hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/io/TestHeapSize.java


> String and ConcurrentHashMap sizes change on jdk7; makes TestHeapSize fail; 
> second attempt
> --
>
> Key: HBASE-6921
> URL: https://issues.apache.org/jira/browse/HBASE-6921
> Project: HBase
>  Issue Type: Bug
>  Components: test
>Reporter: stack
>Assignee: stack
> Fix For: 0.96.0
>
> Attachments: 6921.txt
>
>
> HBASE-6918 added debug.  Jenkins runs jdk7 23.0-b21.  Locally I have 
> 23.6-b03.  It looks like ConcurrentHashMap is a different size in these two 
> jvm's (To be confirmed).

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-6770) Allow scanner setCaching to specify size instead of number of rows

[Karthik Ranganathan (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-6770?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Karthik Ranganathan updated HBASE-6770:
---

Issue Type: Sub-task  (was: Bug)
Parent: HBASE-6922

> Allow scanner setCaching to specify size instead of number of rows
> --
>
> Key: HBASE-6770
> URL: https://issues.apache.org/jira/browse/HBASE-6770
> Project: HBase
>  Issue Type: Sub-task
>  Components: Client, regionserver
>Reporter: Karthik Ranganathan
>Assignee: Michal Gregorczyk
>
> Currently, we have the following api's to customize the behavior of scans:
> setCaching() - how many rows to cache on client to speed up scans
> setBatch() - max columns per row to return per row to prevent a very large 
> response.
> Ideally, we should be able to specify a memory buffer size because:
> 1. that would take care of both of these use cases.
> 2. it does not need any knowledge of the size of the rows or cells, as the 
> final thing we are worried about is the available memory.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-6920) On timeout connecting to master, client can get stuck and never make progress

[Gregory Chanan (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-6920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13467937#comment-13467937
 ] 

Gregory Chanan commented on HBASE-6920:
---

Lars -- you have an idea for how to do the test without making RPC changes?  
I'd prefer that, but couldn't think of a way.

> On timeout connecting to master, client can get stuck and never make progress
> -
>
> Key: HBASE-6920
> URL: https://issues.apache.org/jira/browse/HBASE-6920
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 0.94.2
>Reporter: Gregory Chanan
>Assignee: Gregory Chanan
>Priority: Critical
> Attachments: HBASE-6920.patch, HBASE-6920-v2.patch
>
>
> HBASE-5058 appears to have introduced an issue where a timeout in 
> HConnection.getMaster() can cause the client to never be able to connect to 
> the master.  So, for example, an HBaseAdmin object can never successfully be 
> initialized.
> The issue is here:
> {code}
> if (tryMaster.isMasterRunning()) {
>   this.master = tryMaster;
>   this.masterLock.notifyAll();
>   break;
> }
> {code}
> If isMasterRunning times out, it throws an UndeclaredThrowableException, 
> which is already not ideal, because it can be returned to the application.
>  But if the first call to getMaster succeeds, it will set masterChecked = 
> true, which makes us never try to reconnect; that is, we will set this.master 
> = null and just throw MasterNotRunningExceptions, without even trying to 
> connect.
> I tried out a 94 client (actually a 92 client with some 94 patches) on a 
> cluster with some network issues, and it would constantly get stuck as 
> described above.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-6707) TEST org.apache.hadoop.hbase.backup.example.TestZooKeeperTableArchiveClient.testMultipleTables flaps

[stack (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-6707?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13467939#comment-13467939
 ] 

stack commented on HBASE-6707:
--

[~jesse_yates] I must have done something wrong.  Sorry about that.  Seems fine 
running it in a loop here locally.  Waiting on Ted's review before committing.

> TEST 
> org.apache.hadoop.hbase.backup.example.TestZooKeeperTableArchiveClient.testMultipleTables
>  flaps
> 
>
> Key: HBASE-6707
> URL: https://issues.apache.org/jira/browse/HBASE-6707
> Project: HBase
>  Issue Type: Bug
>  Components: test
>Reporter: Sameer Vaishampayan
>Assignee: Jesse Yates
>Priority: Critical
> Fix For: 0.96.0
>
> Attachments: hbase-6707-v0.patch, hbase-6707-v1.patch, 
> hbase-6707-v2.patch, hbase-6707-v3.patch, hbase-6707-v4.patch
>
>
> https://builds.apache.org/job/HBase-TRUNK/3293/
> Error Message
> Archived HFiles 
> (hdfs://localhost:59986/user/jenkins/hbase/.archive/otherTable/01ced3b55d7220a9c460273a4a57b198/fam)
>  should have gotten deleted, but didn't, remaining 
> files:[hdfs://localhost:59986/user/jenkins/hbase/.archive/otherTable/01ced3b55d7220a9c460273a4a57b198/fam/fc872572a1f5443eb55b6e2567cfeb1c]
> Stacktrace
> java.lang.AssertionError: Archived HFiles 
> (hdfs://localhost:59986/user/jenkins/hbase/.archive/otherTable/01ced3b55d7220a9c460273a4a57b198/fam)
>  should have gotten deleted, but didn't, remaining 
> files:[hdfs://localhost:59986/user/jenkins/hbase/.archive/otherTable/01ced3b55d7220a9c460273a4a57b198/fam/fc872572a1f5443eb55b6e2567cfeb1c]
>   at org.junit.Assert.fail(Assert.java:93)
>   at org.junit.Assert.assertTrue(Assert.java:43)
>   at org.junit.Assert.assertNull(Assert.java:551)
>   at 
> org.apache.hadoop.hbase.backup.example.TestZooKeeperTableArchiveClient.testMultipleTables(TestZooKeeperTableArchiveClient.java:291)
>   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>   at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>   at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-6066) some low hanging read path improvement ideas

[Karthik Ranganathan (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-6066?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Karthik Ranganathan updated HBASE-6066:
---

Issue Type: Sub-task  (was: Improvement)
Parent: HBASE-6922

> some low hanging read path improvement ideas 
> -
>
> Key: HBASE-6066
> URL: https://issues.apache.org/jira/browse/HBASE-6066
> Project: HBase
>  Issue Type: Sub-task
>  Components: Performance
>Reporter: Kannan Muthukkaruppan
>Assignee: Michal Gregorczyk
>Priority: Critical
>  Labels: noob
> Fix For: 0.96.0
>
> Attachments: 
> 0001-jira-HBASE-6066-89-fb-Some-read-performance-improvem.patch, 
> metric-stringbuilder-fix.patch
>
>
> I was running some single threaded scan performance tests for a table with 
> small sized rows that is fully cached. Some observations...
> We seem to be doing several wasteful iterations over and/or building of 
> temporary lists.
> 1) One such is the following code in HRegionServer.next():
> {code}
>boolean moreRows = s.next(values, HRegion.METRIC_NEXTSIZE);
>if (!values.isEmpty()) {
>  for (KeyValue kv : values) {  -->  wasteful in most 
> cases
>currentScanResultSize += kv.heapSize();
>}
>results.add(new Result(values));
> {code}
> By default the "maxScannerResultSize" is Long.MAX_VALUE. In those cases,
> we can avoid the unnecessary iteration to compute currentScanResultSize.
> 2) An example of a wasteful temporary array, is "results" in
> RegionScanner.next().
> {code}
>   results.clear();
>   boolean returnResult = nextInternal(limit, metric);
>   outResults.addAll(results);
> {code}
> results then gets copied over to outResults via an addAll(). Not sure why we 
> can not directly collect the results in outResults.
> 3) Another almost similar exmaple of a wasteful array is "results" in 
> StoreScanner.next(), which eventually also copies its results into 
> "outResults".
> 4) Reduce overhead of "size metric" maintained in StoreScanner.next().
> {code}
>   if (metric != null) {
>  HRegion.incrNumericMetric(this.metricNamePrefix + metric,
>copyKv.getLength());
>   }
>   results.add(copyKv);
> {code}
> A single call to next() might fetch a lot of KVs. We can first add up the 
> size of those KVs in a local variable and then in a finally clause increment 
> the metric one shot, rather than updating AtomicLongs for each KV.
> 5) RegionScanner.next() calls a helper RegionScanner.next() on the same 
> object. Both are synchronized methods. Synchronized methods calling nested 
> synchronized methods on the same object are probably adding some small 
> overhead. The inner next() calls isFilterDone() which is a also a 
> synchronized method. We should factor the code to avoid these nested 
> synchronized methods.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-6912) Filters are not properly applied in certain cases

[Alex Newman (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-6912?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13467947#comment-13467947
 ] 

Alex Newman commented on HBASE-6912:


Agreed, it would be unfortunate if we attempted, for instance to apply 
timestamp filters at that part of the code. 

> Filters are not properly applied in certain cases
> -
>
> Key: HBASE-6912
> URL: https://issues.apache.org/jira/browse/HBASE-6912
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 0.94.1
>Reporter: Alex Newman
> Fix For: 0.94.3, 0.96.0
>
> Attachments: minimalTest.java
>
>
> Steps to reproduce:
> Create a table, load data into it. Flush the table.
> Do a scan with
> 1. Some filter which should not match the first entry in the scan
> 2. Where one specifies a family and column.
> You will notice that the first entry is returned even though it doesn't match 
> the filter.
> It looks like the when the first KeyValue of a scan in the column from the 
> point of view of the code
> HRegion.java
> {code}
> } else if (kv != null && !kv.isInternal() && filterRowKey(currentRow)) {
> {code}
> Is generated by
> {code}
> public static KeyValue createLastOnRow(final byte [] row,
> final int roffset, final int rlength, final byte [] family,
> final int foffset, final int flength, final byte [] qualifier,
> final int qoffset, final int qlength) { return new KeyValue(row, roffset, 
> rlength, family, foffset, flength, qualifier, qoffset, qlength, 
> HConstants.OLDEST_TIMESTAMP, Type.Minimum, null, 0, 0); }
> {code}
> So it is always internal from that point of the code.
> Only later from within
> StoreScanner.java
> {code}
> public synchronized boolean next(List outResult, int limit, String 
> metric) throws IOException {
> 
> LOOP: while((kv = this.heap.peek()) != null) {
> {code}
> ( The second time through)
> Do we get the actual kv, with a proper type and timestamp. This seems to mess 
> with filtering.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-6912) Filters are not properly applied in certain cases

[Lars Hofhansl (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-6912?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13467946#comment-13467946
 ] 

Lars Hofhansl commented on HBASE-6912:
--

I have a patch, which fixes RowFilter. But I do not like it.
Each "real" KV should through filterRowKey before it goes through 
filterKeyValue. That seems to be broken in some cases (which has to do with the 
lazy seeking we're doing now).

> Filters are not properly applied in certain cases
> -
>
> Key: HBASE-6912
> URL: https://issues.apache.org/jira/browse/HBASE-6912
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 0.94.1
>Reporter: Alex Newman
> Fix For: 0.94.3, 0.96.0
>
> Attachments: minimalTest.java
>
>
> Steps to reproduce:
> Create a table, load data into it. Flush the table.
> Do a scan with
> 1. Some filter which should not match the first entry in the scan
> 2. Where one specifies a family and column.
> You will notice that the first entry is returned even though it doesn't match 
> the filter.
> It looks like the when the first KeyValue of a scan in the column from the 
> point of view of the code
> HRegion.java
> {code}
> } else if (kv != null && !kv.isInternal() && filterRowKey(currentRow)) {
> {code}
> Is generated by
> {code}
> public static KeyValue createLastOnRow(final byte [] row,
> final int roffset, final int rlength, final byte [] family,
> final int foffset, final int flength, final byte [] qualifier,
> final int qoffset, final int qlength) { return new KeyValue(row, roffset, 
> rlength, family, foffset, flength, qualifier, qoffset, qlength, 
> HConstants.OLDEST_TIMESTAMP, Type.Minimum, null, 0, 0); }
> {code}
> So it is always internal from that point of the code.
> Only later from within
> StoreScanner.java
> {code}
> public synchronized boolean next(List outResult, int limit, String 
> metric) throws IOException {
> 
> LOOP: while((kv = this.heap.peek()) != null) {
> {code}
> ( The second time through)
> Do we get the actual kv, with a proper type and timestamp. This seems to mess 
> with filtering.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-6923) Create scanner benchmark

[Todd Lipcon (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-6923?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Todd Lipcon updated HBASE-6923:
---

Attachment: TestStorePerformance.java

Hey Karthik. Here's a test I wrote recently which may help you out. I also have 
some interesting results I'm hoping to share later this week.

> Create scanner benchmark
> 
>
> Key: HBASE-6923
> URL: https://issues.apache.org/jira/browse/HBASE-6923
> Project: HBase
>  Issue Type: Sub-task
>Reporter: Karthik Ranganathan
>Assignee: Karthik Ranganathan
> Attachments: TestStorePerformance.java
>
>
> Create a simple program to benchmark performance/throughput of scanners, and 
> print some results at the end.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-6914) Scans/Gets/Mutations don't give a good error if the table is disabled.

[Elliott Clark (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-6914?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Elliott Clark updated HBASE-6914:
-

Attachment: HBASE-6914-094-3.patch
HBASE-6914-092-3.patch

Attaching 0.92 and 0.94 patch versions before commit.

> Scans/Gets/Mutations don't give a good error if the table is disabled.
> --
>
> Key: HBASE-6914
> URL: https://issues.apache.org/jira/browse/HBASE-6914
> Project: HBase
>  Issue Type: Improvement
>  Components: Client
>Reporter: Elliott Clark
>Assignee: Elliott Clark
> Fix For: 0.92.3, 0.94.3, 0.96.0
>
> Attachments: HBASE-6914-092-3.patch, HBASE-6914-094-3.patch, 
> HBASE-6914-0.patch, HBASE-6914-1.patch, HBASE-6914-2.patch, HBASE-6914-3.patch
>
>
> Scan a table that is disabled will have the client retry multiple times and 
> then will error out with NotServingRegionException.  If the table is disabled 
> there's no need to re-try and the message should be more explicit.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-6914) Scans/Gets/Mutations don't give a good error if the table is disabled.

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-6914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13467956#comment-13467956
 ] 

Hadoop QA commented on HBASE-6914:
--

{color:red}-1 overall{color}.  Here are the results of testing the latest 
attachment 
  
http://issues.apache.org/jira/secure/attachment/12547421/HBASE-6914-094-3.patch
  against trunk revision .

{color:green}+1 @author{color}.  The patch does not contain any @author 
tags.

{color:green}+1 tests included{color}.  The patch appears to include 3 new 
or modified tests.

{color:red}-1 patch{color}.  The patch command could not apply the patch.

Console output: 
https://builds.apache.org/job/PreCommit-HBASE-Build/2988//console

This message is automatically generated.

> Scans/Gets/Mutations don't give a good error if the table is disabled.
> --
>
> Key: HBASE-6914
> URL: https://issues.apache.org/jira/browse/HBASE-6914
> Project: HBase
>  Issue Type: Improvement
>  Components: Client
>Reporter: Elliott Clark
>Assignee: Elliott Clark
> Fix For: 0.92.3, 0.94.3, 0.96.0
>
> Attachments: HBASE-6914-092-3.patch, HBASE-6914-094-3.patch, 
> HBASE-6914-0.patch, HBASE-6914-1.patch, HBASE-6914-2.patch, HBASE-6914-3.patch
>
>
> Scan a table that is disabled will have the client retry multiple times and 
> then will error out with NotServingRegionException.  If the table is disabled 
> there's no need to re-try and the message should be more explicit.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-6878) DistributerLogSplit can fail to resubmit a task done if there is an exception during the log archiving

[nkeywal (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-6878?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13467980#comment-13467980
 ] 

nkeywal commented on HBASE-6878:


Thanks a lot Prakash. I understand I could change the CHECK to FORCE, but it 
won't actually change much in real cases. So let's mark this to wont fix :-).

> DistributerLogSplit can fail to resubmit a task done if there is an exception 
> during the log archiving
> --
>
> Key: HBASE-6878
> URL: https://issues.apache.org/jira/browse/HBASE-6878
> Project: HBase
>  Issue Type: Bug
>  Components: master
>Reporter: nkeywal
>Priority: Minor
>
> The code in SplitLogManager# getDataSetWatchSuccess is:
> {code}
> if (slt.isDone()) {
>   LOG.info("task " + path + " entered state: " + slt.toString());
>   if (taskFinisher != null && !ZKSplitLog.isRescanNode(watcher, path)) {
> if (taskFinisher.finish(slt.getServerName(), 
> ZKSplitLog.getFileName(path)) == Status.DONE) {
>   setDone(path, SUCCESS);
> } else {
>   resubmitOrFail(path, CHECK);
> }
>   } else {
> setDone(path, SUCCESS);
>   }
> {code}
>   resubmitOrFail(path, CHECK);
> should be 
>   resubmitOrFail(path, FORCE);
> Without it, the task won't be resubmitted if the delay is not reached, and 
> the task will be marked as failed.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Resolved] (HBASE-6878) DistributerLogSplit can fail to resubmit a task done if there is an exception during the log archiving

[nkeywal (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-6878?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

nkeywal resolved HBASE-6878.


Resolution: Won't Fix

> DistributerLogSplit can fail to resubmit a task done if there is an exception 
> during the log archiving
> --
>
> Key: HBASE-6878
> URL: https://issues.apache.org/jira/browse/HBASE-6878
> Project: HBase
>  Issue Type: Bug
>  Components: master
>Reporter: nkeywal
>Priority: Minor
>
> The code in SplitLogManager# getDataSetWatchSuccess is:
> {code}
> if (slt.isDone()) {
>   LOG.info("task " + path + " entered state: " + slt.toString());
>   if (taskFinisher != null && !ZKSplitLog.isRescanNode(watcher, path)) {
> if (taskFinisher.finish(slt.getServerName(), 
> ZKSplitLog.getFileName(path)) == Status.DONE) {
>   setDone(path, SUCCESS);
> } else {
>   resubmitOrFail(path, CHECK);
> }
>   } else {
> setDone(path, SUCCESS);
>   }
> {code}
>   resubmitOrFail(path, CHECK);
> should be 
>   resubmitOrFail(path, FORCE);
> Without it, the task won't be resubmitted if the delay is not reached, and 
> the task will be marked as failed.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5937) Refactor HLog into an interface.

[stack (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-5937?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stack updated HBASE-5937:
-

   Resolution: Fixed
Fix Version/s: 0.96.0
 Hadoop Flags: Reviewed
   Status: Resolved  (was: Patch Available)

Committed to trunk after running full suite and having it pass.  Good on you 
Flavio.

Please create new issues for problems found reviewing this patch.

> Refactor HLog into an interface.
> 
>
> Key: HBASE-5937
> URL: https://issues.apache.org/jira/browse/HBASE-5937
> Project: HBase
>  Issue Type: Sub-task
>Reporter: Li Pi
>Assignee: Flavio Junqueira
>Priority: Minor
> Fix For: 0.96.0
>
> Attachments: 5937-hlog-with-javadoc.txt, HBASE-5937.patch, 
> HBASE-5937.patch, HBASE-5937.patch, HBASE-5937.patch, HBASE-5937.patch, 
> HBASE-5937.v2.patch, HBASE-5937.v3.patch, 
> org.apache.hadoop.hbase.client.TestMultiParallel-output.txt
>
>
> What the summary says. Create HLog interface. Make current implementation use 
> it.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (HBASE-6924) HBase Master spews into a loop if a user attempts to create a snappy table when snappy isn't properly configured

[Alex Newman (JIRA)]

Alex Newman created HBASE-6924:
--

 Summary: HBase Master spews into a loop if a user attempts to 
create a snappy table when snappy isn't properly configured
 Key: HBASE-6924
 URL: https://issues.apache.org/jira/browse/HBASE-6924
 Project: HBase
  Issue Type: Bug
Reporter: Alex Newman


If a user attempts to create a table, for instance
 create 't1',{ NAME=>'c1', COMPRESSION=>'snappy'}
on stock HBase(Without snappy setup), 
the master will spew this error in a loop

12/10/02 12:41:38 INFO handler.OpenRegionHandler: Opening of region {NAME => 
't1,,1349206881317.2d34e32205ffe677496b03faa7e66063.', STARTKEY => '', ENDKEY 
=> '', ENCODED => 2d34e32205ffe677496b03faa7e66063,} failed, marking as 
FAILED_OPEN in ZK
12/10/02 12:41:38 INFO regionserver.HRegionServer: Received request to open 
region: t1,,1349206881317.2d34e32205ffe677496b03faa7e66063.
12/10/02 12:41:38 INFO regionserver.HRegion: Setting up tabledescriptor config 
now ...
12/10/02 12:41:38 ERROR handler.OpenRegionHandler: Failed open of 
region=t1,,1349206881317.2d34e32205ffe677496b03faa7e66063., starting to roll 
back the global memstore size.
java.io.IOException: Compression algorithm 'snappy' previously failed test.
at 
org.apache.hadoop.hbase.util.CompressionTest.testCompression(CompressionTest.java:78)
at 
org.apache.hadoop.hbase.regionserver.HRegion.checkCompressionCodecs(HRegion.java:3822)
at 
org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:3811)
at 
org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:3761)
at 
org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.openRegion(OpenRegionHandler.java:332)
at 
org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.process(OpenRegionHandler.java:108)
at 
org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:169)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
Even after the shell is killed.

In fact, even after a hbase reboot we will endlessly spew this painful and high 
overhead error.



--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-6924) HBase Master spews into a loop if a user attempts to create a snappy table when snappy isn't properly configured

[Alex Newman (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-6924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13468012#comment-13468012
 ] 

Alex Newman commented on HBASE-6924:


I assume the issue is that opening a region can often fail and we want some 
retry logic.

> HBase Master spews into a loop if a user attempts to create a snappy table 
> when snappy isn't properly configured
> 
>
> Key: HBASE-6924
> URL: https://issues.apache.org/jira/browse/HBASE-6924
> Project: HBase
>  Issue Type: Bug
>Reporter: Alex Newman
>
> If a user attempts to create a table, for instance
>  create 't1',{ NAME=>'c1', COMPRESSION=>'snappy'}
> on stock HBase(Without snappy setup), 
> the master will spew this error in a loop
> 12/10/02 12:41:38 INFO handler.OpenRegionHandler: Opening of region {NAME => 
> 't1,,1349206881317.2d34e32205ffe677496b03faa7e66063.', STARTKEY => '', ENDKEY 
> => '', ENCODED => 2d34e32205ffe677496b03faa7e66063,} failed, marking as 
> FAILED_OPEN in ZK
> 12/10/02 12:41:38 INFO regionserver.HRegionServer: Received request to open 
> region: t1,,1349206881317.2d34e32205ffe677496b03faa7e66063.
> 12/10/02 12:41:38 INFO regionserver.HRegion: Setting up tabledescriptor 
> config now ...
> 12/10/02 12:41:38 ERROR handler.OpenRegionHandler: Failed open of 
> region=t1,,1349206881317.2d34e32205ffe677496b03faa7e66063., starting to roll 
> back the global memstore size.
> java.io.IOException: Compression algorithm 'snappy' previously failed test.
> at 
> org.apache.hadoop.hbase.util.CompressionTest.testCompression(CompressionTest.java:78)
> at 
> org.apache.hadoop.hbase.regionserver.HRegion.checkCompressionCodecs(HRegion.java:3822)
> at 
> org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:3811)
> at 
> org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:3761)
> at 
> org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.openRegion(OpenRegionHandler.java:332)
> at 
> org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.process(OpenRegionHandler.java:108)
> at 
> org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:169)
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:662)
> Even after the shell is killed.
> In fact, even after a hbase reboot we will endlessly spew this painful and 
> high overhead error.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-6707) TEST org.apache.hadoop.hbase.backup.example.TestZooKeeperTableArchiveClient.testMultipleTables flaps

[Ted Yu (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-6707?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13468033#comment-13468033
 ] 

Ted Yu commented on HBASE-6707:
---

The gist of this JIRA is about making sure that HFiles for table not configured 
to endure table drop should be deleted.
I had a question w.r.t. the above which I noted on review board.

> TEST 
> org.apache.hadoop.hbase.backup.example.TestZooKeeperTableArchiveClient.testMultipleTables
>  flaps
> 
>
> Key: HBASE-6707
> URL: https://issues.apache.org/jira/browse/HBASE-6707
> Project: HBase
>  Issue Type: Bug
>  Components: test
>Reporter: Sameer Vaishampayan
>Assignee: Jesse Yates
>Priority: Critical
> Fix For: 0.96.0
>
> Attachments: hbase-6707-v0.patch, hbase-6707-v1.patch, 
> hbase-6707-v2.patch, hbase-6707-v3.patch, hbase-6707-v4.patch
>
>
> https://builds.apache.org/job/HBase-TRUNK/3293/
> Error Message
> Archived HFiles 
> (hdfs://localhost:59986/user/jenkins/hbase/.archive/otherTable/01ced3b55d7220a9c460273a4a57b198/fam)
>  should have gotten deleted, but didn't, remaining 
> files:[hdfs://localhost:59986/user/jenkins/hbase/.archive/otherTable/01ced3b55d7220a9c460273a4a57b198/fam/fc872572a1f5443eb55b6e2567cfeb1c]
> Stacktrace
> java.lang.AssertionError: Archived HFiles 
> (hdfs://localhost:59986/user/jenkins/hbase/.archive/otherTable/01ced3b55d7220a9c460273a4a57b198/fam)
>  should have gotten deleted, but didn't, remaining 
> files:[hdfs://localhost:59986/user/jenkins/hbase/.archive/otherTable/01ced3b55d7220a9c460273a4a57b198/fam/fc872572a1f5443eb55b6e2567cfeb1c]
>   at org.junit.Assert.fail(Assert.java:93)
>   at org.junit.Assert.assertTrue(Assert.java:43)
>   at org.junit.Assert.assertNull(Assert.java:551)
>   at 
> org.apache.hadoop.hbase.backup.example.TestZooKeeperTableArchiveClient.testMultipleTables(TestZooKeeperTableArchiveClient.java:291)
>   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>   at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>   at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-6923) Create scanner benchmark

[Karthik Ranganathan (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-6923?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13468039#comment-13468039
 ] 

Karthik Ranganathan commented on HBASE-6923:


Hey Todd, nice! I too have written a benchmark with interesting results. Would 
be interesting to compare :)

> Create scanner benchmark
> 
>
> Key: HBASE-6923
> URL: https://issues.apache.org/jira/browse/HBASE-6923
> Project: HBase
>  Issue Type: Sub-task
>Reporter: Karthik Ranganathan
>Assignee: Karthik Ranganathan
> Attachments: TestStorePerformance.java
>
>
> Create a simple program to benchmark performance/throughput of scanners, and 
> print some results at the end.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (HBASE-6925) Change socket write size from 8K to 64K for HBaseServer

[Karthik Ranganathan (JIRA)]

Karthik Ranganathan created HBASE-6925:
--

 Summary: Change socket write size from 8K to 64K for HBaseServer
 Key: HBASE-6925
 URL: https://issues.apache.org/jira/browse/HBASE-6925
 Project: HBase
  Issue Type: Improvement
Reporter: Karthik Ranganathan


Creating a JIRA for this, but the change is trivial: change NIO_BUFFER_LIMIT 
from 8K to 64K in HBaseServer. This seems to increase scan throughput.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-6925) Change socket write size from 8K to 64K for HBaseServer

[Karthik Ranganathan (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-6925?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Karthik Ranganathan updated HBASE-6925:
---

Issue Type: Sub-task  (was: Improvement)
Parent: HBASE-6922

> Change socket write size from 8K to 64K for HBaseServer
> ---
>
> Key: HBASE-6925
> URL: https://issues.apache.org/jira/browse/HBASE-6925
> Project: HBase
>  Issue Type: Sub-task
>Reporter: Karthik Ranganathan
>Assignee: Karthik Ranganathan
>
> Creating a JIRA for this, but the change is trivial: change NIO_BUFFER_LIMIT 
> from 8K to 64K in HBaseServer. This seems to increase scan throughput.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-6707) TEST org.apache.hadoop.hbase.backup.example.TestZooKeeperTableArchiveClient.testMultipleTables flaps

[Ted Yu (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-6707?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13468042#comment-13468042
 ] 

Ted Yu commented on HBASE-6707:
---

I am sorry Jesse: HBASE-5699 has gone in and patch v4 is stale now.

Once you provide rebased patch, I will run the test and confirm that the fix 
works.

Thanks for your patience.

> TEST 
> org.apache.hadoop.hbase.backup.example.TestZooKeeperTableArchiveClient.testMultipleTables
>  flaps
> 
>
> Key: HBASE-6707
> URL: https://issues.apache.org/jira/browse/HBASE-6707
> Project: HBase
>  Issue Type: Bug
>  Components: test
>Reporter: Sameer Vaishampayan
>Assignee: Jesse Yates
>Priority: Critical
> Fix For: 0.96.0
>
> Attachments: hbase-6707-v0.patch, hbase-6707-v1.patch, 
> hbase-6707-v2.patch, hbase-6707-v3.patch, hbase-6707-v4.patch
>
>
> https://builds.apache.org/job/HBase-TRUNK/3293/
> Error Message
> Archived HFiles 
> (hdfs://localhost:59986/user/jenkins/hbase/.archive/otherTable/01ced3b55d7220a9c460273a4a57b198/fam)
>  should have gotten deleted, but didn't, remaining 
> files:[hdfs://localhost:59986/user/jenkins/hbase/.archive/otherTable/01ced3b55d7220a9c460273a4a57b198/fam/fc872572a1f5443eb55b6e2567cfeb1c]
> Stacktrace
> java.lang.AssertionError: Archived HFiles 
> (hdfs://localhost:59986/user/jenkins/hbase/.archive/otherTable/01ced3b55d7220a9c460273a4a57b198/fam)
>  should have gotten deleted, but didn't, remaining 
> files:[hdfs://localhost:59986/user/jenkins/hbase/.archive/otherTable/01ced3b55d7220a9c460273a4a57b198/fam/fc872572a1f5443eb55b6e2567cfeb1c]
>   at org.junit.Assert.fail(Assert.java:93)
>   at org.junit.Assert.assertTrue(Assert.java:43)
>   at org.junit.Assert.assertNull(Assert.java:551)
>   at 
> org.apache.hadoop.hbase.backup.example.TestZooKeeperTableArchiveClient.testMultipleTables(TestZooKeeperTableArchiveClient.java:291)
>   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>   at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>   at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Assigned] (HBASE-6925) Change socket write size from 8K to 64K for HBaseServer

[Karthik Ranganathan (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-6925?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Karthik Ranganathan reassigned HBASE-6925:
--

Assignee: Karthik Ranganathan

> Change socket write size from 8K to 64K for HBaseServer
> ---
>
> Key: HBASE-6925
> URL: https://issues.apache.org/jira/browse/HBASE-6925
> Project: HBase
>  Issue Type: Improvement
>Reporter: Karthik Ranganathan
>Assignee: Karthik Ranganathan
>
> Creating a JIRA for this, but the change is trivial: change NIO_BUFFER_LIMIT 
> from 8K to 64K in HBaseServer. This seems to increase scan throughput.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-6707) TEST org.apache.hadoop.hbase.backup.example.TestZooKeeperTableArchiveClient.testMultipleTables flaps

[Jesse Yates (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-6707?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13468043#comment-13468043
 ] 

Jesse Yates commented on HBASE-6707:


@Ted - which jira? That one's still open... I want to make sure my changes make 
sense.

> TEST 
> org.apache.hadoop.hbase.backup.example.TestZooKeeperTableArchiveClient.testMultipleTables
>  flaps
> 
>
> Key: HBASE-6707
> URL: https://issues.apache.org/jira/browse/HBASE-6707
> Project: HBase
>  Issue Type: Bug
>  Components: test
>Reporter: Sameer Vaishampayan
>Assignee: Jesse Yates
>Priority: Critical
> Fix For: 0.96.0
>
> Attachments: hbase-6707-v0.patch, hbase-6707-v1.patch, 
> hbase-6707-v2.patch, hbase-6707-v3.patch, hbase-6707-v4.patch
>
>
> https://builds.apache.org/job/HBase-TRUNK/3293/
> Error Message
> Archived HFiles 
> (hdfs://localhost:59986/user/jenkins/hbase/.archive/otherTable/01ced3b55d7220a9c460273a4a57b198/fam)
>  should have gotten deleted, but didn't, remaining 
> files:[hdfs://localhost:59986/user/jenkins/hbase/.archive/otherTable/01ced3b55d7220a9c460273a4a57b198/fam/fc872572a1f5443eb55b6e2567cfeb1c]
> Stacktrace
> java.lang.AssertionError: Archived HFiles 
> (hdfs://localhost:59986/user/jenkins/hbase/.archive/otherTable/01ced3b55d7220a9c460273a4a57b198/fam)
>  should have gotten deleted, but didn't, remaining 
> files:[hdfs://localhost:59986/user/jenkins/hbase/.archive/otherTable/01ced3b55d7220a9c460273a4a57b198/fam/fc872572a1f5443eb55b6e2567cfeb1c]
>   at org.junit.Assert.fail(Assert.java:93)
>   at org.junit.Assert.assertTrue(Assert.java:43)
>   at org.junit.Assert.assertNull(Assert.java:551)
>   at 
> org.apache.hadoop.hbase.backup.example.TestZooKeeperTableArchiveClient.testMultipleTables(TestZooKeeperTableArchiveClient.java:291)
>   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>   at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>   at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-6707) TEST org.apache.hadoop.hbase.backup.example.TestZooKeeperTableArchiveClient.testMultipleTables flaps

[Ted Yu (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-6707?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13468047#comment-13468047
 ] 

Ted Yu commented on HBASE-6707:
---

Pardon me. Looks like Stack entered wrong JIRA:
{code}
r1393126 | stack | 2012-10-02 12:29:19 -0700 (Tue, 02 Oct 2012) | 1 line

HBASE-5699 Refactor HLog into an interface
{code}
What he integrated was HBASE-5937

> TEST 
> org.apache.hadoop.hbase.backup.example.TestZooKeeperTableArchiveClient.testMultipleTables
>  flaps
> 
>
> Key: HBASE-6707
> URL: https://issues.apache.org/jira/browse/HBASE-6707
> Project: HBase
>  Issue Type: Bug
>  Components: test
>Reporter: Sameer Vaishampayan
>Assignee: Jesse Yates
>Priority: Critical
> Fix For: 0.96.0
>
> Attachments: hbase-6707-v0.patch, hbase-6707-v1.patch, 
> hbase-6707-v2.patch, hbase-6707-v3.patch, hbase-6707-v4.patch
>
>
> https://builds.apache.org/job/HBase-TRUNK/3293/
> Error Message
> Archived HFiles 
> (hdfs://localhost:59986/user/jenkins/hbase/.archive/otherTable/01ced3b55d7220a9c460273a4a57b198/fam)
>  should have gotten deleted, but didn't, remaining 
> files:[hdfs://localhost:59986/user/jenkins/hbase/.archive/otherTable/01ced3b55d7220a9c460273a4a57b198/fam/fc872572a1f5443eb55b6e2567cfeb1c]
> Stacktrace
> java.lang.AssertionError: Archived HFiles 
> (hdfs://localhost:59986/user/jenkins/hbase/.archive/otherTable/01ced3b55d7220a9c460273a4a57b198/fam)
>  should have gotten deleted, but didn't, remaining 
> files:[hdfs://localhost:59986/user/jenkins/hbase/.archive/otherTable/01ced3b55d7220a9c460273a4a57b198/fam/fc872572a1f5443eb55b6e2567cfeb1c]
>   at org.junit.Assert.fail(Assert.java:93)
>   at org.junit.Assert.assertTrue(Assert.java:43)
>   at org.junit.Assert.assertNull(Assert.java:551)
>   at 
> org.apache.hadoop.hbase.backup.example.TestZooKeeperTableArchiveClient.testMultipleTables(TestZooKeeperTableArchiveClient.java:291)
>   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>   at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>   at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (HBASE-6926) Cleanup some of the javadoc warnings.

[stack (JIRA)]

stack created HBASE-6926:


 Summary: Cleanup some of the javadoc warnings.
 Key: HBASE-6926
 URL: https://issues.apache.org/jira/browse/HBASE-6926
 Project: HBase
  Issue Type: Bug
  Components: documentation
Reporter: stack
Assignee: stack
 Fix For: 0.96.0
 Attachments: javadocleanup.txt



--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-6926) Cleanup some of the javadoc warnings.

[stack (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-6926?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stack updated HBASE-6926:
-

Attachment: javadocleanup.txt

Makes all but hbase-server module clean.  Still a bunch to do.

> Cleanup some of the javadoc warnings.
> -
>
> Key: HBASE-6926
> URL: https://issues.apache.org/jira/browse/HBASE-6926
> Project: HBase
>  Issue Type: Bug
>  Components: documentation
>Reporter: stack
>Assignee: stack
> Fix For: 0.96.0
>
> Attachments: javadocleanup.txt
>
>


--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Resolved] (HBASE-6926) Cleanup some of the javadoc warnings.

[stack (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-6926?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stack resolved HBASE-6926.
--

Resolution: Fixed

I committed these fixups.

> Cleanup some of the javadoc warnings.
> -
>
> Key: HBASE-6926
> URL: https://issues.apache.org/jira/browse/HBASE-6926
> Project: HBase
>  Issue Type: Bug
>  Components: documentation
>Reporter: stack
>Assignee: stack
> Fix For: 0.96.0
>
> Attachments: javadocleanup.txt
>
>


--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-6914) Scans/Gets/Mutations don't give a good error if the table is disabled.

[Elliott Clark (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-6914?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Elliott Clark updated HBASE-6914:
-

  Resolution: Fixed
Release Note: Scans/Gets/Mutations will now throw a DoNotRetryException, 
with an explanation, when the tageted table has been disabled.
Hadoop Flags: Reviewed
  Status: Resolved  (was: Patch Available)

> Scans/Gets/Mutations don't give a good error if the table is disabled.
> --
>
> Key: HBASE-6914
> URL: https://issues.apache.org/jira/browse/HBASE-6914
> Project: HBase
>  Issue Type: Improvement
>  Components: Client
>Reporter: Elliott Clark
>Assignee: Elliott Clark
> Fix For: 0.92.3, 0.94.3, 0.96.0
>
> Attachments: HBASE-6914-092-3.patch, HBASE-6914-094-3.patch, 
> HBASE-6914-0.patch, HBASE-6914-1.patch, HBASE-6914-2.patch, HBASE-6914-3.patch
>
>
> Scan a table that is disabled will have the client retry multiple times and 
> then will error out with NotServingRegionException.  If the table is disabled 
> there's no need to re-try and the message should be more explicit.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Assigned] (HBASE-3661) Handle empty qualifier better in shell for increments

[Michael Drzal (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-3661?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Michael Drzal reassigned HBASE-3661:


Assignee: Michael Drzal

> Handle empty qualifier better in shell for increments
> -
>
> Key: HBASE-3661
> URL: https://issues.apache.org/jira/browse/HBASE-3661
> Project: HBase
>  Issue Type: Improvement
>  Components: shell
>Affects Versions: 0.92.0
>Reporter: Lars George
>Assignee: Michael Drzal
>Priority: Minor
>
> When trying to increment a counter using the examples, which specify no 
> *explicit* qualifier you get an error:
> {code}
> hbase(main):014:0> incr 'testtable', 'cnt1', 'colfam1', 1
> ERROR: org.apache.hadoop.hbase.client.RetriesExhaustedException: Trying to 
> contact region server 10.0.0.57:51640 for region 
> testtable,,1300267113942.cd2e7925140eb414d519621e384fb654., row 'cnt1', but 
> failed after 7 attempts.
> Exceptions:
> java.io.IOException: java.io.IOException: java.lang.NullPointerException
> at 
> org.apache.hadoop.hbase.regionserver.ColumnCount.(ColumnCount.java:47)
> at 
> org.apache.hadoop.hbase.regionserver.ExplicitColumnTracker.(ExplicitColumnTracker.java:69)
> at 
> org.apache.hadoop.hbase.regionserver.ScanQueryMatcher.(ScanQueryMatcher.java:93)
> at 
> org.apache.hadoop.hbase.regionserver.StoreScanner.(StoreScanner.java:65)
> at 
> org.apache.hadoop.hbase.regionserver.Store.getScanner(Store.java:1436)
> at 
> org.apache.hadoop.hbase.regionserver.HRegion$RegionScanner.(HRegion.java:2412)
> at 
> org.apache.hadoop.hbase.regionserver.HRegion.instantiateInternalScanner(HRegion.java:1185)
> at 
> org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:1171)
> at 
> org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:1155)
> at 
> org.apache.hadoop.hbase.regionserver.HRegion.getLastIncrement(HRegion.java:3087)
> at 
> org.apache.hadoop.hbase.regionserver.HRegion.incrementColumnValue(HRegion.java:3312)
> at 
> org.apache.hadoop.hbase.regionserver.HRegionServer.incrementColumnValue(HRegionServer.java:2570)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at 
> org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:309)
> at 
> org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1060)
> Here is some help for this command:
> Increments a cell 'value' at specified table/row/column coordinates.
> To increment a cell value in table 't1' at row 'r1' under column
> 'c1' by 1 (can be omitted) or 10 do:
>   hbase> incr 't1', 'r1', 'c1'
>   hbase> incr 't1', 'r1', 'c1', 1
>   hbase> incr 't1', 'r1', 'c1', 10
> {code}
> Handle this more gracefully (printing 5 stacktraces is ugly), improve the 
> help to specify what is needed more clearly. Or fix the server side to 
> support this, if this makes sense, and therefore never triggering this issue.
> Adding a qualifier makes it work:
> {code}
> hbase(main):015:0> incr 'testtable', 'cnt1', 'colfam1:test', 1
> COUNTER VALUE = 1
> hbase(main):016:0> incr 'testtable', 'cnt1', 'colfam1:test', 1
> COUNTER VALUE = 2
> {code} 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Resolved] (HBASE-6921) String and ConcurrentHashMap sizes change on jdk7; makes TestHeapSize fail; second attempt

[stack (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-6921?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stack resolved HBASE-6921.
--

Resolution: Fixed

This seems to fix the jdk7 issues.  Resolving.

> String and ConcurrentHashMap sizes change on jdk7; makes TestHeapSize fail; 
> second attempt
> --
>
> Key: HBASE-6921
> URL: https://issues.apache.org/jira/browse/HBASE-6921
> Project: HBase
>  Issue Type: Bug
>  Components: test
>Reporter: stack
>Assignee: stack
> Fix For: 0.96.0
>
> Attachments: 6921.txt
>
>
> HBASE-6918 added debug.  Jenkins runs jdk7 23.0-b21.  Locally I have 
> 23.6-b03.  It looks like ConcurrentHashMap is a different size in these two 
> jvm's (To be confirmed).

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-6707) TEST org.apache.hadoop.hbase.backup.example.TestZooKeeperTableArchiveClient.testMultipleTables flaps

[Jesse Yates (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-6707?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13468063#comment-13468063
 ] 

Jesse Yates commented on HBASE-6707:


[~te...@apache.org][~stack] looks like stack accidentally rolled in the latest 
version of this patch with HBASE-5937. So feel free to test it Ted(?) since its 
in. We can always revert if not good. Only thing missing from the patch to 
trunk is removal of CheckedArchivingHFileCleaner.java

> TEST 
> org.apache.hadoop.hbase.backup.example.TestZooKeeperTableArchiveClient.testMultipleTables
>  flaps
> 
>
> Key: HBASE-6707
> URL: https://issues.apache.org/jira/browse/HBASE-6707
> Project: HBase
>  Issue Type: Bug
>  Components: test
>Reporter: Sameer Vaishampayan
>Assignee: Jesse Yates
>Priority: Critical
> Fix For: 0.96.0
>
> Attachments: hbase-6707-v0.patch, hbase-6707-v1.patch, 
> hbase-6707-v2.patch, hbase-6707-v3.patch, hbase-6707-v4.patch
>
>
> https://builds.apache.org/job/HBase-TRUNK/3293/
> Error Message
> Archived HFiles 
> (hdfs://localhost:59986/user/jenkins/hbase/.archive/otherTable/01ced3b55d7220a9c460273a4a57b198/fam)
>  should have gotten deleted, but didn't, remaining 
> files:[hdfs://localhost:59986/user/jenkins/hbase/.archive/otherTable/01ced3b55d7220a9c460273a4a57b198/fam/fc872572a1f5443eb55b6e2567cfeb1c]
> Stacktrace
> java.lang.AssertionError: Archived HFiles 
> (hdfs://localhost:59986/user/jenkins/hbase/.archive/otherTable/01ced3b55d7220a9c460273a4a57b198/fam)
>  should have gotten deleted, but didn't, remaining 
> files:[hdfs://localhost:59986/user/jenkins/hbase/.archive/otherTable/01ced3b55d7220a9c460273a4a57b198/fam/fc872572a1f5443eb55b6e2567cfeb1c]
>   at org.junit.Assert.fail(Assert.java:93)
>   at org.junit.Assert.assertTrue(Assert.java:43)
>   at org.junit.Assert.assertNull(Assert.java:551)
>   at 
> org.apache.hadoop.hbase.backup.example.TestZooKeeperTableArchiveClient.testMultipleTables(TestZooKeeperTableArchiveClient.java:291)
>   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>   at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>   at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5937) Refactor HLog into an interface.

[Ted Yu (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-5937?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13468066#comment-13468066
 ] 

Ted Yu commented on HBASE-5937:
---

I am confused as to the scope of the commit:
{code}
$ svn log 
hbase-server/src/main/java/org/apache/hadoop/hbase/backup/example/LongTermArchivingHFileCleaner.java
 | head

r1393126 | stack | 2012-10-02 12:29:19 -0700 (Tue, 02 Oct 2012) | 1 line

HBASE-5699 Refactor HLog into an interface
{code}
First, JIRA number was wrong.
Second, LongTermArchivingHFileCleaner.java wasn't even in Flavio's patch.

Maybe one version of patch for HBASE-6707 was integrated by chance ?

> Refactor HLog into an interface.
> 
>
> Key: HBASE-5937
> URL: https://issues.apache.org/jira/browse/HBASE-5937
> Project: HBase
>  Issue Type: Sub-task
>Reporter: Li Pi
>Assignee: Flavio Junqueira
>Priority: Minor
> Fix For: 0.96.0
>
> Attachments: 5937-hlog-with-javadoc.txt, HBASE-5937.patch, 
> HBASE-5937.patch, HBASE-5937.patch, HBASE-5937.patch, HBASE-5937.patch, 
> HBASE-5937.v2.patch, HBASE-5937.v3.patch, 
> org.apache.hadoop.hbase.client.TestMultiParallel-output.txt
>
>
> What the summary says. Create HLog interface. Make current implementation use 
> it.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-4060) Making region assignment more robust

[stack (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-4060?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stack updated HBASE-4060:
-

Fix Version/s: (was: 0.96.0)

Moving out of 0.96.0.  Needs to be worked through more.  Won't be done for 
0.96.0

> Making region assignment more robust
> 
>
> Key: HBASE-4060
> URL: https://issues.apache.org/jira/browse/HBASE-4060
> Project: HBase
>  Issue Type: Bug
>Reporter: Ted Yu
>Assignee: ramkrishna.s.vasudevan
>
> From Eran Kutner:
> My concern is that the region allocation process seems to rely too much on
> timing considerations and doesn't seem to take enough measures to guarantee
> conflicts do not occur. I understand that in a distributed environment, when
> you don't get a timely response from a remote machine you can't know for
> sure if it did or did not receive the request, however there are things that
> can be done to mitigate this and reduce the conflict time significantly. For
> example, when I run dbck it knows that some regions are multiply assigned,
> the master could do the same and try to resolve the conflict. Another
> approach would be to handle late responses, even if the response from the
> remote machine arrives after it was assumed to be dead the master should
> have enough information to know it had created a conflict by assigning the
> region to another server. An even better solution, I think, is for the RS to
> periodically test that it is indeed the rightful owner of every region it
> holds and relinquish control over the region if it's not.
> Obviously a state where two RSs hold the same region is pathological and can
> lead to data loss, as demonstrated in my case. The system should be able to
> actively protect itself against such a scenario. It probably doesn't need
> saying but there is really nothing worse for a data storage system than data
> loss.
> In my case the problem didn't happen in the initial phase but after
> disabling and enabling a table with about 12K regions.
> For more background information, see 'Errors after major compaction' 
> discussion on u...@hbase.apache.org

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-6480) If callQueueSize exceed maxQueueSize, all call will be rejected, do not reject priorityCall

[stack (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-6480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13468069#comment-13468069
 ] 

stack commented on HBASE-6480:
--

[~aoxiang] These should be some upper bound.  What is your operational 
experience running with this patch?  The idea is good.  This call is probably 
expensive ' callSize = call.getSize();'  What happens when client gets 
'callTooBig'?  It retries?  Can it wait before it retries?

> If callQueueSize exceed maxQueueSize, all call will be rejected, do not 
> reject priorityCall 
> 
>
> Key: HBASE-6480
> URL: https://issues.apache.org/jira/browse/HBASE-6480
> Project: HBase
>  Issue Type: Bug
>Reporter: binlijin
> Fix For: 0.94.3, 0.96.0
>
> Attachments: HBASE-6480-94.patch, HBASE-6480-trunk.patch
>
>
> Current if the callQueueSize exceed maxQueueSize, all call will be rejected, 
> Should we let the priority Call pass through?
> Current:
> {code}
> if ((callSize + callQueueSize.get()) > maxQueueSize) {
>   Call callTooBig = xxx
>   return ;
> }
> if (priorityCallQueue != null && getQosLevel(param) > highPriorityLevel) {
>   priorityCallQueue.put(call);
>   updateCallQueueLenMetrics(priorityCallQueue);
> } else {
>   callQueue.put(call);  // queue the call; maybe blocked here
>   updateCallQueueLenMetrics(callQueue);
> }
> {code}
> Should we change it to :
> {code}
> if (priorityCallQueue != null && getQosLevel(param) > highPriorityLevel) {
>   priorityCallQueue.put(call);
>   updateCallQueueLenMetrics(priorityCallQueue);
> } else {
>   if ((callSize + callQueueSize.get()) > maxQueueSize) {
>Call callTooBig = xxx
>return ;
>   }
>   callQueue.put(call);  // queue the call; maybe blocked here
>   updateCallQueueLenMetrics(callQueue);
> }
> {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-6491) add limit function at ClientScanner

[stack (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-6491?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stack updated HBASE-6491:
-

Fix Version/s: (was: 0.96.0)

Moving out till questions answered.

> add limit function at ClientScanner
> ---
>
> Key: HBASE-6491
> URL: https://issues.apache.org/jira/browse/HBASE-6491
> Project: HBase
>  Issue Type: New Feature
>  Components: Client
>Affects Versions: 0.96.0
>Reporter: ronghai.ma
>Assignee: ronghai.ma
>  Labels: patch
> Attachments: ClientScanner.java, HBASE-6491.patch
>
>
> Add a new method in ClientScanner to implement a function like LIMIT in MySQL.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-6920) On timeout connecting to master, client can get stuck and never make progress

[Lars Hofhansl (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-6920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13468084#comment-13468084
 ] 

Lars Hofhansl commented on HBASE-6920:
--

Hmm... Not offhand.

Looking at the patch again... Will this actually fix the problem? The problem 
is that masterChecked is true when we successfully connected to a master once. 
If true, we do not try again.
So we were successful once, we will never enter the retry loop (since 
masterChecked == true)... Right?

I wonder whether another approach would be to set masterChecked = false at the 
same spot where master is set to null, but only do that if master was not-null 
before.


> On timeout connecting to master, client can get stuck and never make progress
> -
>
> Key: HBASE-6920
> URL: https://issues.apache.org/jira/browse/HBASE-6920
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 0.94.2
>Reporter: Gregory Chanan
>Assignee: Gregory Chanan
>Priority: Critical
> Attachments: HBASE-6920.patch, HBASE-6920-v2.patch
>
>
> HBASE-5058 appears to have introduced an issue where a timeout in 
> HConnection.getMaster() can cause the client to never be able to connect to 
> the master.  So, for example, an HBaseAdmin object can never successfully be 
> initialized.
> The issue is here:
> {code}
> if (tryMaster.isMasterRunning()) {
>   this.master = tryMaster;
>   this.masterLock.notifyAll();
>   break;
> }
> {code}
> If isMasterRunning times out, it throws an UndeclaredThrowableException, 
> which is already not ideal, because it can be returned to the application.
>  But if the first call to getMaster succeeds, it will set masterChecked = 
> true, which makes us never try to reconnect; that is, we will set this.master 
> = null and just throw MasterNotRunningExceptions, without even trying to 
> connect.
> I tried out a 94 client (actually a 92 client with some 94 patches) on a 
> cluster with some network issues, and it would constantly get stuck as 
> described above.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-6805) Extend co-processor framework to provide observers for filter operations

[Gary Helmling (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-6805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13468086#comment-13468086
 ] 

Gary Helmling commented on HBASE-6805:
--

Looking at the encryption example here, it seems like you could provide that 
with the existing coprocessor hooks:

Option A:
# In {{EncryptingRegionObserver.preScannerOpen()}}, if any Filter is set on the 
Scan object, wrap it in a custom {{DecryptingFilterWrapper}}.  This would just 
decrypt the KVs before passing them on to the client provided Filter, 
essentially doing the same work your example preFilterXXX methods are doing.
# In {{EncryptingRegionObserver.postScannerNext()}}, again decrypt the final 
KVs being returned to the client, same as your example.

The duplicate decryption here seems unnecessary, but it should give you the 
same results as your provided example, without the need to add a batch of 
pre/postFilterXXX hooks to RegionObservers.

Option B:
# In {{EncryptingRegionObserver.preStoreScannerOpen()}} return a custom 
KeyValueScanner implementation that extends or wraps the default StoreScanner 
implementation.  Note that this would still be a little tricky since filters 
are applied down in ScanQueryMatcher.  For decryption what you would really 
want is to hook in above the StoreFileScanners and MemStoreScanners used 
internally by StoreScanner, but below the ScanQueryMatcher operations, so that 
you can decrypt each KV once as it's read.  Seems like that would currently 
require duplicating a fair amount of StoreScanner functionality.  Maybe 
something needs to be added to better hook in to this data reading layer?

The main issue I see is that the added hooks fuzz the line between Filters and 
RegionObservers and their areas of responsibility.  It doesn't seem like we 
should really need pre/postFilterXXX hooks, because that's what filters are 
supposed to provide.  And of course adding more Observer hooks does have a cost 
in increasing complexity of the coprocessor interfaces and added overhead 
(especially in hot code paths).

Are there really cases that require the pre/postFilter hooks that can't be 
accomplished by having a RegionObserver wrap gets/scans with it's own Filter 
implementation that coordinates with the RegionObserver instance?

> Extend co-processor framework to provide observers for filter operations
> 
>
> Key: HBASE-6805
> URL: https://issues.apache.org/jira/browse/HBASE-6805
> Project: HBase
>  Issue Type: Sub-task
>  Components: Coprocessors
>Affects Versions: 0.96.0
>Reporter: Jason Dai
> Attachments: extend_coprocessor.patch
>
>
> There are several filter operations (e.g., filterKeyValue, filterRow, 
> transform, etc.) at the region server side that either exclude KVs from the 
> returned results, or transform the returned KV. We need to provide observers 
> (e.g., preFilterKeyValue and postFilterKeyValue) for these operations in the 
> same way as the observers for other data access operations (e.g., preGet and 
> postGet). This extension is needed to support DOT (e.g., extracting 
> individual fields from the document in the observers before passing them to 
> the related filter operations) 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5699) Run with > 1 WAL in HRegionServer

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-5699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13468087#comment-13468087
 ] 

Hudson commented on HBASE-5699:
---

Integrated in HBase-TRUNK #3408 (See 
[https://builds.apache.org/job/HBase-TRUNK/3408/])
HBASE-5699 Refactor HLog into an interface (Revision 1393126)

 Result = FAILURE
stack : 
Files : 
* 
/hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/backup/example/LongTermArchivingHFileCleaner.java
* 
/hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/fs/HFileSystem.java
* 
/hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/mapreduce/HLogInputFormat.java
* 
/hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/mapreduce/WALPlayer.java
* 
/hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/master/MasterFileSystem.java
* 
/hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/master/cleaner/CleanerChore.java
* 
/hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/master/cleaner/HFileCleaner.java
* 
/hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/master/cleaner/LogCleaner.java
* 
/hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/protobuf/ProtobufUtil.java
* 
/hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
* 
/hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
* 
/hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/metrics/RegionServerMetrics.java
* 
/hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/Compressor.java
* 
/hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/FSHLog.java
* 
/hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/HLog.java
* 
/hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/HLogFactory.java
* 
/hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/HLogMetrics.java
* 
/hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/HLogPrettyPrinter.java
* 
/hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/HLogSplitter.java
* 
/hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/HLogUtil.java
* 
/hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/SequenceFileLogReader.java
* 
/hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/SequenceFileLogWriter.java
* 
/hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/WALActionsListener.java
* 
/hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/WALCoprocessorHost.java
* 
/hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationSource.java
* 
/hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java
* 
/hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/util/HFileArchiveUtil.java
* 
/hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/util/HMerge.java
* 
/hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/util/MetaUtils.java
* 
/hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/backup/example/TestZooKeeperTableArchiveClient.java
* 
/hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/coprocessor/TestRowProcessorEndpoint.java
* 
/hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/coprocessor/TestWALObserver.java
* 
/hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/fs/TestBlockReorder.java
* 
/hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHLogRecordReader.java
* 
/hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestDistributedLogSplitting.java
* 
/hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestCacheOnWriteInSchema.java
* 
/hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestCompactSelection.java
* 
/hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegion.java
* 
/hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestSplitTransaction.java
* 
/hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestStore.java
* 
/hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/wal/FaultySequenceFileLogReader.java
* 
/hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/wal/HLogPerformanceEvaluation.java
* 
/hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/wal/HLogUtilsForTests.java
* 
/hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/wal/TestHLog.java
* 
/hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/wal/TestHLo

[jira] [Commented] (HBASE-6914) Scans/Gets/Mutations don't give a good error if the table is disabled.

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-6914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13468088#comment-13468088
 ] 

Hudson commented on HBASE-6914:
---

Integrated in HBase-TRUNK #3408 (See 
[https://builds.apache.org/job/HBase-TRUNK/3408/])
HBASE-6914 Scans/Gets/Mutations don't give a good error if the table is 
disabled. (Revision 1393163)

 Result = FAILURE
eclark : 
Files : 
* 
/hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java
* 
/hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestAdmin.java


> Scans/Gets/Mutations don't give a good error if the table is disabled.
> --
>
> Key: HBASE-6914
> URL: https://issues.apache.org/jira/browse/HBASE-6914
> Project: HBase
>  Issue Type: Improvement
>  Components: Client
>Reporter: Elliott Clark
>Assignee: Elliott Clark
> Fix For: 0.92.3, 0.94.3, 0.96.0
>
> Attachments: HBASE-6914-092-3.patch, HBASE-6914-094-3.patch, 
> HBASE-6914-0.patch, HBASE-6914-1.patch, HBASE-6914-2.patch, HBASE-6914-3.patch
>
>
> Scan a table that is disabled will have the client retry multiple times and 
> then will error out with NotServingRegionException.  If the table is disabled 
> there's no need to re-try and the message should be more explicit.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-6926) Cleanup some of the javadoc warnings.

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-6926?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13468089#comment-13468089
 ] 

Hudson commented on HBASE-6926:
---

Integrated in HBase-TRUNK #3408 (See 
[https://builds.apache.org/job/HBase-TRUNK/3408/])
HBASE-6926 Cleanup some of the javadoc warnings (Revision 1393168)

 Result = FAILURE
stack : 
Files : 
* 
/hbase/trunk/hbase-common/src/main/java/org/apache/hadoop/hbase/HConstants.java
* /hbase/trunk/hbase-common/src/main/java/org/apache/hadoop/hbase/KeyValue.java
* 
/hbase/trunk/hbase-common/src/main/java/org/apache/hadoop/hbase/io/hfile/BlockType.java
* 
/hbase/trunk/hbase-hadoop1-compat/src/main/java/org/apache/hadoop/hbase/metrics/BaseMetricsSourceImpl.java
* 
/hbase/trunk/hbase-hadoop2-compat/src/main/java/org/apache/hadoop/metrics2/lib/DynamicMetricsRegistry.java
* /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/ClusterId.java
* 
/hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/HBaseException.java
* 
/hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/HBaseIOException.java
* 
/hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/HColumnDescriptor.java
* 
/hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/HRegionInfo.java
* 
/hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/HServerInfo.java
* 
/hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/HTableDescriptor.java
* 
/hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/MasterAdminProtocol.java
* 
/hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/RegionTransition.java
* 
/hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java
* 
/hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/client/HConnection.java


> Cleanup some of the javadoc warnings.
> -
>
> Key: HBASE-6926
> URL: https://issues.apache.org/jira/browse/HBASE-6926
> Project: HBase
>  Issue Type: Bug
>  Components: documentation
>Reporter: stack
>Assignee: stack
> Fix For: 0.96.0
>
> Attachments: javadocleanup.txt
>
>


--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-6479) HFileReaderV1 caching the same parent META block could cause server abot when splitting

[stack (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-6479?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stack updated HBASE-6479:
-

Resolution: Fixed
Status: Resolved  (was: Patch Available)

Committed to trunk.  Thanks Chunhui (would need more work getting this into 
0.94)

> HFileReaderV1 caching the same parent META block could cause server abot when 
> splitting
> ---
>
> Key: HBASE-6479
> URL: https://issues.apache.org/jira/browse/HBASE-6479
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 0.94.0
>Reporter: chunhui shen
>Assignee: chunhui shen
> Fix For: 0.96.0
>
> Attachments: HBASE-6479.patch, HBASE-6479v2.patch, test.patch
>
>
> If the hfile's version is 1 now, when splitting, two daughters would 
> loadBloomfilter concurrently in the open progress. Because their META block 
> is the same one(parent's META block),  the following expection would be 
> thrown when doing HFileReaderV1#getMetaBlock
> {code}
> java.io.IOException: Failed 
> null-daughterOpener=af73f8c9a9b409531ac211a9a7f92eba
>   at 
> org.apache.hadoop.hbase.regionserver.SplitTransaction.openDaughters(SplitTransaction.java:367)
>   at 
> org.apache.hadoop.hbase.regionserver.SplitTransaction.execute(SplitTransaction.java:453)
>   at 
> org.apache.hadoop.hbase.regionserver.TestSplitTransaction.testWholesomeSplit(TestSplitTransaction.java:225)
>   at 
> org.apache.hadoop.hbase.regionserver.TestSplitTransaction.testWholesomeSplitWithHFileV1(TestSplitTransaction.java:203)
>   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>   at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>   at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>   at java.lang.reflect.Method.invoke(Method.java:597)
>   at 
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45)
>   at 
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
>   at 
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42)
>   at 
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
>   at 
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28)
>   at 
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:30)
>   at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:47)
>   at org.junit.rules.RunRules.evaluate(RunRules.java:18)
>   at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:263)
>   at 
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:68)
>   at 
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:47)
>   at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231)
>   at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:60)
>   at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:229)
>   at org.junit.runners.ParentRunner.access$000(ParentRunner.java:50)
>   at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:222)
>   at org.junit.runners.ParentRunner.run(ParentRunner.java:300)
>   at 
> org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference.run(JUnit4TestReference.java:49)
>   at 
> org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:38)
>   at 
> org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:467)
>   at 
> org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:683)
>   at 
> org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:390)
>   at 
> org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:197)
> Caused by: java.io.IOException: java.io.IOException: 
> java.lang.RuntimeException: Cached an already cached block
>   at 
> org.apache.hadoop.hbase.regionserver.HRegion.initializeRegionInternals(HRegion.java:540)
>   at 
> org.apache.hadoop.hbase.regionserver.HRegion.initialize(HRegion.java:463)
>   at 
> org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:3784)
>   at 
> org.apache.hadoop.hbase.regionserver.SplitTransaction.openDaughterRegion(SplitTransaction.java:506)
>   at 
> org.apache.hadoop.hbase.regionserver.SplitTransaction$DaughterOpener.run(SplitTransaction.java:486)
>   at java.lang.Thread.run(Thread.java:662)
> Caused by: java.io.IOException: java.lang.RuntimeException: Cached an already 
> cached block
>   at 
> org.apache.hadoop.hbase.regionserver.Store.loadStoreFiles(Store.java:424)
>   at org.apache.hadoop.hbase.regionserver.

[jira] [Updated] (HBASE-6919) Remove unnecessary cast from Bytes.readVLong

[Lars Hofhansl (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-6919?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lars Hofhansl updated HBASE-6919:
-

Fix Version/s: 0.96.0
   0.94.3

> Remove unnecessary cast from Bytes.readVLong
> 
>
> Key: HBASE-6919
> URL: https://issues.apache.org/jira/browse/HBASE-6919
> Project: HBase
>  Issue Type: Bug
>Reporter: James Taylor
>Priority: Minor
> Fix For: 0.94.3, 0.96.0
>
>
> Remove the throws IOException so that caller doesn't have to catch and ignore.
>   public static long readVLong(final byte [] buffer, final int offset)
>   throws IOException
> Also, add
>   public static int readVInt(final byte [] buffer, final int offset)
>   throws IOException {
> return (int)readVLong(buffer,offset);
>   }
> and these are useful too:
> /**
>  * Put long as variable length encoded number at the offset in
>  * the result byte array.
>  * @param vint Integer to make a vint of.
>  * @param result buffer to put vint into
>  * @return Vint length in bytes of vint
>  */
> public static int vintToBytes(byte[] result, int offset, final long vint) 
> {
>   long i = vint;
>   if (i >= -112 && i <= 127) {
> result[offset] = (byte) i;
> return 1;
>   }
>   int len = -112;
>   if (i < 0) {
> i ^= -1L; // take one's complement'
> len = -120;
>   }
>   long tmp = i;
>   while (tmp != 0) {
> tmp = tmp >> 8;
> len--;
>   }
>   result[offset++] = (byte) len;
>   len = (len < -120) ? -(len + 120) : -(len + 112);
>   for (int idx = len; idx != 0; idx--) {
> int shiftbits = (idx - 1) * 8;
> long mask = 0xFFL << shiftbits;
> result[offset++] = (byte)((i & mask) >> shiftbits);
>   }
>   return len + 1;
> }
> /**
>  * Decode a vint from the buffer pointed at to by ptr and
>  * increment the offset of the ptr by the length of the
>  * vint.
>  * @param ptr a pointer to a byte array buffer
>  * @return the decoded vint value as an int
>  */
> public static int vintFromBytes(ImmutableBytesWritable ptr) {
> return (int) vlongFromBytes(ptr);
> }
> 
> /**
>  * Decode a vint from the buffer pointed at to by ptr and
>  * increment the offset of the ptr by the length of the
>  * vint.
>  * @param ptr a pointer to a byte array buffer
>  * @return the decoded vint value as a long
>  */
> public static long vlongFromBytes(ImmutableBytesWritable ptr) {
> final byte [] buffer = ptr.get();
> final int offset = ptr.getOffset();
> byte firstByte = buffer[offset];
> int len = WritableUtils.decodeVIntSize(firstByte);
> if (len == 1) {
> ptr.set(buffer, offset+1, ptr.getLength());
> return firstByte;
> }
> long i = 0;
> for (int idx = 0; idx < len-1; idx++) {
> byte b = buffer[offset + 1 + idx];
> i = i << 8;
> i = i | (b & 0xFF);
> }
> ptr.set(buffer, offset+len, ptr.getLength());
> return (WritableUtils.isNegativeVInt(firstByte) ? ~i : i);
> }

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-6479) HFileReaderV1 caching the same parent META block could cause server abot when splitting

[stack (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-6479?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stack updated HBASE-6479:
-

Attachment: 6479v2.txt

Here is what I committed.  Had to massage a little to get it in.  Thanks for 
the patch Chunhui.

> HFileReaderV1 caching the same parent META block could cause server abot when 
> splitting
> ---
>
> Key: HBASE-6479
> URL: https://issues.apache.org/jira/browse/HBASE-6479
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 0.94.0
>Reporter: chunhui shen
>Assignee: chunhui shen
> Fix For: 0.96.0
>
> Attachments: 6479v2.txt, HBASE-6479.patch, HBASE-6479v2.patch, 
> test.patch
>
>
> If the hfile's version is 1 now, when splitting, two daughters would 
> loadBloomfilter concurrently in the open progress. Because their META block 
> is the same one(parent's META block),  the following expection would be 
> thrown when doing HFileReaderV1#getMetaBlock
> {code}
> java.io.IOException: Failed 
> null-daughterOpener=af73f8c9a9b409531ac211a9a7f92eba
>   at 
> org.apache.hadoop.hbase.regionserver.SplitTransaction.openDaughters(SplitTransaction.java:367)
>   at 
> org.apache.hadoop.hbase.regionserver.SplitTransaction.execute(SplitTransaction.java:453)
>   at 
> org.apache.hadoop.hbase.regionserver.TestSplitTransaction.testWholesomeSplit(TestSplitTransaction.java:225)
>   at 
> org.apache.hadoop.hbase.regionserver.TestSplitTransaction.testWholesomeSplitWithHFileV1(TestSplitTransaction.java:203)
>   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>   at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>   at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>   at java.lang.reflect.Method.invoke(Method.java:597)
>   at 
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45)
>   at 
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
>   at 
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42)
>   at 
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
>   at 
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28)
>   at 
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:30)
>   at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:47)
>   at org.junit.rules.RunRules.evaluate(RunRules.java:18)
>   at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:263)
>   at 
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:68)
>   at 
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:47)
>   at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231)
>   at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:60)
>   at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:229)
>   at org.junit.runners.ParentRunner.access$000(ParentRunner.java:50)
>   at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:222)
>   at org.junit.runners.ParentRunner.run(ParentRunner.java:300)
>   at 
> org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference.run(JUnit4TestReference.java:49)
>   at 
> org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:38)
>   at 
> org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:467)
>   at 
> org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:683)
>   at 
> org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:390)
>   at 
> org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:197)
> Caused by: java.io.IOException: java.io.IOException: 
> java.lang.RuntimeException: Cached an already cached block
>   at 
> org.apache.hadoop.hbase.regionserver.HRegion.initializeRegionInternals(HRegion.java:540)
>   at 
> org.apache.hadoop.hbase.regionserver.HRegion.initialize(HRegion.java:463)
>   at 
> org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:3784)
>   at 
> org.apache.hadoop.hbase.regionserver.SplitTransaction.openDaughterRegion(SplitTransaction.java:506)
>   at 
> org.apache.hadoop.hbase.regionserver.SplitTransaction$DaughterOpener.run(SplitTransaction.java:486)
>   at java.lang.Thread.run(Thread.java:662)
> Caused by: java.io.IOException: java.lang.RuntimeException: Cached an already 
> cached block
>   at 
> org.apache.hadoop.hbase.regionserver.Store.loadStoreFiles(Store.java:424)
>   at org.apache.hadoop.hbase.regionserver.Store.(Store.ja

[jira] [Updated] (HBASE-6479) HFileReaderV1 caching the same parent META block could cause server abort when splitting

[Ted Yu (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-6479?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ted Yu updated HBASE-6479:
--

Summary: HFileReaderV1 caching the same parent META block could cause 
server abort when splitting  (was: HFileReaderV1 caching the same parent META 
block could cause server abot when splitting)

> HFileReaderV1 caching the same parent META block could cause server abort 
> when splitting
> 
>
> Key: HBASE-6479
> URL: https://issues.apache.org/jira/browse/HBASE-6479
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 0.94.0
>Reporter: chunhui shen
>Assignee: chunhui shen
> Fix For: 0.96.0
>
> Attachments: 6479v2.txt, HBASE-6479.patch, HBASE-6479v2.patch, 
> test.patch
>
>
> If the hfile's version is 1 now, when splitting, two daughters would 
> loadBloomfilter concurrently in the open progress. Because their META block 
> is the same one(parent's META block),  the following expection would be 
> thrown when doing HFileReaderV1#getMetaBlock
> {code}
> java.io.IOException: Failed 
> null-daughterOpener=af73f8c9a9b409531ac211a9a7f92eba
>   at 
> org.apache.hadoop.hbase.regionserver.SplitTransaction.openDaughters(SplitTransaction.java:367)
>   at 
> org.apache.hadoop.hbase.regionserver.SplitTransaction.execute(SplitTransaction.java:453)
>   at 
> org.apache.hadoop.hbase.regionserver.TestSplitTransaction.testWholesomeSplit(TestSplitTransaction.java:225)
>   at 
> org.apache.hadoop.hbase.regionserver.TestSplitTransaction.testWholesomeSplitWithHFileV1(TestSplitTransaction.java:203)
>   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>   at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>   at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>   at java.lang.reflect.Method.invoke(Method.java:597)
>   at 
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45)
>   at 
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
>   at 
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42)
>   at 
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
>   at 
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28)
>   at 
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:30)
>   at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:47)
>   at org.junit.rules.RunRules.evaluate(RunRules.java:18)
>   at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:263)
>   at 
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:68)
>   at 
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:47)
>   at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231)
>   at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:60)
>   at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:229)
>   at org.junit.runners.ParentRunner.access$000(ParentRunner.java:50)
>   at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:222)
>   at org.junit.runners.ParentRunner.run(ParentRunner.java:300)
>   at 
> org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference.run(JUnit4TestReference.java:49)
>   at 
> org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:38)
>   at 
> org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:467)
>   at 
> org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:683)
>   at 
> org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:390)
>   at 
> org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:197)
> Caused by: java.io.IOException: java.io.IOException: 
> java.lang.RuntimeException: Cached an already cached block
>   at 
> org.apache.hadoop.hbase.regionserver.HRegion.initializeRegionInternals(HRegion.java:540)
>   at 
> org.apache.hadoop.hbase.regionserver.HRegion.initialize(HRegion.java:463)
>   at 
> org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:3784)
>   at 
> org.apache.hadoop.hbase.regionserver.SplitTransaction.openDaughterRegion(SplitTransaction.java:506)
>   at 
> org.apache.hadoop.hbase.regionserver.SplitTransaction$DaughterOpener.run(SplitTransaction.java:486)
>   at java.lang.Thread.run(Thread.java:662)
> Caused by: java.io.IOException: java.lang.RuntimeException: Cached an already 
> cached block
>   at 
> org.apache.hadoop.hbase.regionserver.Store.loadStoreFiles(Store.java:4

[jira] [Created] (HBASE-6927) WrongFS using HRegionInfo.getTableDesc() and different fs for hbase.root and fs.defaultFS

[Matteo Bertozzi (JIRA)]

Matteo Bertozzi created HBASE-6927:
--

 Summary: WrongFS using HRegionInfo.getTableDesc() and different fs 
for hbase.root and fs.defaultFS
 Key: HBASE-6927
 URL: https://issues.apache.org/jira/browse/HBASE-6927
 Project: HBase
  Issue Type: Bug
Affects Versions: 0.94.1, 0.92.2, 0.96.0
Reporter: Matteo Bertozzi
Assignee: Matteo Bertozzi


Calling HRegionInfo.getTableDesc() with different fs schema for hbase.root and 
fs.defaultFS raises "IllegalArgumentException: Wrong FS" exception.

HRegionInfo.getTableDesc() is called only by bin/region_mover.rb to get the 
table name and can be easily replaced, getTableDesc() is also deprecated.

The main problem is that getTableDesc() doesn't replace fs.defaultFS with 
hbase.root as all the other hbase code (all the code does this, except 
getTableDesc)
{code}
Configuration c = HBaseConfiguration.create();
c.set("fs.defaultFS", c.get(HConstants.HBASE_DIR));
c.set("fs.default.name", c.get(HConstants.HBASE_DIR));
{code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-6927) WrongFS using HRegionInfo.getTableDesc() and different fs for hbase.root and fs.defaultFS

[Matteo Bertozzi (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-6927?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Matteo Bertozzi updated HBASE-6927:
---

Attachment: HBASE-6927-v0.patch

> WrongFS using HRegionInfo.getTableDesc() and different fs for hbase.root and 
> fs.defaultFS
> -
>
> Key: HBASE-6927
> URL: https://issues.apache.org/jira/browse/HBASE-6927
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 0.92.2, 0.94.1, 0.96.0
>Reporter: Matteo Bertozzi
>Assignee: Matteo Bertozzi
> Attachments: HBASE-6927-v0.patch
>
>
> Calling HRegionInfo.getTableDesc() with different fs schema for hbase.root 
> and fs.defaultFS raises "IllegalArgumentException: Wrong FS" exception.
> HRegionInfo.getTableDesc() is called only by bin/region_mover.rb to get the 
> table name and can be easily replaced, getTableDesc() is also deprecated.
> The main problem is that getTableDesc() doesn't replace fs.defaultFS with 
> hbase.root as all the other hbase code (all the code does this, except 
> getTableDesc)
> {code}
> Configuration c = HBaseConfiguration.create();
> c.set("fs.defaultFS", c.get(HConstants.HBASE_DIR));
> c.set("fs.default.name", c.get(HConstants.HBASE_DIR));
> {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-6927) WrongFS using HRegionInfo.getTableDesc() and different fs for hbase.root and fs.defaultFS

[Matteo Bertozzi (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-6927?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Matteo Bertozzi updated HBASE-6927:
---

Status: Patch Available  (was: Open)

> WrongFS using HRegionInfo.getTableDesc() and different fs for hbase.root and 
> fs.defaultFS
> -
>
> Key: HBASE-6927
> URL: https://issues.apache.org/jira/browse/HBASE-6927
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 0.94.1, 0.92.2, 0.96.0
>Reporter: Matteo Bertozzi
>Assignee: Matteo Bertozzi
> Attachments: HBASE-6927-v0.patch
>
>
> Calling HRegionInfo.getTableDesc() with different fs schema for hbase.root 
> and fs.defaultFS raises "IllegalArgumentException: Wrong FS" exception.
> HRegionInfo.getTableDesc() is called only by bin/region_mover.rb to get the 
> table name and can be easily replaced, getTableDesc() is also deprecated.
> The main problem is that getTableDesc() doesn't replace fs.defaultFS with 
> hbase.root as all the other hbase code (all the code does this, except 
> getTableDesc)
> {code}
> Configuration c = HBaseConfiguration.create();
> c.set("fs.defaultFS", c.get(HConstants.HBASE_DIR));
> c.set("fs.default.name", c.get(HConstants.HBASE_DIR));
> {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Assigned] (HBASE-6770) Allow scanner setCaching to specify size instead of number of rows

[Karthik Ranganathan (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-6770?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Karthik Ranganathan reassigned HBASE-6770:
--

Assignee: Chen Jin  (was: Michal Gregorczyk)

> Allow scanner setCaching to specify size instead of number of rows
> --
>
> Key: HBASE-6770
> URL: https://issues.apache.org/jira/browse/HBASE-6770
> Project: HBase
>  Issue Type: Sub-task
>  Components: Client, regionserver
>Reporter: Karthik Ranganathan
>Assignee: Chen Jin
>
> Currently, we have the following api's to customize the behavior of scans:
> setCaching() - how many rows to cache on client to speed up scans
> setBatch() - max columns per row to return per row to prevent a very large 
> response.
> Ideally, we should be able to specify a memory buffer size because:
> 1. that would take care of both of these use cases.
> 2. it does not need any knowledge of the size of the rows or cells, as the 
> final thing we are worried about is the available memory.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-6914) Scans/Gets/Mutations don't give a good error if the table is disabled.

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-6914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13468113#comment-13468113
 ] 

Hudson commented on HBASE-6914:
---

Integrated in HBase-0.92 #592 (See 
[https://builds.apache.org/job/HBase-0.92/592/])
HBASE-6914 Scans/Gets/Mutations don't give a good error if the table is 
disabled. (Revision 1393166)

 Result = FAILURE
eclark : 
Files : 
* /hbase/branches/0.92/CHANGES.txt
* 
/hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java
* 
/hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/client/TestAdmin.java


> Scans/Gets/Mutations don't give a good error if the table is disabled.
> --
>
> Key: HBASE-6914
> URL: https://issues.apache.org/jira/browse/HBASE-6914
> Project: HBase
>  Issue Type: Improvement
>  Components: Client
>Reporter: Elliott Clark
>Assignee: Elliott Clark
> Fix For: 0.92.3, 0.94.3, 0.96.0
>
> Attachments: HBASE-6914-092-3.patch, HBASE-6914-094-3.patch, 
> HBASE-6914-0.patch, HBASE-6914-1.patch, HBASE-6914-2.patch, HBASE-6914-3.patch
>
>
> Scan a table that is disabled will have the client retry multiple times and 
> then will error out with NotServingRegionException.  If the table is disabled 
> there's no need to re-try and the message should be more explicit.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-6914) Scans/Gets/Mutations don't give a good error if the table is disabled.

[Ted Yu (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-6914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13468115#comment-13468115
 ] 

Ted Yu commented on HBASE-6914:
---

0.92 build #592 seems to indicate that 0.92 build is broken.

> Scans/Gets/Mutations don't give a good error if the table is disabled.
> --
>
> Key: HBASE-6914
> URL: https://issues.apache.org/jira/browse/HBASE-6914
> Project: HBase
>  Issue Type: Improvement
>  Components: Client
>Reporter: Elliott Clark
>Assignee: Elliott Clark
> Fix For: 0.92.3, 0.94.3, 0.96.0
>
> Attachments: HBASE-6914-092-3.patch, HBASE-6914-094-3.patch, 
> HBASE-6914-0.patch, HBASE-6914-1.patch, HBASE-6914-2.patch, HBASE-6914-3.patch
>
>
> Scan a table that is disabled will have the client retry multiple times and 
> then will error out with NotServingRegionException.  If the table is disabled 
> there's no need to re-try and the message should be more explicit.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira