date:20140919


[ 
https://issues.apache.org/jira/browse/HBASE-12025?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14140067#comment-14140067
 ] 

Hudson commented on HBASE-12025:


FAILURE: Integrated in HBase-TRUNK #5530 (See 
[https://builds.apache.org/job/HBase-TRUNK/5530/])
HBASE-12025 TestHttpServerLifecycle.testStartedServerWithRequestLog hangs 
frequently (stack: rev 64a69f0ffca7d588b3835680db0286660c741d1f)
* 
hbase-server/src/test/java/org/apache/hadoop/hbase/http/TestHttpServerLifecycle.java


 TestHttpServerLifecycle.testStartedServerWithRequestLog hangs frequently
 

 Key: HBASE-12025
 URL: https://issues.apache.org/jira/browse/HBASE-12025
 Project: HBase
  Issue Type: Bug
  Components: test
Reporter: stack
Assignee: stack
 Fix For: 2.0.0, 0.99.1

 Attachments: 12025.txt


 Happened over on 
 https://builds.apache.org/job/PreCommit-HBASE-Build/10974/consoleText most 
 recently.  The test is to see if can enable request logging in jetty (we 
 should remove it?).  Looks like some bug in our ancient jetty version.  Let 
 me for now change the stop order so the request listener is removed before we 
 go to shut down.  Here is the bind:
 {code}
 1738904334@qtp-1379460953-1 - Acceptor0 
 SelectChannelConnector@localhost:56958 daemon prio=10 tid=0x7f493884b800 
 nid=0x14b9 waiting for monitor entry [0x7f490a59e000]
java.lang.Thread.State: BLOCKED (on object monitor)
   at 
 org.mortbay.io.nio.SelectorManager$SelectSet.doSelect(SelectorManager.java:464)
   - waiting to lock 0x0007dd6a1028 (a 
 org.mortbay.io.nio.SelectorManager$SelectSet)
   at org.mortbay.io.nio.SelectorManager.doSelect(SelectorManager.java:192)
   at 
 org.mortbay.jetty.nio.SelectChannelConnector.accept(SelectChannelConnector.java:124)
   at 
 org.mortbay.jetty.AbstractConnector$Acceptor.run(AbstractConnector.java:708)
   at 
 org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)
 825895443@qtp-1379460953-0 daemon prio=10 tid=0x7f493885a000 nid=0x14b8 
 in Object.wait() [0x7f490a49d000]
java.lang.Thread.State: TIMED_WAITING (on object monitor)
   at java.lang.Object.wait(Native Method)
   - waiting on 0x0007dcd02220 (a 
 org.mortbay.thread.QueuedThreadPool$PoolThread)
   at 
 org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:626)
   - locked 0x0007dcd02220 (a 
 org.mortbay.thread.QueuedThreadPool$PoolThread)
 Service Thread daemon prio=10 tid=0x7f493829 nid=0x14ab runnable 
 [0x]
java.lang.Thread.State: RUNNABLE
 C2 CompilerThread1 daemon prio=10 tid=0x7f493828e000 nid=0x14aa waiting 
 on condition [0x]
java.lang.Thread.State: RUNNABLE
 C2 CompilerThread0 daemon prio=10 tid=0x7f493828b000 nid=0x14a9 waiting 
 on condition [0x]
java.lang.Thread.State: RUNNABLE
 Signal Dispatcher daemon prio=10 tid=0x7f4938280800 nid=0x14a8 runnable 
 [0x]
java.lang.Thread.State: RUNNABLE
 Finalizer daemon prio=10 tid=0x7f493826b000 nid=0x14a7 in Object.wait() 
 [0x7f490b3f2000]
java.lang.Thread.State: WAITING (on object monitor)
   at java.lang.Object.wait(Native Method)
   - waiting on 0x0007d8685568 (a java.lang.ref.ReferenceQueue$Lock)
   at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:135)
   - locked 0x0007d8685568 (a java.lang.ref.ReferenceQueue$Lock)
   at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:151)
   at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:189)
 Reference Handler daemon prio=10 tid=0x7f4938267000 nid=0x14a6 in 
 Object.wait() [0x7f490b4f3000]
java.lang.Thread.State: WAITING (on object monitor)
   at java.lang.Object.wait(Native Method)
   - waiting on 0x0007d86850f0 (a java.lang.ref.Reference$Lock)
   at java.lang.Object.wait(Object.java:503)
   at java.lang.ref.Reference$ReferenceHandler.run(Reference.java:133)
   - locked 0x0007d86850f0 (a java.lang.ref.Reference$Lock)
 main prio=10 tid=0x7f493800a800 nid=0x1497 runnable [0x7f4941b5f000]
java.lang.Thread.State: RUNNABLE
   at 
 org.mortbay.io.nio.SelectorManager$SelectSet.stop(SelectorManager.java:879)
   - locked 0x0007dd6a1028 (a 
 org.mortbay.io.nio.SelectorManager$SelectSet)
   at org.mortbay.io.nio.SelectorManager.doStop(SelectorManager.java:240)
   at 
 org.mortbay.component.AbstractLifeCycle.stop(AbstractLifeCycle.java:76)
   - locked 0x0007dccf2e60 (a java.lang.Object)
   at 
 org.mortbay.jetty.nio.SelectChannelConnector.close(SelectChannelConnector.java:136)
   - locked 0x0007dccf2cc0 (a 
 org.mortbay.jetty.nio.SelectChannelConnector)
   at

[jira] [Updated] (HBASE-12021) Hbase shell does not respect the HBASE_OPTS set by the user in console

2014-09-19 Thread Ashish Singhi (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-12021?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ashish Singhi updated HBASE-12021:
--
Affects Version/s: (was: 0.98.5)
   2.0.0
   Status: Patch Available  (was: Open)

 Hbase shell does not respect the HBASE_OPTS set by the user in console
 --

 Key: HBASE-12021
 URL: https://issues.apache.org/jira/browse/HBASE-12021
 Project: HBase
  Issue Type: Improvement
  Components: shell
Affects Versions: 2.0.0
Reporter: nijel
Assignee: Ashish Singhi
Priority: Minor
 Attachments: HBASE-12021.patch


 Before starting hbase shell i have set some -D parameters reauired for 
 kerberos authentication in HBASE_OPTS (there is no other variables are used 
 when shell is executed). But while invoking Hbase shell it is not used.
 The code in hbase-env.sh is 
 export HBASE_OPTS=-XX:+UseConcMarkSweepGC
 This will overwrite the the value set in the console
 There can be 2 options to support this
 1. Append the HBASE_OPTS like HBASE_OPTS=$HBASE_OPTS 
 -XX:+UseConcMarkSweepGC.
 2. Have a new variable (HBASE_CLIENT_OPTS) which is can be appended in shell 
 section of hbase file



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-11879) Change TableInputFormatBase to take interface arguments


[ 
https://issues.apache.org/jira/browse/HBASE-11879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14140069#comment-14140069
 ] 

stack commented on HBASE-11879:
---

Took a quick look. I think it looking good.  Looks like old MR jobs will 
continue to work.

nits: Fix spelling in + *  
ConnectionFactory.createConnectin(HBaseConfiguration.create(job));
nits: When you @Deprecated a method, you can add a @deprecated to javadoc too 
and qualify it with what you would have the user call instead.  You actually 
have javadoc already doing '* Use setTable'.  Instead making '* @deprecated Use 
setTable instead'.  Minor.

Is this right (in mapred package -- seems right in mapreduce package)?

-  public void setHTable(Table htable) {
-Configuration conf = htable.getConfiguration();
+  public void setHTable(Table table) {

Should there be a deprecation of setHTable going on here?

What was wrong w/ your 2a suggestion?  That looked nice.  Let this class 
internally go get regionlocators?

Otherwise patch is looking good.

bq. Does it make sense to migrate getRegionLocations() and its dependencies to 
use the new Table/RegionLocator/Connection/Admin interfaces?

That would make sense.  In another JIRA?

Good stuff.


 Change TableInputFormatBase to take interface arguments
 ---

 Key: HBASE-11879
 URL: https://issues.apache.org/jira/browse/HBASE-11879
 Project: HBase
  Issue Type: Improvement
Reporter: Carter
Assignee: Solomon Duskis
 Fix For: 0.99.1

 Attachments: HBASE_11879.patch


 As part of the ongoing interface abstraction work, I'm now investigating 
 {{TableInputFormatBase}}, which has two methods that break encapsulation:
 {code}
 protected HTable getHTable();
 protected void setHTable(HTable table);
 {code}
 While these are protected methods, the base @InterfaceAudience.Public is 
 abstract, meaning that it supports extension by user code.
 I propose deprecating these two methods and replacing them with these four, 
 once the Table interface is merged:
 {code}
 protected Table getTable();
 protected void setTable(Table table);
 protected RegionLocator getRegionLocator();
 protected void setRegionLocator(RegionLocator regionLocator);
 {code}
 Since users will frequently call {{setTable}} and {{setRegionLocator}} 
 together, it probably also makes sense to add the following convenience 
 method:
 {code}
 protected void setTableAndRegionLocator(Table table, RegionLocator 
 regionLocator);
 {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-11462) MetaTableAccessor shouldn't use ZooKeeeper

[
https://issues.apache.org/jira/browse/HBASE-11462?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14140076#comment-14140076
]

Hadoop QA commented on HBASE-11462:
---

{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12669907/HBASE-11462.v4.patch
against trunk revision .
ATTACHMENT ID: 12669907

{color:green}+1 @author{color}. The patch does not contain any @author
tags.

{color:green}+1 tests included{color}. The patch appears to include 18 new
or modified tests.

{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.

{color:red}-1 javadoc{color}. The javadoc tool appears to have generated 2
warning messages.

{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.

{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.

{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100

{color:green}+1 site{color}. The mvn site goal succeeds with this patch.

{color:red}-1 core tests{color}. The patch failed these unit tests:

org.apache.hadoop.hbase.security.visibility.TestVisibilityLabelsWithDeletes

Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/10985//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10985//artifact/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10985//artifact/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10985//artifact/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10985//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10985//artifact/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10985//artifact/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10985//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10985//artifact/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10985//artifact/patchprocess/newPatchFindbugsWarningshbase-common.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/10985//console

This message is automatically generated.

MetaTableAccessor shouldn't use ZooKeeeper
--

Key: HBASE-11462
URL: https://issues.apache.org/jira/browse/HBASE-11462
Project: HBase
Issue Type: Improvement
Components: Client, Zookeeper
Affects Versions: 2.0.0
Reporter: Mikhail Antonov
Assignee: Mikhail Antonov
Fix For: 2.0.0

Attachments: HBASE-11462.v4.patch, HBASE-11462.v4.patch

After committing patch for HBASE-4495, there's an further improvement which
can be made (discussed originally on review board to that jira).
We have MetaTableAccessor and MetaTableLocator classes. First one is used to
access information stored in hbase:meta table. Second one is used to deal
with ZooKeeper state to find out region server hosting hbase:meta, wait for
it to become available and so on.
MetaTableAccessor, in turn, should only operate on the meta table content, so
shouldn't need ZK. The only reason why MetaTableAccessor is using ZK - when
callers request assignment information, they can request location of meta
table itself, which we can't read from meta, so in that case
MetaTableAccessor relays the call to MetaTableLocator. May be the solution
here is to declare that clients of MetaTableAccessor shall not use it to work
with meta table itself (not it's content).

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-10757) Change HTable class doc so it sends people to HCM getting instances


 [ 
https://issues.apache.org/jira/browse/HBASE-10757?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stack updated HBASE-10757:
--
Fix Version/s: 2.0.0
 Assignee: stack
   Status: Patch Available  (was: Open)

 Change HTable class doc so it sends people to HCM getting instances
 ---

 Key: HBASE-10757
 URL: https://issues.apache.org/jira/browse/HBASE-10757
 Project: HBase
  Issue Type: Bug
Reporter: stack
Assignee: stack
Priority: Critical
 Fix For: 2.0.0, 0.99.1

 Attachments: 10757.txt


 The HTable class doc does not reflect the new regime where you are meant to 
 go via cluster connection to get an instance.
 Getting an HTable otherwise also makes for some strange issues:  See 
 Occasional GSSException that brings down region server on mailing list.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-10757) Change HTable class doc so it sends people to HCM getting instances


 [ 
https://issues.apache.org/jira/browse/HBASE-10757?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stack updated HBASE-10757:
--
Attachment: 10757.txt

Purges stale javadoc and points folks instead to new way as described over in 
ConnectionFactory.  Is loud about not creating HTable instances directly.  
Removes all that 'helpful' stuff about connections being shared.

I think this patch is enough to get this blocker closed.  Nicks comment no long 
applies after the extensive Connection and Table refactor.

 Change HTable class doc so it sends people to HCM getting instances
 ---

 Key: HBASE-10757
 URL: https://issues.apache.org/jira/browse/HBASE-10757
 Project: HBase
  Issue Type: Bug
Reporter: stack
Priority: Critical
 Fix For: 2.0.0, 0.99.1

 Attachments: 10757.txt


 The HTable class doc does not reflect the new regime where you are meant to 
 go via cluster connection to get an instance.
 Getting an HTable otherwise also makes for some strange issues:  See 
 Occasional GSSException that brings down region server on mailing list.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-9117) Remove HTablePool and all HConnection pooling related APIs


[ 
https://issues.apache.org/jira/browse/HBASE-9117?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14140097#comment-14140097
 ] 

stack commented on HBASE-9117:
--

How to proceed with this issue [~ndimiduk] given the lay of the land has 
changed under us w/ all the Table/Connection/Admin changes?  Should I go 
through the patch as is and see what can be salvaged?

 Remove HTablePool and all HConnection pooling related APIs
 --

 Key: HBASE-9117
 URL: https://issues.apache.org/jira/browse/HBASE-9117
 Project: HBase
  Issue Type: Bug
Reporter: Lars Hofhansl
Assignee: Nick Dimiduk
Priority: Critical
 Fix For: 0.99.1

 Attachments: HBASE-9117.00.patch, HBASE-9117.01.patch, 
 HBASE-9117.02.patch, HBASE-9117.03.patch, HBASE-9117.04.patch, 
 HBASE-9117.05.patch, HBASE-9117.06.patch


 The recommended way is now:
 # Create an HConnection: HConnectionManager.createConnection(...)
 # Create a light HTable: HConnection.getTable(...)
 # table.close()
 # connection.close()
 All other API and pooling will be removed.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-12025) TestHttpServerLifecycle.testStartedServerWithRequestLog hangs frequently


[ 
https://issues.apache.org/jira/browse/HBASE-12025?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14140102#comment-14140102
 ] 

Hudson commented on HBASE-12025:


FAILURE: Integrated in HBase-1.0 #204 (See 
[https://builds.apache.org/job/HBase-1.0/204/])
HBASE-12025 TestHttpServerLifecycle.testStartedServerWithRequestLog hangs 
frequently (stack: rev 60287ed9833a26f3e40db88f95b496a1f90e71cb)
* 
hbase-server/src/test/java/org/apache/hadoop/hbase/http/TestHttpServerLifecycle.java


 TestHttpServerLifecycle.testStartedServerWithRequestLog hangs frequently
 

 Key: HBASE-12025
 URL: https://issues.apache.org/jira/browse/HBASE-12025
 Project: HBase
  Issue Type: Bug
  Components: test
Reporter: stack
Assignee: stack
 Fix For: 2.0.0, 0.99.1

 Attachments: 12025.txt


 Happened over on 
 https://builds.apache.org/job/PreCommit-HBASE-Build/10974/consoleText most 
 recently.  The test is to see if can enable request logging in jetty (we 
 should remove it?).  Looks like some bug in our ancient jetty version.  Let 
 me for now change the stop order so the request listener is removed before we 
 go to shut down.  Here is the bind:
 {code}
 1738904334@qtp-1379460953-1 - Acceptor0 
 SelectChannelConnector@localhost:56958 daemon prio=10 tid=0x7f493884b800 
 nid=0x14b9 waiting for monitor entry [0x7f490a59e000]
java.lang.Thread.State: BLOCKED (on object monitor)
   at 
 org.mortbay.io.nio.SelectorManager$SelectSet.doSelect(SelectorManager.java:464)
   - waiting to lock 0x0007dd6a1028 (a 
 org.mortbay.io.nio.SelectorManager$SelectSet)
   at org.mortbay.io.nio.SelectorManager.doSelect(SelectorManager.java:192)
   at 
 org.mortbay.jetty.nio.SelectChannelConnector.accept(SelectChannelConnector.java:124)
   at 
 org.mortbay.jetty.AbstractConnector$Acceptor.run(AbstractConnector.java:708)
   at 
 org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)
 825895443@qtp-1379460953-0 daemon prio=10 tid=0x7f493885a000 nid=0x14b8 
 in Object.wait() [0x7f490a49d000]
java.lang.Thread.State: TIMED_WAITING (on object monitor)
   at java.lang.Object.wait(Native Method)
   - waiting on 0x0007dcd02220 (a 
 org.mortbay.thread.QueuedThreadPool$PoolThread)
   at 
 org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:626)
   - locked 0x0007dcd02220 (a 
 org.mortbay.thread.QueuedThreadPool$PoolThread)
 Service Thread daemon prio=10 tid=0x7f493829 nid=0x14ab runnable 
 [0x]
java.lang.Thread.State: RUNNABLE
 C2 CompilerThread1 daemon prio=10 tid=0x7f493828e000 nid=0x14aa waiting 
 on condition [0x]
java.lang.Thread.State: RUNNABLE
 C2 CompilerThread0 daemon prio=10 tid=0x7f493828b000 nid=0x14a9 waiting 
 on condition [0x]
java.lang.Thread.State: RUNNABLE
 Signal Dispatcher daemon prio=10 tid=0x7f4938280800 nid=0x14a8 runnable 
 [0x]
java.lang.Thread.State: RUNNABLE
 Finalizer daemon prio=10 tid=0x7f493826b000 nid=0x14a7 in Object.wait() 
 [0x7f490b3f2000]
java.lang.Thread.State: WAITING (on object monitor)
   at java.lang.Object.wait(Native Method)
   - waiting on 0x0007d8685568 (a java.lang.ref.ReferenceQueue$Lock)
   at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:135)
   - locked 0x0007d8685568 (a java.lang.ref.ReferenceQueue$Lock)
   at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:151)
   at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:189)
 Reference Handler daemon prio=10 tid=0x7f4938267000 nid=0x14a6 in 
 Object.wait() [0x7f490b4f3000]
java.lang.Thread.State: WAITING (on object monitor)
   at java.lang.Object.wait(Native Method)
   - waiting on 0x0007d86850f0 (a java.lang.ref.Reference$Lock)
   at java.lang.Object.wait(Object.java:503)
   at java.lang.ref.Reference$ReferenceHandler.run(Reference.java:133)
   - locked 0x0007d86850f0 (a java.lang.ref.Reference$Lock)
 main prio=10 tid=0x7f493800a800 nid=0x1497 runnable [0x7f4941b5f000]
java.lang.Thread.State: RUNNABLE
   at 
 org.mortbay.io.nio.SelectorManager$SelectSet.stop(SelectorManager.java:879)
   - locked 0x0007dd6a1028 (a 
 org.mortbay.io.nio.SelectorManager$SelectSet)
   at org.mortbay.io.nio.SelectorManager.doStop(SelectorManager.java:240)
   at 
 org.mortbay.component.AbstractLifeCycle.stop(AbstractLifeCycle.java:76)
   - locked 0x0007dccf2e60 (a java.lang.Object)
   at 
 org.mortbay.jetty.nio.SelectChannelConnector.close(SelectChannelConnector.java:136)
   - locked 0x0007dccf2cc0 (a 
 org.mortbay.jetty.nio.SelectChannelConnector)
   at

[jira] [Updated] (HBASE-11462) MetaTableAccessor shouldn't use ZooKeeeper

[
https://issues.apache.org/jira/browse/HBASE-11462?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

stack updated HBASE-11462:
--
Attachment: HBASE-11462.v4.patch

Visibility labels failure strange missing zk node. Is it related [~mantonov]?
Meantime, retrying to see if we can get further.

MetaTableAccessor shouldn't use ZooKeeeper
--

Attachments: HBASE-11462.v4.patch, HBASE-11462.v4.patch,
HBASE-11462.v4.patch

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-5622) Improve efficiency of mapred vesion of RowCounter


[ 
https://issues.apache.org/jira/browse/HBASE-5622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14140112#comment-14140112
 ] 

Hadoop QA commented on HBASE-5622:
--

{color:red}-1 overall{color}.  Here are the results of testing the latest 
attachment 
  http://issues.apache.org/jira/secure/attachment/12669920/HBASE-5622.patch
  against trunk revision .
  ATTACHMENT ID: 12669920

{color:green}+1 @author{color}.  The patch does not contain any @author 
tags.

{color:red}-1 tests included{color}.  The patch doesn't appear to include 
any new or modified tests.
Please justify why no new tests are needed for this 
patch.
Also please list what manual steps were performed to 
verify this patch.

{color:green}+1 javac{color}.  The applied patch does not increase the 
total number of javac compiler warnings.

{color:green}+1 javac{color}.  The applied patch does not increase the 
total number of javac compiler warnings.

{color:red}-1 javadoc{color}.  The javadoc tool appears to have generated 2 
warning messages.

{color:green}+1 findbugs{color}.  The patch does not introduce any new 
Findbugs (version 2.0.3) warnings.

{color:green}+1 release audit{color}.  The applied patch does not increase 
the total number of release audit warnings.

{color:green}+1 lineLengths{color}.  The patch does not introduce lines 
longer than 100

  {color:green}+1 site{color}.  The mvn site goal succeeds with this patch.

{color:green}+1 core tests{color}.  The patch passed unit tests in .

Test results: 
https://builds.apache.org/job/PreCommit-HBASE-Build/10987//testReport/
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/10987//artifact/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/10987//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/10987//artifact/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/10987//artifact/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/10987//artifact/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/10987//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/10987//artifact/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/10987//artifact/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/10987//artifact/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Console output: 
https://builds.apache.org/job/PreCommit-HBASE-Build/10987//console

This message is automatically generated.

 Improve efficiency of mapred vesion of RowCounter
 -

 Key: HBASE-5622
 URL: https://issues.apache.org/jira/browse/HBASE-5622
 Project: HBase
  Issue Type: Sub-task
Reporter: Lars Hofhansl
Assignee: Ashish Singhi
Priority: Minor
 Attachments: HBASE-5622.patch






--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-12021) Hbase shell does not respect the HBASE_OPTS set by the user in console


[ 
https://issues.apache.org/jira/browse/HBASE-12021?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14140113#comment-14140113
 ] 

stack commented on HBASE-12021:
---

lgtm

[~nijel]?

 Hbase shell does not respect the HBASE_OPTS set by the user in console
 --

 Key: HBASE-12021
 URL: https://issues.apache.org/jira/browse/HBASE-12021
 Project: HBase
  Issue Type: Improvement
  Components: shell
Affects Versions: 2.0.0
Reporter: nijel
Assignee: Ashish Singhi
Priority: Minor
 Attachments: HBASE-12021.patch


 Before starting hbase shell i have set some -D parameters reauired for 
 kerberos authentication in HBASE_OPTS (there is no other variables are used 
 when shell is executed). But while invoking Hbase shell it is not used.
 The code in hbase-env.sh is 
 export HBASE_OPTS=-XX:+UseConcMarkSweepGC
 This will overwrite the the value set in the console
 There can be 2 options to support this
 1. Append the HBASE_OPTS like HBASE_OPTS=$HBASE_OPTS 
 -XX:+UseConcMarkSweepGC.
 2. Have a new variable (HBASE_CLIENT_OPTS) which is can be appended in shell 
 section of hbase file



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-11988) AC/VC system table create on postStartMaster fails too often in test


 [ 
https://issues.apache.org/jira/browse/HBASE-11988?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stack updated HBASE-11988:
--
Fix Version/s: 0.99.1
   0.98.7
   2.0.0

 AC/VC system table create on postStartMaster fails too often in test
 

 Key: HBASE-11988
 URL: https://issues.apache.org/jira/browse/HBASE-11988
 Project: HBase
  Issue Type: Bug
Reporter: Anoop Sam John
Assignee: Anoop Sam John
Priority: Critical
 Fix For: 2.0.0, 0.98.7, 0.99.1

 Attachments: 11988.debug.txt, HBaseTestingUtility.patch


 See for example
 {noformat}
 2014-09-16 04:02:08,833 ERROR [ActiveMasterManager] master.HMaster(633): 
 Coprocessor postStartMaster() hook failed
 java.io.IOException: Table Namespace Manager not ready yet, try again later
   at 
 org.apache.hadoop.hbase.master.HMaster.checkNamespaceManagerReady(HMaster.java:1669)
   at 
 org.apache.hadoop.hbase.master.HMaster.getNamespaceDescriptor(HMaster.java:1852)
   at org.apache.hadoop.hbase.master.HMaster.createTable(HMaster.java:1096)
   at 
 org.apache.hadoop.hbase.security.access.AccessControlLists.init(AccessControlLists.java:143)
   at 
 org.apache.hadoop.hbase.security.access.AccessController.postStartMaster(AccessController.java:1059)
   at 
 org.apache.hadoop.hbase.master.MasterCoprocessorHost$58.call(MasterCoprocessorHost.java:692)
   at 
 org.apache.hadoop.hbase.master.MasterCoprocessorHost.execOperation(MasterCoprocessorHost.java:861)
   at 
 org.apache.hadoop.hbase.master.MasterCoprocessorHost.postStartMaster(MasterCoprocessorHost.java:688)
   at 
 org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:631)
   at org.apache.hadoop.hbase.master.HMaster.access$500(HMaster.java:155)
   at org.apache.hadoop.hbase.master.HMaster$1.run(HMaster.java:1244)
   at java.lang.Thread.run(Thread.java:744)
 {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Assigned] (HBASE-5580) Publish Thrift-generated files for other languages

2014-09-19 Thread Dima Spivak (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-5580?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Dima Spivak reassigned HBASE-5580:
--

Assignee: Dima Spivak

 Publish Thrift-generated files for other languages
 --

 Key: HBASE-5580
 URL: https://issues.apache.org/jira/browse/HBASE-5580
 Project: HBase
  Issue Type: New Feature
  Components: Thrift
Affects Versions: 0.90.4
Reporter: Patrick Angeles
Assignee: Dima Spivak
  Labels: thrift, thrift2

 HBase ships with Thrift-generated Java files for use with the ThriftServer. 
 For convenience (and to save users the frustration of having to compile and 
 install the Thrift compiler), HBase can ship with the thrift-generated files 
 for other languages as well.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-11462) MetaTableAccessor shouldn't use ZooKeeeper


[ 
https://issues.apache.org/jira/browse/HBASE-11462?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14140120#comment-14140120
 ] 

Mikhail Antonov commented on HBASE-11462:
-

No, I don't think it's related, this test passed for me locally.

 MetaTableAccessor shouldn't use ZooKeeeper
 --

 Key: HBASE-11462
 URL: https://issues.apache.org/jira/browse/HBASE-11462
 Project: HBase
  Issue Type: Improvement
  Components: Client, Zookeeper
Affects Versions: 2.0.0
Reporter: Mikhail Antonov
Assignee: Mikhail Antonov
 Fix For: 2.0.0

 Attachments: HBASE-11462.v4.patch, HBASE-11462.v4.patch, 
 HBASE-11462.v4.patch


 After committing patch for HBASE-4495, there's an further improvement which 
 can be made (discussed originally on review board to that jira).
 We have MetaTableAccessor and MetaTableLocator classes. First one is used to 
 access information stored in hbase:meta table. Second one is used to deal 
 with ZooKeeper state to find out region server hosting hbase:meta, wait for 
 it to become available and so on.
 MetaTableAccessor, in turn, should only operate on the meta table content, so 
 shouldn't need ZK. The only reason why MetaTableAccessor is using ZK - when 
 callers request assignment information, they can request location of meta 
 table itself, which we can't read from meta, so in that case 
 MetaTableAccessor relays the call to MetaTableLocator.  May be the solution 
 here is to declare that clients of MetaTableAccessor shall not use it to work 
 with meta table itself (not it's content).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-11462) MetaTableAccessor shouldn't use ZooKeeeper


[ 
https://issues.apache.org/jira/browse/HBASE-11462?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14140123#comment-14140123
 ] 

stack commented on HBASE-11462:
---

I'm just trying to get a run w/ most of hbase-server completes [~mantonov]

 MetaTableAccessor shouldn't use ZooKeeeper
 --

 Key: HBASE-11462
 URL: https://issues.apache.org/jira/browse/HBASE-11462
 Project: HBase
  Issue Type: Improvement
  Components: Client, Zookeeper
Affects Versions: 2.0.0
Reporter: Mikhail Antonov
Assignee: Mikhail Antonov
 Fix For: 2.0.0

 Attachments: HBASE-11462.v4.patch, HBASE-11462.v4.patch, 
 HBASE-11462.v4.patch


 After committing patch for HBASE-4495, there's an further improvement which 
 can be made (discussed originally on review board to that jira).
 We have MetaTableAccessor and MetaTableLocator classes. First one is used to 
 access information stored in hbase:meta table. Second one is used to deal 
 with ZooKeeper state to find out region server hosting hbase:meta, wait for 
 it to become available and so on.
 MetaTableAccessor, in turn, should only operate on the meta table content, so 
 shouldn't need ZK. The only reason why MetaTableAccessor is using ZK - when 
 callers request assignment information, they can request location of meta 
 table itself, which we can't read from meta, so in that case 
 MetaTableAccessor relays the call to MetaTableLocator.  May be the solution 
 here is to declare that clients of MetaTableAccessor shall not use it to work 
 with meta table itself (not it's content).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-12010) Use TableName.META_TABLE_NAME instead of indirectly from HTableDescriptor


 [ 
https://issues.apache.org/jira/browse/HBASE-12010?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stack updated HBASE-12010:
--
   Resolution: Fixed
Fix Version/s: 0.99.1
   2.0.0
 Hadoop Flags: Reviewed
   Status: Resolved  (was: Patch Available)

Committed to master and branch-1.  Thanks for the patch [~octo47]

 Use TableName.META_TABLE_NAME instead of indirectly from HTableDescriptor
 -

 Key: HBASE-12010
 URL: https://issues.apache.org/jira/browse/HBASE-12010
 Project: HBase
  Issue Type: Improvement
Affects Versions: 2.0.0
Reporter: Andrey Stepachev
Assignee: Andrey Stepachev
Priority: Trivial
 Fix For: 2.0.0, 0.99.1

 Attachments: 12010.patch, HBASE-12010.patch


 Currently code accessed meta table name via 
 HTableDescriptor.META_TABLEDESC.getTableName(), but we have 
 TableName.META_TABLE_NAME already. It is better to use from one place.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-11462) MetaTableAccessor shouldn't use ZooKeeeper


 [ 
https://issues.apache.org/jira/browse/HBASE-11462?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mikhail Antonov updated HBASE-11462:

Status: Open  (was: Patch Available)

 MetaTableAccessor shouldn't use ZooKeeeper
 --

 Key: HBASE-11462
 URL: https://issues.apache.org/jira/browse/HBASE-11462
 Project: HBase
  Issue Type: Improvement
  Components: Client, Zookeeper
Affects Versions: 2.0.0
Reporter: Mikhail Antonov
Assignee: Mikhail Antonov
 Fix For: 2.0.0

 Attachments: HBASE-11462.v4.patch, HBASE-11462.v4.patch, 
 HBASE-11462.v4.patch


 After committing patch for HBASE-4495, there's an further improvement which 
 can be made (discussed originally on review board to that jira).
 We have MetaTableAccessor and MetaTableLocator classes. First one is used to 
 access information stored in hbase:meta table. Second one is used to deal 
 with ZooKeeper state to find out region server hosting hbase:meta, wait for 
 it to become available and so on.
 MetaTableAccessor, in turn, should only operate on the meta table content, so 
 shouldn't need ZK. The only reason why MetaTableAccessor is using ZK - when 
 callers request assignment information, they can request location of meta 
 table itself, which we can't read from meta, so in that case 
 MetaTableAccessor relays the call to MetaTableLocator.  May be the solution 
 here is to declare that clients of MetaTableAccessor shall not use it to work 
 with meta table itself (not it's content).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-11462) MetaTableAccessor shouldn't use ZooKeeeper


[ 
https://issues.apache.org/jira/browse/HBASE-11462?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14140127#comment-14140127
 ] 

Mikhail Antonov commented on HBASE-11462:
-

The build seems flaky, but so far I couldn't trace it down to this patch..Last 
2 builds I ran today on our internal servers read:

{code}
1)Failed tests: 
  TestMultiParallel.testActiveThreadsCount:157 expected:5 but was:4
Tests in error: 
  
TestEndToEndSplitTransaction.blockUntilRegionSplit:451-blockUntilRegionIsInMeta:474
 NullPointer
Tests run: 1994, Failures: 1, Errors: 1, Skipped: 18

2) Failed tests: 
  TestSplitLogManager.testGetPreviousRecoveryMode:651 null
  TestMasterFailover.testSimpleMasterFailover:154 expected:5 but was:6
Tests in error: 
  TestMasterFailover.testMetaInTransitionWhenMasterFailover:396 »  test timed 
ou...
  
TestEndToEndSplitTransaction.blockUntilRegionSplit:451-blockUntilRegionIsInMeta:474
 NullPointer
Tests run: 1997, Failures: 2, Errors: 2, Skipped: 19
{code}

So I was suspecting TestEndToEndSplitTransaction is reproducible failure, but 
we can see this test did pass in ASF jenkins...



 MetaTableAccessor shouldn't use ZooKeeeper
 --

 Key: HBASE-11462
 URL: https://issues.apache.org/jira/browse/HBASE-11462
 Project: HBase
  Issue Type: Improvement
  Components: Client, Zookeeper
Affects Versions: 2.0.0
Reporter: Mikhail Antonov
Assignee: Mikhail Antonov
 Fix For: 2.0.0

 Attachments: HBASE-11462.v4.patch, HBASE-11462.v4.patch, 
 HBASE-11462.v4.patch


 After committing patch for HBASE-4495, there's an further improvement which 
 can be made (discussed originally on review board to that jira).
 We have MetaTableAccessor and MetaTableLocator classes. First one is used to 
 access information stored in hbase:meta table. Second one is used to deal 
 with ZooKeeper state to find out region server hosting hbase:meta, wait for 
 it to become available and so on.
 MetaTableAccessor, in turn, should only operate on the meta table content, so 
 shouldn't need ZK. The only reason why MetaTableAccessor is using ZK - when 
 callers request assignment information, they can request location of meta 
 table itself, which we can't read from meta, so in that case 
 MetaTableAccessor relays the call to MetaTableLocator.  May be the solution 
 here is to declare that clients of MetaTableAccessor shall not use it to work 
 with meta table itself (not it's content).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-11462) MetaTableAccessor shouldn't use ZooKeeeper


 [ 
https://issues.apache.org/jira/browse/HBASE-11462?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mikhail Antonov updated HBASE-11462:

Status: Patch Available  (was: Open)

 MetaTableAccessor shouldn't use ZooKeeeper
 --

 Key: HBASE-11462
 URL: https://issues.apache.org/jira/browse/HBASE-11462
 Project: HBase
  Issue Type: Improvement
  Components: Client, Zookeeper
Affects Versions: 2.0.0
Reporter: Mikhail Antonov
Assignee: Mikhail Antonov
 Fix For: 2.0.0

 Attachments: HBASE-11462.v4.patch, HBASE-11462.v4.patch, 
 HBASE-11462.v4.patch


 After committing patch for HBASE-4495, there's an further improvement which 
 can be made (discussed originally on review board to that jira).
 We have MetaTableAccessor and MetaTableLocator classes. First one is used to 
 access information stored in hbase:meta table. Second one is used to deal 
 with ZooKeeper state to find out region server hosting hbase:meta, wait for 
 it to become available and so on.
 MetaTableAccessor, in turn, should only operate on the meta table content, so 
 shouldn't need ZK. The only reason why MetaTableAccessor is using ZK - when 
 callers request assignment information, they can request location of meta 
 table itself, which we can't read from meta, so in that case 
 MetaTableAccessor relays the call to MetaTableLocator.  May be the solution 
 here is to declare that clients of MetaTableAccessor shall not use it to work 
 with meta table itself (not it's content).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-11462) MetaTableAccessor shouldn't use ZooKeeeper

[
https://issues.apache.org/jira/browse/HBASE-11462?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14140130#comment-14140130
]

Mikhail Antonov commented on HBASE-11462:
-

[~stack] sure, I understand. I'm looking into any single test which could fail
repeatedly when running standalone.

MetaTableAccessor shouldn't use ZooKeeeper
--

Attachments: HBASE-11462.v4.patch, HBASE-11462.v4.patch,
HBASE-11462.v4.patch

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-12021) Hbase shell does not respect the HBASE_OPTS set by the user in console


[ 
https://issues.apache.org/jira/browse/HBASE-12021?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14140147#comment-14140147
 ] 

Hadoop QA commented on HBASE-12021:
---

{color:red}-1 overall{color}.  Here are the results of testing the latest 
attachment 
  http://issues.apache.org/jira/secure/attachment/12669931/HBASE-12021.patch
  against trunk revision .
  ATTACHMENT ID: 12669931

{color:green}+1 @author{color}.  The patch does not contain any @author 
tags.

{color:red}-1 tests included{color}.  The patch doesn't appear to include 
any new or modified tests.
Please justify why no new tests are needed for this 
patch.
Also please list what manual steps were performed to 
verify this patch.

{color:green}+1 javac{color}.  The applied patch does not increase the 
total number of javac compiler warnings.

{color:green}+1 javac{color}.  The applied patch does not increase the 
total number of javac compiler warnings.

{color:red}-1 javadoc{color}.  The javadoc tool appears to have generated 2 
warning messages.

{color:green}+1 findbugs{color}.  The patch does not introduce any new 
Findbugs (version 2.0.3) warnings.

{color:green}+1 release audit{color}.  The applied patch does not increase 
the total number of release audit warnings.

{color:green}+1 lineLengths{color}.  The patch does not introduce lines 
longer than 100

  {color:green}+1 site{color}.  The mvn site goal succeeds with this patch.

 {color:red}-1 core tests{color}.  The patch failed these unit tests:
 

 {color:red}-1 core zombie tests{color}.  There are 1 zombie test(s):   
at 
org.apache.hadoop.hbase.regionserver.TestEncryptionKeyRotation.testCFKeyRotation(TestEncryptionKeyRotation.java:119)

Test results: 
https://builds.apache.org/job/PreCommit-HBASE-Build/10989//testReport/
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/10989//artifact/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/10989//artifact/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/10989//artifact/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/10989//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/10989//artifact/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/10989//artifact/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/10989//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/10989//artifact/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/10989//artifact/patchprocess/newPatchFindbugsWarningshbase-common.html
Console output: 
https://builds.apache.org/job/PreCommit-HBASE-Build/10989//console

This message is automatically generated.

 Hbase shell does not respect the HBASE_OPTS set by the user in console
 --

 Key: HBASE-12021
 URL: https://issues.apache.org/jira/browse/HBASE-12021
 Project: HBase
  Issue Type: Improvement
  Components: shell
Affects Versions: 2.0.0
Reporter: nijel
Assignee: Ashish Singhi
Priority: Minor
 Attachments: HBASE-12021.patch


 Before starting hbase shell i have set some -D parameters reauired for 
 kerberos authentication in HBASE_OPTS (there is no other variables are used 
 when shell is executed). But while invoking Hbase shell it is not used.
 The code in hbase-env.sh is 
 export HBASE_OPTS=-XX:+UseConcMarkSweepGC
 This will overwrite the the value set in the console
 There can be 2 options to support this
 1. Append the HBASE_OPTS like HBASE_OPTS=$HBASE_OPTS 
 -XX:+UseConcMarkSweepGC.
 2. Have a new variable (HBASE_CLIENT_OPTS) which is can be appended in shell 
 section of hbase file



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-10757) Change HTable class doc so it sends people to HCM getting instances

[
https://issues.apache.org/jira/browse/HBASE-10757?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14140171#comment-14140171
]

Hadoop QA commented on HBASE-10757:
---

{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12669936/10757.txt
against trunk revision .
ATTACHMENT ID: 12669936

{color:green}+1 @author{color}. The patch does not contain any @author
tags.

{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.

{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.

{color:red}-1 javadoc{color}. The javadoc tool appears to have generated 2
warning messages.

{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.

{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.

{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100

{color:green}+1 site{color}. The mvn site goal succeeds with this patch.

{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.util.TestProcessBasedCluster
org.apache.hadoop.hbase.mapreduce.TestImportExport

Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/10990//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10990//artifact/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10990//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10990//artifact/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10990//artifact/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10990//artifact/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10990//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10990//artifact/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10990//artifact/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10990//artifact/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/10990//console

This message is automatically generated.

Change HTable class doc so it sends people to HCM getting instances
---

Key: HBASE-10757
URL: https://issues.apache.org/jira/browse/HBASE-10757
Project: HBase
Issue Type: Bug
Reporter: stack
Assignee: stack
Priority: Critical
Fix For: 2.0.0, 0.99.1

Attachments: 10757.txt

The HTable class doc does not reflect the new regime where you are meant to
go via cluster connection to get an instance.
Getting an HTable otherwise also makes for some strange issues: See
Occasional GSSException that brings down region server on mailing list.

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-12021) Hbase shell does not respect the HBASE_OPTS set by the user in console

2014-09-19 Thread nijel (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-12021?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14140179#comment-14140179
 ] 

nijel commented on HBASE-12021:
---

Thanks ashish for the quick response

+1 for the patch

 Hbase shell does not respect the HBASE_OPTS set by the user in console
 --

 Key: HBASE-12021
 URL: https://issues.apache.org/jira/browse/HBASE-12021
 Project: HBase
  Issue Type: Improvement
  Components: shell
Affects Versions: 2.0.0
Reporter: nijel
Assignee: Ashish Singhi
Priority: Minor
 Attachments: HBASE-12021.patch


 Before starting hbase shell i have set some -D parameters reauired for 
 kerberos authentication in HBASE_OPTS (there is no other variables are used 
 when shell is executed). But while invoking Hbase shell it is not used.
 The code in hbase-env.sh is 
 export HBASE_OPTS=-XX:+UseConcMarkSweepGC
 This will overwrite the the value set in the console
 There can be 2 options to support this
 1. Append the HBASE_OPTS like HBASE_OPTS=$HBASE_OPTS 
 -XX:+UseConcMarkSweepGC.
 2. Have a new variable (HBASE_CLIENT_OPTS) which is can be appended in shell 
 section of hbase file



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-11462) MetaTableAccessor shouldn't use ZooKeeeper


 [ 
https://issues.apache.org/jira/browse/HBASE-11462?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mikhail Antonov updated HBASE-11462:

Status: Patch Available  (was: Open)

 MetaTableAccessor shouldn't use ZooKeeeper
 --

 Key: HBASE-11462
 URL: https://issues.apache.org/jira/browse/HBASE-11462
 Project: HBase
  Issue Type: Improvement
  Components: Client, Zookeeper
Affects Versions: 2.0.0
Reporter: Mikhail Antonov
Assignee: Mikhail Antonov
 Fix For: 2.0.0

 Attachments: HBASE-11462.v4.patch, HBASE-11462.v4.patch, 
 HBASE-11462.v4.patch


 After committing patch for HBASE-4495, there's an further improvement which 
 can be made (discussed originally on review board to that jira).
 We have MetaTableAccessor and MetaTableLocator classes. First one is used to 
 access information stored in hbase:meta table. Second one is used to deal 
 with ZooKeeper state to find out region server hosting hbase:meta, wait for 
 it to become available and so on.
 MetaTableAccessor, in turn, should only operate on the meta table content, so 
 shouldn't need ZK. The only reason why MetaTableAccessor is using ZK - when 
 callers request assignment information, they can request location of meta 
 table itself, which we can't read from meta, so in that case 
 MetaTableAccessor relays the call to MetaTableLocator.  May be the solution 
 here is to declare that clients of MetaTableAccessor shall not use it to work 
 with meta table itself (not it's content).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-11462) MetaTableAccessor shouldn't use ZooKeeeper


 [ 
https://issues.apache.org/jira/browse/HBASE-11462?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mikhail Antonov updated HBASE-11462:

Status: Open  (was: Patch Available)

 MetaTableAccessor shouldn't use ZooKeeeper
 --

 Key: HBASE-11462
 URL: https://issues.apache.org/jira/browse/HBASE-11462
 Project: HBase
  Issue Type: Improvement
  Components: Client, Zookeeper
Affects Versions: 2.0.0
Reporter: Mikhail Antonov
Assignee: Mikhail Antonov
 Fix For: 2.0.0

 Attachments: HBASE-11462.v4.patch, HBASE-11462.v4.patch, 
 HBASE-11462.v4.patch


 After committing patch for HBASE-4495, there's an further improvement which 
 can be made (discussed originally on review board to that jira).
 We have MetaTableAccessor and MetaTableLocator classes. First one is used to 
 access information stored in hbase:meta table. Second one is used to deal 
 with ZooKeeper state to find out region server hosting hbase:meta, wait for 
 it to become available and so on.
 MetaTableAccessor, in turn, should only operate on the meta table content, so 
 shouldn't need ZK. The only reason why MetaTableAccessor is using ZK - when 
 callers request assignment information, they can request location of meta 
 table itself, which we can't read from meta, so in that case 
 MetaTableAccessor relays the call to MetaTableLocator.  May be the solution 
 here is to declare that clients of MetaTableAccessor shall not use it to work 
 with meta table itself (not it's content).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-11462) MetaTableAccessor shouldn't use ZooKeeeper

[
https://issues.apache.org/jira/browse/HBASE-11462?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14140217#comment-14140217
]

Hadoop QA commented on HBASE-11462:
---

{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12669938/HBASE-11462.v4.patch
against trunk revision .
ATTACHMENT ID: 12669938

{color:green}+1 @author{color}. The patch does not contain any @author
tags.

{color:green}+1 tests included{color}. The patch appears to include 18 new
or modified tests.

{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.

{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.

{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.

{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.

{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100

{color:green}+1 site{color}. The mvn site goal succeeds with this patch.

{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.mapreduce.TestImportExport
org.apache.hadoop.hbase.util.TestProcessBasedCluster
org.apache.hadoop.hbase.master.TestSplitLogManager

{color:red}-1 core zombie tests{color}. There are 2 zombie test(s):
at org.apache.bugs.AMQ1730Test.testRedelivery(AMQ1730Test.java:141)
at
org.apache.blur.lucene.search.FacetQueryTest.testFacetQueryPerformance2(FacetQueryTest.java:141)

Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/10991//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10991//artifact/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10991//artifact/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10991//artifact/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10991//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10991//artifact/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10991//artifact/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10991//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10991//artifact/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10991//artifact/patchprocess/newPatchFindbugsWarningshbase-common.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/10991//console

This message is automatically generated.

MetaTableAccessor shouldn't use ZooKeeeper
--

Attachments: HBASE-11462.v4.patch, HBASE-11462.v4.patch,
HBASE-11462.v4.patch

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-11462) MetaTableAccessor shouldn't use ZooKeeeper


 [ 
https://issues.apache.org/jira/browse/HBASE-11462?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mikhail Antonov updated HBASE-11462:

Status: Open  (was: Patch Available)

 MetaTableAccessor shouldn't use ZooKeeeper
 --

 Key: HBASE-11462
 URL: https://issues.apache.org/jira/browse/HBASE-11462
 Project: HBase
  Issue Type: Improvement
  Components: Client, Zookeeper
Affects Versions: 2.0.0
Reporter: Mikhail Antonov
Assignee: Mikhail Antonov
 Fix For: 2.0.0

 Attachments: HBASE-11462.v4.patch, HBASE-11462.v4.patch, 
 HBASE-11462.v4.patch


 After committing patch for HBASE-4495, there's an further improvement which 
 can be made (discussed originally on review board to that jira).
 We have MetaTableAccessor and MetaTableLocator classes. First one is used to 
 access information stored in hbase:meta table. Second one is used to deal 
 with ZooKeeper state to find out region server hosting hbase:meta, wait for 
 it to become available and so on.
 MetaTableAccessor, in turn, should only operate on the meta table content, so 
 shouldn't need ZK. The only reason why MetaTableAccessor is using ZK - when 
 callers request assignment information, they can request location of meta 
 table itself, which we can't read from meta, so in that case 
 MetaTableAccessor relays the call to MetaTableLocator.  May be the solution 
 here is to declare that clients of MetaTableAccessor shall not use it to work 
 with meta table itself (not it's content).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-11462) MetaTableAccessor shouldn't use ZooKeeeper


 [ 
https://issues.apache.org/jira/browse/HBASE-11462?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mikhail Antonov updated HBASE-11462:

Status: Patch Available  (was: Open)

 MetaTableAccessor shouldn't use ZooKeeeper
 --

 Key: HBASE-11462
 URL: https://issues.apache.org/jira/browse/HBASE-11462
 Project: HBase
  Issue Type: Improvement
  Components: Client, Zookeeper
Affects Versions: 2.0.0
Reporter: Mikhail Antonov
Assignee: Mikhail Antonov
 Fix For: 2.0.0

 Attachments: HBASE-11462.v4.patch, HBASE-11462.v4.patch, 
 HBASE-11462.v4.patch


 After committing patch for HBASE-4495, there's an further improvement which 
 can be made (discussed originally on review board to that jira).
 We have MetaTableAccessor and MetaTableLocator classes. First one is used to 
 access information stored in hbase:meta table. Second one is used to deal 
 with ZooKeeper state to find out region server hosting hbase:meta, wait for 
 it to become available and so on.
 MetaTableAccessor, in turn, should only operate on the meta table content, so 
 shouldn't need ZK. The only reason why MetaTableAccessor is using ZK - when 
 callers request assignment information, they can request location of meta 
 table itself, which we can't read from meta, so in that case 
 MetaTableAccessor relays the call to MetaTableLocator.  May be the solution 
 here is to declare that clients of MetaTableAccessor shall not use it to work 
 with meta table itself (not it's content).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-11462) MetaTableAccessor shouldn't use ZooKeeeper

[
https://issues.apache.org/jira/browse/HBASE-11462?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14140222#comment-14140222
]

Mikhail Antonov commented on HBASE-11462:
-

Retrying. that doesn't look related either.

{code}
Failed tests:
TestProcessBasedCluster.testHomePath:92
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build%402/hbase-server/pom.xml
does not exist
TestSplitLogManager.testGetPreviousRecoveryMode:651 null

Tests in error:
TestImportExport.testImport94Table:224 ? FileNotFound File
/home/jenkins/jenki...

Tests run: 2021, Failures: 2, Errors: 1, Skipped: 19
{code}

MetaTableAccessor shouldn't use ZooKeeeper
--

Attachments: HBASE-11462.v4.patch, HBASE-11462.v4.patch,
HBASE-11462.v4.patch

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (HBASE-12026) Hbase Ave Load work heavily ??

2014-09-19 Thread david_dong (JIRA)

david_dong created HBASE-12026:
--

 Summary: Hbase Ave Load work heavily ??
 Key: HBASE-12026
 URL: https://issues.apache.org/jira/browse/HBASE-12026
 Project: HBase
  Issue Type: Bug
  Components: hadoop2
Affects Versions: 0.98.0
 Environment: Hadoop 2.4.0.2.1.1.0-385
Reporter: david_dong
Priority: Critical


hi,all!
My Hadoop works very well execpt the HBASE.
It displayed that Hbase Ave Load work heavily,but i cann't find out which 
area is hot ..






--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-11644) External MOB compaction tools

[
https://issues.apache.org/jira/browse/HBASE-11644?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Jingcheng Du updated HBASE-11644:
-
Attachment: HBASE-11644-Sep-19.diff

Update the patch based on the patch in handle mob in compaction.

External MOB compaction tools
-

Key: HBASE-11644
URL: https://issues.apache.org/jira/browse/HBASE-11644
Project: HBase
Issue Type: Sub-task
Components: Compaction, master
Affects Versions: hbase-11339
Reporter: Jingcheng Du
Assignee: Jingcheng Du
Fix For: hbase-11339

Attachments: HBASE-11644-Sep-15.diff, HBASE-11644-Sep-16.diff,
HBASE-11644-Sep-16.diff, HBASE-11644-Sep-18.diff, HBASE-11644-Sep-19.diff,
HBASE-11644.diff, HBASE-11646-0918-bad.patch

From the design doc, mob files are not involved in the normal HBase
compaction process. This means deleted mobs would still take up space and
that we never really merge mob files that accrue over time. Currently, MOBs
depend on two external tools:
1) A TTL cleaner that removes mobs that have passed their TTL or exceeded
minVersions.
2) A 'sweep tool' cleaner that remove mobs that have had their references
deleted and merges small files into larger ones.
Today the tools are triggered by admins. The longer term goal would be to
integrate them into hbase such that by default mobs are cleaned. The tools
will be preserved however so that advanced admins can disable automatic
cleanups and manually trigger these compaction like operaitons. #1 would
likely be a chore in the master while #2 requires some design work to
integrate into hbase.

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-12010) Use TableName.META_TABLE_NAME instead of indirectly from HTableDescriptor


[ 
https://issues.apache.org/jira/browse/HBASE-12010?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14140226#comment-14140226
 ] 

Hudson commented on HBASE-12010:


FAILURE: Integrated in HBase-TRUNK #5531 (See 
[https://builds.apache.org/job/HBase-TRUNK/5531/])
HBASE-12010 Use TableName.META_TABLE_NAME instead of indirectly from 
HTableDescriptor (stack: rev e05f78ec01a568f1c3580b08215f55d9f144c1c7)
* hbase-client/src/main/java/org/apache/hadoop/hbase/MetaTableAccessor.java
* 
hbase-server/src/main/java/org/apache/hadoop/hbase/util/FSTableDescriptors.java
* hbase-server/src/test/java/org/apache/hadoop/hbase/util/TestHBaseFsck.java
* 
hbase-server/src/test/java/org/apache/hadoop/hbase/replication/TestReplicationWALEntryFilters.java
* hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestAdmin.java
* 
hbase-server/src/test/java/org/apache/hadoop/hbase/util/hbck/OfflineMetaRebuildTestCore.java
* 
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/wal/TestLogRollingNoCluster.java


 Use TableName.META_TABLE_NAME instead of indirectly from HTableDescriptor
 -

 Key: HBASE-12010
 URL: https://issues.apache.org/jira/browse/HBASE-12010
 Project: HBase
  Issue Type: Improvement
Affects Versions: 2.0.0
Reporter: Andrey Stepachev
Assignee: Andrey Stepachev
Priority: Trivial
 Fix For: 2.0.0, 0.99.1

 Attachments: 12010.patch, HBASE-12010.patch


 Currently code accessed meta table name via 
 HTableDescriptor.META_TABLEDESC.getTableName(), but we have 
 TableName.META_TABLE_NAME already. It is better to use from one place.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-11644) External MOB compaction tools

[
https://issues.apache.org/jira/browse/HBASE-11644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14140231#comment-14140231
]

Hadoop QA commented on HBASE-11644:
---

{color:red}-1 overall{color}. Here are the results of testing the latest
attachment

http://issues.apache.org/jira/secure/attachment/12669957/HBASE-11644-Sep-19.diff
against trunk revision .
ATTACHMENT ID: 12669957

{color:green}+1 @author{color}. The patch does not contain any @author
tags.

{color:green}+1 tests included{color}. The patch appears to include 19 new
or modified tests.

{color:red}-1 patch{color}. The patch command could not apply the patch.

Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/10992//console

This message is automatically generated.

External MOB compaction tools
-

Attachments: HBASE-11644-Sep-15.diff, HBASE-11644-Sep-16.diff,
HBASE-11644-Sep-16.diff, HBASE-11644-Sep-18.diff, HBASE-11644-Sep-19.diff,
HBASE-11644.diff, HBASE-11646-0918-bad.patch

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-11990) Make setting the start and stop row for a specific prefix easier


 [ 
https://issues.apache.org/jira/browse/HBASE-11990?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Niels Basjes updated HBASE-11990:
-
Status: Open  (was: Patch Available)

Working on a different approach as indicated by [~lhofhansl]

 Make setting the start and stop row for a specific prefix easier
 

 Key: HBASE-11990
 URL: https://issues.apache.org/jira/browse/HBASE-11990
 Project: HBase
  Issue Type: New Feature
  Components: Client
Reporter: Niels Basjes
 Attachments: 11990v4.txt, HBASE-11990-20140916-v2.patch, 
 HBASE-11990-20140916-v3.patch, HBASE-11990-20140916-v5.patch, 
 HBASE-11990-20140916-v6.patch, HBASE-11990-20140916.patch, 
 HBASE-11990-20140917-v7.patch


 If you want to set a scan from your application to scan for a specific row 
 prefix this is actually quite hard.
 As described in several places you can set the startRow to the prefix; yet 
 the stopRow should be set to the prefix '+1'
 If the prefix 'ASCII' put into a byte[] then this is easy because you can 
 simply increment the last byte of the array. 
 But if your application uses real binary rowids you may run into the scenario 
 that your prefix is something like 
 {code}{ 0x12, 0x23, 0xFF, 0xFF }{code} Then the increment should be {code}{ 
 0x12, 0x24 }{code}
 I have prepared a proposed patch that makes setting these values correctly a 
 lot easier.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-11644) External MOB compaction tools

[
https://issues.apache.org/jira/browse/HBASE-11644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14140235#comment-14140235
]

Jingcheng Du commented on HBASE-11644:
--

And use mvn clean test -Dtest=TestMob* to pass the testing in the patch.

External MOB compaction tools
-

Attachments: HBASE-11644-Sep-15.diff, HBASE-11644-Sep-16.diff,
HBASE-11644-Sep-16.diff, HBASE-11644-Sep-18.diff, HBASE-11644-Sep-19.diff,
HBASE-11644.diff, HBASE-11646-0918-bad.patch

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-11990) Make setting the start and stop row for a specific prefix easier


[ 
https://issues.apache.org/jira/browse/HBASE-11990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14140240#comment-14140240
 ] 

Niels Basjes commented on HBASE-11990:
--

I could use some help here.
I'm working on a different implementation of Scan to include a method
{code}public Scan addRowPrefixFilter(byte [] rowPrefix){code}  
How do I create a sensible Unit test for this?
In hbase-client I cannot use the HBaseTestingUtility because that is part of 
hbase-server and hbase-server depends on hbase-client.
I haven't been able to find anything else yet other than external things like 
this https://gist.github.com/agaoglu/613217

What is the correct way to do this?

 Make setting the start and stop row for a specific prefix easier
 

 Key: HBASE-11990
 URL: https://issues.apache.org/jira/browse/HBASE-11990
 Project: HBase
  Issue Type: New Feature
  Components: Client
Reporter: Niels Basjes
 Attachments: 11990v4.txt, HBASE-11990-20140916-v2.patch, 
 HBASE-11990-20140916-v3.patch, HBASE-11990-20140916-v5.patch, 
 HBASE-11990-20140916-v6.patch, HBASE-11990-20140916.patch, 
 HBASE-11990-20140917-v7.patch


 If you want to set a scan from your application to scan for a specific row 
 prefix this is actually quite hard.
 As described in several places you can set the startRow to the prefix; yet 
 the stopRow should be set to the prefix '+1'
 If the prefix 'ASCII' put into a byte[] then this is easy because you can 
 simply increment the last byte of the array. 
 But if your application uses real binary rowids you may run into the scenario 
 that your prefix is something like 
 {code}{ 0x12, 0x23, 0xFF, 0xFF }{code} Then the increment should be {code}{ 
 0x12, 0x24 }{code}
 I have prepared a proposed patch that makes setting these values correctly a 
 lot easier.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-11644) External MOB compaction tools

[
https://issues.apache.org/jira/browse/HBASE-11644?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Jingcheng Du updated HBASE-11644:
-
Attachment: HBASE-11644-Sep-19-V2.patch

Generate the patch(HBASE-11644-Sep-19-V2.patch) based on the latest code in
hbase-11339. Thanks Anoop for the help!

External MOB compaction tools
-

Attachments: HBASE-11644-Sep-15.diff, HBASE-11644-Sep-16.diff,
HBASE-11644-Sep-16.diff, HBASE-11644-Sep-18.diff,
HBASE-11644-Sep-19-V2.patch, HBASE-11644-Sep-19.diff, HBASE-11644.diff,
HBASE-11646-0918-bad.patch

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-12010) Use TableName.META_TABLE_NAME instead of indirectly from HTableDescriptor


[ 
https://issues.apache.org/jira/browse/HBASE-12010?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14140255#comment-14140255
 ] 

Hudson commented on HBASE-12010:


FAILURE: Integrated in HBase-1.0 #205 (See 
[https://builds.apache.org/job/HBase-1.0/205/])
HBASE-12010 Use TableName.META_TABLE_NAME instead of indirectly from 
HTableDescriptor (stack: rev 3a5e58001062c1fbb46b421d143b88120c0a0bbd)
* hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestAdmin.java
* hbase-server/src/test/java/org/apache/hadoop/hbase/util/TestHBaseFsck.java
* 
hbase-server/src/main/java/org/apache/hadoop/hbase/util/FSTableDescriptors.java
* 
hbase-server/src/test/java/org/apache/hadoop/hbase/util/hbck/OfflineMetaRebuildTestCore.java
* 
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/wal/TestLogRollingNoCluster.java
* hbase-client/src/main/java/org/apache/hadoop/hbase/MetaTableAccessor.java
* 
hbase-server/src/test/java/org/apache/hadoop/hbase/replication/TestReplicationWALEntryFilters.java


 Use TableName.META_TABLE_NAME instead of indirectly from HTableDescriptor
 -

 Key: HBASE-12010
 URL: https://issues.apache.org/jira/browse/HBASE-12010
 Project: HBase
  Issue Type: Improvement
Affects Versions: 2.0.0
Reporter: Andrey Stepachev
Assignee: Andrey Stepachev
Priority: Trivial
 Fix For: 2.0.0, 0.99.1

 Attachments: 12010.patch, HBASE-12010.patch


 Currently code accessed meta table name via 
 HTableDescriptor.META_TABLEDESC.getTableName(), but we have 
 TableName.META_TABLE_NAME already. It is better to use from one place.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-11644) External MOB compaction tools

[
https://issues.apache.org/jira/browse/HBASE-11644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14140259#comment-14140259
]

Hadoop QA commented on HBASE-11644:
---

{color:red}-1 overall{color}. Here are the results of testing the latest
attachment

http://issues.apache.org/jira/secure/attachment/12669962/HBASE-11644-Sep-19-V2.patch
against trunk revision .
ATTACHMENT ID: 12669962

{color:green}+1 @author{color}. The patch does not contain any @author
tags.

{color:green}+1 tests included{color}. The patch appears to include 19 new
or modified tests.

{color:red}-1 patch{color}. The patch command could not apply the patch.

Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/10993//console

This message is automatically generated.

External MOB compaction tools
-

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (HBASE-12026) Hbase Ave Load work heavily ??

2014-09-19 Thread Jonathan Hsieh (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-12026?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Hsieh resolved HBASE-12026.

Resolution: Invalid

hi [~david_dong1985].  This jira is for tracking bugs and new features.  If you 
have questions, or need help  trying to how to figure out what is going on on 
an hbase deploy, please send email to u...@hbase.apache.org to get responses. 

I'm going to close this since it seems to be a question, as opposed to 
something that can directly lead to code.

 Hbase Ave Load work heavily ??
 --

 Key: HBASE-12026
 URL: https://issues.apache.org/jira/browse/HBASE-12026
 Project: HBase
  Issue Type: Bug
  Components: hadoop2
Affects Versions: 0.98.0
 Environment: Hadoop 2.4.0.2.1.1.0-385
Reporter: david_dong
Priority: Critical

 hi,all!
 My Hadoop works very well execpt the HBASE.
 It displayed that Hbase Ave Load work heavily,but i cann't find out which 
 area is hot ..



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-11644) External MOB compaction tools

2014-09-19 Thread Jonathan Hsieh (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-11644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14140274#comment-14140274
 ] 

Jonathan Hsieh commented on HBASE-11644:


the sept19 v2 version compiles and seems to pass for with the 'mvn clean test 
-Dtest=test\*Mob\*'

 External MOB compaction tools
 -

 Key: HBASE-11644
 URL: https://issues.apache.org/jira/browse/HBASE-11644
 Project: HBase
  Issue Type: Sub-task
  Components: Compaction, master
Affects Versions: hbase-11339
Reporter: Jingcheng Du
Assignee: Jingcheng Du
 Fix For: hbase-11339

 Attachments: HBASE-11644-Sep-15.diff, HBASE-11644-Sep-16.diff, 
 HBASE-11644-Sep-16.diff, HBASE-11644-Sep-18.diff, 
 HBASE-11644-Sep-19-V2.patch, HBASE-11644-Sep-19.diff, HBASE-11644.diff, 
 HBASE-11646-0918-bad.patch


 From the design doc,  mob files are not involved in the normal HBase 
 compaction process.  This means deleted mobs would still take up space and 
 that we never really merge mob files that accrue over time.   Currently, MOBs 
 depend on two external tools:
 1) A TTL cleaner that removes mobs that have passed their TTL or exceeded 
 minVersions.
 2) A 'sweep tool' cleaner that remove mobs that have had their references 
 deleted and merges small files into larger ones.  
 Today the tools are triggered by admins.  The longer term goal would be to 
 integrate them into hbase such that by default mobs are cleaned.  The tools 
 will be preserved however so that advanced admins can disable automatic 
 cleanups and manually trigger these compaction like operaitons.  #1 would 
 likely be a chore in the master while #2 requires some design work to 
 integrate into hbase.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-11644) External MOB compaction tools

2014-09-19 Thread Jonathan Hsieh (JIRA)

[
https://issues.apache.org/jira/browse/HBASE-11644?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Jonathan Hsieh updated HBASE-11644:
---
Resolution: Fixed
Hadoop Flags: Reviewed
Status: Resolved (was: Patch Available)

thanks jingcheng, and thans for reviews anoop. I've committed to the
hbase-11339 branch.

External MOB compaction tools
-

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-11990) Make setting the start and stop row for a specific prefix easier


 [ 
https://issues.apache.org/jira/browse/HBASE-11990?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Niels Basjes updated HBASE-11990:
-
Attachment: HBASE-11990-20140919-v8.patch

First version of the new approach.
No unit tests yet because I do not know how to build them.

 Make setting the start and stop row for a specific prefix easier
 

 Key: HBASE-11990
 URL: https://issues.apache.org/jira/browse/HBASE-11990
 Project: HBase
  Issue Type: New Feature
  Components: Client
Reporter: Niels Basjes
 Attachments: 11990v4.txt, HBASE-11990-20140916-v2.patch, 
 HBASE-11990-20140916-v3.patch, HBASE-11990-20140916-v5.patch, 
 HBASE-11990-20140916-v6.patch, HBASE-11990-20140916.patch, 
 HBASE-11990-20140917-v7.patch, HBASE-11990-20140919-v8.patch


 If you want to set a scan from your application to scan for a specific row 
 prefix this is actually quite hard.
 As described in several places you can set the startRow to the prefix; yet 
 the stopRow should be set to the prefix '+1'
 If the prefix 'ASCII' put into a byte[] then this is easy because you can 
 simply increment the last byte of the array. 
 But if your application uses real binary rowids you may run into the scenario 
 that your prefix is something like 
 {code}{ 0x12, 0x23, 0xFF, 0xFF }{code} Then the increment should be {code}{ 
 0x12, 0x24 }{code}
 I have prepared a proposed patch that makes setting these values correctly a 
 lot easier.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-11990) Make setting the start and stop row for a specific prefix easier


 [ 
https://issues.apache.org/jira/browse/HBASE-11990?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Niels Basjes updated HBASE-11990:
-
Status: Patch Available  (was: Open)

 Make setting the start and stop row for a specific prefix easier
 

 Key: HBASE-11990
 URL: https://issues.apache.org/jira/browse/HBASE-11990
 Project: HBase
  Issue Type: New Feature
  Components: Client
Reporter: Niels Basjes
 Attachments: 11990v4.txt, HBASE-11990-20140916-v2.patch, 
 HBASE-11990-20140916-v3.patch, HBASE-11990-20140916-v5.patch, 
 HBASE-11990-20140916-v6.patch, HBASE-11990-20140916.patch, 
 HBASE-11990-20140917-v7.patch, HBASE-11990-20140919-v8.patch


 If you want to set a scan from your application to scan for a specific row 
 prefix this is actually quite hard.
 As described in several places you can set the startRow to the prefix; yet 
 the stopRow should be set to the prefix '+1'
 If the prefix 'ASCII' put into a byte[] then this is easy because you can 
 simply increment the last byte of the array. 
 But if your application uses real binary rowids you may run into the scenario 
 that your prefix is something like 
 {code}{ 0x12, 0x23, 0xFF, 0xFF }{code} Then the increment should be {code}{ 
 0x12, 0x24 }{code}
 I have prepared a proposed patch that makes setting these values correctly a 
 lot easier.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-9779) IntegrationTestLoadAndVerify fails deleting IntegrationTestLoadAndVerify table

2014-09-19 Thread chendihao (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-9779?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14140287#comment-14140287
 ] 

chendihao commented on HBASE-9779:
--

Why Stopping catalog tracker and establish sessions so frequently? Is it 
related to this issue? [~ndimiduk] [~stack]

 IntegrationTestLoadAndVerify fails deleting IntegrationTestLoadAndVerify 
 table 
 ---

 Key: HBASE-9779
 URL: https://issues.apache.org/jira/browse/HBASE-9779
 Project: HBase
  Issue Type: Bug
  Components: test
Affects Versions: 0.96.0
Reporter: stack
Assignee: stack
Priority: Critical
 Attachments: 9779part.txt


 As part of the test, we want to delete the created table to restore cluster 
 state.  Interestingly we can disable the table successfully but then 
 immediately after we fail the delete because we cannot get the table 
 descriptor -- getting the file descriptor is used to test if table is present.
 The test for getDescriptor is kinda broke because it throws base IOE which 
 causes clients to retry over and over again as though the descriptor was 
 going to come back.
 This bug is kinda ugly because in at least one case it caused our 
 long-running hbase-it suite run to fail so would be good to fix.
 Here is sample from a test run:
 {code}
 Disabling table IntegrationTestLoadAndVerify 2013-10-11 18:27:53,485 INFO  
 [main] client.HBaseAdmin: Started disable of IntegrationTestLoadAndVerify
 2013-10-11 18:27:53,526 INFO  [main] zookeeper.ZooKeeper: Initiating client 
 connection, connectString=a1805.halxg.cloudera.com:2181 sessionTimeout=9 
 watcher=catalogtracker-on-hconnection-0x5a7e666f
 2013-10-11 18:27:53,527 INFO  [main] zookeeper.RecoverableZooKeeper: Process 
 identifier=catalogtracker-on-hconnection-0x5a7e666f connecting to ZooKeeper 
 ensemble=a1805.halxg.cloudera.com:2181
 2013-10-11 18:27:53,527 INFO  
 [main-SendThread(a1805.halxg.cloudera.com:2181)] zookeeper.ClientCnxn: 
 Opening socket connection to server 
 a1805.halxg.cloudera.com/10.20.200.105:2181. Will not attempt to authenticate 
 using SASL (unknown error)
 2013-10-11 18:27:53,527 DEBUG [main] catalog.CatalogTracker: Starting catalog 
 tracker org.apache.hadoop.hbase.catalog.CatalogTracker@4ace08a5
 2013-10-11 18:27:53,529 INFO  
 [main-SendThread(a1805.halxg.cloudera.com:2181)] zookeeper.ClientCnxn: Socket 
 connection established to a1805.halxg.cloudera.com/10.20.200.105:2181, 
 initiating session
 2013-10-11 18:27:53,539 INFO  
 [main-SendThread(a1805.halxg.cloudera.com:2181)] zookeeper.ClientCnxn: 
 Session establishment complete on server 
 a1805.halxg.cloudera.com/10.20.200.105:2181, sessionid = 0x1412d47f53a5c70, 
 negotiated timeout = 4
 2013-10-11 18:27:53,602 DEBUG [main] catalog.CatalogTracker: Stopping catalog 
 tracker org.apache.hadoop.hbase.catalog.CatalogTracker@4ace08a5
 2013-10-11 18:27:53,662 INFO  [main] zookeeper.ZooKeeper: Session: 
 0x1412d47f53a5c70 closed
 2013-10-11 18:27:53,662 INFO  [main-EventThread] zookeeper.ClientCnxn: 
 EventThread shut down
 .2013-10-11 18:27:54,666 INFO  [main] zookeeper.ZooKeeper: Initiating client 
 connection, connectString=a1805.halxg.cloudera.com:2181 sessionTimeout=9 
 watcher=catalogtracker-on-hconnection-0x5a7e666f
 2013-10-11 18:27:54,667 INFO  [main] zookeeper.RecoverableZooKeeper: Process 
 identifier=catalogtracker-on-hconnection-0x5a7e666f connecting to ZooKeeper 
 ensemble=a1805.halxg.cloudera.com:2181
 2013-10-11 18:27:54,667 INFO  
 [main-SendThread(a1805.halxg.cloudera.com:2181)] zookeeper.ClientCnxn: 
 Opening socket connection to server 
 a1805.halxg.cloudera.com/10.20.200.105:2181. Will not attempt to authenticate 
 using SASL (unknown error)
 2013-10-11 18:27:54,667 DEBUG [main] catalog.CatalogTracker: Starting catalog 
 tracker org.apache.hadoop.hbase.catalog.CatalogTracker@692c0c5d
 2013-10-11 18:27:54,667 INFO  
 [main-SendThread(a1805.halxg.cloudera.com:2181)] zookeeper.ClientCnxn: Socket 
 connection established to a1805.halxg.cloudera.com/10.20.200.105:2181, 
 initiating session
 2013-10-11 18:27:54,696 INFO  
 [main-SendThread(a1805.halxg.cloudera.com:2181)] zookeeper.ClientCnxn: 
 Session establishment complete on server 
 a1805.halxg.cloudera.com/10.20.200.105:2181, sessionid = 0x1412d47f53a5c71, 
 negotiated timeout = 4
 2013-10-11 18:27:54,821 DEBUG [main] catalog.CatalogTracker: Stopping catalog 
 tracker org.apache.hadoop.hbase.catalog.CatalogTracker@692c0c5d
 2013-10-11 18:27:54,871 INFO  [main] zookeeper.ZooKeeper: Session: 
 0x1412d47f53a5c71 closed
 2013-10-11 18:27:54,871 INFO  [main-EventThread] zookeeper.ClientCnxn: 
 EventThread shut down
 .2013-10-11 18:27:55,890 INFO  [main] zookeeper.ZooKeeper: Initiating client 
 connection, connectString=a1805.halxg.cloudera.com:2181

[jira] [Created] (HBASE-12027) The ZooKeeperWatcher in HMobStore only uses the default conf

Jingcheng Du created HBASE-12027:


 Summary: The ZooKeeperWatcher in HMobStore only uses the default 
conf
 Key: HBASE-12027
 URL: https://issues.apache.org/jira/browse/HBASE-12027
 Project: HBase
  Issue Type: Bug
Affects Versions: hbase-11339
Reporter: Jingcheng Du
Assignee: Jingcheng Du
 Fix For: hbase-11339


In HMobStore we need to create a ZooKeeperWatcher for a major compaction.
There the conf of store is directly passed into the ZookeeperWatcher, this conf 
is a CompoundConfiguration.
In ZookeeperWatcher, this conf is converted into a Configuration by new 
Configuration(compoundConfiguration), and the zk-related conf are lost, only 
default settings are used by the zookeeper.
In the HMobStore, we should pass the region.getBaseConf() which is a 
Configuration to the ZookeeperWatcher.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-12027) The ZooKeeperWatcher in HMobStore only uses the default conf


 [ 
https://issues.apache.org/jira/browse/HBASE-12027?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jingcheng Du updated HBASE-12027:
-
Attachment: HBASE-12027.diff

Upload the patch.
Directly to use region.getBaseConf() instead of the conf of the HStore.

 The ZooKeeperWatcher in HMobStore only uses the default conf
 

 Key: HBASE-12027
 URL: https://issues.apache.org/jira/browse/HBASE-12027
 Project: HBase
  Issue Type: Bug
Affects Versions: hbase-11339
Reporter: Jingcheng Du
Assignee: Jingcheng Du
 Fix For: hbase-11339

 Attachments: HBASE-12027.diff


 In HMobStore we need to create a ZooKeeperWatcher for a major compaction.
 There the conf of store is directly passed into the ZookeeperWatcher, this 
 conf is a CompoundConfiguration.
 In ZookeeperWatcher, this conf is converted into a Configuration by new 
 Configuration(compoundConfiguration), and the zk-related conf are lost, only 
 default settings are used by the zookeeper.
 In the HMobStore, we should pass the region.getBaseConf() which is a 
 Configuration to the ZookeeperWatcher.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-12027) The ZooKeeperWatcher in HMobStore only uses the default conf


 [ 
https://issues.apache.org/jira/browse/HBASE-12027?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jingcheng Du updated HBASE-12027:
-
Status: Patch Available  (was: Open)

 The ZooKeeperWatcher in HMobStore only uses the default conf
 

 Key: HBASE-12027
 URL: https://issues.apache.org/jira/browse/HBASE-12027
 Project: HBase
  Issue Type: Bug
Affects Versions: hbase-11339
Reporter: Jingcheng Du
Assignee: Jingcheng Du
 Fix For: hbase-11339

 Attachments: HBASE-12027.diff


 In HMobStore we need to create a ZooKeeperWatcher for a major compaction.
 There the conf of store is directly passed into the ZookeeperWatcher, this 
 conf is a CompoundConfiguration.
 In ZookeeperWatcher, this conf is converted into a Configuration by new 
 Configuration(compoundConfiguration), and the zk-related conf are lost, only 
 default settings are used by the zookeeper.
 In the HMobStore, we should pass the region.getBaseConf() which is a 
 Configuration to the ZookeeperWatcher.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-12027) The ZooKeeperWatcher in HMobStore only uses the default conf

[
https://issues.apache.org/jira/browse/HBASE-12027?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14140324#comment-14140324
]

Hadoop QA commented on HBASE-12027:
---

{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12669978/HBASE-12027.diff
against trunk revision .
ATTACHMENT ID: 12669978

{color:green}+1 @author{color}. The patch does not contain any @author
tags.

{color:red}-1 patch{color}. The patch command could not apply the patch.

Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/10995//console

This message is automatically generated.

The ZooKeeperWatcher in HMobStore only uses the default conf

Key: HBASE-12027
URL: https://issues.apache.org/jira/browse/HBASE-12027
Project: HBase
Issue Type: Bug
Affects Versions: hbase-11339
Reporter: Jingcheng Du
Assignee: Jingcheng Du
Fix For: hbase-11339

Attachments: HBASE-12027.diff

In HMobStore we need to create a ZooKeeperWatcher for a major compaction.
There the conf of store is directly passed into the ZookeeperWatcher, this
conf is a CompoundConfiguration.
In ZookeeperWatcher, this conf is converted into a Configuration by new
Configuration(compoundConfiguration), and the zk-related conf are lost, only
default settings are used by the zookeeper.
In the HMobStore, we should pass the region.getBaseConf() which is a
Configuration to the ZookeeperWatcher.

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-11990) Make setting the start and stop row for a specific prefix easier


[ 
https://issues.apache.org/jira/browse/HBASE-11990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14140404#comment-14140404
 ] 

Hadoop QA commented on HBASE-11990:
---

{color:red}-1 overall{color}.  Here are the results of testing the latest 
attachment 
  
http://issues.apache.org/jira/secure/attachment/12669970/HBASE-11990-20140919-v8.patch
  against trunk revision .
  ATTACHMENT ID: 12669970

{color:green}+1 @author{color}.  The patch does not contain any @author 
tags.

{color:red}-1 tests included{color}.  The patch doesn't appear to include 
any new or modified tests.
Please justify why no new tests are needed for this 
patch.
Also please list what manual steps were performed to 
verify this patch.

{color:green}+1 javac{color}.  The applied patch does not increase the 
total number of javac compiler warnings.

{color:green}+1 javac{color}.  The applied patch does not increase the 
total number of javac compiler warnings.

{color:red}-1 javadoc{color}.  The javadoc tool appears to have generated 2 
warning messages.

{color:green}+1 findbugs{color}.  The patch does not introduce any new 
Findbugs (version 2.0.3) warnings.

{color:green}+1 release audit{color}.  The applied patch does not increase 
the total number of release audit warnings.

{color:green}+1 lineLengths{color}.  The patch does not introduce lines 
longer than 100

  {color:green}+1 site{color}.  The mvn site goal succeeds with this patch.

 {color:red}-1 core tests{color}.  The patch failed these unit tests:
   
org.apache.hadoop.hbase.replication.regionserver.TestRegionReplicaReplicationEndpoint
  org.apache.hadoop.hbase.mapreduce.TestImportExport
  org.apache.hadoop.hbase.util.TestProcessBasedCluster

 {color:red}-1 core zombie tests{color}.  There are 1 zombie test(s):   
at org.apache.hadoop.hbase.client.TestHCM.testClusterStatus(TestHCM.java:251)

Test results: 
https://builds.apache.org/job/PreCommit-HBASE-Build/10994//testReport/
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/10994//artifact/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/10994//artifact/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/10994//artifact/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/10994//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/10994//artifact/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/10994//artifact/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/10994//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/10994//artifact/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/10994//artifact/patchprocess/newPatchFindbugsWarningshbase-common.html
Console output: 
https://builds.apache.org/job/PreCommit-HBASE-Build/10994//console

This message is automatically generated.

 Make setting the start and stop row for a specific prefix easier
 

 Key: HBASE-11990
 URL: https://issues.apache.org/jira/browse/HBASE-11990
 Project: HBase
  Issue Type: New Feature
  Components: Client
Reporter: Niels Basjes
 Attachments: 11990v4.txt, HBASE-11990-20140916-v2.patch, 
 HBASE-11990-20140916-v3.patch, HBASE-11990-20140916-v5.patch, 
 HBASE-11990-20140916-v6.patch, HBASE-11990-20140916.patch, 
 HBASE-11990-20140917-v7.patch, HBASE-11990-20140919-v8.patch


 If you want to set a scan from your application to scan for a specific row 
 prefix this is actually quite hard.
 As described in several places you can set the startRow to the prefix; yet 
 the stopRow should be set to the prefix '+1'
 If the prefix 'ASCII' put into a byte[] then this is easy because you can 
 simply increment the last byte of the array. 
 But if your application uses real binary rowids you may run into the scenario 
 that your prefix is something like 
 {code}{ 0x12, 0x23, 0xFF, 0xFF }{code} Then the increment should be {code}{ 
 0x12, 0x24 }{code}
 I have prepared a proposed patch that makes setting these values correctly a 
 lot easier.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-11879) Change TableInputFormatBase to take interface arguments

2014-09-19 Thread Solomon Duskis (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-11879?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Solomon Duskis updated HBASE-11879:
---
Attachment: HBASE_11879_v1.patch

[~stack]'s comments were incorporated.  Good suggestions.

 Change TableInputFormatBase to take interface arguments
 ---

 Key: HBASE-11879
 URL: https://issues.apache.org/jira/browse/HBASE-11879
 Project: HBase
  Issue Type: Improvement
Reporter: Carter
Assignee: Solomon Duskis
 Fix For: 0.99.1

 Attachments: HBASE_11879.patch, HBASE_11879_v1.patch


 As part of the ongoing interface abstraction work, I'm now investigating 
 {{TableInputFormatBase}}, which has two methods that break encapsulation:
 {code}
 protected HTable getHTable();
 protected void setHTable(HTable table);
 {code}
 While these are protected methods, the base @InterfaceAudience.Public is 
 abstract, meaning that it supports extension by user code.
 I propose deprecating these two methods and replacing them with these four, 
 once the Table interface is merged:
 {code}
 protected Table getTable();
 protected void setTable(Table table);
 protected RegionLocator getRegionLocator();
 protected void setRegionLocator(RegionLocator regionLocator);
 {code}
 Since users will frequently call {{setTable}} and {{setRegionLocator}} 
 together, it probably also makes sense to add the following convenience 
 method:
 {code}
 protected void setTableAndRegionLocator(Table table, RegionLocator 
 regionLocator);
 {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (HBASE-12028) Abort the RegionServer, when one of it's handler threads die

2014-09-19 Thread Sudarshan Kadambi (JIRA)

Sudarshan Kadambi created HBASE-12028:
-

 Summary: Abort the RegionServer, when one of it's handler threads 
die
 Key: HBASE-12028
 URL: https://issues.apache.org/jira/browse/HBASE-12028
 Project: HBase
  Issue Type: Bug
  Components: regionserver
Reporter: Sudarshan Kadambi


Over in HBase-11813, a user identified an issue where in all the RPC handler 
threads would exit with StackOverflow errors due to an unchecked 
recursion-terminating condition. Our clusters demonstrated the same trace. 
While the patch posted for HBASE-11813 got our clusters to be merry again, the 
breakdown surfaced some larger issues.

When the RegionServer had all it's RPC handler threads dead, it continued to 
have regions assigned it. Clearly, it wouldn't be able to serve reads and 
writes on those regions. A second issue was that when a user tried to disable 
or drop a table, the master would try to communicate to the regionserver for 
region unassignment. Since the same handler threads seem to be used for master 
- RS communication as well, the master ended up hanging on the RS 
indefinitely. Eventually, the master stopped responding to all table 
meta-operations.

A handler thread should never exit, and if it does, it seems like the more 
prudent thing to do would be for the RS to abort. This way, atleast recovery 
can be undertaken and the regions could be reassigned elsewhere. I also think 
that the master-RS communication should get its own exclusive threadpool, but 
I'll wait until this issue has been sufficiently discussed before opening an 
issue ticket for that.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-12007) StochasticBalancer should avoid putting user regions on master

2014-09-19 Thread Jimmy Xiang (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-12007?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14140753#comment-14140753
 ] 

Jimmy Xiang commented on HBASE-12007:
-

bq. We have two mechanisms for distributing regions on a cluster?
Regions on master are handled with balanceMasterRegions which is internal to 
the balancer.
bq. In branch-1, master can carry regions? DIdn't we disable that for branch-1?
Yes, it still can, if configured. It doesn't host any region by default.
bq. Or is it not possible to undo M being a RS?
Very hard, and it is useful for most of use cases.
bq. We could assign to backup master?
Yes, that's right.

 StochasticBalancer should avoid putting user regions on master
 --

 Key: HBASE-12007
 URL: https://issues.apache.org/jira/browse/HBASE-12007
 Project: HBase
  Issue Type: Bug
Reporter: Jimmy Xiang
Assignee: Jimmy Xiang
 Attachments: hbase-12007.patch


 We should enhance how StochasticBalancer picks up a random server so that it 
 can avoid putting user regions on master, because regions on master are 
 handled differently in a separate method.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-12028) Abort the RegionServer, when one of it's handler threads die

2014-09-19 Thread Sudarshan Kadambi (JIRA)

[
https://issues.apache.org/jira/browse/HBASE-12028?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14140767#comment-14140767
]

Sudarshan Kadambi commented on HBASE-12028:
---

It's also the case that if the handler thread exited because of any
peculiarities in a given region (I'm still unclear about the root cause for
HBASE-11813), moving that region off by aborting the RS, could end up taking
down the entire cluster rather than keep it localized to a single RS.

Abort the RegionServer, when one of it's handler threads die

Key: HBASE-12028
URL: https://issues.apache.org/jira/browse/HBASE-12028
Project: HBase
Issue Type: Bug
Components: regionserver
Reporter: Sudarshan Kadambi

Over in HBase-11813, a user identified an issue where in all the RPC handler
threads would exit with StackOverflow errors due to an unchecked
recursion-terminating condition. Our clusters demonstrated the same trace.
While the patch posted for HBASE-11813 got our clusters to be merry again,
the breakdown surfaced some larger issues.
When the RegionServer had all it's RPC handler threads dead, it continued to
have regions assigned it. Clearly, it wouldn't be able to serve reads and
writes on those regions. A second issue was that when a user tried to disable
or drop a table, the master would try to communicate to the regionserver for
region unassignment. Since the same handler threads seem to be used for
master - RS communication as well, the master ended up hanging on the RS
indefinitely. Eventually, the master stopped responding to all table
meta-operations.
A handler thread should never exit, and if it does, it seems like the more
prudent thing to do would be for the RS to abort. This way, atleast recovery
can be undertaken and the regions could be reassigned elsewhere. I also think
that the master-RS communication should get its own exclusive threadpool,
but I'll wait until this issue has been sufficiently discussed before opening
an issue ticket for that.

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-11974) When a disabled table is scanned, NotServingRegionException is thrown instead of TableNotEnabledException


[ 
https://issues.apache.org/jira/browse/HBASE-11974?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14140766#comment-14140766
 ] 

Ted Yu commented on HBASE-11974:


There is no difference in this regard between patch v6 and v7.

 When a disabled table is scanned, NotServingRegionException is thrown instead 
 of TableNotEnabledException
 -

 Key: HBASE-11974
 URL: https://issues.apache.org/jira/browse/HBASE-11974
 Project: HBase
  Issue Type: Bug
Reporter: Ted Yu
Assignee: Ted Yu
 Attachments: 11974-test.patch, 11974-v1.txt, 11974-v2.txt, 
 11974-v3.txt, 11974-v4.txt, 11974-v5.txt, 11974-v6.txt, 11974-v7.txt


 When a disabled table is scanned, TableNotEnabledException should be thrown.
 However, currently NotServingRegionException is thrown.
 Thanks to Romil Choksi who discovered this problem.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-12005) Split/merge fails if master restarts before PONR

2014-09-19 Thread Jimmy Xiang (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-12005?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14140774#comment-14140774
 ] 

Jimmy Xiang commented on HBASE-12005:
-

Sure, will add a helper method to make the checking easy to understand.

bq. If either is null in below, it is a fatal condition?
It is not. If the master restarts, these region states are lost since they are 
not persisted before PONR.

bq.  yet you test them individually?
We can check them together. 

 Split/merge fails if master restarts before PONR
 

 Key: HBASE-12005
 URL: https://issues.apache.org/jira/browse/HBASE-12005
 Project: HBase
  Issue Type: Bug
Affects Versions: 2.0.0
Reporter: Jimmy Xiang
Assignee: Jimmy Xiang
 Attachments: hbase-12005.patch, hbase-12005_v1.1.patch


 This applies to RPC based assignment. The root cause is that we don't persist 
 the state of the new region(s). If we do persist it, we need to clean it up 
 if the split/merge fails.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (HBASE-12029) Use Table and RegionLocator in HTable.getRegionLocations()

2014-09-19 Thread Solomon Duskis (JIRA)

Solomon Duskis created HBASE-12029:
--

 Summary: Use Table and RegionLocator in 
HTable.getRegionLocations() 
 Key: HBASE-12029
 URL: https://issues.apache.org/jira/browse/HBASE-12029
 Project: HBase
  Issue Type: Bug
Affects Versions: 0.99.0, 2.0.0
Reporter: Solomon Duskis
Assignee: Solomon Duskis


HTable.getRegionLocations() is an internal deprecated API.  There are plenty of 
internal uses of the method, and it would be ideal to use the Table and 
RegionLocator interfaces instead of the HTable method.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-10602) Cleanup HTable public interface

2014-09-19 Thread Solomon Duskis (JIRA)

[
https://issues.apache.org/jira/browse/HBASE-10602?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14140800#comment-14140800
]

Solomon Duskis commented on HBASE-10602:

HTable.getRegionLocations() is an internal deprecated API that's very useful
internally. Switch the implementation to use Table and RegionLocator.

Cleanup HTable public interface
---

Key: HBASE-10602
URL: https://issues.apache.org/jira/browse/HBASE-10602
Project: HBase
Issue Type: Umbrella
Components: Client, Usability
Reporter: Nick Dimiduk
Assignee: Enis Soztutar
Priority: Blocker
Fix For: 0.99.1

Attachments: hbase-10602_v1.patch

HBASE-6580 replaced the preferred means of HTableInterface acquisition to the
HConnection#getTable factory methods. HBASE-9117 removes the HConnection
cache, placing the burden of responsible connection cleanup on whomever
acquires it.
The remaining HTable constructors use a Connection instance and manage their
own HConnection on the callers behalf. This is convenient but also a
surprising source of poor performance for anyone accustomed to the previous
connection caching behavior. I propose deprecating those remaining
constructors for 0.98/0.96 and removing them for 1.0.
While I'm at it, I suggest we pursue some API hygiene in general and convert
HTable into an interface. I'm sure there are method overloads for accepting
String/byte[]/TableName where just TableName is sufficient. Can that be done
for 1.0 as well?

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-11980) Change sync to hsync, remove unused InfoServer, and reference our httpserver instead of hadoops

2014-09-19 Thread Eric Charles (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-11980?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14140820#comment-14140820
 ] 

Eric Charles commented on HBASE-11980:
--

[~stack] patch looks good, build fine, but I fail running hbase on hadoop3. I 
suspect more a config issue on my side, but want to fix it before casting +1. 
Should be ok by this weekend.

 Change sync to hsync, remove unused InfoServer, and reference our httpserver 
 instead of hadoops
 ---

 Key: HBASE-11980
 URL: https://issues.apache.org/jira/browse/HBASE-11980
 Project: HBase
  Issue Type: Sub-task
Reporter: stack
Assignee: stack
 Fix For: 2.0.0, 0.99.1

 Attachments: 
 0001-HBASE-11980-Change-sync-to-hsync-remove-unused-InfoS.patch


 Subtask that fixes a few issues with our building against hadoop3.  Get it in.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-11405) Multiple invocations of hbck in parallel disables balancer permanently


[ 
https://issues.apache.org/jira/browse/HBASE-11405?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14140830#comment-14140830
 ] 

Ted Yu commented on HBASE-11405:


With rebased patch for branch-1:
{code}
Running org.apache.hadoop.hbase.util.TestHBaseFsck
Tests run: 44, Failures: 0, Errors: 0, Skipped: 1, Time elapsed: 334.241 sec
{code}

 Multiple invocations of hbck in parallel disables balancer permanently 
 ---

 Key: HBASE-11405
 URL: https://issues.apache.org/jira/browse/HBASE-11405
 Project: HBase
  Issue Type: Bug
  Components: Balancer, hbck
Affects Versions: 0.99.0
Reporter: bharath v
Assignee: bharath v
 Fix For: 2.0.0

 Attachments: 11405-v3.txt, 11405-v5.patch, 11405-v6.patch, 
 11405v7.patch, HBASE-11405-trunk-rebased.patch, HBASE-11405-trunk.patch, 
 HBASE-11405-trunk.patch.1, hbase-11405.rebase.140911.patch


 This is because of the following piece of code in hbck
 {code:borderStyle=solid}
   boolean oldBalancer = admin.setBalancerRunning(false, true);
 try {
   onlineConsistencyRepair();
 }
 finally {
   admin.setBalancerRunning(oldBalancer, false);
 }
 {code}
 Newer invocations set oldBalancer to false as it was disabled by previous 
 invocations and this disables balancer permanently unless its manually turned 
 on by the user. Easy to reproduce, just run hbck 100 times in a loop in 2 
 different sessions and you can see that balancer is set to false in the 
 HMaster logs.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-11405) Multiple invocations of hbck in parallel disables balancer permanently


 [ 
https://issues.apache.org/jira/browse/HBASE-11405?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ted Yu updated HBASE-11405:
---
Attachment: 11405-1.0.txt

 Multiple invocations of hbck in parallel disables balancer permanently 
 ---

 Key: HBASE-11405
 URL: https://issues.apache.org/jira/browse/HBASE-11405
 Project: HBase
  Issue Type: Bug
  Components: Balancer, hbck
Affects Versions: 0.99.0
Reporter: bharath v
Assignee: bharath v
 Fix For: 2.0.0

 Attachments: 11405-1.0.txt, 11405-v3.txt, 11405-v5.patch, 
 11405-v6.patch, 11405v7.patch, HBASE-11405-trunk-rebased.patch, 
 HBASE-11405-trunk.patch, HBASE-11405-trunk.patch.1, 
 hbase-11405.rebase.140911.patch


 This is because of the following piece of code in hbck
 {code:borderStyle=solid}
   boolean oldBalancer = admin.setBalancerRunning(false, true);
 try {
   onlineConsistencyRepair();
 }
 finally {
   admin.setBalancerRunning(oldBalancer, false);
 }
 {code}
 Newer invocations set oldBalancer to false as it was disabled by previous 
 invocations and this disables balancer permanently unless its manually turned 
 on by the user. Easy to reproduce, just run hbck 100 times in a loop in 2 
 different sessions and you can see that balancer is set to false in the 
 HMaster logs.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-11960) Provide a sample to show how to use Thrift client authentication

2014-09-19 Thread Jimmy Xiang (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-11960?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jimmy Xiang updated HBASE-11960:

   Resolution: Fixed
Fix Version/s: 0.99.1
   2.0.0
 Hadoop Flags: Reviewed
   Status: Resolved  (was: Patch Available)

Thanks Dima for the review. The trailing whitespace is not introduced by this 
patch. I fixed it any way. README.txt is also updated. Integrated into branch 
0.98, 1, and master. 

 Provide a sample to show how to use Thrift client authentication
 

 Key: HBASE-11960
 URL: https://issues.apache.org/jira/browse/HBASE-11960
 Project: HBase
  Issue Type: Task
Reporter: Jimmy Xiang
Assignee: Jimmy Xiang
 Fix For: 2.0.0, 0.99.1

 Attachments: hbase-11960.patch


 Now Thrift server can authenticate clients. However, many people asked how to 
 use it. Although we have some info in the refguide, it doesn't mention how to 
 change the client to turn it on. A sample should help.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (HBASE-11405) Multiple invocations of hbck in parallel disables balancer permanently


 [ 
https://issues.apache.org/jira/browse/HBASE-11405?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ted Yu resolved HBASE-11405.

   Resolution: Fixed
Fix Version/s: 0.99.1

Integrated to branch-1 again.

 Multiple invocations of hbck in parallel disables balancer permanently 
 ---

 Key: HBASE-11405
 URL: https://issues.apache.org/jira/browse/HBASE-11405
 Project: HBase
  Issue Type: Bug
  Components: Balancer, hbck
Affects Versions: 0.99.0
Reporter: bharath v
Assignee: bharath v
 Fix For: 2.0.0, 0.99.1

 Attachments: 11405-1.0.txt, 11405-v3.txt, 11405-v5.patch, 
 11405-v6.patch, 11405v7.patch, HBASE-11405-trunk-rebased.patch, 
 HBASE-11405-trunk.patch, HBASE-11405-trunk.patch.1, 
 hbase-11405.rebase.140911.patch


 This is because of the following piece of code in hbck
 {code:borderStyle=solid}
   boolean oldBalancer = admin.setBalancerRunning(false, true);
 try {
   onlineConsistencyRepair();
 }
 finally {
   admin.setBalancerRunning(oldBalancer, false);
 }
 {code}
 Newer invocations set oldBalancer to false as it was disabled by previous 
 invocations and this disables balancer permanently unless its manually turned 
 on by the user. Easy to reproduce, just run hbck 100 times in a loop in 2 
 different sessions and you can see that balancer is set to false in the 
 HMaster logs.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-12023) HRegion.applyFamilyMapToMemstore creates too many iterator objects...

2014-09-19 Thread Vladimir Rodionov (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-12023?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14140854#comment-14140854
 ] 

Vladimir Rodionov commented on HBASE-12023:
---

{quote}
-1 core tests. The patch failed these unit tests:
org.apache.hadoop.hbase.regionserver.TestEncryptionKeyRotation
org.apache.hadoop.hbase.mapreduce.TestImportExport
org.apache.hadoop.hbase.util.TestProcessBasedCluster
{quote}

TestEncryptionKeyRotation and TestProcessBasedCluster have some transient 
issues - they PASSED in my local environment. TestImportExport fails constantly 
with and without patch. Let me know what else needs to be done to resolve this 
JIRA?

 HRegion.applyFamilyMapToMemstore creates too many iterator objects...
 -

 Key: HBASE-12023
 URL: https://issues.apache.org/jira/browse/HBASE-12023
 Project: HBase
  Issue Type: Bug
Affects Versions: 0.98.5, 0.94.23
Reporter: Vladimir Rodionov
Assignee: Vladimir Rodionov
Priority: Minor
 Attachments: HBASE-12023-0.94.patch, HBASE-12023.patch, 
 applyFamilyMapToMemstore (1).tiff


 for ... loop (creates iterator) inside another for loop. Produces a lot of 
 garbage. See attached picture.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-12023) HRegion.applyFamilyMapToMemstore creates too many iterator objects...

2014-09-19 Thread Andrew Purtell (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-12023?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14140879#comment-14140879
 ] 

Andrew Purtell commented on HBASE-12023:


We can proceed. Let me commit this now.

 HRegion.applyFamilyMapToMemstore creates too many iterator objects...
 -

 Key: HBASE-12023
 URL: https://issues.apache.org/jira/browse/HBASE-12023
 Project: HBase
  Issue Type: Bug
Affects Versions: 0.98.5, 0.94.23
Reporter: Vladimir Rodionov
Assignee: Vladimir Rodionov
Priority: Minor
 Fix For: 0.94.24

 Attachments: HBASE-12023-0.94.patch, HBASE-12023.patch, 
 applyFamilyMapToMemstore (1).tiff


 for ... loop (creates iterator) inside another for loop. Produces a lot of 
 garbage. See attached picture.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-12023) HRegion.applyFamilyMapToMemstore creates too many iterator objects...


 [ 
https://issues.apache.org/jira/browse/HBASE-12023?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lars Hofhansl updated HBASE-12023:
--
Fix Version/s: 0.94.24

 HRegion.applyFamilyMapToMemstore creates too many iterator objects...
 -

 Key: HBASE-12023
 URL: https://issues.apache.org/jira/browse/HBASE-12023
 Project: HBase
  Issue Type: Bug
Affects Versions: 0.98.5, 0.94.23
Reporter: Vladimir Rodionov
Assignee: Vladimir Rodionov
Priority: Minor
 Fix For: 0.94.24

 Attachments: HBASE-12023-0.94.patch, HBASE-12023.patch, 
 applyFamilyMapToMemstore (1).tiff


 for ... loop (creates iterator) inside another for loop. Produces a lot of 
 garbage. See attached picture.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-12023) HRegion.applyFamilyMapToMemstore creates too many iterator objects...


[ 
https://issues.apache.org/jira/browse/HBASE-12023?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14140878#comment-14140878
 ] 

Lars Hofhansl commented on HBASE-12023:
---

What about my comment about not being able to guarantee that the lists passed 
around are not guaranteed to be random accessible?
Not a problem, because they just happen to be (in Mutation they are ArrayLists)?

 HRegion.applyFamilyMapToMemstore creates too many iterator objects...
 -

 Key: HBASE-12023
 URL: https://issues.apache.org/jira/browse/HBASE-12023
 Project: HBase
  Issue Type: Bug
Affects Versions: 0.98.5, 0.94.23
Reporter: Vladimir Rodionov
Assignee: Vladimir Rodionov
Priority: Minor
 Fix For: 0.94.24

 Attachments: HBASE-12023-0.94.patch, HBASE-12023.patch, 
 applyFamilyMapToMemstore (1).tiff


 for ... loop (creates iterator) inside another for loop. Produces a lot of 
 garbage. See attached picture.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-12020) String formatting on each RPC Invoke


[ 
https://issues.apache.org/jira/browse/HBASE-12020?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14140880#comment-14140880
 ] 

Lars Hofhansl commented on HBASE-12020:
---

+1

Will commit in a bit.

 String formatting on each RPC Invoke
 

 Key: HBASE-12020
 URL: https://issues.apache.org/jira/browse/HBASE-12020
 Project: HBase
  Issue Type: Bug
  Components: IPC/RPC
Affects Versions: 0.94.23
Reporter: Vladimir Rodionov
Assignee: Vladimir Rodionov
Priority: Minor
 Fix For: 0.94.24

 Attachments: HBASE-12020.patch


 ExecRPCInvoker.inoker -  debug call needs to be wrapped by a disabled check. 
 For each invocation, the Bytes.toStringBinary is being called.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-12023) HRegion.applyFamilyMapToMemstore creates too many iterator objects...

2014-09-19 Thread Andrew Purtell (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-12023?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14140881#comment-14140881
 ] 

Andrew Purtell commented on HBASE-12023:


JIRA comment race. Holding off commit.

 HRegion.applyFamilyMapToMemstore creates too many iterator objects...
 -

 Key: HBASE-12023
 URL: https://issues.apache.org/jira/browse/HBASE-12023
 Project: HBase
  Issue Type: Bug
Affects Versions: 0.98.5, 0.94.23
Reporter: Vladimir Rodionov
Assignee: Vladimir Rodionov
Priority: Minor
 Fix For: 0.94.24

 Attachments: HBASE-12023-0.94.patch, HBASE-12023.patch, 
 applyFamilyMapToMemstore (1).tiff


 for ... loop (creates iterator) inside another for loop. Produces a lot of 
 garbage. See attached picture.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-12022) Payloads on Failure attempt to serialize the byte[] into strings...


[ 
https://issues.apache.org/jira/browse/HBASE-12022?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14140882#comment-14140882
 ] 

Lars Hofhansl commented on HBASE-12022:
---

lgtm.

Will commit soon.

 Payloads on Failure attempt to serialize the byte[] into strings...
 ---

 Key: HBASE-12022
 URL: https://issues.apache.org/jira/browse/HBASE-12022
 Project: HBase
  Issue Type: Bug
  Components: IPC/RPC
Affects Versions: 0.94.23
Reporter: Vladimir Rodionov
Assignee: Vladimir Rodionov
Priority: Trivial
 Fix For: 0.94.24

 Attachments: HBASE-12022.patch


 HBaseServer$HandlerRun
 LOG.debug(getName(), call +call: error:  + e, e);
 errorClass = e.getClass().getName();
 error = StringUtils.stringifyException(e);
 Whether or not we are at a debug level, we will serialize the call... Our 
 call has a shit load of KVPairs as byte[]... These need to be at least 
 wrapped in LOG.isDebugEnabled blocks.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-11964) Improve spreading replication load from failed regionservers


[ 
https://issues.apache.org/jira/browse/HBASE-11964?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14140885#comment-14140885
 ] 

Lars Hofhansl commented on HBASE-11964:
---

Should we default to the new configs or document them (or do nothing here)?


 Improve spreading replication load from failed regionservers
 

 Key: HBASE-11964
 URL: https://issues.apache.org/jira/browse/HBASE-11964
 Project: HBase
  Issue Type: Sub-task
Reporter: Andrew Purtell
Assignee: Andrew Purtell
 Fix For: 2.0.0, 0.98.7, 0.94.24, 0.99.1

 Attachments: HBASE-11964-0.98.patch, HBASE-11964-0.98.patch, 
 HBASE-11964.patch, HBASE-11964.patch


 Improve replication source thread handling. Improve fanout when transferring 
 queues. Ensure replication sources terminate properly.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-11964) Improve spreading replication load from failed regionservers


 [ 
https://issues.apache.org/jira/browse/HBASE-11964?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lars Hofhansl updated HBASE-11964:
--
Fix Version/s: (was: 0.94.24)
   0.94.25

 Improve spreading replication load from failed regionservers
 

 Key: HBASE-11964
 URL: https://issues.apache.org/jira/browse/HBASE-11964
 Project: HBase
  Issue Type: Sub-task
Reporter: Andrew Purtell
Assignee: Andrew Purtell
 Fix For: 2.0.0, 0.98.7, 0.99.1, 0.94.25

 Attachments: HBASE-11964-0.98.patch, HBASE-11964-0.98.patch, 
 HBASE-11964.patch, HBASE-11964.patch


 Improve replication source thread handling. Improve fanout when transferring 
 queues. Ensure replication sources terminate properly.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-12019) hbase-daemon.sh overwrite HBASE_ROOT_LOGGER and HBASE_SECURITY_LOGGER variables


 [ 
https://issues.apache.org/jira/browse/HBASE-12019?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lars Hofhansl updated HBASE-12019:
--
Assignee: Sebastien Barrier

 hbase-daemon.sh overwrite HBASE_ROOT_LOGGER and HBASE_SECURITY_LOGGER 
 variables
 ---

 Key: HBASE-12019
 URL: https://issues.apache.org/jira/browse/HBASE-12019
 Project: HBase
  Issue Type: Bug
  Components: scripts
Affects Versions: 0.99.0, 0.94.23, 0.98.6
 Environment: Linux
Reporter: Sebastien Barrier
Assignee: Sebastien Barrier
Priority: Minor
  Labels: patch
 Fix For: 2.0.0, 0.98.7, 0.94.24, 0.99.1

 Attachments: HBASE-12019.patch


 hbase-env.sh is supposed to be used to set environment variables like 
 HBASE_ROOT_LOGGER and HBASE_SECURITY_LOGGER but hbase-daemon.sh overwrite the 
 export of thoses, the benefit of hbase-env.sh is lost.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-12019) hbase-daemon.sh overwrite HBASE_ROOT_LOGGER and HBASE_SECURITY_LOGGER variables


[ 
https://issues.apache.org/jira/browse/HBASE-12019?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14140886#comment-14140886
 ] 

Lars Hofhansl commented on HBASE-12019:
---

Added you as contributor and credited this to you [~sbarrier].

 hbase-daemon.sh overwrite HBASE_ROOT_LOGGER and HBASE_SECURITY_LOGGER 
 variables
 ---

 Key: HBASE-12019
 URL: https://issues.apache.org/jira/browse/HBASE-12019
 Project: HBase
  Issue Type: Bug
  Components: scripts
Affects Versions: 0.99.0, 0.94.23, 0.98.6
 Environment: Linux
Reporter: Sebastien Barrier
Assignee: Sebastien Barrier
Priority: Minor
  Labels: patch
 Fix For: 2.0.0, 0.98.7, 0.94.24, 0.99.1

 Attachments: HBASE-12019.patch


 hbase-env.sh is supposed to be used to set environment variables like 
 HBASE_ROOT_LOGGER and HBASE_SECURITY_LOGGER but hbase-daemon.sh overwrite the 
 export of thoses, the benefit of hbase-env.sh is lost.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-12023) HRegion.applyFamilyMapToMemstore creates too many iterator objects...

2014-09-19 Thread Vladimir Rodionov (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-12023?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14140889#comment-14140889
 ] 

Vladimir Rodionov commented on HBASE-12023:
---

[~lhofhansl]:
{quote}
What about my comment about not being able to guarantee that the lists passed 
around are not guaranteed to be random accessible?
Not a problem, because they just happen to be (in Mutation they are ArrayLists)?
{quote}

Not sure I am following you here, Lars. List is random accessible, by default ( 
get(int index) API). As you mentioned already, in mutations we use ArrayList - 
not LinkedList. LinkedLIst is not efficient, but (hopefully) will never be used 
in HBase on a critical read/write paths. 

 HRegion.applyFamilyMapToMemstore creates too many iterator objects...
 -

 Key: HBASE-12023
 URL: https://issues.apache.org/jira/browse/HBASE-12023
 Project: HBase
  Issue Type: Bug
Affects Versions: 0.98.5, 0.94.23
Reporter: Vladimir Rodionov
Assignee: Vladimir Rodionov
Priority: Minor
 Fix For: 0.94.24

 Attachments: HBASE-12023-0.94.patch, HBASE-12023.patch, 
 applyFamilyMapToMemstore (1).tiff


 for ... loop (creates iterator) inside another for loop. Produces a lot of 
 garbage. See attached picture.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-12020) String formatting on each RPC Invoke


 [ 
https://issues.apache.org/jira/browse/HBASE-12020?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lars Hofhansl updated HBASE-12020:
--
  Resolution: Fixed
Hadoop Flags: Reviewed
  Status: Resolved  (was: Patch Available)

Committed to 0.94. Thanks [~vrodionov].

 String formatting on each RPC Invoke
 

 Key: HBASE-12020
 URL: https://issues.apache.org/jira/browse/HBASE-12020
 Project: HBase
  Issue Type: Bug
  Components: IPC/RPC
Affects Versions: 0.94.23
Reporter: Vladimir Rodionov
Assignee: Vladimir Rodionov
Priority: Minor
 Fix For: 0.94.24

 Attachments: HBASE-12020.patch


 ExecRPCInvoker.inoker -  debug call needs to be wrapped by a disabled check. 
 For each invocation, the Bytes.toStringBinary is being called.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-12022) Payloads on Failure attempt to serialize the byte[] into strings.


 [ 
https://issues.apache.org/jira/browse/HBASE-12022?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lars Hofhansl updated HBASE-12022:
--
Summary: Payloads on Failure attempt to serialize the byte[] into strings.  
(was: Payloads on Failure attempt to serialize the byte[] into strings...)

 Payloads on Failure attempt to serialize the byte[] into strings.
 -

 Key: HBASE-12022
 URL: https://issues.apache.org/jira/browse/HBASE-12022
 Project: HBase
  Issue Type: Bug
  Components: IPC/RPC
Affects Versions: 0.94.23
Reporter: Vladimir Rodionov
Assignee: Vladimir Rodionov
Priority: Trivial
 Fix For: 0.94.24

 Attachments: HBASE-12022.patch


 HBaseServer$HandlerRun
 LOG.debug(getName(), call +call: error:  + e, e);
 errorClass = e.getClass().getName();
 error = StringUtils.stringifyException(e);
 Whether or not we are at a debug level, we will serialize the call... Our 
 call has a shit load of KVPairs as byte[]... These need to be at least 
 wrapped in LOG.isDebugEnabled blocks.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-12022) Payloads on Failure attempt to serialize the byte[] into strings.


 [ 
https://issues.apache.org/jira/browse/HBASE-12022?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lars Hofhansl updated HBASE-12022:
--
  Resolution: Fixed
Hadoop Flags: Reviewed
  Status: Resolved  (was: Patch Available)

Committed. Thanks [~vrodionov].

 Payloads on Failure attempt to serialize the byte[] into strings.
 -

 Key: HBASE-12022
 URL: https://issues.apache.org/jira/browse/HBASE-12022
 Project: HBase
  Issue Type: Bug
  Components: IPC/RPC
Affects Versions: 0.94.23
Reporter: Vladimir Rodionov
Assignee: Vladimir Rodionov
Priority: Trivial
 Fix For: 0.94.24

 Attachments: HBASE-12022.patch


 HBaseServer$HandlerRun
 LOG.debug(getName(), call +call: error:  + e, e);
 errorClass = e.getClass().getName();
 error = StringUtils.stringifyException(e);
 Whether or not we are at a debug level, we will serialize the call... Our 
 call has a shit load of KVPairs as byte[]... These need to be at least 
 wrapped in LOG.isDebugEnabled blocks.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-11980) Change sync to hsync, remove unused InfoServer, and reference our httpserver instead of hadoops


[ 
https://issues.apache.org/jira/browse/HBASE-11980?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14140902#comment-14140902
 ] 

stack commented on HBASE-11980:
---

[~echarles] Yeah, parent issue is about fixing the failed compile of h3.  I was 
hoping to commit this issue because it removes an unused file, fixes a wrong 
reference to httpserver, moves us to ordained way of calling FS sync, and just 
generally moves the h3 project further along.  What you think boss?

 Change sync to hsync, remove unused InfoServer, and reference our httpserver 
 instead of hadoops
 ---

 Key: HBASE-11980
 URL: https://issues.apache.org/jira/browse/HBASE-11980
 Project: HBase
  Issue Type: Sub-task
Reporter: stack
Assignee: stack
 Fix For: 2.0.0, 0.99.1

 Attachments: 
 0001-HBASE-11980-Change-sync-to-hsync-remove-unused-InfoS.patch


 Subtask that fixes a few issues with our building against hadoop3.  Get it in.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-12023) HRegion.applyFamilyMapToMemstore creates too many iterator objects...


[ 
https://issues.apache.org/jira/browse/HBASE-12023?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14140908#comment-14140908
 ] 

Lars Hofhansl commented on HBASE-12023:
---

I mean random accessible as in tagged with the RandomAccess interface. 
LinkedList has a get(index) but using that here would be very inefficient.

bq. (hopefully) will never be used in HBase
That's the worry I have. I've seen this happening in projects where folks 
expect an ArrayList but sometimes get LinkedLists and then the performance 
tanks.

Or in other words we now created an implicit dependency on the Lists here being 
ArrayList.

The correct way would be to change the definition in Mutation to 
{code}
  protected NavigableMapbyte [], ArrayListCell familyMap =
new TreeMapbyte [], ArrayListCell(Bytes.BYTES_COMPARATOR);
{code}

Then we're structurally guaranteed that we have ArrayLists and made the 
dependency explicit.
Of course now that will break binary compatibility and so we cannot just change 
it.

Anyway... If I'm the only one worrying about this, let's commit.


 HRegion.applyFamilyMapToMemstore creates too many iterator objects...
 -

 Key: HBASE-12023
 URL: https://issues.apache.org/jira/browse/HBASE-12023
 Project: HBase
  Issue Type: Bug
Affects Versions: 0.98.5, 0.94.23
Reporter: Vladimir Rodionov
Assignee: Vladimir Rodionov
Priority: Minor
 Fix For: 0.94.24

 Attachments: HBASE-12023-0.94.patch, HBASE-12023.patch, 
 applyFamilyMapToMemstore (1).tiff


 for ... loop (creates iterator) inside another for loop. Produces a lot of 
 garbage. See attached picture.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-12005) Split/merge fails if master restarts before PONR


[ 
https://issues.apache.org/jira/browse/HBASE-12005?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14140909#comment-14140909
 ] 

stack commented on HBASE-12005:
---

bq. We can check them together.

You don't have to. Just add note that could be individually null?

+1 on helper method.  Then +1 on patch. Just go for it I'd say Jimmy.

 Split/merge fails if master restarts before PONR
 

 Key: HBASE-12005
 URL: https://issues.apache.org/jira/browse/HBASE-12005
 Project: HBase
  Issue Type: Bug
Affects Versions: 2.0.0
Reporter: Jimmy Xiang
Assignee: Jimmy Xiang
 Attachments: hbase-12005.patch, hbase-12005_v1.1.patch


 This applies to RPC based assignment. The root cause is that we don't persist 
 the state of the new region(s). If we do persist it, we need to clean it up 
 if the split/merge fails.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-10924) [region_mover]: Adjust region_mover script to retry unloading a server a configurable number of times in case of region splits/merges

[
https://issues.apache.org/jira/browse/HBASE-10924?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Lars Hofhansl updated HBASE-10924:
--
Fix Version/s: (was: 0.94.24)
0.94.25

[region_mover]: Adjust region_mover script to retry unloading a server a
configurable number of times in case of region splits/merges
-

Key: HBASE-10924
URL: https://issues.apache.org/jira/browse/HBASE-10924
Project: HBase
Issue Type: Bug
Components: Region Assignment
Affects Versions: 0.94.15
Reporter: Aleksandr Shulman
Assignee: Aleksandr Shulman
Labels: region_mover, rolling_upgrade
Fix For: 0.94.25

Attachments: HBASE-10924-0.94-v2.patch, HBASE-10924-0.94-v3.patch

Observed behavior:
In about 5% of cases, my rolling upgrade tests fail because of stuck regions
during a region server unload. My theory is that this occurs when region
assignment information changes between the time the region list is generated,
and the time when the region is to be moved.
An example of such a region information change is a split or merge.
Example:
Regionserver A has 100 regions (#0-#99). The balancer is turned off and the
regionmover script is called to unload this regionserver. The regionmover
script will generate the list of 100 regions to be moved and then proceed
down that list, moving the regions off in series. However, there is a region,
#84, that has split into two daughter regions while regions 0-83 were moved.
The script will be stuck trying to move #84, timeout, and then the failure
will bubble up (attempt 1 failed).
Proposed solution:
This specific failure mode should be caught and the region_mover script
should now attempt to move off all the regions. Now, it will have 16+1 (due
to split) regions to move. There is a good chance that it will be able to
move all 17 off without issues. However, should it encounter this same issue
(attempt 2 failed), it will retry again. This process will continue until the
maximum number of unload retry attempts has been reached.
This is not foolproof, but let's say for the sake of argument that 5% of
unload attempts hit this issue, then with a retry count of 3, it will reduce
the unload failure probability from 0.05 to 0.000125 (0.05^3).
Next steps:
I am looking for feedback on this approach. If it seems like a sensible
approach, I will create a strawman patch and test it.

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-12023) HRegion.applyFamilyMapToMemstore creates too many iterator objects...

2014-09-19 Thread Andrew Purtell (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-12023?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14140917#comment-14140917
 ] 

Andrew Purtell commented on HBASE-12023:


Can we add a test somewhere out of hot code that checks for ? extends ArrayList 
or similar and throws a runtime exception if not? That's not ideal like static 
type checking but would document the expectation and cause all kinds of issues 
if violated. Or an assertion, but asserts can be turned off.

 HRegion.applyFamilyMapToMemstore creates too many iterator objects...
 -

 Key: HBASE-12023
 URL: https://issues.apache.org/jira/browse/HBASE-12023
 Project: HBase
  Issue Type: Bug
Affects Versions: 0.98.5, 0.94.23
Reporter: Vladimir Rodionov
Assignee: Vladimir Rodionov
Priority: Minor
 Fix For: 0.94.24

 Attachments: HBASE-12023-0.94.patch, HBASE-12023.patch, 
 applyFamilyMapToMemstore (1).tiff


 for ... loop (creates iterator) inside another for loop. Produces a lot of 
 garbage. See attached picture.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-11957) Backport HBASE-5974 to 0.94


[ 
https://issues.apache.org/jira/browse/HBASE-11957?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14140921#comment-14140921
 ] 

Lars Hofhansl commented on HBASE-11957:
---

[~apurtell], [~stack], any opinions? Looks good to me. Would need to be sure 
that we maintain binary compatibility for coprocessors.

 Backport HBASE-5974 to 0.94
 ---

 Key: HBASE-11957
 URL: https://issues.apache.org/jira/browse/HBASE-11957
 Project: HBase
  Issue Type: Bug
Reporter: Liu Shaohui
Assignee: Liu Shaohui
Priority: Critical
 Fix For: 0.94.24

 Attachments: HBASE-5974-0.94-v1.diff, verify-test.patch


 HBASE-5974:Scanner retry behavior with RPC timeout on next() seems incorrect, 
 which cause data missing in hbase scan.
 I think we should fix it in 0.94.
 [~lhofhansl]



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-12023) HRegion.applyFamilyMapToMemstore creates too many iterator objects...


[ 
https://issues.apache.org/jira/browse/HBASE-12023?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14140928#comment-14140928
 ] 

Lars Hofhansl commented on HBASE-12023:
---

An assert is a good idea. We run tests with asserts on, so we'll catch it.
Can add at commit time. 

+1



 HRegion.applyFamilyMapToMemstore creates too many iterator objects...
 -

 Key: HBASE-12023
 URL: https://issues.apache.org/jira/browse/HBASE-12023
 Project: HBase
  Issue Type: Bug
Affects Versions: 0.98.5, 0.94.23
Reporter: Vladimir Rodionov
Assignee: Vladimir Rodionov
Priority: Minor
 Fix For: 0.94.24

 Attachments: HBASE-12023-0.94.patch, HBASE-12023.patch, 
 applyFamilyMapToMemstore (1).tiff


 for ... loop (creates iterator) inside another for loop. Produces a lot of 
 garbage. See attached picture.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-12028) Abort the RegionServer, when one of it's handler threads die

[
https://issues.apache.org/jira/browse/HBASE-12028?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14140931#comment-14140931
]

stack commented on HBASE-12028:
---

bq. I'm still unclear about the root cause for HBASE-11813

A query that returned thousands of empty results turned what was thought an
harmless recursion (triggered by empty result) pathological... it recursed so
much it overflowed the allocated stack.

You have a couple of points both that we are mortally wounded if we lose a
handler and that killing the RS could end up taking down the whole cluster.

Abort the RegionServer, when one of it's handler threads die

Key: HBASE-12028
URL: https://issues.apache.org/jira/browse/HBASE-12028
Project: HBase
Issue Type: Bug
Components: regionserver
Reporter: Sudarshan Kadambi

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-11405) Multiple invocations of hbck in parallel disables balancer permanently


[ 
https://issues.apache.org/jira/browse/HBASE-11405?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14140932#comment-14140932
 ] 

Lars Hofhansl commented on HBASE-11405:
---

Is this not an issue in 0.94 and 0.98?

 Multiple invocations of hbck in parallel disables balancer permanently 
 ---

 Key: HBASE-11405
 URL: https://issues.apache.org/jira/browse/HBASE-11405
 Project: HBase
  Issue Type: Bug
  Components: Balancer, hbck
Affects Versions: 0.99.0
Reporter: bharath v
Assignee: bharath v
 Fix For: 2.0.0, 0.99.1

 Attachments: 11405-1.0.txt, 11405-v3.txt, 11405-v5.patch, 
 11405-v6.patch, 11405v7.patch, HBASE-11405-trunk-rebased.patch, 
 HBASE-11405-trunk.patch, HBASE-11405-trunk.patch.1, 
 hbase-11405.rebase.140911.patch


 This is because of the following piece of code in hbck
 {code:borderStyle=solid}
   boolean oldBalancer = admin.setBalancerRunning(false, true);
 try {
   onlineConsistencyRepair();
 }
 finally {
   admin.setBalancerRunning(oldBalancer, false);
 }
 {code}
 Newer invocations set oldBalancer to false as it was disabled by previous 
 invocations and this disables balancer permanently unless its manually turned 
 on by the user. Easy to reproduce, just run hbck 100 times in a loop in 2 
 different sessions and you can see that balancer is set to false in the 
 HMaster logs.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-11405) Multiple invocations of hbck in parallel disables balancer permanently

2014-09-19 Thread Sean Busbey (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-11405?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14140937#comment-14140937
 ] 

Sean Busbey commented on HBASE-11405:
-

definitely present in both 0.94 and 0.98. Do you prefer a backport or known 
issues text? I can pull together either.

 Multiple invocations of hbck in parallel disables balancer permanently 
 ---

 Key: HBASE-11405
 URL: https://issues.apache.org/jira/browse/HBASE-11405
 Project: HBase
  Issue Type: Bug
  Components: Balancer, hbck
Affects Versions: 0.99.0
Reporter: bharath v
Assignee: bharath v
 Fix For: 2.0.0, 0.99.1

 Attachments: 11405-1.0.txt, 11405-v3.txt, 11405-v5.patch, 
 11405-v6.patch, 11405v7.patch, HBASE-11405-trunk-rebased.patch, 
 HBASE-11405-trunk.patch, HBASE-11405-trunk.patch.1, 
 hbase-11405.rebase.140911.patch


 This is because of the following piece of code in hbck
 {code:borderStyle=solid}
   boolean oldBalancer = admin.setBalancerRunning(false, true);
 try {
   onlineConsistencyRepair();
 }
 finally {
   admin.setBalancerRunning(oldBalancer, false);
 }
 {code}
 Newer invocations set oldBalancer to false as it was disabled by previous 
 invocations and this disables balancer permanently unless its manually turned 
 on by the user. Easy to reproduce, just run hbck 100 times in a loop in 2 
 different sessions and you can see that balancer is set to false in the 
 HMaster logs.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-11957) Backport HBASE-5974 to 0.94


[ 
https://issues.apache.org/jira/browse/HBASE-11957?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14140941#comment-14140941
 ] 

stack commented on HBASE-11957:
---

My opinion is its an important fix.  How you think it could break CP API?  I 
don't see it.

 Backport HBASE-5974 to 0.94
 ---

 Key: HBASE-11957
 URL: https://issues.apache.org/jira/browse/HBASE-11957
 Project: HBase
  Issue Type: Bug
Reporter: Liu Shaohui
Assignee: Liu Shaohui
Priority: Critical
 Fix For: 0.94.24

 Attachments: HBASE-5974-0.94-v1.diff, verify-test.patch


 HBASE-5974:Scanner retry behavior with RPC timeout on next() seems incorrect, 
 which cause data missing in hbase scan.
 I think we should fix it in 0.94.
 [~lhofhansl]



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-12007) StochasticBalancer should avoid putting user regions on master


[ 
https://issues.apache.org/jira/browse/HBASE-12007?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14140951#comment-14140951
 ] 

stack commented on HBASE-12007:
---

bq. We could assign to backup master?

But it is off by default in branch-1?

bq. Very hard, and it is useful for most of use cases.

But it is off in branch-1 (smile).  I think it fine that M can be a RS but that 
it is not on by default.

bq. Regions on master are handled with balanceMasterRegions which is internal 
to the balancer.
... ok, so only one way of distributing regions... the balancer?

Thanks for bringing me along [~jxiang]



 StochasticBalancer should avoid putting user regions on master
 --

 Key: HBASE-12007
 URL: https://issues.apache.org/jira/browse/HBASE-12007
 Project: HBase
  Issue Type: Bug
Reporter: Jimmy Xiang
Assignee: Jimmy Xiang
 Attachments: hbase-12007.patch


 We should enhance how StochasticBalancer picks up a random server so that it 
 can avoid putting user regions on master, because regions on master are 
 handled differently in a separate method.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-11405) Multiple invocations of hbck in parallel disables balancer permanently


[ 
https://issues.apache.org/jira/browse/HBASE-11405?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14140957#comment-14140957
 ] 

Ted Yu commented on HBASE-11405:


11405-1.0.txt applies cleanly on 0.98
[~apurtell]:
Do you want this ?

 Multiple invocations of hbck in parallel disables balancer permanently 
 ---

 Key: HBASE-11405
 URL: https://issues.apache.org/jira/browse/HBASE-11405
 Project: HBase
  Issue Type: Bug
  Components: Balancer, hbck
Affects Versions: 0.99.0
Reporter: bharath v
Assignee: bharath v
 Fix For: 2.0.0, 0.99.1

 Attachments: 11405-1.0.txt, 11405-v3.txt, 11405-v5.patch, 
 11405-v6.patch, 11405v7.patch, HBASE-11405-trunk-rebased.patch, 
 HBASE-11405-trunk.patch, HBASE-11405-trunk.patch.1, 
 hbase-11405.rebase.140911.patch


 This is because of the following piece of code in hbck
 {code:borderStyle=solid}
   boolean oldBalancer = admin.setBalancerRunning(false, true);
 try {
   onlineConsistencyRepair();
 }
 finally {
   admin.setBalancerRunning(oldBalancer, false);
 }
 {code}
 Newer invocations set oldBalancer to false as it was disabled by previous 
 invocations and this disables balancer permanently unless its manually turned 
 on by the user. Easy to reproduce, just run hbck 100 times in a loop in 2 
 different sessions and you can see that balancer is set to false in the 
 HMaster logs.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-11879) Change TableInputFormatBase to take interface arguments


[ 
https://issues.apache.org/jira/browse/HBASE-11879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14140960#comment-14140960
 ] 

stack commented on HBASE-11879:
---

+1 I like it.  Clean.

[~ndimiduk] You are our TIFB man.  What you think?

 Change TableInputFormatBase to take interface arguments
 ---

 Key: HBASE-11879
 URL: https://issues.apache.org/jira/browse/HBASE-11879
 Project: HBase
  Issue Type: Improvement
Reporter: Carter
Assignee: Solomon Duskis
 Fix For: 0.99.1

 Attachments: HBASE_11879.patch, HBASE_11879_v1.patch


 As part of the ongoing interface abstraction work, I'm now investigating 
 {{TableInputFormatBase}}, which has two methods that break encapsulation:
 {code}
 protected HTable getHTable();
 protected void setHTable(HTable table);
 {code}
 While these are protected methods, the base @InterfaceAudience.Public is 
 abstract, meaning that it supports extension by user code.
 I propose deprecating these two methods and replacing them with these four, 
 once the Table interface is merged:
 {code}
 protected Table getTable();
 protected void setTable(Table table);
 protected RegionLocator getRegionLocator();
 protected void setRegionLocator(RegionLocator regionLocator);
 {code}
 Since users will frequently call {{setTable}} and {{setRegionLocator}} 
 together, it probably also makes sense to add the following convenience 
 method:
 {code}
 protected void setTableAndRegionLocator(Table table, RegionLocator 
 regionLocator);
 {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-11957) Backport HBASE-5974 to 0.94


[ 
https://issues.apache.org/jira/browse/HBASE-11957?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14140964#comment-14140964
 ] 

Lars Hofhansl commented on HBASE-11957:
---

Thought maybe this:
{code}
+  final MapString, RegionScannerHolder scanners =
+new ConcurrentHashMapString, RegionScannerHolder();
{code}
But it's not public and a reference to it does not leak into the APIs.

Allright then. Going to commit. Thanks [~liushaohui].

 Backport HBASE-5974 to 0.94
 ---

 Key: HBASE-11957
 URL: https://issues.apache.org/jira/browse/HBASE-11957
 Project: HBase
  Issue Type: Bug
Reporter: Liu Shaohui
Assignee: Liu Shaohui
Priority: Critical
 Fix For: 0.94.24

 Attachments: HBASE-5974-0.94-v1.diff, verify-test.patch


 HBASE-5974:Scanner retry behavior with RPC timeout on next() seems incorrect, 
 which cause data missing in hbase scan.
 I think we should fix it in 0.94.
 [~lhofhansl]



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-11990) Make setting the start and stop row for a specific prefix easier

[
https://issues.apache.org/jira/browse/HBASE-11990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14140979#comment-14140979
]

stack commented on HBASE-11990:
---

Yeah, its a bit of a chicken and egg problem. There is the TestScan class but
just does the Scan itself. There are the filter tests in the filter packages?
There is TestFilterWithScanLimits? Add a test in here? On patch so far, it
looks good -- especially this consideration...
combineUserFilterAndRowPrefixFilter. Javadoc on the new method could do w/ an
example... carry-over the one you had previous? Good stuff [~nielsbasjes]

Make setting the start and stop row for a specific prefix easier

Key: HBASE-11990
URL: https://issues.apache.org/jira/browse/HBASE-11990
Project: HBase
Issue Type: New Feature
Components: Client
Reporter: Niels Basjes
Attachments: 11990v4.txt, HBASE-11990-20140916-v2.patch,
HBASE-11990-20140916-v3.patch, HBASE-11990-20140916-v5.patch,
HBASE-11990-20140916-v6.patch, HBASE-11990-20140916.patch,
HBASE-11990-20140917-v7.patch, HBASE-11990-20140919-v8.patch

If you want to set a scan from your application to scan for a specific row
prefix this is actually quite hard.
As described in several places you can set the startRow to the prefix; yet
the stopRow should be set to the prefix '+1'
If the prefix 'ASCII' put into a byte[] then this is easy because you can
simply increment the last byte of the array.
But if your application uses real binary rowids you may run into the scenario
that your prefix is something like
{code}{ 0x12, 0x23, 0xFF, 0xFF }{code} Then the increment should be {code}{
0x12, 0x24 }{code}
I have prepared a proposed patch that makes setting these values correctly a
lot easier.

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-12020) String formatting on each RPC Invoke


[ 
https://issues.apache.org/jira/browse/HBASE-12020?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14140987#comment-14140987
 ] 

Hudson commented on HBASE-12020:


SUCCESS: Integrated in HBase-0.94-security #523 (See 
[https://builds.apache.org/job/HBase-0.94-security/523/])
HBASE-12020 String formatting on each RPC Invoke. (Vladimir Rodionov) (larsh: 
rev cbc08ceed5f0b1bab4adc0aa37dc00495c249634)
* src/main/java/org/apache/hadoop/hbase/ipc/ExecRPCInvoker.java


 String formatting on each RPC Invoke
 

 Key: HBASE-12020
 URL: https://issues.apache.org/jira/browse/HBASE-12020
 Project: HBase
  Issue Type: Bug
  Components: IPC/RPC
Affects Versions: 0.94.23
Reporter: Vladimir Rodionov
Assignee: Vladimir Rodionov
Priority: Minor
 Fix For: 0.94.24

 Attachments: HBASE-12020.patch


 ExecRPCInvoker.inoker -  debug call needs to be wrapped by a disabled check. 
 For each invocation, the Bytes.toStringBinary is being called.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-12022) Payloads on Failure attempt to serialize the byte[] into strings.


[ 
https://issues.apache.org/jira/browse/HBASE-12022?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14140988#comment-14140988
 ] 

Hudson commented on HBASE-12022:


SUCCESS: Integrated in HBase-0.94-security #523 (See 
[https://builds.apache.org/job/HBase-0.94-security/523/])
HBASE-12022 Payloads on Failure attempt to serialize the byte[] into strings. 
(Vladimir Rodionov) (larsh: rev 0058e7aacee0a10d020590ce2c2485c53f54e270)
* src/main/java/org/apache/hadoop/hbase/ipc/HBaseServer.java


 Payloads on Failure attempt to serialize the byte[] into strings.
 -

 Key: HBASE-12022
 URL: https://issues.apache.org/jira/browse/HBASE-12022
 Project: HBase
  Issue Type: Bug
  Components: IPC/RPC
Affects Versions: 0.94.23
Reporter: Vladimir Rodionov
Assignee: Vladimir Rodionov
Priority: Trivial
 Fix For: 0.94.24

 Attachments: HBASE-12022.patch


 HBaseServer$HandlerRun
 LOG.debug(getName(), call +call: error:  + e, e);
 errorClass = e.getClass().getName();
 error = StringUtils.stringifyException(e);
 Whether or not we are at a debug level, we will serialize the call... Our 
 call has a shit load of KVPairs as byte[]... These need to be at least 
 wrapped in LOG.isDebugEnabled blocks.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-11957) Backport to 0.94 HBASE-5974 Scanner retry behavior with RPC timeout on next() seems incorrect


 [ 
https://issues.apache.org/jira/browse/HBASE-11957?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lars Hofhansl updated HBASE-11957:
--
Summary: Backport to 0.94 HBASE-5974 Scanner retry behavior with RPC 
timeout on next() seems incorrect  (was: Backport HBASE-5974 to 0.94)

 Backport to 0.94 HBASE-5974 Scanner retry behavior with RPC timeout on next() 
 seems incorrect
 -

 Key: HBASE-11957
 URL: https://issues.apache.org/jira/browse/HBASE-11957
 Project: HBase
  Issue Type: Bug
Reporter: Liu Shaohui
Assignee: Liu Shaohui
Priority: Critical
 Fix For: 0.94.24

 Attachments: HBASE-5974-0.94-v1.diff, verify-test.patch


 HBASE-5974:Scanner retry behavior with RPC timeout on next() seems incorrect, 
 which cause data missing in hbase scan.
 I think we should fix it in 0.94.
 [~lhofhansl]



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-11462) MetaTableAccessor shouldn't use ZooKeeeper

[
https://issues.apache.org/jira/browse/HBASE-11462?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

stack updated HBASE-11462:
--
Resolution: Fixed
Hadoop Flags: Reviewed
Status: Resolved (was: Patch Available)

Applied. Sorry it took a while to get in [~mantonov] Nice work.

MetaTableAccessor shouldn't use ZooKeeeper
--

Attachments: HBASE-11462.v4.patch, HBASE-11462.v4.patch,
HBASE-11462.v4.patch

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-12021) Hbase shell does not respect the HBASE_OPTS set by the user in console