date:20120508

[jira] [Commented] (HBASE-5828) TestLogRolling fails in 0.90 branch killing the test suite up on jenkins

2012-05-08 Thread ramkrishna.s.vasudevan (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-5828?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13271162#comment-13271162
 ] 

ramkrishna.s.vasudevan commented on HBASE-5828:
---

bq.Found the meta region and user region be deployed on different RS.It may 
cause the problem.
Can we always ensure the meta and user regions are always in the same server? 

> TestLogRolling fails in 0.90 branch killing the test suite up on jenkins
> 
>
> Key: HBASE-5828
> URL: https://issues.apache.org/jira/browse/HBASE-5828
> Project: HBase
>  Issue Type: Bug
>Reporter: stack
>Assignee: xufeng
> Attachments: TestLogRolling.java, 
> org.apache.hadoop.hbase.regionserver.wal.TestLogRolling-output.rar
>
>
> See recent 0.90 builds up on jenkins: 
> https://builds.apache.org/view/G-L/view/HBase/job/hbase-0.90/471/console

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5944) Convert rs, shutdown, and table dir content to pb

2012-05-08 Thread Hadoop QA (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-5944?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13271161#comment-13271161
 ] 

Hadoop QA commented on HBASE-5944:
--

-1 overall.  Here are the results of testing the latest attachment 
  http://issues.apache.org/jira/secure/attachment/12526107/5944v2.txt
  against trunk revision .

+1 @author.  The patch does not contain any @author tags.

+1 tests included.  The patch appears to include 3 new or modified tests.

+1 hadoop23.  The patch compiles against the hadoop 0.23.x profile.

+1 javadoc.  The javadoc tool did not generate any warning messages.

+1 javac.  The applied patch does not increase the total number of javac 
compiler warnings.

+1 findbugs.  The patch does not introduce any new Findbugs (version 1.3.9) 
warnings.

+1 release audit.  The applied patch does not increase the total number of 
release audit warnings.

 -1 core tests.  The patch failed these unit tests:
 

Test results: 
https://builds.apache.org/job/PreCommit-HBASE-Build/1809//testReport/
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/1809//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
Console output: 
https://builds.apache.org/job/PreCommit-HBASE-Build/1809//console

This message is automatically generated.

> Convert rs, shutdown, and table dir content to pb
> -
>
> Key: HBASE-5944
> URL: https://issues.apache.org/jira/browse/HBASE-5944
> Project: HBase
>  Issue Type: Task
>Reporter: stack
>Assignee: stack
> Attachments: 5944.txt, 5944v2.txt
>
>


--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5916) RS restart just before master intialization we make the cluster non operative

2012-05-08 Thread ramkrishna.s.vasudevan (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-5916?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13271160#comment-13271160
 ] 

ramkrishna.s.vasudevan commented on HBASE-5916:
---

Atleast i can be sure of the RS and master time clocks right?

> RS restart just before master intialization we make the cluster non operative
> -
>
> Key: HBASE-5916
> URL: https://issues.apache.org/jira/browse/HBASE-5916
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 0.92.1, 0.94.0
>Reporter: ramkrishna.s.vasudevan
>Assignee: ramkrishna.s.vasudevan
>Priority: Critical
> Fix For: 0.94.1
>
> Attachments: HBASE-5916_trunk.patch, HBASE-5916_trunk_1.patch, 
> HBASE-5916_trunk_1.patch, HBASE-5916_trunk_2.patch, HBASE-5916_trunk_3.patch
>
>
> Consider a case where my master is getting restarted.  RS that was alive when 
> the master restart started, gets restarted before the master initializes the 
> ServerShutDownHandler.
> {code}
> serverShutdownHandlerEnabled = true;
> {code}
> In this case when the RS tries to register with the master, the master will 
> try to expire the server but the server cannot be expired as still the 
> serverShutdownHandler is not enabled.
> This case may happen when i have only one RS gets restarted or all the RS 
> gets restarted at the same time.(before assignRootandMeta).
> {code}
> LOG.info(message);
>   if (existingServer.getStartcode() < serverName.getStartcode()) {
> LOG.info("Triggering server recovery; existingServer " +
>   existingServer + " looks stale, new server:" + serverName);
> expireServer(existingServer);
>   }
> {code}
> If another RS is brought up then the cluster comes back to normalcy.
> May be a very corner case.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5916) RS restart just before master intialization we make the cluster non operative

2012-05-08 Thread ramkrishna.s.vasudevan (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-5916?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13271158#comment-13271158
 ] 

ramkrishna.s.vasudevan commented on HBASE-5916:
---

bq.if a difference between master and namenode clocks
I had similar doubts in using time factor.  Let me check if something better is 
available.

> RS restart just before master intialization we make the cluster non operative
> -
>
> Key: HBASE-5916
> URL: https://issues.apache.org/jira/browse/HBASE-5916
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 0.92.1, 0.94.0
>Reporter: ramkrishna.s.vasudevan
>Assignee: ramkrishna.s.vasudevan
>Priority: Critical
> Fix For: 0.94.1
>
> Attachments: HBASE-5916_trunk.patch, HBASE-5916_trunk_1.patch, 
> HBASE-5916_trunk_1.patch, HBASE-5916_trunk_2.patch, HBASE-5916_trunk_3.patch
>
>
> Consider a case where my master is getting restarted.  RS that was alive when 
> the master restart started, gets restarted before the master initializes the 
> ServerShutDownHandler.
> {code}
> serverShutdownHandlerEnabled = true;
> {code}
> In this case when the RS tries to register with the master, the master will 
> try to expire the server but the server cannot be expired as still the 
> serverShutdownHandler is not enabled.
> This case may happen when i have only one RS gets restarted or all the RS 
> gets restarted at the same time.(before assignRootandMeta).
> {code}
> LOG.info(message);
>   if (existingServer.getStartcode() < serverName.getStartcode()) {
> LOG.info("Triggering server recovery; existingServer " +
>   existingServer + " looks stale, new server:" + serverName);
> expireServer(existingServer);
>   }
> {code}
> If another RS is brought up then the cluster comes back to normalcy.
> May be a very corner case.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5963) ClassCastException: FileSystem$Cache$ClientFinalizer cannot be cast to Thread

2012-05-08 Thread Zhihong Yu (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-5963?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Zhihong Yu updated HBASE-5963:
--

Attachment: 5963-v3.txt

Patch v3 incorporates Andy's feedback about using 
ShutdownHookManager.addShutdownHook

> ClassCastException: FileSystem$Cache$ClientFinalizer cannot be cast to Thread
> -
>
> Key: HBASE-5963
> URL: https://issues.apache.org/jira/browse/HBASE-5963
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 0.94.0, 0.96.0
> Environment: Bigtop 0.4.0-incubating-SNAPSHOT, Hadoop 
> 2.0.0-alpha-SNAPSHOT, HBase 0.94.0-SNAPSHOT
>Reporter: Andrew Purtell
>Assignee: Andrew Purtell
> Attachments: 5963-v2.txt, 5963-v3.txt, HBASE-5963.patch, 
> HBASE-5963.patch
>
>
> {code}
> 12/05/08 19:49:26 INFO regionserver.HRegionServer: STOPPED: Failed 
> initialization
> Exception in thread "main" java.lang.ClassCastException: 
> org.apache.hadoop.fs.FileSystem$Cache$ClientFinalizer cannot be cast to 
> java.lang.Thread
>   at 
> org.apache.hadoop.hbase.regionserver.ShutdownHook.suppressHdfsShutdownHook(ShutdownHook.java:181)
>   at 
> org.apache.hadoop.hbase.regionserver.ShutdownHook.install(ShutdownHook.java:82)
>   at 
> org.apache.hadoop.hbase.regionserver.HRegionServer.startRegionServer(HRegionServer.java:3601)
>   at 
> org.apache.hadoop.hbase.regionserver.HRegionServer.startRegionServer(HRegionServer.java:3585)
>   at 
> org.apache.hadoop.hbase.regionserver.HRegionServerCommandLine.start(HRegionServerCommandLine.java:61)
>   at 
> org.apache.hadoop.hbase.regionserver.HRegionServerCommandLine.run(HRegionServerCommandLine.java:75)
>   at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
>   at 
> org.apache.hadoop.hbase.util.ServerCommandLine.doMain(ServerCommandLine.java:76)
>   at 
> org.apache.hadoop.hbase.regionserver.HRegionServer.main(HRegionServer.java:3645)
> {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5970) Improve the AssignmentManager#updateTimer and speed up handling opened event

2012-05-08 Thread Zhihong Yu (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-5970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13271153#comment-13271153
 ] 

Zhihong Yu commented on HBASE-5970:
---

The above assumption is reasonable.

> Improve the AssignmentManager#updateTimer and speed up handling opened event
> 
>
> Key: HBASE-5970
> URL: https://issues.apache.org/jira/browse/HBASE-5970
> Project: HBase
>  Issue Type: Improvement
>  Components: master
>Reporter: chunhui shen
>Assignee: chunhui shen
> Attachments: HBASE-5970.patch
>
>
> We found handing opened event very slow in the environment with lots of 
> regions.
> The problem is the slow AssignmentManager#updateTimer.
> We do the test for bulk assigning 10w (i.e. 100k) regions, the whole process 
> of bulk assigning took 1 hours.
> 2012-05-06 20:31:49,201 INFO 
> org.apache.hadoop.hbase.master.AssignmentManager: Bulk assigning 10 
> region(s) round-robin across 5 server(s)
> 2012-05-06 21:26:32,103 INFO 
> org.apache.hadoop.hbase.master.AssignmentManager: Bulk assigning done
> I think we could do the improvement for the AssignmentManager#updateTimer: 
> Make a thread do this work.
> After the improvement, it took only 4.5mins
> 2012-05-07 11:03:36,581 INFO 
> org.apache.hadoop.hbase.master.AssignmentManager: Bulk assigning 10 
> region(s) across 5 server(s), retainAssignment=true 
> 2012-05-07 11:07:57,073 INFO 
> org.apache.hadoop.hbase.master.AssignmentManager: Bulk assigning done 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5828) TestLogRolling fails in 0.90 branch killing the test suite up on jenkins

2012-05-08 Thread xufeng (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-5828?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13271146#comment-13271146
 ] 

xufeng commented on HBASE-5828:
---

@Rama
>>Do you think doing as in trunk adding @before and @after will solve this 
>>problem?
I think yes.In @Before,It will create a new cluster,new datanodes,new pipeline.

>>Because the way the pipeline is formed may not be in our control right?
Yes, but in this problem testLogRollOnPipelineRestart kill the datanodes in 
pipeline,So I think that this is a testcase problem.

> TestLogRolling fails in 0.90 branch killing the test suite up on jenkins
> 
>
> Key: HBASE-5828
> URL: https://issues.apache.org/jira/browse/HBASE-5828
> Project: HBase
>  Issue Type: Bug
>Reporter: stack
>Assignee: xufeng
> Attachments: TestLogRolling.java, 
> org.apache.hadoop.hbase.regionserver.wal.TestLogRolling-output.rar
>
>
> See recent 0.90 builds up on jenkins: 
> https://builds.apache.org/view/G-L/view/HBase/job/hbase-0.90/471/console

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5965) Move replication znodes to pb

2012-05-08 Thread stack (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-5965?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stack updated HBASE-5965:
-

Attachment: 5965.txt

A start.  Not finished.

> Move replication znodes to pb
> -
>
> Key: HBASE-5965
> URL: https://issues.apache.org/jira/browse/HBASE-5965
> Project: HBase
>  Issue Type: Task
>Reporter: stack
> Attachments: 5965.txt
>
>


--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5951) Combining ColumnPrefixFilters using filterlist.

2012-05-08 Thread ramkrishna.s.vasudevan (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-5951?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13271137#comment-13271137
 ] 

ramkrishna.s.vasudevan commented on HBASE-5951:
---

bq.even though it wont work in sql.
Sorry i missed it.


> Combining ColumnPrefixFilters using filterlist.
> ---
>
> Key: HBASE-5951
> URL: https://issues.apache.org/jira/browse/HBASE-5951
> Project: HBase
>  Issue Type: Bug
>  Components: filters
>Affects Versions: 0.92.1
> Environment: Combining multiple ColumnPrefixFilters using filterlist 
> does not return exact result.
>Reporter: Davis Abraham
>
> FilterList listOfFilters = new FilterList (FilterList.Operator.MUST_PASS_ALL);
> FilterList listOfFilters1 = new FilterList 
> (FilterList.Operator.MUST_PASS_ALL);
> FilterList listOfFilters2 = new FilterList 
> (FilterList.Operator.MUST_PASS_ALL);
> SingleColumnValueFilter SingleFilter1 = new
> SingleColumnValueFilter(Bytes.toBytes("cf"),
> Bytes.toBytes("country"), CompareOp.EQUAL,
> Bytes.toBytes("USA"));
> listOfFilters.addFilter(SingleFilter1);
> ValueFilter VF1= new ValueFilter (CompareOp.EQUAL, 
> new SubstringComparator("ABC"));
> ColumnPrefixFilter CF1= new ColumnPrefixFilter(Bytes.toBytes("name"));
> listOfFilters1.addFilter(CF1);
> listOfFilters1.addFilter(VF1);
> listOfFilters.addFilter(listOfFilters1);
> ValueFilter VF2= new ValueFilter (CompareOp.EQUAL, 
> new SubstringComparator("ED"));
> ColumnPrefixFilter CF2= new ColumnPrefixFilter (Bytes.toBytes("CRS"));
> listOfFilters2.addFilter(CF2);
> listOfFilters2.addFilter(VF2);
> listOfFilters.addFilter(listOfFilters2);
> When i do a combibation of SingleFilter1 and listOfFilters1
>  the result is correct, same way the combination of 
> SingleFilter1 and listOfFilters2 is returing correct result.
> But when all the three is combined im not getting any result..
> Is it the problem with multiple ColumnPrefixFilter??? 
> Value "ABC" exist in name.0 and value "ED" exist in CRS.0 and it is in
> the same row under same Column Family.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5963) ClassCastException: FileSystem$Cache$ClientFinalizer cannot be cast to Thread

2012-05-08 Thread Hadoop QA (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-5963?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13271136#comment-13271136
 ] 

Hadoop QA commented on HBASE-5963:
--

-1 overall.  Here are the results of testing the latest attachment 
  http://issues.apache.org/jira/secure/attachment/12526103/5963-v2.txt
  against trunk revision .

+1 @author.  The patch does not contain any @author tags.

-1 tests included.  The patch doesn't appear to include any new or modified 
tests.
Please justify why no new tests are needed for this 
patch.
Also please list what manual steps were performed to 
verify this patch.

+1 hadoop23.  The patch compiles against the hadoop 0.23.x profile.

+1 javadoc.  The javadoc tool did not generate any warning messages.

+1 javac.  The applied patch does not increase the total number of javac 
compiler warnings.

+1 findbugs.  The patch does not introduce any new Findbugs (version 1.3.9) 
warnings.

+1 release audit.  The applied patch does not increase the total number of 
release audit warnings.

 -1 core tests.  The patch failed these unit tests:
 

Test results: 
https://builds.apache.org/job/PreCommit-HBASE-Build/1808//testReport/
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/1808//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
Console output: 
https://builds.apache.org/job/PreCommit-HBASE-Build/1808//console

This message is automatically generated.

> ClassCastException: FileSystem$Cache$ClientFinalizer cannot be cast to Thread
> -
>
> Key: HBASE-5963
> URL: https://issues.apache.org/jira/browse/HBASE-5963
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 0.94.0, 0.96.0
> Environment: Bigtop 0.4.0-incubating-SNAPSHOT, Hadoop 
> 2.0.0-alpha-SNAPSHOT, HBase 0.94.0-SNAPSHOT
>Reporter: Andrew Purtell
>Assignee: Andrew Purtell
> Attachments: 5963-v2.txt, HBASE-5963.patch, HBASE-5963.patch
>
>
> {code}
> 12/05/08 19:49:26 INFO regionserver.HRegionServer: STOPPED: Failed 
> initialization
> Exception in thread "main" java.lang.ClassCastException: 
> org.apache.hadoop.fs.FileSystem$Cache$ClientFinalizer cannot be cast to 
> java.lang.Thread
>   at 
> org.apache.hadoop.hbase.regionserver.ShutdownHook.suppressHdfsShutdownHook(ShutdownHook.java:181)
>   at 
> org.apache.hadoop.hbase.regionserver.ShutdownHook.install(ShutdownHook.java:82)
>   at 
> org.apache.hadoop.hbase.regionserver.HRegionServer.startRegionServer(HRegionServer.java:3601)
>   at 
> org.apache.hadoop.hbase.regionserver.HRegionServer.startRegionServer(HRegionServer.java:3585)
>   at 
> org.apache.hadoop.hbase.regionserver.HRegionServerCommandLine.start(HRegionServerCommandLine.java:61)
>   at 
> org.apache.hadoop.hbase.regionserver.HRegionServerCommandLine.run(HRegionServerCommandLine.java:75)
>   at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
>   at 
> org.apache.hadoop.hbase.util.ServerCommandLine.doMain(ServerCommandLine.java:76)
>   at 
> org.apache.hadoop.hbase.regionserver.HRegionServer.main(HRegionServer.java:3645)
> {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5970) Improve the AssignmentManager#updateTimer and speed up handling opened event

2012-05-08 Thread chunhui shen (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-5970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13271127#comment-13271127
 ] 

chunhui shen commented on HBASE-5970:
-

I think we don't need to update the timer so active.
Waiting for the next chore() could reduce much consume if there are lots of 
opened event, because they may update the same server. 

> Improve the AssignmentManager#updateTimer and speed up handling opened event
> 
>
> Key: HBASE-5970
> URL: https://issues.apache.org/jira/browse/HBASE-5970
> Project: HBase
>  Issue Type: Improvement
>  Components: master
>Reporter: chunhui shen
>Assignee: chunhui shen
> Attachments: HBASE-5970.patch
>
>
> We found handing opened event very slow in the environment with lots of 
> regions.
> The problem is the slow AssignmentManager#updateTimer.
> We do the test for bulk assigning 10w (i.e. 100k) regions, the whole process 
> of bulk assigning took 1 hours.
> 2012-05-06 20:31:49,201 INFO 
> org.apache.hadoop.hbase.master.AssignmentManager: Bulk assigning 10 
> region(s) round-robin across 5 server(s)
> 2012-05-06 21:26:32,103 INFO 
> org.apache.hadoop.hbase.master.AssignmentManager: Bulk assigning done
> I think we could do the improvement for the AssignmentManager#updateTimer: 
> Make a thread do this work.
> After the improvement, it took only 4.5mins
> 2012-05-07 11:03:36,581 INFO 
> org.apache.hadoop.hbase.master.AssignmentManager: Bulk assigning 10 
> region(s) across 5 server(s), retainAssignment=true 
> 2012-05-07 11:07:57,073 INFO 
> org.apache.hadoop.hbase.master.AssignmentManager: Bulk assigning done 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5944) Convert rs, shutdown, and table dir content to pb

2012-05-08 Thread stack (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-5944?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stack updated HBASE-5944:
-

Status: Patch Available  (was: Open)

> Convert rs, shutdown, and table dir content to pb
> -
>
> Key: HBASE-5944
> URL: https://issues.apache.org/jira/browse/HBASE-5944
> Project: HBase
>  Issue Type: Task
>Reporter: stack
>Assignee: stack
> Attachments: 5944.txt, 5944v2.txt
>
>


--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5970) Improve the AssignmentManager#updateTimer and speed up handling opened event

2012-05-08 Thread Zhihong Yu (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-5970?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Zhihong Yu updated HBASE-5970:
--

 Description: 
We found handing opened event very slow in the environment with lots of regions.

The problem is the slow AssignmentManager#updateTimer.

We do the test for bulk assigning 10w (i.e. 100k) regions, the whole process of 
bulk assigning took 1 hours.

2012-05-06 20:31:49,201 INFO org.apache.hadoop.hbase.master.AssignmentManager: 
Bulk assigning 10 region(s) round-robin across 5 server(s)
2012-05-06 21:26:32,103 INFO org.apache.hadoop.hbase.master.AssignmentManager: 
Bulk assigning done

I think we could do the improvement for the AssignmentManager#updateTimer: Make 
a thread do this work.
After the improvement, it took only 4.5mins

2012-05-07 11:03:36,581 INFO org.apache.hadoop.hbase.master.AssignmentManager: 
Bulk assigning 10 region(s) across 5 server(s), retainAssignment=true 
2012-05-07 11:07:57,073 INFO org.apache.hadoop.hbase.master.AssignmentManager: 
Bulk assigning done 

  was:
We found handing opened event very slow in the environment with lots of regions.

The problem is the slow AssignmentManager#updateTimer.

We do the test for bulk assigning 10w regions, the whole process of bulk 
assigning took 1 hours.

2012-05-06 20:31:49,201 INFO org.apache.hadoop.hbase.master.AssignmentManager: 
Bulk assigning 10 region(s) round-robin across 5 server(s)
2012-05-06 21:26:32,103 INFO org.apache.hadoop.hbase.master.AssignmentManager: 
Bulk assigning done

I think we could do the improvement for the AssignmentManager#updateTimer: Make 
a thread do this work.
After the improvement, it took only 4.5mins

2012-05-07 11:03:36,581 INFO org.apache.hadoop.hbase.master.AssignmentManager: 
Bulk assigning 10 region(s) across 5 server(s), retainAssignment=true 
2012-05-07 11:07:57,073 INFO org.apache.hadoop.hbase.master.AssignmentManager: 
Bulk assigning done 

Hadoop Flags: Reviewed

> Improve the AssignmentManager#updateTimer and speed up handling opened event
> 
>
> Key: HBASE-5970
> URL: https://issues.apache.org/jira/browse/HBASE-5970
> Project: HBase
>  Issue Type: Improvement
>  Components: master
>Reporter: chunhui shen
>Assignee: chunhui shen
> Attachments: HBASE-5970.patch
>
>
> We found handing opened event very slow in the environment with lots of 
> regions.
> The problem is the slow AssignmentManager#updateTimer.
> We do the test for bulk assigning 10w (i.e. 100k) regions, the whole process 
> of bulk assigning took 1 hours.
> 2012-05-06 20:31:49,201 INFO 
> org.apache.hadoop.hbase.master.AssignmentManager: Bulk assigning 10 
> region(s) round-robin across 5 server(s)
> 2012-05-06 21:26:32,103 INFO 
> org.apache.hadoop.hbase.master.AssignmentManager: Bulk assigning done
> I think we could do the improvement for the AssignmentManager#updateTimer: 
> Make a thread do this work.
> After the improvement, it took only 4.5mins
> 2012-05-07 11:03:36,581 INFO 
> org.apache.hadoop.hbase.master.AssignmentManager: Bulk assigning 10 
> region(s) across 5 server(s), retainAssignment=true 
> 2012-05-07 11:07:57,073 INFO 
> org.apache.hadoop.hbase.master.AssignmentManager: Bulk assigning done 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5944) Convert rs, shutdown, and table dir content to pb

2012-05-08 Thread stack (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-5944?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stack updated HBASE-5944:
-

Attachment: 5944v2.txt

Missing a pb deserialization in ZKTable.

> Convert rs, shutdown, and table dir content to pb
> -
>
> Key: HBASE-5944
> URL: https://issues.apache.org/jira/browse/HBASE-5944
> Project: HBase
>  Issue Type: Task
>Reporter: stack
>Assignee: stack
> Attachments: 5944.txt, 5944v2.txt
>
>


--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5925) Issue with only using the old config param hbase.hstore.compactionThreshold but not the corresponding new one

2012-05-08 Thread Hudson (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-5925?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13271124#comment-13271124
 ] 

Hudson commented on HBASE-5925:
---

Integrated in HBase-TRUNK-security #196 (See 
[https://builds.apache.org/job/HBase-TRUNK-security/196/])
HBASE-5925 Issue with only using the old config param 
hbase.hstore.compactionThreshold but not the corresponding new one (Revision 
1335738)

 Result = FAILURE
stack : 
Files : 
* 
/hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/MemStoreFlusher.java


> Issue with only using the old config param hbase.hstore.compactionThreshold 
> but not the corresponding new one
> -
>
> Key: HBASE-5925
> URL: https://issues.apache.org/jira/browse/HBASE-5925
> Project: HBase
>  Issue Type: Bug
>Reporter: Anoop Sam John
>Assignee: Anoop Sam John
>Priority: Minor
> Fix For: 0.96.0
>
> Attachments: HBASE-5925.patch
>
>
> One observation while going through the code:-
> In MemStoreFlusher constructor
> {code}
> this.blockingStoreFilesNumber =
>   conf.getInt("hbase.hstore.blockingStoreFiles", 7);
> if (this.blockingStoreFilesNumber == -1) {
>   this.blockingStoreFilesNumber = 1 +
> conf.getInt("hbase.hstore.compactionThreshold", 3);
> }
> {code}
> Here as per the code if hbase.hstore.blockingStoreFiles is configured as -1, 
> we are making this value to be 1+ min files to compact
> But here we read the old config item only!
> Here also we need to read the new config 1st and if not there then the old 
> one.. Is this a miss?
> Like
> conf.getInt("hbase.hstore.compaction.min",
>  conf.getInt("hbase.hstore.compactionThreshold", 3))

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5963) ClassCastException: FileSystem$Cache$ClientFinalizer cannot be cast to Thread

2012-05-08 Thread Zhihong Yu (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-5963?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13271123#comment-13271123
 ] 

Zhihong Yu commented on HBASE-5963:
---

@Andy:
I am running the following on Linux:
{code}
nohup mvn test -Dhadoop.profile=23 -Dhadoop.version=2.0.0-SNAPSHOT 
-PrunAllTests > suite.txt &
{code}
Will check results tomorrow.
For hadoop 1.0, PreCommit build 1808 would tell us the results.

For ShutdownHookManagerV2.addShutdownHook(), current code reflects what was in 
patch v1. Your suggestion is plausible.

> ClassCastException: FileSystem$Cache$ClientFinalizer cannot be cast to Thread
> -
>
> Key: HBASE-5963
> URL: https://issues.apache.org/jira/browse/HBASE-5963
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 0.94.0, 0.96.0
> Environment: Bigtop 0.4.0-incubating-SNAPSHOT, Hadoop 
> 2.0.0-alpha-SNAPSHOT, HBase 0.94.0-SNAPSHOT
>Reporter: Andrew Purtell
>Assignee: Andrew Purtell
> Attachments: 5963-v2.txt, HBASE-5963.patch, HBASE-5963.patch
>
>
> {code}
> 12/05/08 19:49:26 INFO regionserver.HRegionServer: STOPPED: Failed 
> initialization
> Exception in thread "main" java.lang.ClassCastException: 
> org.apache.hadoop.fs.FileSystem$Cache$ClientFinalizer cannot be cast to 
> java.lang.Thread
>   at 
> org.apache.hadoop.hbase.regionserver.ShutdownHook.suppressHdfsShutdownHook(ShutdownHook.java:181)
>   at 
> org.apache.hadoop.hbase.regionserver.ShutdownHook.install(ShutdownHook.java:82)
>   at 
> org.apache.hadoop.hbase.regionserver.HRegionServer.startRegionServer(HRegionServer.java:3601)
>   at 
> org.apache.hadoop.hbase.regionserver.HRegionServer.startRegionServer(HRegionServer.java:3585)
>   at 
> org.apache.hadoop.hbase.regionserver.HRegionServerCommandLine.start(HRegionServerCommandLine.java:61)
>   at 
> org.apache.hadoop.hbase.regionserver.HRegionServerCommandLine.run(HRegionServerCommandLine.java:75)
>   at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
>   at 
> org.apache.hadoop.hbase.util.ServerCommandLine.doMain(ServerCommandLine.java:76)
>   at 
> org.apache.hadoop.hbase.regionserver.HRegionServer.main(HRegionServer.java:3645)
> {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5944) Convert rs, shutdown, and table dir content to pb

2012-05-08 Thread stack (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-5944?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stack updated HBASE-5944:
-

Status: Open  (was: Patch Available)

> Convert rs, shutdown, and table dir content to pb
> -
>
> Key: HBASE-5944
> URL: https://issues.apache.org/jira/browse/HBASE-5944
> Project: HBase
>  Issue Type: Task
>Reporter: stack
>Assignee: stack
> Attachments: 5944.txt
>
>


--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5963) ClassCastException: FileSystem$Cache$ClientFinalizer cannot be cast to Thread

2012-05-08 Thread Andrew Purtell (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-5963?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13271121#comment-13271121
 ] 

Andrew Purtell commented on HBASE-5963:
---

Additionally, shouldn't addShutdownHook in ShutdownHookManagerV2 delegate to 
the Hadoop 2.x shutdown hook manager?


> ClassCastException: FileSystem$Cache$ClientFinalizer cannot be cast to Thread
> -
>
> Key: HBASE-5963
> URL: https://issues.apache.org/jira/browse/HBASE-5963
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 0.94.0, 0.96.0
> Environment: Bigtop 0.4.0-incubating-SNAPSHOT, Hadoop 
> 2.0.0-alpha-SNAPSHOT, HBase 0.94.0-SNAPSHOT
>Reporter: Andrew Purtell
>Assignee: Andrew Purtell
> Attachments: 5963-v2.txt, HBASE-5963.patch, HBASE-5963.patch
>
>
> {code}
> 12/05/08 19:49:26 INFO regionserver.HRegionServer: STOPPED: Failed 
> initialization
> Exception in thread "main" java.lang.ClassCastException: 
> org.apache.hadoop.fs.FileSystem$Cache$ClientFinalizer cannot be cast to 
> java.lang.Thread
>   at 
> org.apache.hadoop.hbase.regionserver.ShutdownHook.suppressHdfsShutdownHook(ShutdownHook.java:181)
>   at 
> org.apache.hadoop.hbase.regionserver.ShutdownHook.install(ShutdownHook.java:82)
>   at 
> org.apache.hadoop.hbase.regionserver.HRegionServer.startRegionServer(HRegionServer.java:3601)
>   at 
> org.apache.hadoop.hbase.regionserver.HRegionServer.startRegionServer(HRegionServer.java:3585)
>   at 
> org.apache.hadoop.hbase.regionserver.HRegionServerCommandLine.start(HRegionServerCommandLine.java:61)
>   at 
> org.apache.hadoop.hbase.regionserver.HRegionServerCommandLine.run(HRegionServerCommandLine.java:75)
>   at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
>   at 
> org.apache.hadoop.hbase.util.ServerCommandLine.doMain(ServerCommandLine.java:76)
>   at 
> org.apache.hadoop.hbase.regionserver.HRegionServer.main(HRegionServer.java:3645)
> {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5970) Improve the AssignmentManager#updateTimer and speed up handling opened event

2012-05-08 Thread Zhihong Yu (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-5970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13271119#comment-13271119
 ] 

Zhihong Yu commented on HBASE-5970:
---

That's right. But I wonder why we need to wait for the next chore() ?

> Improve the AssignmentManager#updateTimer and speed up handling opened event
> 
>
> Key: HBASE-5970
> URL: https://issues.apache.org/jira/browse/HBASE-5970
> Project: HBase
>  Issue Type: Improvement
>  Components: master
>Reporter: chunhui shen
>Assignee: chunhui shen
> Attachments: HBASE-5970.patch
>
>
> We found handing opened event very slow in the environment with lots of 
> regions.
> The problem is the slow AssignmentManager#updateTimer.
> We do the test for bulk assigning 10w regions, the whole process of bulk 
> assigning took 1 hours.
> 2012-05-06 20:31:49,201 INFO 
> org.apache.hadoop.hbase.master.AssignmentManager: Bulk assigning 10 
> region(s) round-robin across 5 server(s)
> 2012-05-06 21:26:32,103 INFO 
> org.apache.hadoop.hbase.master.AssignmentManager: Bulk assigning done
> I think we could do the improvement for the AssignmentManager#updateTimer: 
> Make a thread do this work.
> After the improvement, it took only 4.5mins
> 2012-05-07 11:03:36,581 INFO 
> org.apache.hadoop.hbase.master.AssignmentManager: Bulk assigning 10 
> region(s) across 5 server(s), retainAssignment=true 
> 2012-05-07 11:07:57,073 INFO 
> org.apache.hadoop.hbase.master.AssignmentManager: Bulk assigning done 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5970) Improve the AssignmentManager#updateTimer and speed up handling opened event

2012-05-08 Thread chunhui shen (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-5970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13271115#comment-13271115
 ] 

chunhui shen commented on HBASE-5970:
-

bq.What would happen if a 'lower' ServerName gets its timers updated, gets 
removed from serversInUpdatingTimer and later is added back to 
serversInUpdatingTimer ?

If later it is added back to serversInUpdatingTimer, we will update timer again 
after next chore()

> Improve the AssignmentManager#updateTimer and speed up handling opened event
> 
>
> Key: HBASE-5970
> URL: https://issues.apache.org/jira/browse/HBASE-5970
> Project: HBase
>  Issue Type: Improvement
>  Components: master
>Reporter: chunhui shen
>Assignee: chunhui shen
> Attachments: HBASE-5970.patch
>
>
> We found handing opened event very slow in the environment with lots of 
> regions.
> The problem is the slow AssignmentManager#updateTimer.
> We do the test for bulk assigning 10w regions, the whole process of bulk 
> assigning took 1 hours.
> 2012-05-06 20:31:49,201 INFO 
> org.apache.hadoop.hbase.master.AssignmentManager: Bulk assigning 10 
> region(s) round-robin across 5 server(s)
> 2012-05-06 21:26:32,103 INFO 
> org.apache.hadoop.hbase.master.AssignmentManager: Bulk assigning done
> I think we could do the improvement for the AssignmentManager#updateTimer: 
> Make a thread do this work.
> After the improvement, it took only 4.5mins
> 2012-05-07 11:03:36,581 INFO 
> org.apache.hadoop.hbase.master.AssignmentManager: Bulk assigning 10 
> region(s) across 5 server(s), retainAssignment=true 
> 2012-05-07 11:07:57,073 INFO 
> org.apache.hadoop.hbase.master.AssignmentManager: Bulk assigning done 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5963) ClassCastException: FileSystem$Cache$ClientFinalizer cannot be cast to Thread

2012-05-08 Thread Andrew Purtell (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-5963?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13271114#comment-13271114
 ] 

Andrew Purtell commented on HBASE-5963:
---

Patch v2 is plausible.

Did you run unit tests with -Dhadoop.profile=23 and 
-Dhadoop.version=2.0.0-snapshot ? They pass except for the MR ones? And tests 
pass for Hadoop 1?


> ClassCastException: FileSystem$Cache$ClientFinalizer cannot be cast to Thread
> -
>
> Key: HBASE-5963
> URL: https://issues.apache.org/jira/browse/HBASE-5963
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 0.94.0, 0.96.0
> Environment: Bigtop 0.4.0-incubating-SNAPSHOT, Hadoop 
> 2.0.0-alpha-SNAPSHOT, HBase 0.94.0-SNAPSHOT
>Reporter: Andrew Purtell
>Assignee: Andrew Purtell
> Attachments: 5963-v2.txt, HBASE-5963.patch, HBASE-5963.patch
>
>
> {code}
> 12/05/08 19:49:26 INFO regionserver.HRegionServer: STOPPED: Failed 
> initialization
> Exception in thread "main" java.lang.ClassCastException: 
> org.apache.hadoop.fs.FileSystem$Cache$ClientFinalizer cannot be cast to 
> java.lang.Thread
>   at 
> org.apache.hadoop.hbase.regionserver.ShutdownHook.suppressHdfsShutdownHook(ShutdownHook.java:181)
>   at 
> org.apache.hadoop.hbase.regionserver.ShutdownHook.install(ShutdownHook.java:82)
>   at 
> org.apache.hadoop.hbase.regionserver.HRegionServer.startRegionServer(HRegionServer.java:3601)
>   at 
> org.apache.hadoop.hbase.regionserver.HRegionServer.startRegionServer(HRegionServer.java:3585)
>   at 
> org.apache.hadoop.hbase.regionserver.HRegionServerCommandLine.start(HRegionServerCommandLine.java:61)
>   at 
> org.apache.hadoop.hbase.regionserver.HRegionServerCommandLine.run(HRegionServerCommandLine.java:75)
>   at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
>   at 
> org.apache.hadoop.hbase.util.ServerCommandLine.doMain(ServerCommandLine.java:76)
>   at 
> org.apache.hadoop.hbase.regionserver.HRegionServer.main(HRegionServer.java:3645)
> {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5970) Improve the AssignmentManager#updateTimer and speed up handling opened event

2012-05-08 Thread Zhihong Yu (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-5970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13271113#comment-13271113
 ] 

Zhihong Yu commented on HBASE-5970:
---

{code}
+  serverToUpdateTimer = serversInUpdatingTimer
+  .higher(serverToUpdateTimer);
+}
+if (serverToUpdateTimer == null) {
+  break;
{code}
What would happen if a 'lower' ServerName gets its timers updated, gets removed 
from serversInUpdatingTimer and later is added back to serversInUpdatingTimer ?
How would this server be picked up ?

> Improve the AssignmentManager#updateTimer and speed up handling opened event
> 
>
> Key: HBASE-5970
> URL: https://issues.apache.org/jira/browse/HBASE-5970
> Project: HBase
>  Issue Type: Improvement
>  Components: master
>Reporter: chunhui shen
>Assignee: chunhui shen
> Attachments: HBASE-5970.patch
>
>
> We found handing opened event very slow in the environment with lots of 
> regions.
> The problem is the slow AssignmentManager#updateTimer.
> We do the test for bulk assigning 10w regions, the whole process of bulk 
> assigning took 1 hours.
> 2012-05-06 20:31:49,201 INFO 
> org.apache.hadoop.hbase.master.AssignmentManager: Bulk assigning 10 
> region(s) round-robin across 5 server(s)
> 2012-05-06 21:26:32,103 INFO 
> org.apache.hadoop.hbase.master.AssignmentManager: Bulk assigning done
> I think we could do the improvement for the AssignmentManager#updateTimer: 
> Make a thread do this work.
> After the improvement, it took only 4.5mins
> 2012-05-07 11:03:36,581 INFO 
> org.apache.hadoop.hbase.master.AssignmentManager: Bulk assigning 10 
> region(s) across 5 server(s), retainAssignment=true 
> 2012-05-07 11:07:57,073 INFO 
> org.apache.hadoop.hbase.master.AssignmentManager: Bulk assigning done 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5970) Improve the AssignmentManager#updateTimer and speed up handling opened event

2012-05-08 Thread chunhui shen (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-5970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13271109#comment-13271109
 ] 

chunhui shen commented on HBASE-5970:
-

@ram
Thanks for the review.
Let's see what others say, and I'll do the modify together.

> Improve the AssignmentManager#updateTimer and speed up handling opened event
> 
>
> Key: HBASE-5970
> URL: https://issues.apache.org/jira/browse/HBASE-5970
> Project: HBase
>  Issue Type: Improvement
>  Components: master
>Reporter: chunhui shen
>Assignee: chunhui shen
> Attachments: HBASE-5970.patch
>
>
> We found handing opened event very slow in the environment with lots of 
> regions.
> The problem is the slow AssignmentManager#updateTimer.
> We do the test for bulk assigning 10w regions, the whole process of bulk 
> assigning took 1 hours.
> 2012-05-06 20:31:49,201 INFO 
> org.apache.hadoop.hbase.master.AssignmentManager: Bulk assigning 10 
> region(s) round-robin across 5 server(s)
> 2012-05-06 21:26:32,103 INFO 
> org.apache.hadoop.hbase.master.AssignmentManager: Bulk assigning done
> I think we could do the improvement for the AssignmentManager#updateTimer: 
> Make a thread do this work.
> After the improvement, it took only 4.5mins
> 2012-05-07 11:03:36,581 INFO 
> org.apache.hadoop.hbase.master.AssignmentManager: Bulk assigning 10 
> region(s) across 5 server(s), retainAssignment=true 
> 2012-05-07 11:07:57,073 INFO 
> org.apache.hadoop.hbase.master.AssignmentManager: Bulk assigning done 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (HBASE-5971) ServerLoad needs redo; can't be pb based

2012-05-08 Thread stack (JIRA)

stack created HBASE-5971:


 Summary: ServerLoad needs redo; can't be pb based
 Key: HBASE-5971
 URL: https://issues.apache.org/jira/browse/HBASE-5971
 Project: HBase
  Issue Type: Bug
  Components: metrics
Reporter: stack
Priority: Blocker
 Fix For: 0.96.0


Here is what happens when we try to register server bean:

{code}
javax.management.NotCompliantMBeanException: 
org.apache.hadoop.hbase.master.MXBean: Method 
org.apache.hadoop.hbase.master.MXBean.getRegionServers has parameter or return 
type that cannot be translated into an open type
at 
com.sun.jmx.mbeanserver.Introspector.throwException(Introspector.java:412)
at com.sun.jmx.mbeanserver.MBeanAnalyzer.(MBeanAnalyzer.java:101)
at com.sun.jmx.mbeanserver.MBeanAnalyzer.analyzer(MBeanAnalyzer.java:87)
at 
com.sun.jmx.mbeanserver.MXBeanIntrospector.getAnalyzer(MXBeanIntrospector.java:53)
at 
com.sun.jmx.mbeanserver.MBeanIntrospector.getPerInterface(MBeanIntrospector.java:163)
at com.sun.jmx.mbeanserver.MBeanSupport.(MBeanSupport.java:147)
at com.sun.jmx.mbeanserver.MXBeanSupport.(MXBeanSupport.java:48)
at 
com.sun.jmx.mbeanserver.Introspector.makeDynamicMBean(Introspector.java:184)
at 
com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerObject(DefaultMBeanServerInterceptor.java:915)
at 
com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerMBean(DefaultMBeanServerInterceptor.java:312)
at 
com.sun.jmx.mbeanserver.JmxMBeanServer.registerMBean(JmxMBeanServer.java:482)
at 
org.apache.hadoop.metrics.util.MBeanUtil.registerMBean(MBeanUtil.java:58)
at 
org.apache.hadoop.hbase.master.HMaster.registerMBean(HMaster.java:1926)
at 
org.apache.hadoop.hbase.master.HMaster.finishInitialization(HMaster.java:617)
at org.apache.hadoop.hbase.master.HMaster.run(HMaster.java:367)
at java.lang.Thread.run(Thread.java:680)
Caused by: java.lang.IllegalArgumentException: Method 
org.apache.hadoop.hbase.master.MXBean.getRegionServers has parameter or return 
type that cannot be translated into an open type
at 
com.sun.jmx.mbeanserver.ConvertingMethod.from(ConvertingMethod.java:32)
at 
com.sun.jmx.mbeanserver.MXBeanIntrospector.mFrom(MXBeanIntrospector.java:63)
at 
com.sun.jmx.mbeanserver.MXBeanIntrospector.mFrom(MXBeanIntrospector.java:33)
at 
com.sun.jmx.mbeanserver.MBeanAnalyzer.initMaps(MBeanAnalyzer.java:118)
at com.sun.jmx.mbeanserver.MBeanAnalyzer.(MBeanAnalyzer.java:99)
... 14 more
Caused by: javax.management.openmbean.OpenDataException: Cannot convert type: 
java.util.Map
at 
com.sun.jmx.mbeanserver.OpenConverter.openDataException(OpenConverter.java:1411)
at 
com.sun.jmx.mbeanserver.OpenConverter.toConverter(OpenConverter.java:264)
at 
com.sun.jmx.mbeanserver.ConvertingMethod.(ConvertingMethod.java:184)
at 
com.sun.jmx.mbeanserver.ConvertingMethod.from(ConvertingMethod.java:27)
... 18 more
Caused by: javax.management.openmbean.OpenDataException: Cannot convert type: 
class org.apache.hadoop.hbase.ServerLoad
at 
com.sun.jmx.mbeanserver.OpenConverter.openDataException(OpenConverter.java:1411)
at 
com.sun.jmx.mbeanserver.OpenConverter.toConverter(OpenConverter.java:264)
at 
com.sun.jmx.mbeanserver.OpenConverter.makeTabularConverter(OpenConverter.java:360)
at 
com.sun.jmx.mbeanserver.OpenConverter.makeParameterizedConverter(OpenConverter.java:402)
at 
com.sun.jmx.mbeanserver.OpenConverter.makeConverter(OpenConverter.java:296)
at 
com.sun.jmx.mbeanserver.OpenConverter.toConverter(OpenConverter.java:262)
... 20 more
Caused by: javax.management.openmbean.OpenDataException: Cannot convert type: 
java.util.List
at 
com.sun.jmx.mbeanserver.OpenConverter.openDataException(OpenConverter.java:1411)
at 
com.sun.jmx.mbeanserver.OpenConverter.toConverter(OpenConverter.java:264)
at 
com.sun.jmx.mbeanserver.OpenConverter.makeCompositeConverter(OpenConverter.java:467)
at 
com.sun.jmx.mbeanserver.OpenConverter.makeConverter(OpenConverter.java:293)
at 
com.sun.jmx.mbeanserver.OpenConverter.toConverter(OpenConverter.java:262)
... 24 more
Caused by: javax.management.openmbean.OpenDataException: Cannot convert type: 
class org.apache.hadoop.hbase.protobuf.generated.HBaseProtos$Coprocessor
at 
com.sun.jmx.mbeanserver.OpenConverter.openDataException(OpenConverter.java:1411)
at 
com.sun.jmx.mbeanserver.OpenConverter.toConverter(OpenConverter.java:264)
at 
com.sun.jmx.mbeanserver.OpenConverter.makeArrayOrCollectionConverter(OpenConverter.java:315)
at 
com.sun.jmx.mbeanserver.OpenConverter.makeParameterizedConverter(OpenConverter.java:393)
at 
com.sun.jmx.mbeanserver.OpenConverter.makeConverter(O

[jira] [Resolved] (HBASE-5337) AM.updateTimers() delays the timeout monitor from assigning regions.

2012-05-08 Thread ramkrishna.s.vasudevan (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-5337?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

ramkrishna.s.vasudevan resolved HBASE-5337.
---

Resolution: Duplicate

HBASE-5970 addresses this issue.

> AM.updateTimers() delays the timeout monitor from assigning regions.
> 
>
> Key: HBASE-5337
> URL: https://issues.apache.org/jira/browse/HBASE-5337
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 0.90.5
>Reporter: ramkrishna.s.vasudevan
>Assignee: ramkrishna.s.vasudevan
> Fix For: 0.90.7
>
>
> AM.updateTimers() is necessary when the new RS has joined the cluster and 
> there are more regions to be assigned to it.  The updateTimer will help the 
> TM from not picking up these regions because they are still in the process of 
> assignment.
> But if in 0.90 we have 30min as TM period then this step of updating the 
> timers further delays the assignment.
> I think we can remove the call for updateTimer in 0.90 till we decide to 
> change the TM period.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5963) ClassCastException: FileSystem$Cache$ClientFinalizer cannot be cast to Thread

2012-05-08 Thread Zhihong Yu (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-5963?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Zhihong Yu updated HBASE-5963:
--

Attachment: 5963-v2.txt

Patch v2 introduces ShutdownHookManager shim.

Please review.

> ClassCastException: FileSystem$Cache$ClientFinalizer cannot be cast to Thread
> -
>
> Key: HBASE-5963
> URL: https://issues.apache.org/jira/browse/HBASE-5963
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 0.94.0, 0.96.0
> Environment: Bigtop 0.4.0-incubating-SNAPSHOT, Hadoop 
> 2.0.0-alpha-SNAPSHOT, HBase 0.94.0-SNAPSHOT
>Reporter: Andrew Purtell
>Assignee: Andrew Purtell
> Attachments: 5963-v2.txt, HBASE-5963.patch, HBASE-5963.patch
>
>
> {code}
> 12/05/08 19:49:26 INFO regionserver.HRegionServer: STOPPED: Failed 
> initialization
> Exception in thread "main" java.lang.ClassCastException: 
> org.apache.hadoop.fs.FileSystem$Cache$ClientFinalizer cannot be cast to 
> java.lang.Thread
>   at 
> org.apache.hadoop.hbase.regionserver.ShutdownHook.suppressHdfsShutdownHook(ShutdownHook.java:181)
>   at 
> org.apache.hadoop.hbase.regionserver.ShutdownHook.install(ShutdownHook.java:82)
>   at 
> org.apache.hadoop.hbase.regionserver.HRegionServer.startRegionServer(HRegionServer.java:3601)
>   at 
> org.apache.hadoop.hbase.regionserver.HRegionServer.startRegionServer(HRegionServer.java:3585)
>   at 
> org.apache.hadoop.hbase.regionserver.HRegionServerCommandLine.start(HRegionServerCommandLine.java:61)
>   at 
> org.apache.hadoop.hbase.regionserver.HRegionServerCommandLine.run(HRegionServerCommandLine.java:75)
>   at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
>   at 
> org.apache.hadoop.hbase.util.ServerCommandLine.doMain(ServerCommandLine.java:76)
>   at 
> org.apache.hadoop.hbase.regionserver.HRegionServer.main(HRegionServer.java:3645)
> {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5970) Improve the AssignmentManager#updateTimer and speed up handling opened event

2012-05-08 Thread ramkrishna.s.vasudevan (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-5970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13271103#comment-13271103
 ] 

ramkrishna.s.vasudevan commented on HBASE-5970:
---

@Chunhui
HBASE-5337-AM.updateTimers() delays the timeout monitor from assigning regions.
I think i can duplicate it with this defect.
Few comments
serversInUpdatingTimer can be changed to updateTimerForServer?
The order in which the updation should happen matters? 
{code}
Threads.setDaemonThreadRunning(timerUpdater.getThread(),
+master.getServerName() + ".timerUpdater");
{code}
Can we just add this to one protected method.  Actually we found that doing 
like that will help you in mocking the master in case of testing these 
scenarios?

Patch looks good.



> Improve the AssignmentManager#updateTimer and speed up handling opened event
> 
>
> Key: HBASE-5970
> URL: https://issues.apache.org/jira/browse/HBASE-5970
> Project: HBase
>  Issue Type: Improvement
>  Components: master
>Reporter: chunhui shen
>Assignee: chunhui shen
> Attachments: HBASE-5970.patch
>
>
> We found handing opened event very slow in the environment with lots of 
> regions.
> The problem is the slow AssignmentManager#updateTimer.
> We do the test for bulk assigning 10w regions, the whole process of bulk 
> assigning took 1 hours.
> 2012-05-06 20:31:49,201 INFO 
> org.apache.hadoop.hbase.master.AssignmentManager: Bulk assigning 10 
> region(s) round-robin across 5 server(s)
> 2012-05-06 21:26:32,103 INFO 
> org.apache.hadoop.hbase.master.AssignmentManager: Bulk assigning done
> I think we could do the improvement for the AssignmentManager#updateTimer: 
> Make a thread do this work.
> After the improvement, it took only 4.5mins
> 2012-05-07 11:03:36,581 INFO 
> org.apache.hadoop.hbase.master.AssignmentManager: Bulk assigning 10 
> region(s) across 5 server(s), retainAssignment=true 
> 2012-05-07 11:07:57,073 INFO 
> org.apache.hadoop.hbase.master.AssignmentManager: Bulk assigning done 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5966) MapReduce based tests broken on Hadoop 2.0.0-alpha

2012-05-08 Thread Ashutosh Jindal (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-5966?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13271101#comment-13271101
 ] 

Ashutosh Jindal commented on HBASE-5966:


This is the exception that i got while running testcases. Some other testcases 
are also failing with the same error.
{code:xml} 
---
Test set: org.apache.hadoop.hbase.mapreduce.TestTableInputFormatScan
---
Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 177.132 sec <<< 
FAILURE!
org.apache.hadoop.hbase.mapreduce.TestTableInputFormatScan  Time elapsed: 0 sec 
 <<< ERROR!
org.apache.hadoop.yarn.YarnException: Failed to Start 
org.apache.hadoop.mapred.MiniMRCluster
at 
org.apache.hadoop.yarn.service.CompositeService.start(CompositeService.java:78)
at 
org.apache.hadoop.mapred.MiniMRClientClusterFactory.create(MiniMRClientClusterFactory.java:67)
at org.apache.hadoop.mapred.MiniMRCluster.(MiniMRCluster.java:180)
at org.apache.hadoop.mapred.MiniMRCluster.(MiniMRCluster.java:170)
at org.apache.hadoop.mapred.MiniMRCluster.(MiniMRCluster.java:162)
at org.apache.hadoop.mapred.MiniMRCluster.(MiniMRCluster.java:154)
at org.apache.hadoop.mapred.MiniMRCluster.(MiniMRCluster.java:147)
at org.apache.hadoop.mapred.MiniMRCluster.(MiniMRCluster.java:140)
at org.apache.hadoop.mapred.MiniMRCluster.(MiniMRCluster.java:133)
at org.apache.hadoop.mapred.MiniMRCluster.(MiniMRCluster.java:128)
at 
org.apache.hadoop.hbase.HBaseTestingUtility.startMiniMapReduceCluster(HBaseTestingUtility.java:1269)
at 
org.apache.hadoop.hbase.HBaseTestingUtility.startMiniMapReduceCluster(HBaseTestingUtility.java:1256)
at 
org.apache.hadoop.hbase.mapreduce.TestTableInputFormatScan.setUpBeforeClass(TestTableInputFormatScan.java:83)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at 
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45)
at 
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
at 
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42)
at 
org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:27)
at 
org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:30)
at org.junit.runners.ParentRunner.run(ParentRunner.java:300)
at 
org.apache.maven.surefire.junit4.JUnit4TestSet.execute(JUnit4TestSet.java:53)
at 
org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:123)
at 
org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:104)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at 
org.apache.maven.surefire.util.ReflectionUtils.invokeMethodWithArray(ReflectionUtils.java:164)
at 
org.apache.maven.surefire.booter.ProviderFactory$ProviderProxy.invoke(ProviderFactory.java:110)
at 
org.apache.maven.surefire.booter.SurefireStarter.invokeProvider(SurefireStarter.java:175)
at 
org.apache.maven.surefire.booter.SurefireStarter.runSuitesInProcessWhenForked(SurefireStarter.java:81)
at 
org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:68)
Caused by: org.apache.hadoop.yarn.YarnException: java.io.IOException: 
ResourceManager failed to start. Final state is INITED
at 
org.apache.hadoop.yarn.server.MiniYARNCluster$ResourceManagerWrapper.start(MiniYARNCluster.java:152)
at 
org.apache.hadoop.yarn.service.CompositeService.start(CompositeService.java:68)
... 34 more
Caused by: java.io.IOException: ResourceManager failed to start. Final state is 
INITED
at 
org.apache.hadoop.yarn.server.MiniYARNCluster$ResourceManagerWrapper.start(MiniYARNCluster.java:146)
... 35 more
{code}

> MapReduce based tests broken on Hadoop 2.0.0-alpha
> --
>
> Key: HBASE-5966
> URL: https://issues.apache.org/jira/browse/HBASE-5966
> Project: HBase
>  Issue Type: Bug
>  Components: mapred, mapreduce, test
>Affects Versions: 0.94.0, 0.96.0
> Environment: Hadoop 2.0.0

[jira] [Updated] (HBASE-5970) Improve the AssignmentManager#updateTimer and speed up handling opened event

2012-05-08 Thread chunhui shen (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-5970?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

chunhui shen updated HBASE-5970:


Attachment: HBASE-5970.patch

> Improve the AssignmentManager#updateTimer and speed up handling opened event
> 
>
> Key: HBASE-5970
> URL: https://issues.apache.org/jira/browse/HBASE-5970
> Project: HBase
>  Issue Type: Improvement
>  Components: master
>Reporter: chunhui shen
>Assignee: chunhui shen
> Attachments: HBASE-5970.patch
>
>
> We found handing opened event very slow in the environment with lots of 
> regions.
> The problem is the slow AssignmentManager#updateTimer.
> We do the test for bulk assigning 10w regions, the whole process of bulk 
> assigning took 1 hours.
> 2012-05-06 20:31:49,201 INFO 
> org.apache.hadoop.hbase.master.AssignmentManager: Bulk assigning 10 
> region(s) round-robin across 5 server(s)
> 2012-05-06 21:26:32,103 INFO 
> org.apache.hadoop.hbase.master.AssignmentManager: Bulk assigning done
> I think we could do the improvement for the AssignmentManager#updateTimer: 
> Make a thread do this work.
> After the improvement, it took only 4.5mins
> 2012-05-07 11:03:36,581 INFO 
> org.apache.hadoop.hbase.master.AssignmentManager: Bulk assigning 10 
> region(s) across 5 server(s), retainAssignment=true 
> 2012-05-07 11:07:57,073 INFO 
> org.apache.hadoop.hbase.master.AssignmentManager: Bulk assigning done 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5951) Combining ColumnPrefixFilters using filterlist.

2012-05-08 Thread Davis Abraham (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-5951?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13271097#comment-13271097
 ] 

Davis Abraham commented on HBASE-5951:
--

Its not possible in normal SQL... While posting the SQL query i had mentioned 
that it wont work in SQl...

> Combining ColumnPrefixFilters using filterlist.
> ---
>
> Key: HBASE-5951
> URL: https://issues.apache.org/jira/browse/HBASE-5951
> Project: HBase
>  Issue Type: Bug
>  Components: filters
>Affects Versions: 0.92.1
> Environment: Combining multiple ColumnPrefixFilters using filterlist 
> does not return exact result.
>Reporter: Davis Abraham
>
> FilterList listOfFilters = new FilterList (FilterList.Operator.MUST_PASS_ALL);
> FilterList listOfFilters1 = new FilterList 
> (FilterList.Operator.MUST_PASS_ALL);
> FilterList listOfFilters2 = new FilterList 
> (FilterList.Operator.MUST_PASS_ALL);
> SingleColumnValueFilter SingleFilter1 = new
> SingleColumnValueFilter(Bytes.toBytes("cf"),
> Bytes.toBytes("country"), CompareOp.EQUAL,
> Bytes.toBytes("USA"));
> listOfFilters.addFilter(SingleFilter1);
> ValueFilter VF1= new ValueFilter (CompareOp.EQUAL, 
> new SubstringComparator("ABC"));
> ColumnPrefixFilter CF1= new ColumnPrefixFilter(Bytes.toBytes("name"));
> listOfFilters1.addFilter(CF1);
> listOfFilters1.addFilter(VF1);
> listOfFilters.addFilter(listOfFilters1);
> ValueFilter VF2= new ValueFilter (CompareOp.EQUAL, 
> new SubstringComparator("ED"));
> ColumnPrefixFilter CF2= new ColumnPrefixFilter (Bytes.toBytes("CRS"));
> listOfFilters2.addFilter(CF2);
> listOfFilters2.addFilter(VF2);
> listOfFilters.addFilter(listOfFilters2);
> When i do a combibation of SingleFilter1 and listOfFilters1
>  the result is correct, same way the combination of 
> SingleFilter1 and listOfFilters2 is returing correct result.
> But when all the three is combined im not getting any result..
> Is it the problem with multiple ColumnPrefixFilter??? 
> Value "ABC" exist in name.0 and value "ED" exist in CRS.0 and it is in
> the same row under same Column Family.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Assigned] (HBASE-5967) ServerLoad OpenDataException

2012-05-08 Thread Gregory Chanan (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-5967?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gregory Chanan reassigned HBASE-5967:
-

Assignee: Gregory Chanan

> ServerLoad OpenDataException
> 
>
> Key: HBASE-5967
> URL: https://issues.apache.org/jira/browse/HBASE-5967
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 0.96.0
>Reporter: Jimmy Xiang
>Assignee: Gregory Chanan
>Priority: Minor
> Fix For: 0.96.0
>
> Attachments: master.log
>
>
> I saw this error in the master log:
> Caused by: java.lang.IllegalArgumentException: Method 
> org.apache.hadoop.hbase.master.MXBean.getRegionServers has parameter or 
> return type that cannot be translated into an open type
> at com.sun.jmx.mbeanserver.ConvertingMethod.from(ConvertingMethod.java:32)
> at 
> com.sun.jmx.mbeanserver.MXBeanIntrospector.mFrom(MXBeanIntrospector.java:63)
> at 
> com.sun.jmx.mbeanserver.MXBeanIntrospector.mFrom(MXBeanIntrospector.java:33)
> at com.sun.jmx.mbeanserver.MBeanAnalyzer.initMaps(MBeanAnalyzer.java:118)
> at com.sun.jmx.mbeanserver.MBeanAnalyzer.(MBeanAnalyzer.java:99)
> ... 14 more
> Caused by: javax.management.openmbean.OpenDataException: Cannot convert type: 
> java.util.Map
> at 
> com.sun.jmx.mbeanserver.OpenConverter.openDataException(OpenConverter.jav

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5967) ServerLoad OpenDataException

2012-05-08 Thread Gregory Chanan (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-5967?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13271092#comment-13271092
 ] 

Gregory Chanan commented on HBASE-5967:
---

Probably caused by HBASE-5444.

I'll take a look.

> ServerLoad OpenDataException
> 
>
> Key: HBASE-5967
> URL: https://issues.apache.org/jira/browse/HBASE-5967
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 0.96.0
>Reporter: Jimmy Xiang
>Priority: Minor
> Fix For: 0.96.0
>
> Attachments: master.log
>
>
> I saw this error in the master log:
> Caused by: java.lang.IllegalArgumentException: Method 
> org.apache.hadoop.hbase.master.MXBean.getRegionServers has parameter or 
> return type that cannot be translated into an open type
> at com.sun.jmx.mbeanserver.ConvertingMethod.from(ConvertingMethod.java:32)
> at 
> com.sun.jmx.mbeanserver.MXBeanIntrospector.mFrom(MXBeanIntrospector.java:63)
> at 
> com.sun.jmx.mbeanserver.MXBeanIntrospector.mFrom(MXBeanIntrospector.java:33)
> at com.sun.jmx.mbeanserver.MBeanAnalyzer.initMaps(MBeanAnalyzer.java:118)
> at com.sun.jmx.mbeanserver.MBeanAnalyzer.(MBeanAnalyzer.java:99)
> ... 14 more
> Caused by: javax.management.openmbean.OpenDataException: Cannot convert type: 
> java.util.Map
> at 
> com.sun.jmx.mbeanserver.OpenConverter.openDataException(OpenConverter.jav

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5828) TestLogRolling fails in 0.90 branch killing the test suite up on jenkins

2012-05-08 Thread ramkrishna.s.vasudevan (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-5828?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13271091#comment-13271091
 ] 

ramkrishna.s.vasudevan commented on HBASE-5828:
---

Xufeng, i can understand the problem here to some extent.
bq.A:Because by @before and @after tag,every case will have the absolute test 
cluster.
Do you think doing as in trunk adding @before and @after will solve this 
problem?
Because the way the pipeline is formed may not be in our control right?


> TestLogRolling fails in 0.90 branch killing the test suite up on jenkins
> 
>
> Key: HBASE-5828
> URL: https://issues.apache.org/jira/browse/HBASE-5828
> Project: HBase
>  Issue Type: Bug
>Reporter: stack
>Assignee: xufeng
> Attachments: TestLogRolling.java, 
> org.apache.hadoop.hbase.regionserver.wal.TestLogRolling-output.rar
>
>
> See recent 0.90 builds up on jenkins: 
> https://builds.apache.org/view/G-L/view/HBase/job/hbase-0.90/471/console

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (HBASE-5970) Improve the AssignmentManager#updateTimer and speed up handling opened event

2012-05-08 Thread chunhui shen (JIRA)

chunhui shen created HBASE-5970:
---

 Summary: Improve the AssignmentManager#updateTimer and speed up 
handling opened event
 Key: HBASE-5970
 URL: https://issues.apache.org/jira/browse/HBASE-5970
 Project: HBase
  Issue Type: Improvement
  Components: master
Reporter: chunhui shen
Assignee: chunhui shen


We found handing opened event very slow in the environment with lots of regions.

The problem is the slow AssignmentManager#updateTimer.

We do the test for bulk assigning 10w regions, the whole process of bulk 
assigning took 1 hours.

2012-05-06 20:31:49,201 INFO org.apache.hadoop.hbase.master.AssignmentManager: 
Bulk assigning 10 region(s) round-robin across 5 server(s)
2012-05-06 21:26:32,103 INFO org.apache.hadoop.hbase.master.AssignmentManager: 
Bulk assigning done

I think we could do the improvement for the AssignmentManager#updateTimer: Make 
a thread do this work.
After the improvement, it took only 4.5mins

2012-05-07 11:03:36,581 INFO org.apache.hadoop.hbase.master.AssignmentManager: 
Bulk assigning 10 region(s) across 5 server(s), retainAssignment=true 
2012-05-07 11:07:57,073 INFO org.apache.hadoop.hbase.master.AssignmentManager: 
Bulk assigning done 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5966) MapReduce based tests broken on Hadoop 2.0.0-alpha

2012-05-08 Thread Ashutosh Jindal (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-5966?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13271084#comment-13271084
 ] 

Ashutosh Jindal commented on HBASE-5966:


I am also facing the same issue. Testcases using Map-Red minicluster are 
failing with the same kind of exception when run with 0.23 version

> MapReduce based tests broken on Hadoop 2.0.0-alpha
> --
>
> Key: HBASE-5966
> URL: https://issues.apache.org/jira/browse/HBASE-5966
> Project: HBase
>  Issue Type: Bug
>  Components: mapred, mapreduce, test
>Affects Versions: 0.94.0, 0.96.0
> Environment: Hadoop 2.0.0-alpha-SNAPSHOT, HBase 0.94.0-SNAPSHOT, 
> Ubuntu 12.04 LTS (GNU/Linux 3.2.0-24-generic x86_64)
>Reporter: Andrew Purtell
>
> Some fairly recent change in Hadoop 2.0.0-alpha has broken our MapReduce test 
> rigging. Below is a representative error, can be easily reproduced with:
> {noformat}
> mvn -PlocalTests -Psecurity \
>   -Dhadoop.profile=23 -Dhadoop.version=2.0.0-SNAPSHOT \
>   clean test \
>   -Dtest=org.apache.hadoop.hbase.mapreduce.TestTableMapReduce
> {noformat}
> And the result:
> {noformat}
> ---
>  T E S T S
> ---
> Running org.apache.hadoop.hbase.mapreduce.TestTableMapReduce
> Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 54.292 sec 
> <<< FAILURE!
> ---
> Test set: org.apache.hadoop.hbase.mapreduce.TestTableMapReduce
> ---
> Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 54.292 sec 
> <<< FAILURE!
> testMultiRegionTable(org.apache.hadoop.hbase.mapreduce.TestTableMapReduce)  
> Time elapsed: 21.935 sec  <<< ERROR!
> java.lang.reflect.UndeclaredThrowableException
>   at 
> org.apache.hadoop.yarn.exceptions.impl.pb.YarnRemoteExceptionPBImpl.unwrapAndThrowException(YarnRemoteExceptionPBImpl.java:135)
>   at 
> org.apache.hadoop.yarn.api.impl.pb.client.ClientRMProtocolPBClientImpl.getNewApplication(ClientRMProtocolPBClientImpl.java:134)
>   at 
> org.apache.hadoop.mapred.ResourceMgrDelegate.getNewJobID(ResourceMgrDelegate.java:183)
>   at org.apache.hadoop.mapred.YARNRunner.getNewJobID(YARNRunner.java:216)
>   at 
> org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:339)
>   at org.apache.hadoop.mapreduce.Job$11.run(Job.java:1226)
>   at org.apache.hadoop.mapreduce.Job$11.run(Job.java:1223)
>   at java.security.AccessController.doPrivileged(Native Method)
>   at javax.security.auth.Subject.doAs(Subject.java:416)
>   at 
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1232)
>   at org.apache.hadoop.mapreduce.Job.submit(Job.java:1223)
>   at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:1244)
>   at 
> org.apache.hadoop.hbase.mapreduce.TestTableMapReduce.runTestOnTable(TestTableMapReduce.java:151)
>   at 
> org.apache.hadoop.hbase.mapreduce.TestTableMapReduce.testMultiRegionTable(TestTableMapReduce.java:129)
>   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>   at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>   at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>   at java.lang.reflect.Method.invoke(Method.java:616)
>   at 
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45)
>   at 
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
>   at 
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42)
>   at 
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
>   at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:47)
>   at org.junit.rules.RunRules.evaluate(RunRules.java:18)
>   at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:263)
>   at 
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:68)
>   at 
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:47)
>   at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231)
>   at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:60)
>   at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:229)
>   at org.junit.runners.ParentRunner.access$000(ParentRunner.java:50)
>   at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:222)
>   at 
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28)
>   at 
> org.junit.intern

[jira] [Commented] (HBASE-5966) MapReduce based tests broken on Hadoop 2.0.0-alpha

2012-05-08 Thread Andrew Purtell (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-5966?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13271074#comment-13271074
 ] 

Andrew Purtell commented on HBASE-5966:
---

It's a maven activation that chooses the 0.23+ dependency set. You'll see it in 
the root POM. 


> MapReduce based tests broken on Hadoop 2.0.0-alpha
> --
>
> Key: HBASE-5966
> URL: https://issues.apache.org/jira/browse/HBASE-5966
> Project: HBase
>  Issue Type: Bug
>  Components: mapred, mapreduce, test
>Affects Versions: 0.94.0, 0.96.0
> Environment: Hadoop 2.0.0-alpha-SNAPSHOT, HBase 0.94.0-SNAPSHOT, 
> Ubuntu 12.04 LTS (GNU/Linux 3.2.0-24-generic x86_64)
>Reporter: Andrew Purtell
>
> Some fairly recent change in Hadoop 2.0.0-alpha has broken our MapReduce test 
> rigging. Below is a representative error, can be easily reproduced with:
> {noformat}
> mvn -PlocalTests -Psecurity \
>   -Dhadoop.profile=23 -Dhadoop.version=2.0.0-SNAPSHOT \
>   clean test \
>   -Dtest=org.apache.hadoop.hbase.mapreduce.TestTableMapReduce
> {noformat}
> And the result:
> {noformat}
> ---
>  T E S T S
> ---
> Running org.apache.hadoop.hbase.mapreduce.TestTableMapReduce
> Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 54.292 sec 
> <<< FAILURE!
> ---
> Test set: org.apache.hadoop.hbase.mapreduce.TestTableMapReduce
> ---
> Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 54.292 sec 
> <<< FAILURE!
> testMultiRegionTable(org.apache.hadoop.hbase.mapreduce.TestTableMapReduce)  
> Time elapsed: 21.935 sec  <<< ERROR!
> java.lang.reflect.UndeclaredThrowableException
>   at 
> org.apache.hadoop.yarn.exceptions.impl.pb.YarnRemoteExceptionPBImpl.unwrapAndThrowException(YarnRemoteExceptionPBImpl.java:135)
>   at 
> org.apache.hadoop.yarn.api.impl.pb.client.ClientRMProtocolPBClientImpl.getNewApplication(ClientRMProtocolPBClientImpl.java:134)
>   at 
> org.apache.hadoop.mapred.ResourceMgrDelegate.getNewJobID(ResourceMgrDelegate.java:183)
>   at org.apache.hadoop.mapred.YARNRunner.getNewJobID(YARNRunner.java:216)
>   at 
> org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:339)
>   at org.apache.hadoop.mapreduce.Job$11.run(Job.java:1226)
>   at org.apache.hadoop.mapreduce.Job$11.run(Job.java:1223)
>   at java.security.AccessController.doPrivileged(Native Method)
>   at javax.security.auth.Subject.doAs(Subject.java:416)
>   at 
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1232)
>   at org.apache.hadoop.mapreduce.Job.submit(Job.java:1223)
>   at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:1244)
>   at 
> org.apache.hadoop.hbase.mapreduce.TestTableMapReduce.runTestOnTable(TestTableMapReduce.java:151)
>   at 
> org.apache.hadoop.hbase.mapreduce.TestTableMapReduce.testMultiRegionTable(TestTableMapReduce.java:129)
>   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>   at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>   at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>   at java.lang.reflect.Method.invoke(Method.java:616)
>   at 
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45)
>   at 
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
>   at 
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42)
>   at 
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
>   at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:47)
>   at org.junit.rules.RunRules.evaluate(RunRules.java:18)
>   at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:263)
>   at 
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:68)
>   at 
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:47)
>   at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231)
>   at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:60)
>   at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:229)
>   at org.junit.runners.ParentRunner.access$000(ParentRunner.java:50)
>   at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:222)
>   at 
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28)
>   at 
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAf

[jira] [Commented] (HBASE-5966) MapReduce based tests broken on Hadoop 2.0.0-alpha

2012-05-08 Thread Arun C Murthy (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-5966?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13271065#comment-13271065
 ] 

Arun C Murthy commented on HBASE-5966:
--

Btw, for my education, -Dhadoop.profile just flips some code in hbase and 
doesn't mean we are getting different versions of hadoop-mapreduce, correct?

> MapReduce based tests broken on Hadoop 2.0.0-alpha
> --
>
> Key: HBASE-5966
> URL: https://issues.apache.org/jira/browse/HBASE-5966
> Project: HBase
>  Issue Type: Bug
>  Components: mapred, mapreduce, test
>Affects Versions: 0.94.0, 0.96.0
> Environment: Hadoop 2.0.0-alpha-SNAPSHOT, HBase 0.94.0-SNAPSHOT, 
> Ubuntu 12.04 LTS (GNU/Linux 3.2.0-24-generic x86_64)
>Reporter: Andrew Purtell
>
> Some fairly recent change in Hadoop 2.0.0-alpha has broken our MapReduce test 
> rigging. Below is a representative error, can be easily reproduced with:
> {noformat}
> mvn -PlocalTests -Psecurity \
>   -Dhadoop.profile=23 -Dhadoop.version=2.0.0-SNAPSHOT \
>   clean test \
>   -Dtest=org.apache.hadoop.hbase.mapreduce.TestTableMapReduce
> {noformat}
> And the result:
> {noformat}
> ---
>  T E S T S
> ---
> Running org.apache.hadoop.hbase.mapreduce.TestTableMapReduce
> Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 54.292 sec 
> <<< FAILURE!
> ---
> Test set: org.apache.hadoop.hbase.mapreduce.TestTableMapReduce
> ---
> Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 54.292 sec 
> <<< FAILURE!
> testMultiRegionTable(org.apache.hadoop.hbase.mapreduce.TestTableMapReduce)  
> Time elapsed: 21.935 sec  <<< ERROR!
> java.lang.reflect.UndeclaredThrowableException
>   at 
> org.apache.hadoop.yarn.exceptions.impl.pb.YarnRemoteExceptionPBImpl.unwrapAndThrowException(YarnRemoteExceptionPBImpl.java:135)
>   at 
> org.apache.hadoop.yarn.api.impl.pb.client.ClientRMProtocolPBClientImpl.getNewApplication(ClientRMProtocolPBClientImpl.java:134)
>   at 
> org.apache.hadoop.mapred.ResourceMgrDelegate.getNewJobID(ResourceMgrDelegate.java:183)
>   at org.apache.hadoop.mapred.YARNRunner.getNewJobID(YARNRunner.java:216)
>   at 
> org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:339)
>   at org.apache.hadoop.mapreduce.Job$11.run(Job.java:1226)
>   at org.apache.hadoop.mapreduce.Job$11.run(Job.java:1223)
>   at java.security.AccessController.doPrivileged(Native Method)
>   at javax.security.auth.Subject.doAs(Subject.java:416)
>   at 
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1232)
>   at org.apache.hadoop.mapreduce.Job.submit(Job.java:1223)
>   at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:1244)
>   at 
> org.apache.hadoop.hbase.mapreduce.TestTableMapReduce.runTestOnTable(TestTableMapReduce.java:151)
>   at 
> org.apache.hadoop.hbase.mapreduce.TestTableMapReduce.testMultiRegionTable(TestTableMapReduce.java:129)
>   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>   at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>   at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>   at java.lang.reflect.Method.invoke(Method.java:616)
>   at 
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45)
>   at 
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
>   at 
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42)
>   at 
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
>   at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:47)
>   at org.junit.rules.RunRules.evaluate(RunRules.java:18)
>   at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:263)
>   at 
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:68)
>   at 
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:47)
>   at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231)
>   at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:60)
>   at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:229)
>   at org.junit.runners.ParentRunner.access$000(ParentRunner.java:50)
>   at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:222)
>   at 
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28)
>   at 
> org.junit

[jira] [Commented] (HBASE-5966) MapReduce based tests broken on Hadoop 2.0.0-alpha

2012-05-08 Thread Andrew Purtell (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-5966?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13271064#comment-13271064
 ] 

Andrew Purtell commented on HBASE-5966:
---

Anything that uses the minicluster fails. I only checked the output of a few 
but all had the same type and order of exceptions. 


> MapReduce based tests broken on Hadoop 2.0.0-alpha
> --
>
> Key: HBASE-5966
> URL: https://issues.apache.org/jira/browse/HBASE-5966
> Project: HBase
>  Issue Type: Bug
>  Components: mapred, mapreduce, test
>Affects Versions: 0.94.0, 0.96.0
> Environment: Hadoop 2.0.0-alpha-SNAPSHOT, HBase 0.94.0-SNAPSHOT, 
> Ubuntu 12.04 LTS (GNU/Linux 3.2.0-24-generic x86_64)
>Reporter: Andrew Purtell
>
> Some fairly recent change in Hadoop 2.0.0-alpha has broken our MapReduce test 
> rigging. Below is a representative error, can be easily reproduced with:
> {noformat}
> mvn -PlocalTests -Psecurity \
>   -Dhadoop.profile=23 -Dhadoop.version=2.0.0-SNAPSHOT \
>   clean test \
>   -Dtest=org.apache.hadoop.hbase.mapreduce.TestTableMapReduce
> {noformat}
> And the result:
> {noformat}
> ---
>  T E S T S
> ---
> Running org.apache.hadoop.hbase.mapreduce.TestTableMapReduce
> Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 54.292 sec 
> <<< FAILURE!
> ---
> Test set: org.apache.hadoop.hbase.mapreduce.TestTableMapReduce
> ---
> Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 54.292 sec 
> <<< FAILURE!
> testMultiRegionTable(org.apache.hadoop.hbase.mapreduce.TestTableMapReduce)  
> Time elapsed: 21.935 sec  <<< ERROR!
> java.lang.reflect.UndeclaredThrowableException
>   at 
> org.apache.hadoop.yarn.exceptions.impl.pb.YarnRemoteExceptionPBImpl.unwrapAndThrowException(YarnRemoteExceptionPBImpl.java:135)
>   at 
> org.apache.hadoop.yarn.api.impl.pb.client.ClientRMProtocolPBClientImpl.getNewApplication(ClientRMProtocolPBClientImpl.java:134)
>   at 
> org.apache.hadoop.mapred.ResourceMgrDelegate.getNewJobID(ResourceMgrDelegate.java:183)
>   at org.apache.hadoop.mapred.YARNRunner.getNewJobID(YARNRunner.java:216)
>   at 
> org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:339)
>   at org.apache.hadoop.mapreduce.Job$11.run(Job.java:1226)
>   at org.apache.hadoop.mapreduce.Job$11.run(Job.java:1223)
>   at java.security.AccessController.doPrivileged(Native Method)
>   at javax.security.auth.Subject.doAs(Subject.java:416)
>   at 
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1232)
>   at org.apache.hadoop.mapreduce.Job.submit(Job.java:1223)
>   at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:1244)
>   at 
> org.apache.hadoop.hbase.mapreduce.TestTableMapReduce.runTestOnTable(TestTableMapReduce.java:151)
>   at 
> org.apache.hadoop.hbase.mapreduce.TestTableMapReduce.testMultiRegionTable(TestTableMapReduce.java:129)
>   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>   at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>   at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>   at java.lang.reflect.Method.invoke(Method.java:616)
>   at 
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45)
>   at 
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
>   at 
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42)
>   at 
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
>   at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:47)
>   at org.junit.rules.RunRules.evaluate(RunRules.java:18)
>   at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:263)
>   at 
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:68)
>   at 
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:47)
>   at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231)
>   at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:60)
>   at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:229)
>   at org.junit.runners.ParentRunner.access$000(ParentRunner.java:50)
>   at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:222)
>   at 
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28)
>   at 
> org.junit.internal.runners.s

[jira] [Commented] (HBASE-5966) MapReduce based tests broken on Hadoop 2.0.0-alpha

2012-05-08 Thread Arun C Murthy (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-5966?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13271059#comment-13271059
 ] 

Arun C Murthy commented on HBASE-5966:
--

Andy, I assume you are seeing more than one test fail with similar errors?

> MapReduce based tests broken on Hadoop 2.0.0-alpha
> --
>
> Key: HBASE-5966
> URL: https://issues.apache.org/jira/browse/HBASE-5966
> Project: HBase
>  Issue Type: Bug
>  Components: mapred, mapreduce, test
>Affects Versions: 0.94.0, 0.96.0
> Environment: Hadoop 2.0.0-alpha-SNAPSHOT, HBase 0.94.0-SNAPSHOT, 
> Ubuntu 12.04 LTS (GNU/Linux 3.2.0-24-generic x86_64)
>Reporter: Andrew Purtell
>
> Some fairly recent change in Hadoop 2.0.0-alpha has broken our MapReduce test 
> rigging. Below is a representative error, can be easily reproduced with:
> {noformat}
> mvn -PlocalTests -Psecurity \
>   -Dhadoop.profile=23 -Dhadoop.version=2.0.0-SNAPSHOT \
>   clean test \
>   -Dtest=org.apache.hadoop.hbase.mapreduce.TestTableMapReduce
> {noformat}
> And the result:
> {noformat}
> ---
>  T E S T S
> ---
> Running org.apache.hadoop.hbase.mapreduce.TestTableMapReduce
> Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 54.292 sec 
> <<< FAILURE!
> ---
> Test set: org.apache.hadoop.hbase.mapreduce.TestTableMapReduce
> ---
> Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 54.292 sec 
> <<< FAILURE!
> testMultiRegionTable(org.apache.hadoop.hbase.mapreduce.TestTableMapReduce)  
> Time elapsed: 21.935 sec  <<< ERROR!
> java.lang.reflect.UndeclaredThrowableException
>   at 
> org.apache.hadoop.yarn.exceptions.impl.pb.YarnRemoteExceptionPBImpl.unwrapAndThrowException(YarnRemoteExceptionPBImpl.java:135)
>   at 
> org.apache.hadoop.yarn.api.impl.pb.client.ClientRMProtocolPBClientImpl.getNewApplication(ClientRMProtocolPBClientImpl.java:134)
>   at 
> org.apache.hadoop.mapred.ResourceMgrDelegate.getNewJobID(ResourceMgrDelegate.java:183)
>   at org.apache.hadoop.mapred.YARNRunner.getNewJobID(YARNRunner.java:216)
>   at 
> org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:339)
>   at org.apache.hadoop.mapreduce.Job$11.run(Job.java:1226)
>   at org.apache.hadoop.mapreduce.Job$11.run(Job.java:1223)
>   at java.security.AccessController.doPrivileged(Native Method)
>   at javax.security.auth.Subject.doAs(Subject.java:416)
>   at 
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1232)
>   at org.apache.hadoop.mapreduce.Job.submit(Job.java:1223)
>   at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:1244)
>   at 
> org.apache.hadoop.hbase.mapreduce.TestTableMapReduce.runTestOnTable(TestTableMapReduce.java:151)
>   at 
> org.apache.hadoop.hbase.mapreduce.TestTableMapReduce.testMultiRegionTable(TestTableMapReduce.java:129)
>   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>   at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>   at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>   at java.lang.reflect.Method.invoke(Method.java:616)
>   at 
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45)
>   at 
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
>   at 
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42)
>   at 
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
>   at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:47)
>   at org.junit.rules.RunRules.evaluate(RunRules.java:18)
>   at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:263)
>   at 
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:68)
>   at 
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:47)
>   at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231)
>   at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:60)
>   at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:229)
>   at org.junit.runners.ParentRunner.access$000(ParentRunner.java:50)
>   at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:222)
>   at 
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28)
>   at 
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:30)
>   at

[jira] [Commented] (HBASE-5929) HBaseAdmin.compact and flush are giving confusing errors for ROOT, META, and regions that don't exist

2012-05-08 Thread Jean-Daniel Cryans (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-5929?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13271054#comment-13271054
 ] 

Jean-Daniel Cryans commented on HBASE-5929:
---

I spawed HBASE-5969 for the apparent randomness of the issue with the region 
that's pre 0.89

> HBaseAdmin.compact and flush are giving confusing errors for ROOT, META, and 
> regions that don't exist
> -
>
> Key: HBASE-5929
> URL: https://issues.apache.org/jira/browse/HBASE-5929
> Project: HBase
>  Issue Type: Bug
>  Components: client, shell
>Affects Versions: 0.92.1
> Environment: Linux Ubuntu Lucid 64bit
>Reporter: Aravind Gottipati
>Priority: Minor
>
> I have been noticing that calls to HBaseAdmin.majorCompact throws exceptions 
> randomly for some regions.  I could not find a pattern to these exception.  
> The code I have simply does this 
> admin.majorCompact(region.getRegionNameAsString()).  admin is an instance of 
> HBaseAdmin and region is an instance of HRegionInfo.  The exception I get is 
> org.apache.hadoop.hbase.TableNotFoundException: -ROOT-,,0
> at 
> org.apache.hadoop.hbase.client.HBaseAdmin.tableNameString(HBaseAdmin.java:1473)
>  ~[hbase-0.92.1.jar:0.92.1]
> at 
> org.apache.hadoop.hbase.client.HBaseAdmin.compact(HBaseAdmin.java:1235) 
> ~[hbase-0.92.1.jar:0.92.1]
> at 
> org.apache.hadoop.hbase.client.HBaseAdmin.majorCompact(HBaseAdmin.java:1209) 
> ~[hbase-0.92.1.jar:0.92.1]
> at com.stumbleupon.hbaseadmin.HBaseCompact.compactAllServers(Unknown 
> Source) [hbase_compact.jar:na]
> In this case it's the root region, but I get similar exceptions for other 
> tables, like this.
> 2012-05-03 19:03:42,994 WARN  [main] HBaseCompact: Could not compact:
> org.apache.hadoop.hbase.TableNotFoundException: 
> ad_daily,49842:2009-07-10,1269763588508.1997607018
> at 
> org.apache.hadoop.hbase.client.HBaseAdmin.tableNameString(HBaseAdmin.java:1473)
>  ~[hbase-0.92.1.jar:0.92.1]
> at 
> org.apache.hadoop.hbase.client.HBaseAdmin.compact(HBaseAdmin.java:1235) 
> ~[hbase-0.92.1.jar:0.92.1]
> at 
> org.apache.hadoop.hbase.client.HBaseAdmin.majorCompact(HBaseAdmin.java:1209) 
> ~[hbase-0.92.1.jar:0.92.1]
> at 
> org.apache.hadoop.hbase.client.HBaseAdmin.majorCompact(HBaseAdmin.java:1196) 
> ~[hbase-0.92.1.jar:0.92.1]
> at com.stumbleupon.hbaseadmin.HBaseCompact.compactAllServers(Unknown 
> Source) [hbase_compact.jar:na]
> at com.stumbleupon.hbaseadmin.HBaseCompact.main(Unknown Source) 
> [hbase_compact.jar:na]
> I see this on hbase shell as well.  However, I don't see these exceptions if 
> I use admin.majorCompact(region.getRegionName()), so it looks like something 
> gets lost when I use getRegionNameAsString().
> Let me know if I can provide more information.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (HBASE-5969) HRI.getRegionName/AsString are inconsistent for regions with the old encoding scheme

2012-05-08 Thread Jean-Daniel Cryans (JIRA)

Jean-Daniel Cryans created HBASE-5969:
-

 Summary: HRI.getRegionName/AsString are inconsistent for regions 
with the old encoding scheme
 Key: HBASE-5969
 URL: https://issues.apache.org/jira/browse/HBASE-5969
 Project: HBase
  Issue Type: Bug
Affects Versions: 0.92.1, 0.90.6
Reporter: Jean-Daniel Cryans


HRI.getRegionName and getRegionNameAsString don't give consistent results for 
regions created with the old encoding scheme. See Aravind's comment in 
HBASE-5929 on how a region was appended with ".1290583321" when its HRI was 
queried with getRegionNameAsString and, once passed to HBA, wasn't able to 
compact it (I checked .META. and it doesn't have that last part so it must come 
from getRegionName).

This one might be a little hard to fix if we're already dependent on the broken 
behavior of getRegionName.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5927) AM#unassign should handle local exceptions after calling sendRegionClose

2012-05-08 Thread Jieshan Bean (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-5927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13271046#comment-13271046
 ] 

Jieshan Bean commented on HBASE-5927:
-

I didn't do anything to ensure that test case can reproduce it each time. Just 
give a simple test case which can reproduced with a high probability. 
We have an idea on how to fix this, and will provide a patch after 
verification. Thank you.

> AM#unassign should handle local exceptions after calling sendRegionClose
> 
>
> Key: HBASE-5927
> URL: https://issues.apache.org/jira/browse/HBASE-5927
> Project: HBase
>  Issue Type: Bug
>  Components: master
>Affects Versions: 0.92.1, 0.96.0, 0.94.1
>Reporter: Jieshan Bean
>Assignee: Jieshan Bean
> Fix For: 0.92.2, 0.96.0, 0.94.1
>
> Attachments: TestCaseForReProduce.txt
>
>
> A possible exception: If the related regionserver was just killed(But HMaster 
> has not perceived that), then we will get a local exception "Connection reset 
> by peer". If this region belongs to a disabling table. what will happen?
> ServerShutdownHandler will remove this region from AM#regions. So this region 
> is still existing in RIT. TimeoutMonitor will take care of it after it got 
> timeout. Then invoke unassign again. Since this region has been removed from 
> AM#regions, it will return directly due to the below code:
> {code}
> synchronized (this.regions) {
>   // Check if this region is currently assigned
>   if (!regions.containsKey(region)) {
> LOG.debug("Attempted to unassign region " +
>   region.getRegionNameAsString() + " but it is not " +
>   "currently assigned anywhere");
> return;
>   }
> }
> {code}
> Then it leads to an end-less loop.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5957) Mismatch with config param name in xml and code

2012-05-08 Thread Anoop Sam John (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-5957?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13271038#comment-13271038
 ] 

Anoop Sam John commented on HBASE-5957:
---

{quote}-1 tests included. The patch doesn't appear to include any new or 
modified tests.
Please justify why no new tests are needed for this patch.
Also please list what manual steps were performed to verify this patch.
{quote}
Change is wrt correcting the xml entry in the hbase-default.xml

{quote}
-1 core tests. The patch failed these unit tests:
org.apache.hadoop.hbase.regionserver.wal.TestHLog
{quote}
Patch has nothing to do with HLog. Failure is not related to this patch I 
think. Run tests in TestHLog locally and all passed

> Mismatch with config param name in xml and code
> ---
>
> Key: HBASE-5957
> URL: https://issues.apache.org/jira/browse/HBASE-5957
> Project: HBase
>  Issue Type: Bug
>Reporter: Anoop Sam John
> Attachments: HBASE-5957.patch
>
>
> In hbase-default.xml there listed one param "io.storefile.bloom.cacheonwrite"
> {code}
> 
>   io.storefile.bloom.cacheonwrite
>   false
>   
> Enables cache-on-write for inline blocks of a compound Bloom filter.
>   
> 
> {code}
> No place in the code this config is getting used.
> Instead in CacheConfig there is a config param name 
> {code}
> /**
>* Configuration key to cache compound bloom filter blocks on write.
>*/
>   public static final String CACHE_BLOOM_BLOCKS_ON_WRITE_KEY =
>   "hfile.block.bloom.cacheonwrite";
> {code}
> Seems issue with entry in the xml file.
> We can correct the xml with the config name as hfile.block.bloom.cacheonwrite 
> ?

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5754) data lost with gora continuous ingest test (goraci)

2012-05-08 Thread Enis Soztutar (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-5754?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13271036#comment-13271036
 ] 

Enis Soztutar commented on HBASE-5754:
--

In one of my 0.92.x tests on a 10 node cluster, 250M inserts, I did manage to 
get the verify to fail: 
{code}
12/05/08 11:11:18 INFO mapred.JobClient:   goraci.Verify$Counts
12/05/08 11:11:18 INFO mapred.JobClient: UNDEFINED=972506
12/05/08 11:11:18 INFO mapred.JobClient: REFERENCED=248051318
12/05/08 11:11:18 INFO mapred.JobClient: UNREFERENCED=972506
12/05/08 11:11:18 INFO mapred.JobClient:   Map-Reduce Framework
12/05/08 11:11:18 INFO mapred.JobClient: Map input records=249023824
{code}

Notice that map input records is 1M less that 250M, which indicates that the 
inputformat did not provide all records in the table. The missing rows all 
belong to the single region. I rerun the test again after a couple of hours, 
and it passed. But the failed test created 244 maps, instead of 246, which is 
the current region count, so I am suspecting there is something wrong in the 
split calculation or in the supposed transactional behavior for split/balance 
operations in the meta table. I am still inspecting the code and the logs, but 
any pointers are welcome. 

> data lost with gora continuous ingest test (goraci)
> ---
>
> Key: HBASE-5754
> URL: https://issues.apache.org/jira/browse/HBASE-5754
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 0.92.1
> Environment: 10 node test cluster
>Reporter: Eric Newton
>Assignee: stack
>
> Keith Turner re-wrote the accumulo continuous ingest test using gora, which 
> has both hbase and accumulo back-ends.
> I put a billion entries into HBase, and ran the Verify map/reduce job.  The 
> verification failed because about 21K entries were missing.  The goraci 
> [README|https://github.com/keith-turner/goraci] explains the test, and how it 
> detects missing data.
> I re-ran the test with 100 million entries, and it verified successfully.  
> Both of the times I tested using a billion entries, the verification failed.
> If I run the verification step twice, the results are consistent, so the 
> problem is
> probably not on the verify step.
> Here's the versions of the various packages:
> ||package||version||
> |hadoop|0.20.205.0|
> |hbase|0.92.1|
> |gora|http://svn.apache.org/repos/asf/gora/trunk r1311277|
> |goraci|https://github.com/ericnewton/goraci  tagged 2012-04-08|
> The change I made to goraci was to configure it for hbase and to allow it to 
> build properly.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5963) ClassCastException: FileSystem$Cache$ClientFinalizer cannot be cast to Thread

2012-05-08 Thread Andrew Purtell (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-5963?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13271033#comment-13271033
 ] 

Andrew Purtell commented on HBASE-5963:
---

The bulk of the work here isn't managing the shutdown hook, it is dealing with 
the different Hadoop versions through special casing and reflection. If you'd 
like to make a more involved change here that introduces a shim and works more 
nicely with 2.0, that would be great. I opted for lower risk of breaking 1.0, 
CDH, etc. It's definitely a debatable choice. 


> ClassCastException: FileSystem$Cache$ClientFinalizer cannot be cast to Thread
> -
>
> Key: HBASE-5963
> URL: https://issues.apache.org/jira/browse/HBASE-5963
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 0.94.0, 0.96.0
> Environment: Bigtop 0.4.0-incubating-SNAPSHOT, Hadoop 
> 2.0.0-alpha-SNAPSHOT, HBase 0.94.0-SNAPSHOT
>Reporter: Andrew Purtell
>Assignee: Andrew Purtell
> Attachments: HBASE-5963.patch, HBASE-5963.patch
>
>
> {code}
> 12/05/08 19:49:26 INFO regionserver.HRegionServer: STOPPED: Failed 
> initialization
> Exception in thread "main" java.lang.ClassCastException: 
> org.apache.hadoop.fs.FileSystem$Cache$ClientFinalizer cannot be cast to 
> java.lang.Thread
>   at 
> org.apache.hadoop.hbase.regionserver.ShutdownHook.suppressHdfsShutdownHook(ShutdownHook.java:181)
>   at 
> org.apache.hadoop.hbase.regionserver.ShutdownHook.install(ShutdownHook.java:82)
>   at 
> org.apache.hadoop.hbase.regionserver.HRegionServer.startRegionServer(HRegionServer.java:3601)
>   at 
> org.apache.hadoop.hbase.regionserver.HRegionServer.startRegionServer(HRegionServer.java:3585)
>   at 
> org.apache.hadoop.hbase.regionserver.HRegionServerCommandLine.start(HRegionServerCommandLine.java:61)
>   at 
> org.apache.hadoop.hbase.regionserver.HRegionServerCommandLine.run(HRegionServerCommandLine.java:75)
>   at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
>   at 
> org.apache.hadoop.hbase.util.ServerCommandLine.doMain(ServerCommandLine.java:76)
>   at 
> org.apache.hadoop.hbase.regionserver.HRegionServer.main(HRegionServer.java:3645)
> {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5963) ClassCastException: FileSystem$Cache$ClientFinalizer cannot be cast to Thread

2012-05-08 Thread Zhihong Yu (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-5963?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13271031#comment-13271031
 ] 

Zhihong Yu commented on HBASE-5963:
---

ShutdownHookManager from hadoop 2.0 is only 181 lines long.
It shouldn't be hard to produce our ShutdownHookManager shim.

> ClassCastException: FileSystem$Cache$ClientFinalizer cannot be cast to Thread
> -
>
> Key: HBASE-5963
> URL: https://issues.apache.org/jira/browse/HBASE-5963
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 0.94.0, 0.96.0
> Environment: Bigtop 0.4.0-incubating-SNAPSHOT, Hadoop 
> 2.0.0-alpha-SNAPSHOT, HBase 0.94.0-SNAPSHOT
>Reporter: Andrew Purtell
>Assignee: Andrew Purtell
> Attachments: HBASE-5963.patch, HBASE-5963.patch
>
>
> {code}
> 12/05/08 19:49:26 INFO regionserver.HRegionServer: STOPPED: Failed 
> initialization
> Exception in thread "main" java.lang.ClassCastException: 
> org.apache.hadoop.fs.FileSystem$Cache$ClientFinalizer cannot be cast to 
> java.lang.Thread
>   at 
> org.apache.hadoop.hbase.regionserver.ShutdownHook.suppressHdfsShutdownHook(ShutdownHook.java:181)
>   at 
> org.apache.hadoop.hbase.regionserver.ShutdownHook.install(ShutdownHook.java:82)
>   at 
> org.apache.hadoop.hbase.regionserver.HRegionServer.startRegionServer(HRegionServer.java:3601)
>   at 
> org.apache.hadoop.hbase.regionserver.HRegionServer.startRegionServer(HRegionServer.java:3585)
>   at 
> org.apache.hadoop.hbase.regionserver.HRegionServerCommandLine.start(HRegionServerCommandLine.java:61)
>   at 
> org.apache.hadoop.hbase.regionserver.HRegionServerCommandLine.run(HRegionServerCommandLine.java:75)
>   at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
>   at 
> org.apache.hadoop.hbase.util.ServerCommandLine.doMain(ServerCommandLine.java:76)
>   at 
> org.apache.hadoop.hbase.regionserver.HRegionServer.main(HRegionServer.java:3645)
> {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5968) Proper html escaping for region names

2012-05-08 Thread Enis Soztutar (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-5968?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Enis Soztutar updated HBASE-5968:
-

Description: 
I noticed that we are not doing html escaping for the rs/master web interfaces, 
so you can end up generating html like: 
{code}

  ci,,\xEEp/
  
  
hostname
  
  
  ,\xEEp/
  -n\xA8\xE0\x15\xDD\x80!
  2966724

{code}

This obviously does not render properly. 

Also, my crazy theory is that it can be a security risk. Since the region name 
is computed from table rows, which are most of the time user input. Thus if  
the rows contain a "

[jira] [Created] (HBASE-5968) Proper html escaping for region names

2012-05-08 Thread Enis Soztutar (JIRA)

Enis Soztutar created HBASE-5968:


 Summary: Proper html escaping for region names
 Key: HBASE-5968
 URL: https://issues.apache.org/jira/browse/HBASE-5968
 Project: HBase
  Issue Type: Bug
  Components: util
Affects Versions: 0.96.0
Reporter: Enis Soztutar
Assignee: Enis Soztutar


I noticed that we are not doing html escaping for the rs/master web interfaces, 
so you can end up generating html like: 
{code}

  ci,,\xEEp/
  
  
http://hrt24n06.cc1.ygridcore.net:60030/";>hrt24n06.cc1.ygridcore.net:60030
  
  
  ,\xEEp/
  -n\xA8\xE0\x15\xDD\x80!
  2966724

{code}

This obviously does not render properly. 

Also, my crazy theory is that it can be a security risk. Since the region name 
is computed from table rows, which are most of the time user input. Thus if  
the rows contain a "

[jira] [Created] (HBASE-5967) ServerLoad OpenDataException

2012-05-08 Thread Jimmy Xiang (JIRA)

Jimmy Xiang created HBASE-5967:
--

 Summary: ServerLoad OpenDataException
 Key: HBASE-5967
 URL: https://issues.apache.org/jira/browse/HBASE-5967
 Project: HBase
  Issue Type: Bug
Affects Versions: 0.96.0
Reporter: Jimmy Xiang
Priority: Minor
 Fix For: 0.96.0
 Attachments: master.log

I saw this error in the master log:

Caused by: java.lang.IllegalArgumentException: Method 
org.apache.hadoop.hbase.master.MXBean.getRegionServers has parameter or return 
type that cannot be translated into an open type
at com.sun.jmx.mbeanserver.ConvertingMethod.from(ConvertingMethod.java:32)
at 
com.sun.jmx.mbeanserver.MXBeanIntrospector.mFrom(MXBeanIntrospector.java:63)
at 
com.sun.jmx.mbeanserver.MXBeanIntrospector.mFrom(MXBeanIntrospector.java:33)
at com.sun.jmx.mbeanserver.MBeanAnalyzer.initMaps(MBeanAnalyzer.java:118)
at com.sun.jmx.mbeanserver.MBeanAnalyzer.(MBeanAnalyzer.java:99)
... 14 more
Caused by: javax.management.openmbean.OpenDataException: Cannot convert type: 
java.util.Map
at com.sun.jmx.mbeanserver.OpenConverter.openDataException(OpenConverter.jav



--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5967) ServerLoad OpenDataException

2012-05-08 Thread Jimmy Xiang (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-5967?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jimmy Xiang updated HBASE-5967:
---

Attachment: master.log

> ServerLoad OpenDataException
> 
>
> Key: HBASE-5967
> URL: https://issues.apache.org/jira/browse/HBASE-5967
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 0.96.0
>Reporter: Jimmy Xiang
>Priority: Minor
> Fix For: 0.96.0
>
> Attachments: master.log
>
>
> I saw this error in the master log:
> Caused by: java.lang.IllegalArgumentException: Method 
> org.apache.hadoop.hbase.master.MXBean.getRegionServers has parameter or 
> return type that cannot be translated into an open type
> at com.sun.jmx.mbeanserver.ConvertingMethod.from(ConvertingMethod.java:32)
> at 
> com.sun.jmx.mbeanserver.MXBeanIntrospector.mFrom(MXBeanIntrospector.java:63)
> at 
> com.sun.jmx.mbeanserver.MXBeanIntrospector.mFrom(MXBeanIntrospector.java:33)
> at com.sun.jmx.mbeanserver.MBeanAnalyzer.initMaps(MBeanAnalyzer.java:118)
> at com.sun.jmx.mbeanserver.MBeanAnalyzer.(MBeanAnalyzer.java:99)
> ... 14 more
> Caused by: javax.management.openmbean.OpenDataException: Cannot convert type: 
> java.util.Map
> at 
> com.sun.jmx.mbeanserver.OpenConverter.openDataException(OpenConverter.jav

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5964) HFileSystem: "No FileSystem for scheme: hdfs"

2012-05-08 Thread Zhihong Yu (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-5964?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13271019#comment-13271019
 ] 

Zhihong Yu commented on HBASE-5964:
---

Probably it was specific to the build machine.
Resubmitting the patch would give us some idea.

> HFileSystem: "No FileSystem for scheme: hdfs"
> -
>
> Key: HBASE-5964
> URL: https://issues.apache.org/jira/browse/HBASE-5964
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 0.94.0, 0.96.0
> Environment: Hadoop 2.0.0-alpha-SNAPSHOT, HBase 0.94.0-SNAPSHOT
>Reporter: Andrew Purtell
>Assignee: Andrew Purtell
> Attachments: HBASE-5964.patch
>
>
> I've been seeing this with Hadoop 2.0.0-alpha-SNAPSHOT and HBase 
> 0.94.0-SNAPSHOT:
> {noformat}
> 2012-05-08 15:18:00,692 FATAL 
> [RegionServer:0;acer.localdomain,48307,1336515479011] 
> regionserver.HRegionServer(1674): ABORTING region server 
> acer.localdomain,48307,1336515479011: Unhandled exception: No FileSystem for 
> scheme: hdfs
> java.io.IOException: No FileSystem for scheme: hdfs
>   at 
> org.apache.hadoop.hbase.fs.HFileSystem.newInstanceFileSystem(HFileSystem.java:146)
>   at org.apache.hadoop.hbase.fs.HFileSystem.(HFileSystem.java:75)
>   at 
> org.apache.hadoop.hbase.regionserver.HRegionServer.handleReportForDutyResponse(HRegionServer.java:973)
>   at 
> org.apache.hadoop.hbase.MiniHBaseCluster$MiniHBaseClusterRegionServer.handleReportForDutyResponse(MiniHBaseCluster.java:110)
>   at 
> org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:671)
>   at 
> org.apache.hadoop.hbase.MiniHBaseCluster$MiniHBaseClusterRegionServer.runRegionServer(MiniHBaseCluster.java:136)
>   at 
> org.apache.hadoop.hbase.MiniHBaseCluster$MiniHBaseClusterRegionServer.access$000(MiniHBaseCluster.java:89)
>   at 
> org.apache.hadoop.hbase.MiniHBaseCluster$MiniHBaseClusterRegionServer$1.run(MiniHBaseCluster.java:120)
>   at java.security.AccessController.doPrivileged(Native Method)
>   at javax.security.auth.Subject.doAs(Subject.java:357)
>   at 
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1212)
>   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>   at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>   at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>   at java.lang.reflect.Method.invoke(Method.java:616)
>   at org.apache.hadoop.hbase.util.Methods.call(Methods.java:37)
>   at org.apache.hadoop.hbase.security.User.call(User.java:586)
>   at org.apache.hadoop.hbase.security.User.access$700(User.java:50)
>   at 
> org.apache.hadoop.hbase.security.User$SecureHadoopUser.runAs(User.java:426)
>   at 
> org.apache.hadoop.hbase.MiniHBaseCluster$MiniHBaseClusterRegionServer.run(MiniHBaseCluster.java:118)
>   at java.lang.Thread.run(Thread.java:679)
> {noformat}
> Not sure precisely when it started. First I thought it might be due to 
> HBASE-5955 but I cherry picked that change over from trunk. Then I got 
> HBASE-5963 out of the way.  

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5828) TestLogRolling fails in 0.90 branch killing the test suite up on jenkins

2012-05-08 Thread xufeng (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-5828?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

xufeng updated HBASE-5828:
--

Attachment: TestLogRolling.java

org.apache.hadoop.hbase.regionserver.wal.TestLogRolling-output.rar

1.The reproduce result.
2.The test file thar be added some debug code.

> TestLogRolling fails in 0.90 branch killing the test suite up on jenkins
> 
>
> Key: HBASE-5828
> URL: https://issues.apache.org/jira/browse/HBASE-5828
> Project: HBase
>  Issue Type: Bug
>Reporter: stack
>Assignee: xufeng
> Attachments: TestLogRolling.java, 
> org.apache.hadoop.hbase.regionserver.wal.TestLogRolling-output.rar
>
>
> See recent 0.90 builds up on jenkins: 
> https://builds.apache.org/view/G-L/view/HBase/job/hbase-0.90/471/console

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Assigned] (HBASE-5828) TestLogRolling fails in 0.90 branch killing the test suite up on jenkins

2012-05-08 Thread xufeng (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-5828?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

xufeng reassigned HBASE-5828:
-

Assignee: xufeng

> TestLogRolling fails in 0.90 branch killing the test suite up on jenkins
> 
>
> Key: HBASE-5828
> URL: https://issues.apache.org/jira/browse/HBASE-5828
> Project: HBase
>  Issue Type: Bug
>Reporter: stack
>Assignee: xufeng
>
> See recent 0.90 builds up on jenkins: 
> https://builds.apache.org/view/G-L/view/HBase/job/hbase-0.90/471/console

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5828) TestLogRolling fails in 0.90 branch killing the test suite up on jenkins

2012-05-08 Thread xufeng (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-5828?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13271013#comment-13271013
 ] 

xufeng commented on HBASE-5828:
---

My Conclusion:
The the log writer pipeline of meta regionserver has been destroyed by 
testLogRollOnDatanodeDeath().
when operate the Hlog on meta regionserver it will failed.

I think this is not a bug just a test problem.Like the other versions,We scan 
segregate the test cluster by @before and @after tag.


My My Analysis::
1.I check the log from 
https://builds.apache.org/view/G-L/view/HBase/job/hbase-0.90/472/artifact/trunk/target/surefire-reports/org.apache.hadoop.hbase.regionserver.wal.TestLogRolling-output.txt
Found the meta region and user region be deployed on different RS.It may cause 
the problem.
{noformat}
Line 226: 2012-04-15 09:19:11,325 INFO  
[MASTER_OPEN_REGION-hemera.apache.org:45266-0] 
handler.OpenedRegionHandler(137): The master has opened the region 
-ROOT-,,0.70236052 that was online on 
serverName=hemera.apache.org,46368,1334481545408, load=(requests=0, regions=0, 
usedHeap=71, maxHeap=1244)
Line 288: 2012-04-15 09:19:11,379 INFO  
[MASTER_OPEN_REGION-hemera.apache.org:45266-1] 
handler.OpenedRegionHandler(137): The master has opened the region 
.META.,,1.1028785192 that was online on 
serverName=hemera.apache.org,46368,1334481545408, load=(requests=0, regions=0, 
usedHeap=71, maxHeap=1244)
Line 504: 2012-04-15 09:19:21,513 DEBUG 
[MASTER_OPEN_REGION-hemera.apache.org:45266-2] 
handler.OpenedRegionHandler(139): The master has opened the region 
TestLogRolling,,1334481561377.369bae2d9411e5836a17df5e31b07b5e. that was online 
on serverName=hemera.apache.org,46368,1334481545408, load=(requests=0, 
regions=2, usedHeap=76, maxHeap=1244)
Line 8652: 2012-04-15 09:20:09,099 DEBUG 
[MASTER_OPEN_REGION-hemera.apache.org:45266-3] 
handler.OpenedRegionHandler(139): The master has opened the region 
TestLogRolling,,1334481609001.198321df4a9c90eebcf670a2218d29f0. that was online 
on serverName=hemera.apache.org,34725,1334481545432, load=(requests=0, 
regions=0, usedHeap=129, maxHeap=1244)
{noformat}

2.In this issue we should find in case where the problem happened. 
I reproduce this issue again，this time I added the debug code in 
testLogRollOnPipelineRestart().
if (admin.tableExists(tableName)) {
  admin.disableTable(tableName);
  +LOG.info("xufeng7>before delete table!");
  admin.deleteTable(tableName);
  +LOG.info("xufeng7>after delete table!");
}

3.The test be killed And I just found the "before delete table!" no "after 
delete table!" string in log.
It can ensure that the prroblem happened in admin.deleteTable(tableName).
This is the region assigned info 
{noformat}
Line 226: 2012-05-08 16:45:17,603 INFO  
[MASTER_OPEN_REGION-HADOOP-CI-AGENT-A:42383-0] 
handler.OpenedRegionHandler(137): The master has opened the region 
-ROOT-,,0.70236052 that was online on 
serverName=HADOOP-CI-AGENT-A,49991,1336466711048, load=(requests=0, regions=0, 
usedHeap=49, maxHeap=1348)
Line 288: 2012-05-08 16:45:17,696 INFO  
[MASTER_OPEN_REGION-HADOOP-CI-AGENT-A:42383-1] 
handler.OpenedRegionHandler(137): The master has opened the region 
.META.,,1.1028785192 that was online on 
serverName=HADOOP-CI-AGENT-A,49991,1336466711048, load=(requests=0, regions=0, 
usedHeap=49, maxHeap=1348)
Line 505: 2012-05-08 16:45:28,099 DEBUG 
[MASTER_OPEN_REGION-HADOOP-CI-AGENT-A:42383-2] 
handler.OpenedRegionHandler(139): The master has opened the region 
TestLogRolling,,1336466727648.b119d8cad80bc39cdd38b04421a67280. that was online 
on serverName=HADOOP-CI-AGENT-A,57227,1336466711078, load=(requests=0, 
regions=0, usedHeap=54, maxHeap=1348)
Line 8495: 2012-05-08 16:46:18,169 DEBUG 
[MASTER_OPEN_REGION-HADOOP-CI-AGENT-A:42383-3] 
handler.OpenedRegionHandler(139): The master has opened the region 
TestLogRolling,,1336466777847.0117c81b3e59fdad15c4a4390156a558. that was online 
on serverName=HADOOP-CI-AGENT-A,57227,1336466711078, load=(requests=0, 
regions=0, usedHeap=68, maxHeap=1348)
{noformat}
This is my debug info:
{noformat}
Line 8526: 2012-05-08 16:46:28,138 INFO  [main] 
wal.TestLogRolling(339): 
xufeng1>cluster.getRegionServer(0)serverName=HADOOP-CI-AGENT-A,49991,1336466711048,
 load=(requests=0, regions=2, usedHeap=72, maxHeap=1348)
Line 8527: 2012-05-08 16:46:28,138 INFO  [main] 
wal.TestLogRolling(344): xufeng1>127.0.0.1:36932
Line 8528: 2012-05-08 16:46:28,138 INFO  [main] 
wal.TestLogRolling(345): xufeng1>127.0.0.1:42887
Line 8529: 2012-05-08 16:46:28,138 INFO  [main] 
wal.TestLogRolling(351): 
xufeng2>cluster.getRegionServer(1)serverName=HADOOP-CI-AGENT-A,57227,1336466711078,
 load=(requests=0, regions=1, usedHeap=72, maxHeap=1348)
Line 8530: 2012-05-08 16:46:28,138 INFO  [main] 
wal.TestLogRolling(356): x

[jira] [Commented] (HBASE-5964) HFileSystem: "No FileSystem for scheme: hdfs"

2012-05-08 Thread Andrew Purtell (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-5964?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13271008#comment-13271008
 ] 

Andrew Purtell commented on HBASE-5964:
---

You think that somehow related to this patch Ted?

> HFileSystem: "No FileSystem for scheme: hdfs"
> -
>
> Key: HBASE-5964
> URL: https://issues.apache.org/jira/browse/HBASE-5964
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 0.94.0, 0.96.0
> Environment: Hadoop 2.0.0-alpha-SNAPSHOT, HBase 0.94.0-SNAPSHOT
>Reporter: Andrew Purtell
>Assignee: Andrew Purtell
> Attachments: HBASE-5964.patch
>
>
> I've been seeing this with Hadoop 2.0.0-alpha-SNAPSHOT and HBase 
> 0.94.0-SNAPSHOT:
> {noformat}
> 2012-05-08 15:18:00,692 FATAL 
> [RegionServer:0;acer.localdomain,48307,1336515479011] 
> regionserver.HRegionServer(1674): ABORTING region server 
> acer.localdomain,48307,1336515479011: Unhandled exception: No FileSystem for 
> scheme: hdfs
> java.io.IOException: No FileSystem for scheme: hdfs
>   at 
> org.apache.hadoop.hbase.fs.HFileSystem.newInstanceFileSystem(HFileSystem.java:146)
>   at org.apache.hadoop.hbase.fs.HFileSystem.(HFileSystem.java:75)
>   at 
> org.apache.hadoop.hbase.regionserver.HRegionServer.handleReportForDutyResponse(HRegionServer.java:973)
>   at 
> org.apache.hadoop.hbase.MiniHBaseCluster$MiniHBaseClusterRegionServer.handleReportForDutyResponse(MiniHBaseCluster.java:110)
>   at 
> org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:671)
>   at 
> org.apache.hadoop.hbase.MiniHBaseCluster$MiniHBaseClusterRegionServer.runRegionServer(MiniHBaseCluster.java:136)
>   at 
> org.apache.hadoop.hbase.MiniHBaseCluster$MiniHBaseClusterRegionServer.access$000(MiniHBaseCluster.java:89)
>   at 
> org.apache.hadoop.hbase.MiniHBaseCluster$MiniHBaseClusterRegionServer$1.run(MiniHBaseCluster.java:120)
>   at java.security.AccessController.doPrivileged(Native Method)
>   at javax.security.auth.Subject.doAs(Subject.java:357)
>   at 
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1212)
>   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>   at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>   at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>   at java.lang.reflect.Method.invoke(Method.java:616)
>   at org.apache.hadoop.hbase.util.Methods.call(Methods.java:37)
>   at org.apache.hadoop.hbase.security.User.call(User.java:586)
>   at org.apache.hadoop.hbase.security.User.access$700(User.java:50)
>   at 
> org.apache.hadoop.hbase.security.User$SecureHadoopUser.runAs(User.java:426)
>   at 
> org.apache.hadoop.hbase.MiniHBaseCluster$MiniHBaseClusterRegionServer.run(MiniHBaseCluster.java:118)
>   at java.lang.Thread.run(Thread.java:679)
> {noformat}
> Not sure precisely when it started. First I thought it might be due to 
> HBASE-5955 but I cherry picked that change over from trunk. Then I got 
> HBASE-5963 out of the way.  

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5964) HFileSystem: "No FileSystem for scheme: hdfs"

2012-05-08 Thread Zhihong Yu (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-5964?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13271006#comment-13271006
 ] 

Zhihong Yu commented on HBASE-5964:
---

My script wasn't able to identify the hanging test.
I saw the following from 
https://builds.apache.org/job/PreCommit-HBASE-Build/1805/console:
{code}
Running org.apache.hadoop.hbase.io.hfile.TestLruBlockCache
Tests run: 16, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.823 sec
#
# There is insufficient memory for the Java Runtime Environment to continue.
# Native memory allocation (malloc) failed to allocate 32756 bytes for 
ChunkPool::allocate
# An error report file with more information is saved as:
# 
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/trunk/hs_err_pid19988.log
Aborted
org.apache.maven.surefire.booter.SurefireBooterForkException: Error occurred in 
starting fork, check output in log
at 
org.apache.maven.plugin.surefire.booterclient.ForkStarter.fork(ForkStarter.java:238)
at 
org.apache.maven.plugin.surefire.booterclient.ForkStarter.access$000(ForkStarter.java:64)
at 
org.apache.maven.plugin.surefire.booterclient.ForkStarter$ParallelFork.call(ForkStarter.java:303)
at 
org.apache.maven.plugin.surefire.booterclient.ForkStarter$ParallelFork.call(ForkStarter.java:285)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
Running org.apache.hadoop.hbase.io.hfile.TestHFileSeek
{code}

> HFileSystem: "No FileSystem for scheme: hdfs"
> -
>
> Key: HBASE-5964
> URL: https://issues.apache.org/jira/browse/HBASE-5964
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 0.94.0, 0.96.0
> Environment: Hadoop 2.0.0-alpha-SNAPSHOT, HBase 0.94.0-SNAPSHOT
>Reporter: Andrew Purtell
>Assignee: Andrew Purtell
> Attachments: HBASE-5964.patch
>
>
> I've been seeing this with Hadoop 2.0.0-alpha-SNAPSHOT and HBase 
> 0.94.0-SNAPSHOT:
> {noformat}
> 2012-05-08 15:18:00,692 FATAL 
> [RegionServer:0;acer.localdomain,48307,1336515479011] 
> regionserver.HRegionServer(1674): ABORTING region server 
> acer.localdomain,48307,1336515479011: Unhandled exception: No FileSystem for 
> scheme: hdfs
> java.io.IOException: No FileSystem for scheme: hdfs
>   at 
> org.apache.hadoop.hbase.fs.HFileSystem.newInstanceFileSystem(HFileSystem.java:146)
>   at org.apache.hadoop.hbase.fs.HFileSystem.(HFileSystem.java:75)
>   at 
> org.apache.hadoop.hbase.regionserver.HRegionServer.handleReportForDutyResponse(HRegionServer.java:973)
>   at 
> org.apache.hadoop.hbase.MiniHBaseCluster$MiniHBaseClusterRegionServer.handleReportForDutyResponse(MiniHBaseCluster.java:110)
>   at 
> org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:671)
>   at 
> org.apache.hadoop.hbase.MiniHBaseCluster$MiniHBaseClusterRegionServer.runRegionServer(MiniHBaseCluster.java:136)
>   at 
> org.apache.hadoop.hbase.MiniHBaseCluster$MiniHBaseClusterRegionServer.access$000(MiniHBaseCluster.java:89)
>   at 
> org.apache.hadoop.hbase.MiniHBaseCluster$MiniHBaseClusterRegionServer$1.run(MiniHBaseCluster.java:120)
>   at java.security.AccessController.doPrivileged(Native Method)
>   at javax.security.auth.Subject.doAs(Subject.java:357)
>   at 
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1212)
>   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>   at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>   at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>   at java.lang.reflect.Method.invoke(Method.java:616)
>   at org.apache.hadoop.hbase.util.Methods.call(Methods.java:37)
>   at org.apache.hadoop.hbase.security.User.call(User.java:586)
>   at org.apache.hadoop.hbase.security.User.access$700(User.java:50)
>   at 
> org.apache.hadoop.hbase.security.User$SecureHadoopUser.runAs(User.java:426)
>   at 
> org.apache.hadoop.hbase.MiniHBaseCluster$MiniHBaseClusterRegionServer.run(MiniHBaseCluster.java:118)
>   at java.lang.Thread.run(Thread.java:679)
> {noformat}
> Not sure precisely when it started. First I thought it might be due to 
> HBASE-5955 but I cherry picked that change over from trunk. Then I got 
> HBASE-5963 out of the way.  

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.o

[jira] [Commented] (HBASE-5342) Grant/Revoke global permissions

2012-05-08 Thread Andrew Purtell (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-5342?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13271001#comment-13271001
 ] 

Andrew Purtell commented on HBASE-5342:
---

@Jon: Noted, thanks for the clarification!

> Grant/Revoke global permissions
> ---
>
> Key: HBASE-5342
> URL: https://issues.apache.org/jira/browse/HBASE-5342
> Project: HBase
>  Issue Type: Sub-task
>Reporter: Enis Soztutar
>Assignee: Matteo Bertozzi
> Attachments: HBASE-5342-draft.patch, HBASE-5342-v0.patch, 
> HBASE-5342-v1.patch, HBASE-5342-v2.patch
>
>
> HBASE-3025 introduced simple ACLs based on coprocessors. It defines 
> global/table/cf/cq level permissions. However, there is no way to 
> grant/revoke global level permissions, other than the hbase.superuser conf 
> setting. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Assigned] (HBASE-5955) Guava 11 drops MapEvictionListener and Hadoop 2.0.0-alpha requires it

2012-05-08 Thread Andrew Purtell (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-5955?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrew Purtell reassigned HBASE-5955:
-

Assignee: Lars Hofhansl  (was: Andrew Purtell)

Reassign to Lars as it is not my call.

> Guava 11 drops MapEvictionListener and Hadoop 2.0.0-alpha requires it
> -
>
> Key: HBASE-5955
> URL: https://issues.apache.org/jira/browse/HBASE-5955
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 0.94.0
>Reporter: Andrew Purtell
>Assignee: Lars Hofhansl
>
> Hadoop 2.0.0-alpha depends on Guava 11.0.2. Updating HBase dependencies to 
> match produces the following compilation errors:
> {code}
> [ERROR] SingleSizeCache.java:[41,32] cannot find symbol
> [ERROR] symbol  : class MapEvictionListener
> [ERROR] location: package com.google.common.collect
> [ERROR] 
> [ERROR] SingleSizeCache.java:[94,4] cannot find symbol
> [ERROR] symbol  : class MapEvictionListener
> [ERROR] location: class org.apache.hadoop.hbase.io.hfile.slab.SingleSizeCache
> [ERROR] 
> [ERROR] SingleSizeCache.java:[94,69] cannot find symbol
> [ERROR] symbol  : class MapEvictionListener
> [ERROR] location: class org.apache.hadoop.hbase.io.hfile.slab.SingleSizeCache
> {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5963) ClassCastException: FileSystem$Cache$ClientFinalizer cannot be cast to Thread

2012-05-08 Thread Hadoop QA (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-5963?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13270997#comment-13270997
 ] 

Hadoop QA commented on HBASE-5963:
--

-1 overall.  Here are the results of testing the latest attachment 
  http://issues.apache.org/jira/secure/attachment/12526076/HBASE-5963.patch
  against trunk revision .

+1 @author.  The patch does not contain any @author tags.

-1 tests included.  The patch doesn't appear to include any new or modified 
tests.
Please justify why no new tests are needed for this 
patch.
Also please list what manual steps were performed to 
verify this patch.

+1 hadoop23.  The patch compiles against the hadoop 0.23.x profile.

+1 javadoc.  The javadoc tool did not generate any warning messages.

+1 javac.  The applied patch does not increase the total number of javac 
compiler warnings.

+1 findbugs.  The patch does not introduce any new Findbugs (version 1.3.9) 
warnings.

+1 release audit.  The applied patch does not increase the total number of 
release audit warnings.

+1 core tests.  The patch passed unit tests in .

Test results: 
https://builds.apache.org/job/PreCommit-HBASE-Build/1807//testReport/
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/1807//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
Console output: 
https://builds.apache.org/job/PreCommit-HBASE-Build/1807//console

This message is automatically generated.

> ClassCastException: FileSystem$Cache$ClientFinalizer cannot be cast to Thread
> -
>
> Key: HBASE-5963
> URL: https://issues.apache.org/jira/browse/HBASE-5963
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 0.94.0, 0.96.0
> Environment: Bigtop 0.4.0-incubating-SNAPSHOT, Hadoop 
> 2.0.0-alpha-SNAPSHOT, HBase 0.94.0-SNAPSHOT
>Reporter: Andrew Purtell
>Assignee: Andrew Purtell
> Attachments: HBASE-5963.patch, HBASE-5963.patch
>
>
> {code}
> 12/05/08 19:49:26 INFO regionserver.HRegionServer: STOPPED: Failed 
> initialization
> Exception in thread "main" java.lang.ClassCastException: 
> org.apache.hadoop.fs.FileSystem$Cache$ClientFinalizer cannot be cast to 
> java.lang.Thread
>   at 
> org.apache.hadoop.hbase.regionserver.ShutdownHook.suppressHdfsShutdownHook(ShutdownHook.java:181)
>   at 
> org.apache.hadoop.hbase.regionserver.ShutdownHook.install(ShutdownHook.java:82)
>   at 
> org.apache.hadoop.hbase.regionserver.HRegionServer.startRegionServer(HRegionServer.java:3601)
>   at 
> org.apache.hadoop.hbase.regionserver.HRegionServer.startRegionServer(HRegionServer.java:3585)
>   at 
> org.apache.hadoop.hbase.regionserver.HRegionServerCommandLine.start(HRegionServerCommandLine.java:61)
>   at 
> org.apache.hadoop.hbase.regionserver.HRegionServerCommandLine.run(HRegionServerCommandLine.java:75)
>   at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
>   at 
> org.apache.hadoop.hbase.util.ServerCommandLine.doMain(ServerCommandLine.java:76)
>   at 
> org.apache.hadoop.hbase.regionserver.HRegionServer.main(HRegionServer.java:3645)
> {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5963) ClassCastException: FileSystem$Cache$ClientFinalizer cannot be cast to Thread

2012-05-08 Thread Andrew Purtell (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-5963?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13270996#comment-13270996
 ] 

Andrew Purtell commented on HBASE-5963:
---

Ignore Hadoop QA. To verify the patch, you'll have to run tests locally with

{noformat}
mvn -PlocalTests -Psecurity \
  -Dhadoop.profile=23 -Dhadoop.version=2.0.0-SNAPSHOT \
  clean test
{noformat}

Note that all of the MR based tests are currently broken, I opened HBASE-5966.

> ClassCastException: FileSystem$Cache$ClientFinalizer cannot be cast to Thread
> -
>
> Key: HBASE-5963
> URL: https://issues.apache.org/jira/browse/HBASE-5963
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 0.94.0, 0.96.0
> Environment: Bigtop 0.4.0-incubating-SNAPSHOT, Hadoop 
> 2.0.0-alpha-SNAPSHOT, HBase 0.94.0-SNAPSHOT
>Reporter: Andrew Purtell
>Assignee: Andrew Purtell
> Attachments: HBASE-5963.patch, HBASE-5963.patch
>
>
> {code}
> 12/05/08 19:49:26 INFO regionserver.HRegionServer: STOPPED: Failed 
> initialization
> Exception in thread "main" java.lang.ClassCastException: 
> org.apache.hadoop.fs.FileSystem$Cache$ClientFinalizer cannot be cast to 
> java.lang.Thread
>   at 
> org.apache.hadoop.hbase.regionserver.ShutdownHook.suppressHdfsShutdownHook(ShutdownHook.java:181)
>   at 
> org.apache.hadoop.hbase.regionserver.ShutdownHook.install(ShutdownHook.java:82)
>   at 
> org.apache.hadoop.hbase.regionserver.HRegionServer.startRegionServer(HRegionServer.java:3601)
>   at 
> org.apache.hadoop.hbase.regionserver.HRegionServer.startRegionServer(HRegionServer.java:3585)
>   at 
> org.apache.hadoop.hbase.regionserver.HRegionServerCommandLine.start(HRegionServerCommandLine.java:61)
>   at 
> org.apache.hadoop.hbase.regionserver.HRegionServerCommandLine.run(HRegionServerCommandLine.java:75)
>   at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
>   at 
> org.apache.hadoop.hbase.util.ServerCommandLine.doMain(ServerCommandLine.java:76)
>   at 
> org.apache.hadoop.hbase.regionserver.HRegionServer.main(HRegionServer.java:3645)
> {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5964) HFileSystem: "No FileSystem for scheme: hdfs"

2012-05-08 Thread Andrew Purtell (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-5964?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13270992#comment-13270992
 ] 

Andrew Purtell commented on HBASE-5964:
---

The test report for build 1805 shows no test failures, so WTF. For the record 
I'm ignoring Hadoop QA.

> HFileSystem: "No FileSystem for scheme: hdfs"
> -
>
> Key: HBASE-5964
> URL: https://issues.apache.org/jira/browse/HBASE-5964
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 0.94.0, 0.96.0
> Environment: Hadoop 2.0.0-alpha-SNAPSHOT, HBase 0.94.0-SNAPSHOT
>Reporter: Andrew Purtell
>Assignee: Andrew Purtell
> Attachments: HBASE-5964.patch
>
>
> I've been seeing this with Hadoop 2.0.0-alpha-SNAPSHOT and HBase 
> 0.94.0-SNAPSHOT:
> {noformat}
> 2012-05-08 15:18:00,692 FATAL 
> [RegionServer:0;acer.localdomain,48307,1336515479011] 
> regionserver.HRegionServer(1674): ABORTING region server 
> acer.localdomain,48307,1336515479011: Unhandled exception: No FileSystem for 
> scheme: hdfs
> java.io.IOException: No FileSystem for scheme: hdfs
>   at 
> org.apache.hadoop.hbase.fs.HFileSystem.newInstanceFileSystem(HFileSystem.java:146)
>   at org.apache.hadoop.hbase.fs.HFileSystem.(HFileSystem.java:75)
>   at 
> org.apache.hadoop.hbase.regionserver.HRegionServer.handleReportForDutyResponse(HRegionServer.java:973)
>   at 
> org.apache.hadoop.hbase.MiniHBaseCluster$MiniHBaseClusterRegionServer.handleReportForDutyResponse(MiniHBaseCluster.java:110)
>   at 
> org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:671)
>   at 
> org.apache.hadoop.hbase.MiniHBaseCluster$MiniHBaseClusterRegionServer.runRegionServer(MiniHBaseCluster.java:136)
>   at 
> org.apache.hadoop.hbase.MiniHBaseCluster$MiniHBaseClusterRegionServer.access$000(MiniHBaseCluster.java:89)
>   at 
> org.apache.hadoop.hbase.MiniHBaseCluster$MiniHBaseClusterRegionServer$1.run(MiniHBaseCluster.java:120)
>   at java.security.AccessController.doPrivileged(Native Method)
>   at javax.security.auth.Subject.doAs(Subject.java:357)
>   at 
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1212)
>   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>   at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>   at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>   at java.lang.reflect.Method.invoke(Method.java:616)
>   at org.apache.hadoop.hbase.util.Methods.call(Methods.java:37)
>   at org.apache.hadoop.hbase.security.User.call(User.java:586)
>   at org.apache.hadoop.hbase.security.User.access$700(User.java:50)
>   at 
> org.apache.hadoop.hbase.security.User$SecureHadoopUser.runAs(User.java:426)
>   at 
> org.apache.hadoop.hbase.MiniHBaseCluster$MiniHBaseClusterRegionServer.run(MiniHBaseCluster.java:118)
>   at java.lang.Thread.run(Thread.java:679)
> {noformat}
> Not sure precisely when it started. First I thought it might be due to 
> HBASE-5955 but I cherry picked that change over from trunk. Then I got 
> HBASE-5963 out of the way.  

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5966) MapReduce based tests broken on Hadoop 2.0.0-alpha

2012-05-08 Thread Andrew Purtell (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-5966?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13270989#comment-13270989
 ] 

Andrew Purtell commented on HBASE-5966:
---

-Dhadoop.profile=24 gives the same result, the profiles seem to be identical 
(?).

> MapReduce based tests broken on Hadoop 2.0.0-alpha
> --
>
> Key: HBASE-5966
> URL: https://issues.apache.org/jira/browse/HBASE-5966
> Project: HBase
>  Issue Type: Bug
>  Components: mapred, mapreduce, test
>Affects Versions: 0.94.0, 0.96.0
> Environment: Hadoop 2.0.0-alpha-SNAPSHOT, HBase 0.94.0-SNAPSHOT, 
> Ubuntu 12.04 LTS (GNU/Linux 3.2.0-24-generic x86_64)
>Reporter: Andrew Purtell
>
> Some fairly recent change in Hadoop 2.0.0-alpha has broken our MapReduce test 
> rigging. Below is a representative error, can be easily reproduced with:
> {noformat}
> mvn -PlocalTests -Psecurity \
>   -Dhadoop.profile=23 -Dhadoop.version=2.0.0-SNAPSHOT \
>   clean test \
>   -Dtest=org.apache.hadoop.hbase.mapreduce.TestTableMapReduce
> {noformat}
> And the result:
> {noformat}
> ---
>  T E S T S
> ---
> Running org.apache.hadoop.hbase.mapreduce.TestTableMapReduce
> Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 54.292 sec 
> <<< FAILURE!
> ---
> Test set: org.apache.hadoop.hbase.mapreduce.TestTableMapReduce
> ---
> Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 54.292 sec 
> <<< FAILURE!
> testMultiRegionTable(org.apache.hadoop.hbase.mapreduce.TestTableMapReduce)  
> Time elapsed: 21.935 sec  <<< ERROR!
> java.lang.reflect.UndeclaredThrowableException
>   at 
> org.apache.hadoop.yarn.exceptions.impl.pb.YarnRemoteExceptionPBImpl.unwrapAndThrowException(YarnRemoteExceptionPBImpl.java:135)
>   at 
> org.apache.hadoop.yarn.api.impl.pb.client.ClientRMProtocolPBClientImpl.getNewApplication(ClientRMProtocolPBClientImpl.java:134)
>   at 
> org.apache.hadoop.mapred.ResourceMgrDelegate.getNewJobID(ResourceMgrDelegate.java:183)
>   at org.apache.hadoop.mapred.YARNRunner.getNewJobID(YARNRunner.java:216)
>   at 
> org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:339)
>   at org.apache.hadoop.mapreduce.Job$11.run(Job.java:1226)
>   at org.apache.hadoop.mapreduce.Job$11.run(Job.java:1223)
>   at java.security.AccessController.doPrivileged(Native Method)
>   at javax.security.auth.Subject.doAs(Subject.java:416)
>   at 
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1232)
>   at org.apache.hadoop.mapreduce.Job.submit(Job.java:1223)
>   at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:1244)
>   at 
> org.apache.hadoop.hbase.mapreduce.TestTableMapReduce.runTestOnTable(TestTableMapReduce.java:151)
>   at 
> org.apache.hadoop.hbase.mapreduce.TestTableMapReduce.testMultiRegionTable(TestTableMapReduce.java:129)
>   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>   at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>   at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>   at java.lang.reflect.Method.invoke(Method.java:616)
>   at 
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45)
>   at 
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
>   at 
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42)
>   at 
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
>   at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:47)
>   at org.junit.rules.RunRules.evaluate(RunRules.java:18)
>   at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:263)
>   at 
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:68)
>   at 
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:47)
>   at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231)
>   at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:60)
>   at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:229)
>   at org.junit.runners.ParentRunner.access$000(ParentRunner.java:50)
>   at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:222)
>   at 
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28)
>   at 
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:30)

[jira] [Created] (HBASE-5966) MapReduce based tests broken on Hadoop 2.0.0-alpha

2012-05-08 Thread Andrew Purtell (JIRA)

Andrew Purtell created HBASE-5966:
-

 Summary: MapReduce based tests broken on Hadoop 2.0.0-alpha
 Key: HBASE-5966
 URL: https://issues.apache.org/jira/browse/HBASE-5966
 Project: HBase
  Issue Type: Bug
  Components: mapred, mapreduce, test
Affects Versions: 0.94.0, 0.96.0
 Environment: Hadoop 2.0.0-alpha-SNAPSHOT, HBase 0.94.0-SNAPSHOT, 
Ubuntu 12.04 LTS (GNU/Linux 3.2.0-24-generic x86_64)
Reporter: Andrew Purtell


Some fairly recent change in Hadoop 2.0.0-alpha has broken our MapReduce test 
rigging. Below is a representative error, can be easily reproduced with:

{noformat}
mvn -PlocalTests -Psecurity \
  -Dhadoop.profile=23 -Dhadoop.version=2.0.0-SNAPSHOT \
  clean test \
  -Dtest=org.apache.hadoop.hbase.mapreduce.TestTableMapReduce
{noformat}

And the result:

{noformat}
---
 T E S T S
---
Running org.apache.hadoop.hbase.mapreduce.TestTableMapReduce
Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 54.292 sec <<< 
FAILURE!

---
Test set: org.apache.hadoop.hbase.mapreduce.TestTableMapReduce
---
Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 54.292 sec <<< 
FAILURE!
testMultiRegionTable(org.apache.hadoop.hbase.mapreduce.TestTableMapReduce)  
Time elapsed: 21.935 sec  <<< ERROR!
java.lang.reflect.UndeclaredThrowableException
at 
org.apache.hadoop.yarn.exceptions.impl.pb.YarnRemoteExceptionPBImpl.unwrapAndThrowException(YarnRemoteExceptionPBImpl.java:135)
at 
org.apache.hadoop.yarn.api.impl.pb.client.ClientRMProtocolPBClientImpl.getNewApplication(ClientRMProtocolPBClientImpl.java:134)
at 
org.apache.hadoop.mapred.ResourceMgrDelegate.getNewJobID(ResourceMgrDelegate.java:183)
at org.apache.hadoop.mapred.YARNRunner.getNewJobID(YARNRunner.java:216)
at 
org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:339)
at org.apache.hadoop.mapreduce.Job$11.run(Job.java:1226)
at org.apache.hadoop.mapreduce.Job$11.run(Job.java:1223)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:416)
at 
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1232)
at org.apache.hadoop.mapreduce.Job.submit(Job.java:1223)
at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:1244)
at 
org.apache.hadoop.hbase.mapreduce.TestTableMapReduce.runTestOnTable(TestTableMapReduce.java:151)
at 
org.apache.hadoop.hbase.mapreduce.TestTableMapReduce.testMultiRegionTable(TestTableMapReduce.java:129)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:616)
at 
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45)
at 
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
at 
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42)
at 
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:47)
at org.junit.rules.RunRules.evaluate(RunRules.java:18)
at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:263)
at 
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:68)
at 
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:47)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:60)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:229)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:50)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:222)
at 
org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28)
at 
org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:30)
at org.junit.runners.ParentRunner.run(ParentRunner.java:300)
at 
org.apache.maven.surefire.junit4.JUnit4TestSet.execute(JUnit4TestSet.java:53)
at 
org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:123)
at 
org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:104)
at sun.reflect.NativeMethodAccessorImpl.invoke

[jira] [Commented] (HBASE-5342) Grant/Revoke global permissions

2012-05-08 Thread Jonathan Hsieh (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-5342?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13270987#comment-13270987
 ] 

Jonathan Hsieh commented on HBASE-5342:
---

Andrew: I don't think the order matters anymore -- I checked this when I added 
the offline method to 0.90.  Rpcs actually send their names across the wire!  

See here on HBASE-5589. 
https://issues.apache.org/jira/browse/HBASE-5589?focusedCommentId=13233923&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13233923



> Grant/Revoke global permissions
> ---
>
> Key: HBASE-5342
> URL: https://issues.apache.org/jira/browse/HBASE-5342
> Project: HBase
>  Issue Type: Sub-task
>Reporter: Enis Soztutar
>Assignee: Matteo Bertozzi
> Attachments: HBASE-5342-draft.patch, HBASE-5342-v0.patch, 
> HBASE-5342-v1.patch, HBASE-5342-v2.patch
>
>
> HBASE-3025 introduced simple ACLs based on coprocessors. It defines 
> global/table/cf/cq level permissions. However, there is no way to 
> grant/revoke global level permissions, other than the hbase.superuser conf 
> setting. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5944) Convert rs, shutdown, and table dir content to pb

2012-05-08 Thread Hadoop QA (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-5944?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13270982#comment-13270982
 ] 

Hadoop QA commented on HBASE-5944:
--

-1 overall.  Here are the results of testing the latest attachment 
  http://issues.apache.org/jira/secure/attachment/12526070/5944.txt
  against trunk revision .

+1 @author.  The patch does not contain any @author tags.

-1 tests included.  The patch doesn't appear to include any new or modified 
tests.
Please justify why no new tests are needed for this 
patch.
Also please list what manual steps were performed to 
verify this patch.

+1 hadoop23.  The patch compiles against the hadoop 0.23.x profile.

+1 javadoc.  The javadoc tool did not generate any warning messages.

+1 javac.  The applied patch does not increase the total number of javac 
compiler warnings.

+1 findbugs.  The patch does not introduce any new Findbugs (version 1.3.9) 
warnings.

+1 release audit.  The applied patch does not increase the total number of 
release audit warnings.

 -1 core tests.  The patch failed these unit tests:
   org.apache.hadoop.hbase.thrift.TestThriftServer
  org.apache.hadoop.hbase.rest.client.TestRemoteAdmin
  org.apache.hadoop.hbase.coprocessor.TestRowProcessorEndpoint
  org.apache.hadoop.hbase.rest.TestSchemaResource
  org.apache.hadoop.hbase.master.TestDistributedLogSplitting
  org.apache.hadoop.hbase.TestZooKeeper
  org.apache.hadoop.hbase.regionserver.TestRegionServerMetrics
  org.apache.hadoop.hbase.client.TestHCM
  
org.apache.hadoop.hbase.client.TestHTablePool$TestHTableThreadLocalPool
  org.apache.hadoop.hbase.coprocessor.TestClassLoading
  org.apache.hadoop.hbase.mapreduce.TestMultithreadedTableMapper
  org.apache.hadoop.hbase.master.TestMasterTransitions
  
org.apache.hadoop.hbase.master.TestMasterRestartAfterDisablingTable
  
org.apache.hadoop.hbase.client.TestHTablePool$TestHTableReusablePool
  org.apache.hadoop.hbase.thrift.TestThriftServerCmdLine
  org.apache.hadoop.hbase.util.TestHBaseFsck
  
org.apache.hadoop.hbase.io.encoding.TestLoadAndSwitchEncodeOnDisk
  org.apache.hadoop.hbase.avro.TestAvroServer
  org.apache.hadoop.hbase.regionserver.TestServerCustomProtocol
  org.apache.hadoop.hbase.constraint.TestConstraint
  
org.apache.hadoop.hbase.coprocessor.TestRegionServerCoprocessorExceptionWithRemove
  org.apache.hadoop.hbase.master.TestRollingRestart
  org.apache.hadoop.hbase.io.encoding.TestChangingEncoding
  org.apache.hadoop.hbase.TestGlobalMemStoreSize
  org.apache.hadoop.hbase.TestFullLogReconstruction
  org.apache.hadoop.hbase.util.hbck.TestOfflineMetaRebuildHole
  
org.apache.hadoop.hbase.coprocessor.TestRegionServerCoprocessorExceptionWithAbort
  org.apache.hadoop.hbase.client.TestFromClientSide
  org.apache.hadoop.hbase.master.TestRestartCluster
  org.apache.hadoop.hbase.master.TestAssignmentManager
  org.apache.hadoop.hbase.client.TestAdmin
  org.apache.hadoop.hbase.TestMultiVersions
  
org.apache.hadoop.hbase.util.hbck.TestOfflineMetaRebuildOverlap
  org.apache.hadoop.hbase.coprocessor.TestAggregateProtocol
  org.apache.hadoop.hbase.util.hbck.TestOfflineMetaRebuildBase
  org.apache.hadoop.hbase.replication.TestReplication
  org.apache.hadoop.hbase.mapreduce.TestTableInputFormatScan
  org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat
  org.apache.hadoop.hbase.catalog.TestMetaReaderEditor
  org.apache.hadoop.hbase.client.TestShell
  org.apache.hadoop.hbase.catalog.TestCatalogTrackerOnCluster
  org.apache.hadoop.hbase.TestDrainingServer
  org.apache.hadoop.hbase.coprocessor.TestMasterObserver
  
org.apache.hadoop.hbase.coprocessor.TestRegionObserverInterface
  org.apache.hadoop.hbase.mapred.TestTableMapReduce
  org.apache.hadoop.hbase.util.TestMergeTable
  org.apache.hadoop.hbase.client.TestMultiParallel
  org.apache.hadoop.hbase.coprocessor.TestCoprocessorEndpoint
  org.apache.hadoop.hbase.client.TestMetaScanner
  
org.apache.hadoop.hbase.io.encoding.TestUpgradeFromHFileV1ToEncoding
  org.apache.hadoop.hbase.master.TestMasterFailover
  org

[jira] [Issue Comment Edited] (HBASE-5963) ClassCastException: FileSystem$Cache$ClientFinalizer cannot be cast to Thread

2012-05-08 Thread Andrew Purtell (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-5963?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13270975#comment-13270975
 ] 

Andrew Purtell edited comment on HBASE-5963 at 5/9/12 12:27 AM:


Latest patch seems to do the right thing according to unit tests without 
breaking older functionality.

However, what we are doing here is yanking the HDFS client finalizer Runnable 
away from the Hadoop 2.x ShutdownHookManager and adding it -- wrapped with our 
existing strategy here -- to the Runtime shutdown hook list, to perhaps run 
concurrently with the rest of the ShutdownHookManager registered Runnables. I 
don't think this more problematic than the rest of this horrible code. The 
solution is, someday, to leave Hadoop 1.x behind and transition to using the 
2.x ShutdownHookManager.

  was (Author: apurtell):
Latest patch seems to do the right thing according to unit tests without 
breaking older functionality.

However, what we are doing here is yanking the HDFS client finalizer Runnable 
away from the Hadoop 2.x ShutdownHookManager and adding it -- wrapped with our 
existing strategy here -- to the Runtime shutdown hook list, to perhaps run 
concurrently with the rest of the ShutdownHookManager registered Runnables. I 
don't think this less problematic than the rest of this horrible code. The 
solution is, someday, to leave Hadoop 1.x behind and transition to using the 
2.x ShutdownHookManager.
  
> ClassCastException: FileSystem$Cache$ClientFinalizer cannot be cast to Thread
> -
>
> Key: HBASE-5963
> URL: https://issues.apache.org/jira/browse/HBASE-5963
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 0.94.0, 0.96.0
> Environment: Bigtop 0.4.0-incubating-SNAPSHOT, Hadoop 
> 2.0.0-alpha-SNAPSHOT, HBase 0.94.0-SNAPSHOT
>Reporter: Andrew Purtell
>Assignee: Andrew Purtell
> Attachments: HBASE-5963.patch, HBASE-5963.patch
>
>
> {code}
> 12/05/08 19:49:26 INFO regionserver.HRegionServer: STOPPED: Failed 
> initialization
> Exception in thread "main" java.lang.ClassCastException: 
> org.apache.hadoop.fs.FileSystem$Cache$ClientFinalizer cannot be cast to 
> java.lang.Thread
>   at 
> org.apache.hadoop.hbase.regionserver.ShutdownHook.suppressHdfsShutdownHook(ShutdownHook.java:181)
>   at 
> org.apache.hadoop.hbase.regionserver.ShutdownHook.install(ShutdownHook.java:82)
>   at 
> org.apache.hadoop.hbase.regionserver.HRegionServer.startRegionServer(HRegionServer.java:3601)
>   at 
> org.apache.hadoop.hbase.regionserver.HRegionServer.startRegionServer(HRegionServer.java:3585)
>   at 
> org.apache.hadoop.hbase.regionserver.HRegionServerCommandLine.start(HRegionServerCommandLine.java:61)
>   at 
> org.apache.hadoop.hbase.regionserver.HRegionServerCommandLine.run(HRegionServerCommandLine.java:75)
>   at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
>   at 
> org.apache.hadoop.hbase.util.ServerCommandLine.doMain(ServerCommandLine.java:76)
>   at 
> org.apache.hadoop.hbase.regionserver.HRegionServer.main(HRegionServer.java:3645)
> {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5963) ClassCastException: FileSystem$Cache$ClientFinalizer cannot be cast to Thread

2012-05-08 Thread Andrew Purtell (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-5963?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrew Purtell updated HBASE-5963:
--

Attachment: HBASE-5963.patch

Latest patch seems to do the right thing according to unit tests without 
breaking older functionality.

However, what we are doing here is yanking the HDFS client finalizer Runnable 
away from the Hadoop 2.x ShutdownHookManager and adding it -- wrapped with our 
existing strategy here -- to the Runtime shutdown hook list, to perhaps run 
concurrently with the rest of the ShutdownHookManager registered Runnables. I 
don't think this less problematic than the rest of this horrible code. The 
solution is, someday, to leave Hadoop 1.x behind and transition to using the 
2.x ShutdownHookManager.

> ClassCastException: FileSystem$Cache$ClientFinalizer cannot be cast to Thread
> -
>
> Key: HBASE-5963
> URL: https://issues.apache.org/jira/browse/HBASE-5963
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 0.94.0, 0.96.0
> Environment: Bigtop 0.4.0-incubating-SNAPSHOT, Hadoop 
> 2.0.0-alpha-SNAPSHOT, HBase 0.94.0-SNAPSHOT
>Reporter: Andrew Purtell
>Assignee: Andrew Purtell
> Attachments: HBASE-5963.patch, HBASE-5963.patch
>
>
> {code}
> 12/05/08 19:49:26 INFO regionserver.HRegionServer: STOPPED: Failed 
> initialization
> Exception in thread "main" java.lang.ClassCastException: 
> org.apache.hadoop.fs.FileSystem$Cache$ClientFinalizer cannot be cast to 
> java.lang.Thread
>   at 
> org.apache.hadoop.hbase.regionserver.ShutdownHook.suppressHdfsShutdownHook(ShutdownHook.java:181)
>   at 
> org.apache.hadoop.hbase.regionserver.ShutdownHook.install(ShutdownHook.java:82)
>   at 
> org.apache.hadoop.hbase.regionserver.HRegionServer.startRegionServer(HRegionServer.java:3601)
>   at 
> org.apache.hadoop.hbase.regionserver.HRegionServer.startRegionServer(HRegionServer.java:3585)
>   at 
> org.apache.hadoop.hbase.regionserver.HRegionServerCommandLine.start(HRegionServerCommandLine.java:61)
>   at 
> org.apache.hadoop.hbase.regionserver.HRegionServerCommandLine.run(HRegionServerCommandLine.java:75)
>   at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
>   at 
> org.apache.hadoop.hbase.util.ServerCommandLine.doMain(ServerCommandLine.java:76)
>   at 
> org.apache.hadoop.hbase.regionserver.HRegionServer.main(HRegionServer.java:3645)
> {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5964) HFileSystem: "No FileSystem for scheme: hdfs"

2012-05-08 Thread Hadoop QA (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-5964?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13270974#comment-13270974
 ] 

Hadoop QA commented on HBASE-5964:
--

-1 overall.  Here are the results of testing the latest attachment 
  http://issues.apache.org/jira/secure/attachment/12526067/HBASE-5964.patch
  against trunk revision .

+1 @author.  The patch does not contain any @author tags.

-1 tests included.  The patch doesn't appear to include any new or modified 
tests.
Please justify why no new tests are needed for this 
patch.
Also please list what manual steps were performed to 
verify this patch.

+1 hadoop23.  The patch compiles against the hadoop 0.23.x profile.

+1 javadoc.  The javadoc tool did not generate any warning messages.

+1 javac.  The applied patch does not increase the total number of javac 
compiler warnings.

+1 findbugs.  The patch does not introduce any new Findbugs (version 1.3.9) 
warnings.

+1 release audit.  The applied patch does not increase the total number of 
release audit warnings.

 -1 core tests.  The patch failed these unit tests:
 

Test results: 
https://builds.apache.org/job/PreCommit-HBASE-Build/1805//testReport/
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/1805//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
Console output: 
https://builds.apache.org/job/PreCommit-HBASE-Build/1805//console

This message is automatically generated.

> HFileSystem: "No FileSystem for scheme: hdfs"
> -
>
> Key: HBASE-5964
> URL: https://issues.apache.org/jira/browse/HBASE-5964
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 0.94.0, 0.96.0
> Environment: Hadoop 2.0.0-alpha-SNAPSHOT, HBase 0.94.0-SNAPSHOT
>Reporter: Andrew Purtell
>Assignee: Andrew Purtell
> Attachments: HBASE-5964.patch
>
>
> I've been seeing this with Hadoop 2.0.0-alpha-SNAPSHOT and HBase 
> 0.94.0-SNAPSHOT:
> {noformat}
> 2012-05-08 15:18:00,692 FATAL 
> [RegionServer:0;acer.localdomain,48307,1336515479011] 
> regionserver.HRegionServer(1674): ABORTING region server 
> acer.localdomain,48307,1336515479011: Unhandled exception: No FileSystem for 
> scheme: hdfs
> java.io.IOException: No FileSystem for scheme: hdfs
>   at 
> org.apache.hadoop.hbase.fs.HFileSystem.newInstanceFileSystem(HFileSystem.java:146)
>   at org.apache.hadoop.hbase.fs.HFileSystem.(HFileSystem.java:75)
>   at 
> org.apache.hadoop.hbase.regionserver.HRegionServer.handleReportForDutyResponse(HRegionServer.java:973)
>   at 
> org.apache.hadoop.hbase.MiniHBaseCluster$MiniHBaseClusterRegionServer.handleReportForDutyResponse(MiniHBaseCluster.java:110)
>   at 
> org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:671)
>   at 
> org.apache.hadoop.hbase.MiniHBaseCluster$MiniHBaseClusterRegionServer.runRegionServer(MiniHBaseCluster.java:136)
>   at 
> org.apache.hadoop.hbase.MiniHBaseCluster$MiniHBaseClusterRegionServer.access$000(MiniHBaseCluster.java:89)
>   at 
> org.apache.hadoop.hbase.MiniHBaseCluster$MiniHBaseClusterRegionServer$1.run(MiniHBaseCluster.java:120)
>   at java.security.AccessController.doPrivileged(Native Method)
>   at javax.security.auth.Subject.doAs(Subject.java:357)
>   at 
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1212)
>   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>   at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>   at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>   at java.lang.reflect.Method.invoke(Method.java:616)
>   at org.apache.hadoop.hbase.util.Methods.call(Methods.java:37)
>   at org.apache.hadoop.hbase.security.User.call(User.java:586)
>   at org.apache.hadoop.hbase.security.User.access$700(User.java:50)
>   at 
> org.apache.hadoop.hbase.security.User$SecureHadoopUser.runAs(User.java:426)
>   at 
> org.apache.hadoop.hbase.MiniHBaseCluster$MiniHBaseClusterRegionServer.run(MiniHBaseCluster.java:118)
>   at java.lang.Thread.run(Thread.java:679)
> {noformat}
> Not sure precisely when it started. First I thought it might be due to 
> HBASE-5955 but I cherry picked that change over from trunk. Then I got 
> HBASE-5963 out of the way.  

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5935) Add Region-level PB-based calls to HMasterInterface

2012-05-08 Thread Gregory Chanan (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-5935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13270971#comment-13270971
 ] 

Gregory Chanan commented on HBASE-5935:
---

@Stack: Patch that matches reviewboard was posted and passed Haddop QA.

> Add Region-level PB-based calls to HMasterInterface
> ---
>
> Key: HBASE-5935
> URL: https://issues.apache.org/jira/browse/HBASE-5935
> Project: HBase
>  Issue Type: Task
>  Components: ipc, master, migration
>Reporter: Gregory Chanan
>Assignee: Gregory Chanan
> Fix For: 0.96.0
>
> Attachments: HBASE-5935-v2.patch
>
>
> This should be a subtask of HBASE-5445, but since that is a subtask, I can't 
> also make this a subtask (apparently).
> This is for converting the region-level calls, i.e.:
> moveRegion
> assignRegion
> unassignRegion

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5963) ClassCastException: FileSystem$Cache$ClientFinalizer cannot be cast to Thread

2012-05-08 Thread Hadoop QA (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-5963?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13270963#comment-13270963
 ] 

Hadoop QA commented on HBASE-5963:
--

-1 overall.  Here are the results of testing the latest attachment 
  http://issues.apache.org/jira/secure/attachment/12526060/HBASE-5963.patch
  against trunk revision .

+1 @author.  The patch does not contain any @author tags.

-1 tests included.  The patch doesn't appear to include any new or modified 
tests.
Please justify why no new tests are needed for this 
patch.
Also please list what manual steps were performed to 
verify this patch.

+1 hadoop23.  The patch compiles against the hadoop 0.23.x profile.

+1 javadoc.  The javadoc tool did not generate any warning messages.

+1 javac.  The applied patch does not increase the total number of javac 
compiler warnings.

+1 findbugs.  The patch does not introduce any new Findbugs (version 1.3.9) 
warnings.

+1 release audit.  The applied patch does not increase the total number of 
release audit warnings.

+1 core tests.  The patch passed unit tests in .

Test results: 
https://builds.apache.org/job/PreCommit-HBASE-Build/1804//testReport/
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/1804//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
Console output: 
https://builds.apache.org/job/PreCommit-HBASE-Build/1804//console

This message is automatically generated.

> ClassCastException: FileSystem$Cache$ClientFinalizer cannot be cast to Thread
> -
>
> Key: HBASE-5963
> URL: https://issues.apache.org/jira/browse/HBASE-5963
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 0.94.0, 0.96.0
> Environment: Bigtop 0.4.0-incubating-SNAPSHOT, Hadoop 
> 2.0.0-alpha-SNAPSHOT, HBase 0.94.0-SNAPSHOT
>Reporter: Andrew Purtell
>Assignee: Andrew Purtell
> Attachments: HBASE-5963.patch
>
>
> {code}
> 12/05/08 19:49:26 INFO regionserver.HRegionServer: STOPPED: Failed 
> initialization
> Exception in thread "main" java.lang.ClassCastException: 
> org.apache.hadoop.fs.FileSystem$Cache$ClientFinalizer cannot be cast to 
> java.lang.Thread
>   at 
> org.apache.hadoop.hbase.regionserver.ShutdownHook.suppressHdfsShutdownHook(ShutdownHook.java:181)
>   at 
> org.apache.hadoop.hbase.regionserver.ShutdownHook.install(ShutdownHook.java:82)
>   at 
> org.apache.hadoop.hbase.regionserver.HRegionServer.startRegionServer(HRegionServer.java:3601)
>   at 
> org.apache.hadoop.hbase.regionserver.HRegionServer.startRegionServer(HRegionServer.java:3585)
>   at 
> org.apache.hadoop.hbase.regionserver.HRegionServerCommandLine.start(HRegionServerCommandLine.java:61)
>   at 
> org.apache.hadoop.hbase.regionserver.HRegionServerCommandLine.run(HRegionServerCommandLine.java:75)
>   at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
>   at 
> org.apache.hadoop.hbase.util.ServerCommandLine.doMain(ServerCommandLine.java:76)
>   at 
> org.apache.hadoop.hbase.regionserver.HRegionServer.main(HRegionServer.java:3645)
> {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5929) HBaseAdmin.majorCompact and hbase shell randomly throw exceptions when asked to majorcompact regions.

2012-05-08 Thread Jean-Daniel Cryans (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-5929?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13270943#comment-13270943
 ] 

Jean-Daniel Cryans commented on HBASE-5929:
---

The reason it doesn't work with META is because we don't write the full meta 
row names in ROOT. The row key should be ".META.,,1.1028785192" but if I scan 
ROOT I see:

{quote}
 .META.,,1   
column=info:regioninfo, timestamp=1336512980545, value= 
   
 .META.,,1   
column=info:server, timestamp=1336512994161, 
value=h-25-183.sfo.stumble.net:50063
  
 .META.,,1   
column=info:serverstartcode, timestamp=1336512994161, value=1336512979497   

   
 .META.,,1   column=info:v, 
timestamp=1336512980545, value=\x00\x00
{quote}

It's missing the encoded name. That won't work. Requesting the compaction of a 
root region won't work either because the code that figures out if a region or 
a table is passed depends on MetaReader.getCatalogHTable which doesn't take 
ROOT regions (which is normal since the ROOT address isn't contained in an 
HTable but in ZK).

For the other regions that had that issue, it was a bad interaction in our code 
where the scan of .META. is done possible hours before the call for compaction 
is issued. I was able to confirm that all the regions that were affected had 
been recently split.

The issues that remain:

 - It's impossible to compact the root region directly, but calling compact on 
the table itself works.
 - It's possible to compact the meta region directly but the user needs to pass 
".META.,,1" instead of the full region name.
 - Trying to compact a region that doesn't exist throws a 
TableNotFoundException, which confuses the user.

I'd say that this is minor but we should probably fix for usability.

> HBaseAdmin.majorCompact and hbase shell randomly throw exceptions when asked 
> to majorcompact regions.
> -
>
> Key: HBASE-5929
> URL: https://issues.apache.org/jira/browse/HBASE-5929
> Project: HBase
>  Issue Type: Bug
>  Components: client, shell
>Affects Versions: 0.92.1
> Environment: Linux Ubuntu Lucid 64bit
>Reporter: Aravind Gottipati
>Priority: Minor
>
> I have been noticing that calls to HBaseAdmin.majorCompact throws exceptions 
> randomly for some regions.  I could not find a pattern to these exception.  
> The code I have simply does this 
> admin.majorCompact(region.getRegionNameAsString()).  admin is an instance of 
> HBaseAdmin and region is an instance of HRegionInfo.  The exception I get is 
> org.apache.hadoop.hbase.TableNotFoundException: -ROOT-,,0
> at 
> org.apache.hadoop.hbase.client.HBaseAdmin.tableNameString(HBaseAdmin.java:1473)
>  ~[hbase-0.92.1.jar:0.92.1]
> at 
> org.apache.hadoop.hbase.client.HBaseAdmin.compact(HBaseAdmin.java:1235) 
> ~[hbase-0.92.1.jar:0.92.1]
> at 
> org.apache.hadoop.hbase.client.HBaseAdmin.majorCompact(HBaseAdmin.java:1209) 
> ~[hbase-0.92.1.jar:0.92.1]
> at com.stumbleupon.hbaseadmin.HBaseCompact.compactAllServers(Unknown 
> Source) [hbase_compact.jar:na]
> In this case it's the root region, but I get similar exceptions for other 
> tables, like this.
> 2012-05-03 19:03:42,994 WARN  [main] HBaseCompact: Could not compact:
> org.apache.hadoop.hbase.TableNotFoundException: 
> ad_daily,49842:2009-07-10,1269763588508.1997607018
> at 
> org.apache.hadoop.hbase.client.HBaseAdmin.tableNameString(HBaseAdmin.java:1473)
>  ~[hbase-0.92.1.jar:0.92.1]
> at 
> org.apache.hadoop.hbase.client.HBaseAdmin.compact(HBaseAdmin.java:1235) 
> ~[hbase-0.92.1.jar:0.92.1]
> at 
> org.apache.hadoop.hbase.client.HBaseAdmin.majorCompact(HBaseAdmin.java:1209) 
> ~[hbase-0.92.1.jar:0.92.1]
> at 
> org.apache.hadoop.hbase.client.HBaseAdmin.majorCompact(HBaseAdmin.java:1196) 
> ~[hbase-0.92.1.jar:0.92.1]
> at com.stumbleupon.hbaseadmin.HBaseCompact.compactAllServers(Unknown 
> Source) [hbase_compact.jar:na]
> at com.stumbleupon.hbaseadmin.HBaseCompact.main(Unknown Source) 
> [hbase_compact.jar:na]
> I see this on hbase shell as well.  However, I don't see these exceptions if 
> I use admin.majorCompact(region.getRegionName()), so it looks like something 
> gets lost when I use getRegionNameAsString().
> Let me know if I can provide more inform

[jira] [Updated] (HBASE-5929) HBaseAdmin.compact and flush are giving confusing errors for ROOT, META, and regions that don't exist

2012-05-08 Thread Jean-Daniel Cryans (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-5929?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jean-Daniel Cryans updated HBASE-5929:
--


Fixing the title.

> HBaseAdmin.compact and flush are giving confusing errors for ROOT, META, and 
> regions that don't exist
> -
>
> Key: HBASE-5929
> URL: https://issues.apache.org/jira/browse/HBASE-5929
> Project: HBase
>  Issue Type: Bug
>  Components: client, shell
>Affects Versions: 0.92.1
> Environment: Linux Ubuntu Lucid 64bit
>Reporter: Aravind Gottipati
>Priority: Minor
>
> I have been noticing that calls to HBaseAdmin.majorCompact throws exceptions 
> randomly for some regions.  I could not find a pattern to these exception.  
> The code I have simply does this 
> admin.majorCompact(region.getRegionNameAsString()).  admin is an instance of 
> HBaseAdmin and region is an instance of HRegionInfo.  The exception I get is 
> org.apache.hadoop.hbase.TableNotFoundException: -ROOT-,,0
> at 
> org.apache.hadoop.hbase.client.HBaseAdmin.tableNameString(HBaseAdmin.java:1473)
>  ~[hbase-0.92.1.jar:0.92.1]
> at 
> org.apache.hadoop.hbase.client.HBaseAdmin.compact(HBaseAdmin.java:1235) 
> ~[hbase-0.92.1.jar:0.92.1]
> at 
> org.apache.hadoop.hbase.client.HBaseAdmin.majorCompact(HBaseAdmin.java:1209) 
> ~[hbase-0.92.1.jar:0.92.1]
> at com.stumbleupon.hbaseadmin.HBaseCompact.compactAllServers(Unknown 
> Source) [hbase_compact.jar:na]
> In this case it's the root region, but I get similar exceptions for other 
> tables, like this.
> 2012-05-03 19:03:42,994 WARN  [main] HBaseCompact: Could not compact:
> org.apache.hadoop.hbase.TableNotFoundException: 
> ad_daily,49842:2009-07-10,1269763588508.1997607018
> at 
> org.apache.hadoop.hbase.client.HBaseAdmin.tableNameString(HBaseAdmin.java:1473)
>  ~[hbase-0.92.1.jar:0.92.1]
> at 
> org.apache.hadoop.hbase.client.HBaseAdmin.compact(HBaseAdmin.java:1235) 
> ~[hbase-0.92.1.jar:0.92.1]
> at 
> org.apache.hadoop.hbase.client.HBaseAdmin.majorCompact(HBaseAdmin.java:1209) 
> ~[hbase-0.92.1.jar:0.92.1]
> at 
> org.apache.hadoop.hbase.client.HBaseAdmin.majorCompact(HBaseAdmin.java:1196) 
> ~[hbase-0.92.1.jar:0.92.1]
> at com.stumbleupon.hbaseadmin.HBaseCompact.compactAllServers(Unknown 
> Source) [hbase_compact.jar:na]
> at com.stumbleupon.hbaseadmin.HBaseCompact.main(Unknown Source) 
> [hbase_compact.jar:na]
> I see this on hbase shell as well.  However, I don't see these exceptions if 
> I use admin.majorCompact(region.getRegionName()), so it looks like something 
> gets lost when I use getRegionNameAsString().
> Let me know if I can provide more information.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5929) HBaseAdmin.compact and flush are giving confusing errors for ROOT, META, and regions that don't exist

2012-05-08 Thread Jean-Daniel Cryans (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-5929?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jean-Daniel Cryans updated HBASE-5929:
--

Summary: HBaseAdmin.compact and flush are giving confusing errors for ROOT, 
META, and regions that don't exist  (was: HBaseAdmin.majorCompact and hbase 
shell randomly throw exceptions when asked to majorcompact regions.)

Fixing the title.

> HBaseAdmin.compact and flush are giving confusing errors for ROOT, META, and 
> regions that don't exist
> -
>
> Key: HBASE-5929
> URL: https://issues.apache.org/jira/browse/HBASE-5929
> Project: HBase
>  Issue Type: Bug
>  Components: client, shell
>Affects Versions: 0.92.1
> Environment: Linux Ubuntu Lucid 64bit
>Reporter: Aravind Gottipati
>Priority: Minor
>
> I have been noticing that calls to HBaseAdmin.majorCompact throws exceptions 
> randomly for some regions.  I could not find a pattern to these exception.  
> The code I have simply does this 
> admin.majorCompact(region.getRegionNameAsString()).  admin is an instance of 
> HBaseAdmin and region is an instance of HRegionInfo.  The exception I get is 
> org.apache.hadoop.hbase.TableNotFoundException: -ROOT-,,0
> at 
> org.apache.hadoop.hbase.client.HBaseAdmin.tableNameString(HBaseAdmin.java:1473)
>  ~[hbase-0.92.1.jar:0.92.1]
> at 
> org.apache.hadoop.hbase.client.HBaseAdmin.compact(HBaseAdmin.java:1235) 
> ~[hbase-0.92.1.jar:0.92.1]
> at 
> org.apache.hadoop.hbase.client.HBaseAdmin.majorCompact(HBaseAdmin.java:1209) 
> ~[hbase-0.92.1.jar:0.92.1]
> at com.stumbleupon.hbaseadmin.HBaseCompact.compactAllServers(Unknown 
> Source) [hbase_compact.jar:na]
> In this case it's the root region, but I get similar exceptions for other 
> tables, like this.
> 2012-05-03 19:03:42,994 WARN  [main] HBaseCompact: Could not compact:
> org.apache.hadoop.hbase.TableNotFoundException: 
> ad_daily,49842:2009-07-10,1269763588508.1997607018
> at 
> org.apache.hadoop.hbase.client.HBaseAdmin.tableNameString(HBaseAdmin.java:1473)
>  ~[hbase-0.92.1.jar:0.92.1]
> at 
> org.apache.hadoop.hbase.client.HBaseAdmin.compact(HBaseAdmin.java:1235) 
> ~[hbase-0.92.1.jar:0.92.1]
> at 
> org.apache.hadoop.hbase.client.HBaseAdmin.majorCompact(HBaseAdmin.java:1209) 
> ~[hbase-0.92.1.jar:0.92.1]
> at 
> org.apache.hadoop.hbase.client.HBaseAdmin.majorCompact(HBaseAdmin.java:1196) 
> ~[hbase-0.92.1.jar:0.92.1]
> at com.stumbleupon.hbaseadmin.HBaseCompact.compactAllServers(Unknown 
> Source) [hbase_compact.jar:na]
> at com.stumbleupon.hbaseadmin.HBaseCompact.main(Unknown Source) 
> [hbase_compact.jar:na]
> I see this on hbase shell as well.  However, I don't see these exceptions if 
> I use admin.majorCompact(region.getRegionName()), so it looks like something 
> gets lost when I use getRegionNameAsString().
> Let me know if I can provide more information.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5935) Add Region-level PB-based calls to HMasterInterface

2012-05-08 Thread jirapos...@reviews.apache.org (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-5935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13270942#comment-13270942
 ] 

jirapos...@reviews.apache.org commented on HBASE-5935:
--

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/5060/#review7709
---

Ship it!

+1  Put the patch up in JIRA so we can run it by hadoopqa. Good stuff G.

- Michael

On 2012-05-08 22:49:04, Gregory Chanan wrote:
bq.  
bq.  ---
bq.  This is an automatically generated e-mail. To reply, visit:
bq.  https://reviews.apache.org/r/5060/
bq.  ---
bq.  
bq.  (Updated 2012-05-08 22:49:04)
bq.  
bq.  
bq.  Review request for hbase, Michael Stack and Jimmy Xiang.
bq.  
bq.  
bq.  Summary
bq.  ---
bq.  
bq.  Converted all the region-level calls in HMasterInterface to use Protobufs 
(move, assign, unassign).
bq.  
bq.  I've put the new generated calls in HMasterInterface rather than a new 
class, because it is easier to have one class for now, until I convert 
everything.  This will only be temporary, but let me know how you want to 
handle.  Bump the version number?
bq.  
bq.  
bq.  This addresses bug HBASE-5935.
bq.  https://issues.apache.org/jira/browse/HBASE-5935
bq.  
bq.  
bq.  Diffs
bq.  -
bq.  
bq.src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java 5d4be3f 
bq.src/main/java/org/apache/hadoop/hbase/ipc/HMasterInterface.java ce81547 
bq.src/main/java/org/apache/hadoop/hbase/master/HMaster.java ccc7119 
bq.src/main/java/org/apache/hadoop/hbase/protobuf/RequestConverter.java 
a5e03e1 
bq.
src/main/java/org/apache/hadoop/hbase/protobuf/generated/MasterProtos.java 
PRE-CREATION 
bq.src/main/protobuf/Master.proto PRE-CREATION 
bq.src/main/protobuf/RegionServerStatus.proto 9d7728f 
bq.src/test/java/org/apache/hadoop/hbase/TestDrainingServer.java e5de603 
bq.
src/test/java/org/apache/hadoop/hbase/coprocessor/TestMasterObserver.java 
83297cc 
bq.  
bq.  Diff: https://reviews.apache.org/r/5060/diff
bq.  
bq.  
bq.  Testing
bq.  ---
bq.  
bq.  
bq.  Thanks,
bq.  
bq.  Gregory
bq.  
bq.

> Add Region-level PB-based calls to HMasterInterface
> ---
>
> Key: HBASE-5935
> URL: https://issues.apache.org/jira/browse/HBASE-5935
> Project: HBase
>  Issue Type: Task
>  Components: ipc, master, migration
>Reporter: Gregory Chanan
>Assignee: Gregory Chanan
> Fix For: 0.96.0
>
> Attachments: HBASE-5935-v2.patch
>
>
> This should be a subtask of HBASE-5445, but since that is a subtask, I can't 
> also make this a subtask (apparently).
> This is for converting the region-level calls, i.e.:
> moveRegion
> assignRegion
> unassignRegion

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5935) Add Region-level PB-based calls to HMasterInterface

2012-05-08 Thread jirapos...@reviews.apache.org (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-5935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13270938#comment-13270938
 ] 

jirapos...@reviews.apache.org commented on HBASE-5935:
--

bq.  On 2012-05-08 04:55:21, Michael Stack wrote:
bq.  > src/main/java/org/apache/hadoop/hbase/ipc/HMasterInterface.java, line 285
bq.  > 
bq.  >
bq.  > It does seem a little messy having to hoist all these pb classes up 
in here AND into HMasterInterface.
bq.  > 
bq.  > HMasterInterface is going to go away?
bq.  
bq.  Gregory Chanan wrote:
bq.  I'm not sure what you mean by "having to hoist all these pb classes up 
in here AND into HMasterInterface." -- this is HMasterInterface?
bq.  
bq.  Also, I don't think PB will be nice and generated usable javadoc for 
you, so I'm not sure how to provide javadoc comments without redeclaring the 
functions?  I admit to being completely ignorant about this area.
bq.  
bq.  Once everything in HMasterInterface is converted to use PB, we can 
either declare a new class for the representation (similar to 
RegionServerStatusProtocol) or just re-purpose HMasterInterface for that.  What 
is your preference?

I was just saying that we have generated pb pollution in both the Interface and 
in the Implementation, but its unavoidable really.

You can't have pb do javadoc (is my understanding) so don't worry about it (Not 
sure what I suggested but it doesn't seem important now)

Lets do what Jimmy did, make a new class and kill the old.

- Michael

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/5060/#review7667
---

On 2012-05-08 22:49:04, Gregory Chanan wrote:
bq.  
bq.  ---
bq.  This is an automatically generated e-mail. To reply, visit:
bq.  https://reviews.apache.org/r/5060/
bq.  ---
bq.  
bq.  (Updated 2012-05-08 22:49:04)
bq.  
bq.  
bq.  Review request for hbase, Michael Stack and Jimmy Xiang.
bq.  
bq.  
bq.  Summary
bq.  ---
bq.  
bq.  Converted all the region-level calls in HMasterInterface to use Protobufs 
(move, assign, unassign).
bq.  
bq.  I've put the new generated calls in HMasterInterface rather than a new 
class, because it is easier to have one class for now, until I convert 
everything.  This will only be temporary, but let me know how you want to 
handle.  Bump the version number?
bq.  
bq.  
bq.  This addresses bug HBASE-5935.
bq.  https://issues.apache.org/jira/browse/HBASE-5935
bq.  
bq.  
bq.  Diffs
bq.  -
bq.  
bq.src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java 5d4be3f 
bq.src/main/java/org/apache/hadoop/hbase/ipc/HMasterInterface.java ce81547 
bq.src/main/java/org/apache/hadoop/hbase/master/HMaster.java ccc7119 
bq.src/main/java/org/apache/hadoop/hbase/protobuf/RequestConverter.java 
a5e03e1 
bq.
src/main/java/org/apache/hadoop/hbase/protobuf/generated/MasterProtos.java 
PRE-CREATION 
bq.src/main/protobuf/Master.proto PRE-CREATION 
bq.src/main/protobuf/RegionServerStatus.proto 9d7728f 
bq.src/test/java/org/apache/hadoop/hbase/TestDrainingServer.java e5de603 
bq.
src/test/java/org/apache/hadoop/hbase/coprocessor/TestMasterObserver.java 
83297cc 
bq.  
bq.  Diff: https://reviews.apache.org/r/5060/diff
bq.  
bq.  
bq.  Testing
bq.  ---
bq.  
bq.  
bq.  Thanks,
bq.  
bq.  Gregory
bq.  
bq.

> Add Region-level PB-based calls to HMasterInterface
> ---
>
> Key: HBASE-5935
> URL: https://issues.apache.org/jira/browse/HBASE-5935
> Project: HBase
>  Issue Type: Task
>  Components: ipc, master, migration
>Reporter: Gregory Chanan
>Assignee: Gregory Chanan
> Fix For: 0.96.0
>
> Attachments: HBASE-5935-v2.patch
>
>
> This should be a subtask of HBASE-5445, but since that is a subtask, I can't 
> also make this a subtask (apparently).
> This is for converting the region-level calls, i.e.:
> moveRegion
> assignRegion
> unassignRegion

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5732) Remove the SecureRPCEngine and merge the security-related logic in the core engine

2012-05-08 Thread Zhihong Yu (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-5732?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13270936#comment-13270936
 ] 

Zhihong Yu commented on HBASE-5732:
---

I looped TestMXBean 4 times with patch v9 and they (master and regionserver) 
passed.

> Remove the SecureRPCEngine and merge the security-related logic in the core 
> engine
> --
>
> Key: HBASE-5732
> URL: https://issues.apache.org/jira/browse/HBASE-5732
> Project: HBase
>  Issue Type: Improvement
>Reporter: Devaraj Das
>Assignee: Devaraj Das
> Attachments: 5732-rpcengine-merge.7.patch, rpcengine-merge.3.patch, 
> rpcengine-merge.4.patch, rpcengine-merge.9.patch, rpcengine-merge.patch
>
>
> Remove the SecureRPCEngine and merge the security-related logic in the core 
> engine. Follow up to HBASE-5727.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (HBASE-5965) Move replication znodes to pb

2012-05-08 Thread stack (JIRA)

stack created HBASE-5965:


 Summary: Move replication znodes to pb
 Key: HBASE-5965
 URL: https://issues.apache.org/jira/browse/HBASE-5965
 Project: HBase
  Issue Type: Task
Reporter: stack




--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5944) Convert rs, shutdown, and table dir content to pb

2012-05-08 Thread stack (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-5944?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stack updated HBASE-5944:
-

Attachment: 5944.txt

Brings over the table directory znodes.

There is nothing to do for draining servers and for rs.

{code}
M src/main/java/org/apache/hadoop/hbase/zookeeper/ClusterStatusTracker.java

  Rename getZNodeData as toByteArray to match naming elsewhere getting znode 
byte content.

M src/main/java/org/apache/hadoop/hbase/zookeeper/ZKLeaderManager.java

  Deprecated.  Not used.

M src/main/java/org/apache/hadoop/hbase/zookeeper/ZKTable.java
  Use pbs writing znode data.  Use the pb state instead of the enum we used to 
have in here.

M src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java
  Use the RootRegionLocation.getRootRegionLocation static method rather than do 
custom undoing of znode content.

M src/main/protobuf/ZooKeeper.proto
  Added Table message.
{code}

> Convert rs, shutdown, and table dir content to pb
> -
>
> Key: HBASE-5944
> URL: https://issues.apache.org/jira/browse/HBASE-5944
> Project: HBase
>  Issue Type: Task
>Reporter: stack
> Attachments: 5944.txt
>
>


--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5935) Add Region-level PB-based calls to HMasterInterface

2012-05-08 Thread Hadoop QA (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-5935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13270929#comment-13270929
 ] 

Hadoop QA commented on HBASE-5935:
--

+1 overall.  Here are the results of testing the latest attachment 
  http://issues.apache.org/jira/secure/attachment/12526063/HBASE-5935-v2.patch
  against trunk revision .

+1 @author.  The patch does not contain any @author tags.

+1 tests included.  The patch appears to include 6 new or modified tests.

+1 hadoop23.  The patch compiles against the hadoop 0.23.x profile.

+1 javadoc.  The javadoc tool did not generate any warning messages.

+1 javac.  The applied patch does not increase the total number of javac 
compiler warnings.

+1 findbugs.  The patch does not introduce any new Findbugs (version 1.3.9) 
warnings.

+1 release audit.  The applied patch does not increase the total number of 
release audit warnings.

+1 core tests.  The patch passed unit tests in .

Test results: 
https://builds.apache.org/job/PreCommit-HBASE-Build/1803//testReport/
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/1803//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
Console output: 
https://builds.apache.org/job/PreCommit-HBASE-Build/1803//console

This message is automatically generated.

> Add Region-level PB-based calls to HMasterInterface
> ---
>
> Key: HBASE-5935
> URL: https://issues.apache.org/jira/browse/HBASE-5935
> Project: HBase
>  Issue Type: Task
>  Components: ipc, master, migration
>Reporter: Gregory Chanan
>Assignee: Gregory Chanan
> Fix For: 0.96.0
>
> Attachments: HBASE-5935-v2.patch
>
>
> This should be a subtask of HBASE-5445, but since that is a subtask, I can't 
> also make this a subtask (apparently).
> This is for converting the region-level calls, i.e.:
> moveRegion
> assignRegion
> unassignRegion

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5944) Convert rs, shutdown, and table dir content to pb

2012-05-08 Thread stack (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-5944?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stack updated HBASE-5944:
-

Assignee: stack
  Status: Patch Available  (was: Open)

> Convert rs, shutdown, and table dir content to pb
> -
>
> Key: HBASE-5944
> URL: https://issues.apache.org/jira/browse/HBASE-5944
> Project: HBase
>  Issue Type: Task
>Reporter: stack
>Assignee: stack
> Attachments: 5944.txt
>
>


--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5944) Convert rs, shutdown, and table dir content to pb

2012-05-08 Thread stack (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-5944?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13270927#comment-13270927
 ] 

stack commented on HBASE-5944:
--

Found more to convert

{code}

[splitlog, unassigned, replication, root-region-server, rs, backup-masters, 
table, draining, master, shutdown, hbaseid]


[state, peers, rs]

{code}

Need to bring over replication and draining servers.

> Convert rs, shutdown, and table dir content to pb
> -
>
> Key: HBASE-5944
> URL: https://issues.apache.org/jira/browse/HBASE-5944
> Project: HBase
>  Issue Type: Task
>Reporter: stack
>


--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-4216) IllegalArgumentException prefetching from META

2012-05-08 Thread Dave Latham (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-4216?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13270925#comment-13270925
 ] 

Dave Latham commented on HBASE-4216:


Got the same thing on 0.92.1 in production.

> IllegalArgumentException prefetching from META
> --
>
> Key: HBASE-4216
> URL: https://issues.apache.org/jira/browse/HBASE-4216
> Project: HBase
>  Issue Type: Bug
>  Components: client
>Affects Versions: 0.92.0
>Reporter: Todd Lipcon
>Priority: Critical
>
> Received one of these while doing a YCSB test on 26 nodes on trunk:
> java.io.IOException: java.lang.IllegalArgumentException: hostname can't be 
> null

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5964) HFileSystem: "No FileSystem for scheme: hdfs"

2012-05-08 Thread Andrew Purtell (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-5964?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrew Purtell updated HBASE-5964:
--

Attachment: HBASE-5964.patch

> HFileSystem: "No FileSystem for scheme: hdfs"
> -
>
> Key: HBASE-5964
> URL: https://issues.apache.org/jira/browse/HBASE-5964
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 0.94.0, 0.96.0
> Environment: Hadoop 2.0.0-alpha-SNAPSHOT, HBase 0.94.0-SNAPSHOT
>Reporter: Andrew Purtell
>Assignee: Andrew Purtell
> Attachments: HBASE-5964.patch
>
>
> I've been seeing this with Hadoop 2.0.0-alpha-SNAPSHOT and HBase 
> 0.94.0-SNAPSHOT:
> {noformat}
> 2012-05-08 15:18:00,692 FATAL 
> [RegionServer:0;acer.localdomain,48307,1336515479011] 
> regionserver.HRegionServer(1674): ABORTING region server 
> acer.localdomain,48307,1336515479011: Unhandled exception: No FileSystem for 
> scheme: hdfs
> java.io.IOException: No FileSystem for scheme: hdfs
>   at 
> org.apache.hadoop.hbase.fs.HFileSystem.newInstanceFileSystem(HFileSystem.java:146)
>   at org.apache.hadoop.hbase.fs.HFileSystem.(HFileSystem.java:75)
>   at 
> org.apache.hadoop.hbase.regionserver.HRegionServer.handleReportForDutyResponse(HRegionServer.java:973)
>   at 
> org.apache.hadoop.hbase.MiniHBaseCluster$MiniHBaseClusterRegionServer.handleReportForDutyResponse(MiniHBaseCluster.java:110)
>   at 
> org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:671)
>   at 
> org.apache.hadoop.hbase.MiniHBaseCluster$MiniHBaseClusterRegionServer.runRegionServer(MiniHBaseCluster.java:136)
>   at 
> org.apache.hadoop.hbase.MiniHBaseCluster$MiniHBaseClusterRegionServer.access$000(MiniHBaseCluster.java:89)
>   at 
> org.apache.hadoop.hbase.MiniHBaseCluster$MiniHBaseClusterRegionServer$1.run(MiniHBaseCluster.java:120)
>   at java.security.AccessController.doPrivileged(Native Method)
>   at javax.security.auth.Subject.doAs(Subject.java:357)
>   at 
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1212)
>   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>   at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>   at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>   at java.lang.reflect.Method.invoke(Method.java:616)
>   at org.apache.hadoop.hbase.util.Methods.call(Methods.java:37)
>   at org.apache.hadoop.hbase.security.User.call(User.java:586)
>   at org.apache.hadoop.hbase.security.User.access$700(User.java:50)
>   at 
> org.apache.hadoop.hbase.security.User$SecureHadoopUser.runAs(User.java:426)
>   at 
> org.apache.hadoop.hbase.MiniHBaseCluster$MiniHBaseClusterRegionServer.run(MiniHBaseCluster.java:118)
>   at java.lang.Thread.run(Thread.java:679)
> {noformat}
> Not sure precisely when it started. First I thought it might be due to 
> HBASE-5955 but I cherry picked that change over from trunk. Then I got 
> HBASE-5963 out of the way.  

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5964) HFileSystem: "No FileSystem for scheme: hdfs"

2012-05-08 Thread Andrew Purtell (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-5964?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrew Purtell updated HBASE-5964:
--

Status: Patch Available  (was: Open)

> HFileSystem: "No FileSystem for scheme: hdfs"
> -
>
> Key: HBASE-5964
> URL: https://issues.apache.org/jira/browse/HBASE-5964
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 0.94.0, 0.96.0
> Environment: Hadoop 2.0.0-alpha-SNAPSHOT, HBase 0.94.0-SNAPSHOT
>Reporter: Andrew Purtell
>Assignee: Andrew Purtell
> Attachments: HBASE-5964.patch
>
>
> I've been seeing this with Hadoop 2.0.0-alpha-SNAPSHOT and HBase 
> 0.94.0-SNAPSHOT:
> {noformat}
> 2012-05-08 15:18:00,692 FATAL 
> [RegionServer:0;acer.localdomain,48307,1336515479011] 
> regionserver.HRegionServer(1674): ABORTING region server 
> acer.localdomain,48307,1336515479011: Unhandled exception: No FileSystem for 
> scheme: hdfs
> java.io.IOException: No FileSystem for scheme: hdfs
>   at 
> org.apache.hadoop.hbase.fs.HFileSystem.newInstanceFileSystem(HFileSystem.java:146)
>   at org.apache.hadoop.hbase.fs.HFileSystem.(HFileSystem.java:75)
>   at 
> org.apache.hadoop.hbase.regionserver.HRegionServer.handleReportForDutyResponse(HRegionServer.java:973)
>   at 
> org.apache.hadoop.hbase.MiniHBaseCluster$MiniHBaseClusterRegionServer.handleReportForDutyResponse(MiniHBaseCluster.java:110)
>   at 
> org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:671)
>   at 
> org.apache.hadoop.hbase.MiniHBaseCluster$MiniHBaseClusterRegionServer.runRegionServer(MiniHBaseCluster.java:136)
>   at 
> org.apache.hadoop.hbase.MiniHBaseCluster$MiniHBaseClusterRegionServer.access$000(MiniHBaseCluster.java:89)
>   at 
> org.apache.hadoop.hbase.MiniHBaseCluster$MiniHBaseClusterRegionServer$1.run(MiniHBaseCluster.java:120)
>   at java.security.AccessController.doPrivileged(Native Method)
>   at javax.security.auth.Subject.doAs(Subject.java:357)
>   at 
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1212)
>   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>   at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>   at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>   at java.lang.reflect.Method.invoke(Method.java:616)
>   at org.apache.hadoop.hbase.util.Methods.call(Methods.java:37)
>   at org.apache.hadoop.hbase.security.User.call(User.java:586)
>   at org.apache.hadoop.hbase.security.User.access$700(User.java:50)
>   at 
> org.apache.hadoop.hbase.security.User$SecureHadoopUser.runAs(User.java:426)
>   at 
> org.apache.hadoop.hbase.MiniHBaseCluster$MiniHBaseClusterRegionServer.run(MiniHBaseCluster.java:118)
>   at java.lang.Thread.run(Thread.java:679)
> {noformat}
> Not sure precisely when it started. First I thought it might be due to 
> HBASE-5955 but I cherry picked that change over from trunk. Then I got 
> HBASE-5963 out of the way.  

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5963) ClassCastException: FileSystem$Cache$ClientFinalizer cannot be cast to Thread

2012-05-08 Thread Andrew Purtell (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-5963?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrew Purtell updated HBASE-5963:
--

Status: Patch Available  (was: Open)

> ClassCastException: FileSystem$Cache$ClientFinalizer cannot be cast to Thread
> -
>
> Key: HBASE-5963
> URL: https://issues.apache.org/jira/browse/HBASE-5963
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 0.94.0, 0.96.0
> Environment: Bigtop 0.4.0-incubating-SNAPSHOT, Hadoop 
> 2.0.0-alpha-SNAPSHOT, HBase 0.94.0-SNAPSHOT
>Reporter: Andrew Purtell
>Assignee: Andrew Purtell
> Attachments: HBASE-5963.patch
>
>
> {code}
> 12/05/08 19:49:26 INFO regionserver.HRegionServer: STOPPED: Failed 
> initialization
> Exception in thread "main" java.lang.ClassCastException: 
> org.apache.hadoop.fs.FileSystem$Cache$ClientFinalizer cannot be cast to 
> java.lang.Thread
>   at 
> org.apache.hadoop.hbase.regionserver.ShutdownHook.suppressHdfsShutdownHook(ShutdownHook.java:181)
>   at 
> org.apache.hadoop.hbase.regionserver.ShutdownHook.install(ShutdownHook.java:82)
>   at 
> org.apache.hadoop.hbase.regionserver.HRegionServer.startRegionServer(HRegionServer.java:3601)
>   at 
> org.apache.hadoop.hbase.regionserver.HRegionServer.startRegionServer(HRegionServer.java:3585)
>   at 
> org.apache.hadoop.hbase.regionserver.HRegionServerCommandLine.start(HRegionServerCommandLine.java:61)
>   at 
> org.apache.hadoop.hbase.regionserver.HRegionServerCommandLine.run(HRegionServerCommandLine.java:75)
>   at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
>   at 
> org.apache.hadoop.hbase.util.ServerCommandLine.doMain(ServerCommandLine.java:76)
>   at 
> org.apache.hadoop.hbase.regionserver.HRegionServer.main(HRegionServer.java:3645)
> {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5942) HConnnectionManager.getRegionServerWithRetries doesn't call afterCall properly

2012-05-08 Thread Hudson (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-5942?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13270923#comment-13270923
 ] 

Hudson commented on HBASE-5942:
---

Integrated in HBase-0.92 #402 (See 
[https://builds.apache.org/job/HBase-0.92/402/])
HBASE-5942 HConnnectionManager.getRegionServerWithRetries doesn't call 
afterCall properly (Revision 1335734)

 Result = FAILURE
stack : 
Files : 
* /hbase/branches/0.92/CHANGES.txt
* 
/hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java


> HConnnectionManager.getRegionServerWithRetries doesn't call afterCall properly
> --
>
> Key: HBASE-5942
> URL: https://issues.apache.org/jira/browse/HBASE-5942
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 0.92.1
>Reporter: Zhihong Yu
>Assignee: ramkrishna.s.vasudevan
> Fix For: 0.92.2
>
> Attachments: hbase-5942.patch
>
>
> HConnnectionManager.getRegionServerWithRetries:
> {code}
>   return callable.call();
> } catch (Throwable t) {
>   callable.shouldRetry(t);
> {code}
> shouldRetry relies on the proper startTime and endTime to calculate the 
> timeout value. However, callable.afterCall() is called in finally block. Thus 
> callable.callTimeout will be set to negative value in callable.shouldRetry().

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5732) Remove the SecureRPCEngine and merge the security-related logic in the core engine

2012-05-08 Thread Hadoop QA (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-5732?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13270915#comment-13270915
 ] 

Hadoop QA commented on HBASE-5732:
--

-1 overall.  Here are the results of testing the latest attachment 
  
http://issues.apache.org/jira/secure/attachment/12526048/rpcengine-merge.9.patch
  against trunk revision .

+1 @author.  The patch does not contain any @author tags.

+1 tests included.  The patch appears to include 37 new or modified tests.

+1 hadoop23.  The patch compiles against the hadoop 0.23.x profile.

+1 javadoc.  The javadoc tool did not generate any warning messages.

+1 javac.  The applied patch does not increase the total number of javac 
compiler warnings.

-1 findbugs.  The patch appears to introduce 26 new Findbugs (version 
1.3.9) warnings.

+1 release audit.  The applied patch does not increase the total number of 
release audit warnings.

 -1 core tests.  The patch failed these unit tests:
   org.apache.hadoop.hbase.master.TestMXBean

Test results: 
https://builds.apache.org/job/PreCommit-HBASE-Build/1802//testReport/
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/1802//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
Console output: 
https://builds.apache.org/job/PreCommit-HBASE-Build/1802//console

This message is automatically generated.

> Remove the SecureRPCEngine and merge the security-related logic in the core 
> engine
> --
>
> Key: HBASE-5732
> URL: https://issues.apache.org/jira/browse/HBASE-5732
> Project: HBase
>  Issue Type: Improvement
>Reporter: Devaraj Das
>Assignee: Devaraj Das
> Attachments: 5732-rpcengine-merge.7.patch, rpcengine-merge.3.patch, 
> rpcengine-merge.4.patch, rpcengine-merge.9.patch, rpcengine-merge.patch
>
>
> Remove the SecureRPCEngine and merge the security-related logic in the core 
> engine. Follow up to HBASE-5727.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5877) When a query fails because the region has moved, let the regionserver return the new address to the client

2012-05-08 Thread Hadoop QA (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-5877?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13270913#comment-13270913
 ] 

Hadoop QA commented on HBASE-5877:
--

-1 overall.  Here are the results of testing the latest attachment 
  http://issues.apache.org/jira/secure/attachment/12526047/5877.v18.patch
  against trunk revision .

+1 @author.  The patch does not contain any @author tags.

+1 tests included.  The patch appears to include 9 new or modified tests.

+1 hadoop23.  The patch compiles against the hadoop 0.23.x profile.

+1 javadoc.  The javadoc tool did not generate any warning messages.

+1 javac.  The applied patch does not increase the total number of javac 
compiler warnings.

+1 findbugs.  The patch does not introduce any new Findbugs (version 1.3.9) 
warnings.

+1 release audit.  The applied patch does not increase the total number of 
release audit warnings.

 -1 core tests.  The patch failed these unit tests:
   
org.apache.hadoop.hbase.regionserver.TestSplitTransactionOnCluster
  org.apache.hadoop.hbase.replication.TestReplication

Test results: 
https://builds.apache.org/job/PreCommit-HBASE-Build/1801//testReport/
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/1801//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
Console output: 
https://builds.apache.org/job/PreCommit-HBASE-Build/1801//console

This message is automatically generated.

> When a query fails because the region has moved, let the regionserver return 
> the new address to the client
> --
>
> Key: HBASE-5877
> URL: https://issues.apache.org/jira/browse/HBASE-5877
> Project: HBase
>  Issue Type: Improvement
>  Components: client, master, regionserver
>Affects Versions: 0.96.0
>Reporter: nkeywal
>Assignee: nkeywal
>Priority: Minor
> Fix For: 0.96.0
>
> Attachments: 5877-v16.txt, 5877-v17.txt, 5877-v17.txt, 5877.v1.patch, 
> 5877.v12.patch, 5877.v15.patch, 5877.v18.patch, 5877.v6.patch
>
>
> This is mainly useful when we do a rolling restart. This will decrease the 
> load on the master and the network load.
> Note that a region is not immediately opened after a close. So:
> - it seems preferable to wait before retrying on the other server. An 
> optimisation would be to have an heuristic depending on when the region was 
> closed.
> - during a rolling restart, the server moves the regions then stops. So we 
> may have failures when the server is stopped, and this patch won't help.
> The implementation in the first patch does:
> - on the region move, there is an added parameter on the regionserver#close 
> to say where we are sending the region
> - the regionserver keeps a list of what was moved. Each entry is kept 100 
> seconds.
> - the regionserver sends a specific exception when it receives a query on a 
> moved region. This exception contains the new address.
> - the client analyses the exeptions and update its cache accordingly...

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5964) HFileSystem: "No FileSystem for scheme: hdfs"

2012-05-08 Thread Andrew Purtell (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-5964?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrew Purtell updated HBASE-5964:
--

Summary: HFileSystem: "No FileSystem for scheme: hdfs"  (was: Hadoop 
2.0.0-alpha: "No FileSystem for scheme: hdfs")

> HFileSystem: "No FileSystem for scheme: hdfs"
> -
>
> Key: HBASE-5964
> URL: https://issues.apache.org/jira/browse/HBASE-5964
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 0.94.0, 0.96.0
> Environment: Hadoop 2.0.0-alpha-SNAPSHOT, HBase 0.94.0-SNAPSHOT
>Reporter: Andrew Purtell
>Assignee: Andrew Purtell
>
> I've been seeing this with Hadoop 2.0.0-alpha-SNAPSHOT and HBase 
> 0.94.0-SNAPSHOT:
> {noformat}
> 2012-05-08 15:18:00,692 FATAL 
> [RegionServer:0;acer.localdomain,48307,1336515479011] 
> regionserver.HRegionServer(1674): ABORTING region server 
> acer.localdomain,48307,1336515479011: Unhandled exception: No FileSystem for 
> scheme: hdfs
> java.io.IOException: No FileSystem for scheme: hdfs
>   at 
> org.apache.hadoop.hbase.fs.HFileSystem.newInstanceFileSystem(HFileSystem.java:146)
>   at org.apache.hadoop.hbase.fs.HFileSystem.(HFileSystem.java:75)
>   at 
> org.apache.hadoop.hbase.regionserver.HRegionServer.handleReportForDutyResponse(HRegionServer.java:973)
>   at 
> org.apache.hadoop.hbase.MiniHBaseCluster$MiniHBaseClusterRegionServer.handleReportForDutyResponse(MiniHBaseCluster.java:110)
>   at 
> org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:671)
>   at 
> org.apache.hadoop.hbase.MiniHBaseCluster$MiniHBaseClusterRegionServer.runRegionServer(MiniHBaseCluster.java:136)
>   at 
> org.apache.hadoop.hbase.MiniHBaseCluster$MiniHBaseClusterRegionServer.access$000(MiniHBaseCluster.java:89)
>   at 
> org.apache.hadoop.hbase.MiniHBaseCluster$MiniHBaseClusterRegionServer$1.run(MiniHBaseCluster.java:120)
>   at java.security.AccessController.doPrivileged(Native Method)
>   at javax.security.auth.Subject.doAs(Subject.java:357)
>   at 
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1212)
>   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>   at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>   at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>   at java.lang.reflect.Method.invoke(Method.java:616)
>   at org.apache.hadoop.hbase.util.Methods.call(Methods.java:37)
>   at org.apache.hadoop.hbase.security.User.call(User.java:586)
>   at org.apache.hadoop.hbase.security.User.access$700(User.java:50)
>   at 
> org.apache.hadoop.hbase.security.User$SecureHadoopUser.runAs(User.java:426)
>   at 
> org.apache.hadoop.hbase.MiniHBaseCluster$MiniHBaseClusterRegionServer.run(MiniHBaseCluster.java:118)
>   at java.lang.Thread.run(Thread.java:679)
> {noformat}
> Not sure precisely when it started. First I thought it might be due to 
> HBASE-5955 but I cherry picked that change over from trunk. Then I got 
> HBASE-5963 out of the way.  

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5935) Add Region-level PB-based calls to HMasterInterface

2012-05-08 Thread Gregory Chanan (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-5935?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gregory Chanan updated HBASE-5935:
--

Status: Patch Available  (was: Open)

> Add Region-level PB-based calls to HMasterInterface
> ---
>
> Key: HBASE-5935
> URL: https://issues.apache.org/jira/browse/HBASE-5935
> Project: HBase
>  Issue Type: Task
>  Components: ipc, master, migration
>Reporter: Gregory Chanan
>Assignee: Gregory Chanan
> Fix For: 0.96.0
>
> Attachments: HBASE-5935-v2.patch
>
>
> This should be a subtask of HBASE-5445, but since that is a subtask, I can't 
> also make this a subtask (apparently).
> This is for converting the region-level calls, i.e.:
> moveRegion
> assignRegion
> unassignRegion

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5935) Add Region-level PB-based calls to HMasterInterface

2012-05-08 Thread Gregory Chanan (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-5935?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gregory Chanan updated HBASE-5935:
--

Attachment: HBASE-5935-v2.patch

> Add Region-level PB-based calls to HMasterInterface
> ---
>
> Key: HBASE-5935
> URL: https://issues.apache.org/jira/browse/HBASE-5935
> Project: HBase
>  Issue Type: Task
>  Components: ipc, master, migration
>Reporter: Gregory Chanan
>Assignee: Gregory Chanan
> Fix For: 0.96.0
>
> Attachments: HBASE-5935-v2.patch
>
>
> This should be a subtask of HBASE-5445, but since that is a subtask, I can't 
> also make this a subtask (apparently).
> This is for converting the region-level calls, i.e.:
> moveRegion
> assignRegion
> unassignRegion

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Assigned] (HBASE-5964) Hadoop 2.0.0-alpha: "No FileSystem for scheme: hdfs"

2012-05-08 Thread Andrew Purtell (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-5964?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrew Purtell reassigned HBASE-5964:
-

Assignee: Andrew Purtell

> Hadoop 2.0.0-alpha: "No FileSystem for scheme: hdfs"
> 
>
> Key: HBASE-5964
> URL: https://issues.apache.org/jira/browse/HBASE-5964
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 0.94.0, 0.96.0
> Environment: Hadoop 2.0.0-alpha-SNAPSHOT, HBase 0.94.0-SNAPSHOT
>Reporter: Andrew Purtell
>Assignee: Andrew Purtell
>
> I've been seeing this with Hadoop 2.0.0-alpha-SNAPSHOT and HBase 
> 0.94.0-SNAPSHOT:
> {noformat}
> 2012-05-08 15:18:00,692 FATAL 
> [RegionServer:0;acer.localdomain,48307,1336515479011] 
> regionserver.HRegionServer(1674): ABORTING region server 
> acer.localdomain,48307,1336515479011: Unhandled exception: No FileSystem for 
> scheme: hdfs
> java.io.IOException: No FileSystem for scheme: hdfs
>   at 
> org.apache.hadoop.hbase.fs.HFileSystem.newInstanceFileSystem(HFileSystem.java:146)
>   at org.apache.hadoop.hbase.fs.HFileSystem.(HFileSystem.java:75)
>   at 
> org.apache.hadoop.hbase.regionserver.HRegionServer.handleReportForDutyResponse(HRegionServer.java:973)
>   at 
> org.apache.hadoop.hbase.MiniHBaseCluster$MiniHBaseClusterRegionServer.handleReportForDutyResponse(MiniHBaseCluster.java:110)
>   at 
> org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:671)
>   at 
> org.apache.hadoop.hbase.MiniHBaseCluster$MiniHBaseClusterRegionServer.runRegionServer(MiniHBaseCluster.java:136)
>   at 
> org.apache.hadoop.hbase.MiniHBaseCluster$MiniHBaseClusterRegionServer.access$000(MiniHBaseCluster.java:89)
>   at 
> org.apache.hadoop.hbase.MiniHBaseCluster$MiniHBaseClusterRegionServer$1.run(MiniHBaseCluster.java:120)
>   at java.security.AccessController.doPrivileged(Native Method)
>   at javax.security.auth.Subject.doAs(Subject.java:357)
>   at 
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1212)
>   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>   at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>   at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>   at java.lang.reflect.Method.invoke(Method.java:616)
>   at org.apache.hadoop.hbase.util.Methods.call(Methods.java:37)
>   at org.apache.hadoop.hbase.security.User.call(User.java:586)
>   at org.apache.hadoop.hbase.security.User.access$700(User.java:50)
>   at 
> org.apache.hadoop.hbase.security.User$SecureHadoopUser.runAs(User.java:426)
>   at 
> org.apache.hadoop.hbase.MiniHBaseCluster$MiniHBaseClusterRegionServer.run(MiniHBaseCluster.java:118)
>   at java.lang.Thread.run(Thread.java:679)
> {noformat}
> Not sure precisely when it started. First I thought it might be due to 
> HBASE-5955 but I cherry picked that change over from trunk. Then I got 
> HBASE-5963 out of the way.  

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Assigned] (HBASE-5955) Guava 11 drops MapEvictionListener and Hadoop 2.0.0-alpha requires it

2012-05-08 Thread Andrew Purtell (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-5955?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrew Purtell reassigned HBASE-5955:
-

Assignee: Andrew Purtell

> Guava 11 drops MapEvictionListener and Hadoop 2.0.0-alpha requires it
> -
>
> Key: HBASE-5955
> URL: https://issues.apache.org/jira/browse/HBASE-5955
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 0.94.0
>Reporter: Andrew Purtell
>Assignee: Andrew Purtell
>
> Hadoop 2.0.0-alpha depends on Guava 11.0.2. Updating HBase dependencies to 
> match produces the following compilation errors:
> {code}
> [ERROR] SingleSizeCache.java:[41,32] cannot find symbol
> [ERROR] symbol  : class MapEvictionListener
> [ERROR] location: package com.google.common.collect
> [ERROR] 
> [ERROR] SingleSizeCache.java:[94,4] cannot find symbol
> [ERROR] symbol  : class MapEvictionListener
> [ERROR] location: class org.apache.hadoop.hbase.io.hfile.slab.SingleSizeCache
> [ERROR] 
> [ERROR] SingleSizeCache.java:[94,69] cannot find symbol
> [ERROR] symbol  : class MapEvictionListener
> [ERROR] location: class org.apache.hadoop.hbase.io.hfile.slab.SingleSizeCache
> {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Assigned] (HBASE-5963) ClassCastException: FileSystem$Cache$ClientFinalizer cannot be cast to Thread

2012-05-08 Thread Andrew Purtell (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-5963?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrew Purtell reassigned HBASE-5963:
-

Assignee: Andrew Purtell

> ClassCastException: FileSystem$Cache$ClientFinalizer cannot be cast to Thread
> -
>
> Key: HBASE-5963
> URL: https://issues.apache.org/jira/browse/HBASE-5963
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 0.94.0, 0.96.0
> Environment: Bigtop 0.4.0-incubating-SNAPSHOT, Hadoop 
> 2.0.0-alpha-SNAPSHOT, HBase 0.94.0-SNAPSHOT
>Reporter: Andrew Purtell
>Assignee: Andrew Purtell
> Attachments: HBASE-5963.patch
>
>
> {code}
> 12/05/08 19:49:26 INFO regionserver.HRegionServer: STOPPED: Failed 
> initialization
> Exception in thread "main" java.lang.ClassCastException: 
> org.apache.hadoop.fs.FileSystem$Cache$ClientFinalizer cannot be cast to 
> java.lang.Thread
>   at 
> org.apache.hadoop.hbase.regionserver.ShutdownHook.suppressHdfsShutdownHook(ShutdownHook.java:181)
>   at 
> org.apache.hadoop.hbase.regionserver.ShutdownHook.install(ShutdownHook.java:82)
>   at 
> org.apache.hadoop.hbase.regionserver.HRegionServer.startRegionServer(HRegionServer.java:3601)
>   at 
> org.apache.hadoop.hbase.regionserver.HRegionServer.startRegionServer(HRegionServer.java:3585)
>   at 
> org.apache.hadoop.hbase.regionserver.HRegionServerCommandLine.start(HRegionServerCommandLine.java:61)
>   at 
> org.apache.hadoop.hbase.regionserver.HRegionServerCommandLine.run(HRegionServerCommandLine.java:75)
>   at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
>   at 
> org.apache.hadoop.hbase.util.ServerCommandLine.doMain(ServerCommandLine.java:76)
>   at 
> org.apache.hadoop.hbase.regionserver.HRegionServer.main(HRegionServer.java:3645)
> {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5935) Add Region-level PB-based calls to HMasterInterface

2012-05-08 Thread jirapos...@reviews.apache.org (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-5935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13270911#comment-13270911
 ] 

jirapos...@reviews.apache.org commented on HBASE-5935:
--


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/5060/
---

(Updated 2012-05-08 22:49:04.533210)


Review request for hbase, Michael Stack and Jimmy Xiang.


Changes
---

Updated for Jimmy's and stack's comments.


Summary
---

Converted all the region-level calls in HMasterInterface to use Protobufs 
(move, assign, unassign).

I've put the new generated calls in HMasterInterface rather than a new class, 
because it is easier to have one class for now, until I convert everything.  
This will only be temporary, but let me know how you want to handle.  Bump the 
version number?


This addresses bug HBASE-5935.
https://issues.apache.org/jira/browse/HBASE-5935


Diffs (updated)
-

  src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java 5d4be3f 
  src/main/java/org/apache/hadoop/hbase/ipc/HMasterInterface.java ce81547 
  src/main/java/org/apache/hadoop/hbase/master/HMaster.java ccc7119 
  src/main/java/org/apache/hadoop/hbase/protobuf/RequestConverter.java a5e03e1 
  src/main/java/org/apache/hadoop/hbase/protobuf/generated/MasterProtos.java 
PRE-CREATION 
  src/main/protobuf/Master.proto PRE-CREATION 
  src/main/protobuf/RegionServerStatus.proto 9d7728f 
  src/test/java/org/apache/hadoop/hbase/TestDrainingServer.java e5de603 
  src/test/java/org/apache/hadoop/hbase/coprocessor/TestMasterObserver.java 
83297cc 

Diff: https://reviews.apache.org/r/5060/diff


Testing
---


Thanks,

Gregory



> Add Region-level PB-based calls to HMasterInterface
> ---
>
> Key: HBASE-5935
> URL: https://issues.apache.org/jira/browse/HBASE-5935
> Project: HBase
>  Issue Type: Task
>  Components: ipc, master, migration
>Reporter: Gregory Chanan
>Assignee: Gregory Chanan
> Fix For: 0.96.0
>
>
> This should be a subtask of HBASE-5445, but since that is a subtask, I can't 
> also make this a subtask (apparently).
> This is for converting the region-level calls, i.e.:
> moveRegion
> assignRegion
> unassignRegion

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5955) Guava 11 drops MapEvictionListener and Hadoop 2.0.0-alpha requires it

2012-05-08 Thread Andrew Purtell (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-5955?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13270910#comment-13270910
 ] 

Andrew Purtell commented on HBASE-5955:
---

@Lars, how about getting HBASE-5739 into 0.94?

> Guava 11 drops MapEvictionListener and Hadoop 2.0.0-alpha requires it
> -
>
> Key: HBASE-5955
> URL: https://issues.apache.org/jira/browse/HBASE-5955
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 0.94.0
>Reporter: Andrew Purtell
>Assignee: Andrew Purtell
>
> Hadoop 2.0.0-alpha depends on Guava 11.0.2. Updating HBase dependencies to 
> match produces the following compilation errors:
> {code}
> [ERROR] SingleSizeCache.java:[41,32] cannot find symbol
> [ERROR] symbol  : class MapEvictionListener
> [ERROR] location: package com.google.common.collect
> [ERROR] 
> [ERROR] SingleSizeCache.java:[94,4] cannot find symbol
> [ERROR] symbol  : class MapEvictionListener
> [ERROR] location: class org.apache.hadoop.hbase.io.hfile.slab.SingleSizeCache
> [ERROR] 
> [ERROR] SingleSizeCache.java:[94,69] cannot find symbol
> [ERROR] symbol  : class MapEvictionListener
> [ERROR] location: class org.apache.hadoop.hbase.io.hfile.slab.SingleSizeCache
> {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5935) Add Region-level PB-based calls to HMasterInterface

2012-05-08 Thread jirapos...@reviews.apache.org (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-5935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13270906#comment-13270906
 ] 

jirapos...@reviews.apache.org commented on HBASE-5935:
--



bq.  On 2012-05-08 04:55:21, Michael Stack wrote:
bq.  > Looks good.  Some small potatoes below.  On this, "I've put the new 
generated calls in HMasterInterface rather than a new class, because it is 
easier to have one class for now, until I convert everything.  This will only 
be temporary, but let me know how you want to handle.  Bump the version 
number?" ... when will HMasterInterface go away?  What else all is to be done?  
Sure on bump the version number.  Pretty soon they'll be ignored so no harm 
using up a few.  Good stuff G.

Cool, I'll bump the version number for now.

I believe I've answered all your questions below.  Let me know if I missed 
anything.

Thanks for the review.


bq.  On 2012-05-08 04:55:21, Michael Stack wrote:
bq.  > src/main/java/org/apache/hadoop/hbase/master/HMaster.java, line 1213
bq.  > 

bq.  >
bq.  > FYI, spaces between if and open paren... and usually around 
operators.  Just FYI.

Thanks, will do.


bq.  On 2012-05-08 04:55:21, Michael Stack wrote:
bq.  > src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java, line 1394
bq.  > 

bq.  >
bq.  > Is it right catching this here?  You seem to handle it at a higher 
level up in the move invocation?   Won't this clause catch all the possible SEs?
bq.  > 
bq.  > Or I'm suffering myopia  and there is more going on than just this 
call invocation?

Good point, I can catch at a higher level.


bq.  On 2012-05-08 04:55:21, Michael Stack wrote:
bq.  > src/main/java/org/apache/hadoop/hbase/master/HMaster.java, line 1219
bq.  > 

bq.  >
bq.  > This needs paren (though its in the original)

Will do.


bq.  On 2012-05-08 04:55:21, Michael Stack wrote:
bq.  > src/main/protobuf/Master.proto, line 71
bq.  > 
bq.  >
bq.  > Great.  Is this all that is in the master interface?

No, the rest of the functions in HMasterInterface need to be converted.  This 
is the region-level stuff, I still have column-level, table-level and 
miscellaneous stuff to do.  I've filed a JIRA for the column-level stuff; I'll 
file JIRAs for the others once I figure out how I'm going to break down the 
work.


bq.  On 2012-05-08 04:55:21, Michael Stack wrote:
bq.  > src/main/java/org/apache/hadoop/hbase/master/HMaster.java, line 1253
bq.  > 

bq.  >
bq.  > All IOEs now need to get converted to a SE?  I added an 
HBaseException recently.  Its a checked exception.  Should we have an 
HBaseServiceExceptoin?  SE is pb?

ServiceException is coming from pb, that's right.  So when we either get rid of 
HMasterInterface or repurpose it, the interface will define that we throw 
ServiceExceptions.


bq.  On 2012-05-08 04:55:21, Michael Stack wrote:
bq.  > src/main/java/org/apache/hadoop/hbase/ipc/HMasterInterface.java, line 285
bq.  > 
bq.  >
bq.  > It does seem a little messy having to hoist all these pb classes up 
in here AND into HMasterInterface.
bq.  > 
bq.  > HMasterInterface is going to go away?

I'm not sure what you mean by "having to hoist all these pb classes up in here 
AND into HMasterInterface." -- this is HMasterInterface?

Also, I don't think PB will be nice and generated usable javadoc for you, so 
I'm not sure how to provide javadoc comments without redeclaring the functions? 
 I admit to being completely ignorant about this area.

Once everything in HMasterInterface is converted to use PB, we can either 
declare a new class for the representation (similar to 
RegionServerStatusProtocol) or just re-purpose HMasterInterface for that.  What 
is your preference?


- Gregory


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/5060/#review7667
---


On 2012-05-08 01:34:20, Gregory Chanan wrote:
bq.  
bq.  ---
bq.  This is an automatically generated e-mail. To reply, visit:
bq.  https://reviews.apache.org/r/5060/
bq.  ---
bq.  
bq.  (Updated 2012-05-08 01:34:20)
bq.  
bq.  
bq.  Review request for hbase, Michael Stack and Jimmy Xiang.
bq.  
bq.  
bq.  Summary
bq.  ---
bq.  
bq.  Converted all the region-level calls in HMasterInterface

[jira] [Commented] (HBASE-5925) Issue with only using the old config param hbase.hstore.compactionThreshold but not the corresponding new one

2012-05-08 Thread Hudson (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-5925?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13270898#comment-13270898
 ] 

Hudson commented on HBASE-5925:
---

Integrated in HBase-TRUNK #2857 (See 
[https://builds.apache.org/job/HBase-TRUNK/2857/])
HBASE-5925 Issue with only using the old config param 
hbase.hstore.compactionThreshold but not the corresponding new one (Revision 
1335738)

 Result = FAILURE
stack : 
Files : 
* 
/hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/MemStoreFlusher.java


> Issue with only using the old config param hbase.hstore.compactionThreshold 
> but not the corresponding new one
> -
>
> Key: HBASE-5925
> URL: https://issues.apache.org/jira/browse/HBASE-5925
> Project: HBase
>  Issue Type: Bug
>Reporter: Anoop Sam John
>Assignee: Anoop Sam John
>Priority: Minor
> Fix For: 0.96.0
>
> Attachments: HBASE-5925.patch
>
>
> One observation while going through the code:-
> In MemStoreFlusher constructor
> {code}
> this.blockingStoreFilesNumber =
>   conf.getInt("hbase.hstore.blockingStoreFiles", 7);
> if (this.blockingStoreFilesNumber == -1) {
>   this.blockingStoreFilesNumber = 1 +
> conf.getInt("hbase.hstore.compactionThreshold", 3);
> }
> {code}
> Here as per the code if hbase.hstore.blockingStoreFiles is configured as -1, 
> we are making this value to be 1+ min files to compact
> But here we read the old config item only!
> Here also we need to read the new config 1st and if not there then the old 
> one.. Is this a miss?
> Like
> conf.getInt("hbase.hstore.compaction.min",
>  conf.getInt("hbase.hstore.compactionThreshold", 3))

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (HBASE-5964) Hadoop 2.0.0-alpha: "No FileSystem for scheme: hdfs"

2012-05-08 Thread Andrew Purtell (JIRA)

Andrew Purtell created HBASE-5964:
-

 Summary: Hadoop 2.0.0-alpha: "No FileSystem for scheme: hdfs"
 Key: HBASE-5964
 URL: https://issues.apache.org/jira/browse/HBASE-5964
 Project: HBase
  Issue Type: Bug
Affects Versions: 0.94.0, 0.96.0
 Environment: Hadoop 2.0.0-alpha-SNAPSHOT, HBase 0.94.0-SNAPSHOT
Reporter: Andrew Purtell


I've been seeing this with Hadoop 2.0.0-alpha-SNAPSHOT and HBase 
0.94.0-SNAPSHOT:

{noformat}
2012-05-08 15:18:00,692 FATAL 
[RegionServer:0;acer.localdomain,48307,1336515479011] 
regionserver.HRegionServer(1674): ABORTING region server 
acer.localdomain,48307,1336515479011: Unhandled exception: No FileSystem for 
scheme: hdfs
java.io.IOException: No FileSystem for scheme: hdfs
at 
org.apache.hadoop.hbase.fs.HFileSystem.newInstanceFileSystem(HFileSystem.java:146)
at org.apache.hadoop.hbase.fs.HFileSystem.(HFileSystem.java:75)
at 
org.apache.hadoop.hbase.regionserver.HRegionServer.handleReportForDutyResponse(HRegionServer.java:973)
at 
org.apache.hadoop.hbase.MiniHBaseCluster$MiniHBaseClusterRegionServer.handleReportForDutyResponse(MiniHBaseCluster.java:110)
at 
org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:671)
at 
org.apache.hadoop.hbase.MiniHBaseCluster$MiniHBaseClusterRegionServer.runRegionServer(MiniHBaseCluster.java:136)
at 
org.apache.hadoop.hbase.MiniHBaseCluster$MiniHBaseClusterRegionServer.access$000(MiniHBaseCluster.java:89)
at 
org.apache.hadoop.hbase.MiniHBaseCluster$MiniHBaseClusterRegionServer$1.run(MiniHBaseCluster.java:120)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:357)
at 
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1212)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:616)
at org.apache.hadoop.hbase.util.Methods.call(Methods.java:37)
at org.apache.hadoop.hbase.security.User.call(User.java:586)
at org.apache.hadoop.hbase.security.User.access$700(User.java:50)
at 
org.apache.hadoop.hbase.security.User$SecureHadoopUser.runAs(User.java:426)
at 
org.apache.hadoop.hbase.MiniHBaseCluster$MiniHBaseClusterRegionServer.run(MiniHBaseCluster.java:118)
at java.lang.Thread.run(Thread.java:679)
{noformat}

Not sure precisely when it started. First I thought it might be due to 
HBASE-5955 but I cherry picked that change over from trunk. Then I got 
HBASE-5963 out of the way.  

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Issue Comment Edited] (HBASE-5963) ClassCastException: FileSystem$Cache$ClientFinalizer cannot be cast to Thread

2012-05-08 Thread Andrew Purtell (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-5963?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13270892#comment-13270892
 ] 

Andrew Purtell edited comment on HBASE-5963 at 5/8/12 10:22 PM:


The attached patch gets past the problem but I'm not able to test it yet due to 
another issue I'm about to file another JIRA for.

Edit: I mean, I can see that it gets past the ClassCastException, and the 
ShutdownHookManager removes the client finalizer successfully, but I have not 
yet been able to observe if normal RS shutdown is ok.

  was (Author: apurtell):
The attached patch gets past the problem but I'm not able to test it yet 
due to another issue I'm about to file another JIRA for.
  
> ClassCastException: FileSystem$Cache$ClientFinalizer cannot be cast to Thread
> -
>
> Key: HBASE-5963
> URL: https://issues.apache.org/jira/browse/HBASE-5963
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 0.94.0, 0.96.0
> Environment: Bigtop 0.4.0-incubating-SNAPSHOT, Hadoop 
> 2.0.0-alpha-SNAPSHOT, HBase 0.94.0-SNAPSHOT
>Reporter: Andrew Purtell
> Attachments: HBASE-5963.patch
>
>
> {code}
> 12/05/08 19:49:26 INFO regionserver.HRegionServer: STOPPED: Failed 
> initialization
> Exception in thread "main" java.lang.ClassCastException: 
> org.apache.hadoop.fs.FileSystem$Cache$ClientFinalizer cannot be cast to 
> java.lang.Thread
>   at 
> org.apache.hadoop.hbase.regionserver.ShutdownHook.suppressHdfsShutdownHook(ShutdownHook.java:181)
>   at 
> org.apache.hadoop.hbase.regionserver.ShutdownHook.install(ShutdownHook.java:82)
>   at 
> org.apache.hadoop.hbase.regionserver.HRegionServer.startRegionServer(HRegionServer.java:3601)
>   at 
> org.apache.hadoop.hbase.regionserver.HRegionServer.startRegionServer(HRegionServer.java:3585)
>   at 
> org.apache.hadoop.hbase.regionserver.HRegionServerCommandLine.start(HRegionServerCommandLine.java:61)
>   at 
> org.apache.hadoop.hbase.regionserver.HRegionServerCommandLine.run(HRegionServerCommandLine.java:75)
>   at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
>   at 
> org.apache.hadoop.hbase.util.ServerCommandLine.doMain(ServerCommandLine.java:76)
>   at 
> org.apache.hadoop.hbase.regionserver.HRegionServer.main(HRegionServer.java:3645)
> {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

1 2 3 >

1 - 100 of 208 matches

Mail list logo