[jira] [Updated] (HBASE-4847) Activate single jvm for small tests on jenkins

[nkeywal (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-4847?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

nkeywal updated HBASE-4847:
---

Status: Patch Available  (was: Open)

> Activate single jvm for small tests on jenkins
> --
>
> Key: HBASE-4847
> URL: https://issues.apache.org/jira/browse/HBASE-4847
> Project: HBase
>  Issue Type: Improvement
>  Components: build, test
>Affects Versions: 0.94.0
> Environment: build
>Reporter: nkeywal
>Assignee: nkeywal
>Priority: Minor
> Attachments: 4847_all.v4.patch, 4847_all.v5.patch, 4847_all.v6.patch, 
> 4847_all.v6.patch, 4847_all.v7.patch, 4847_all.v7.patch, 4847_all.v7.patch, 
> 4847_all.v7.patch, 4847_all.v8.patch, 4847_all.v8.patch, 4847_pom.patch, 
> 4847_pom.v2.patch, 4847_pom.v2.patch, 4847_pom.v2.patch, 4847_pom.v3.patch
>
>
> This will not revolutionate performances alone. We will win between 1 to 4 
> minutes.
> But we win as well:
>  - it's a step for parallelizing the tests
>  - new tests are less expensive as they do not create a new jvm: it's a 
> continuous win
>  - it will allow to push it on dev env while having the same env on dev & on 
> build, and 3 minutes are 10% of small + medium tests execution time.
> I will do a few "submit patch" to see if it works well before asking for the 
> real commit.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-4847) Activate single jvm for small tests on jenkins

[Hadoop QA (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-4847?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13162070#comment-13162070
 ] 

Hadoop QA commented on HBASE-4847:
--

-1 overall.  Here are the results of testing the latest attachment 
  http://issues.apache.org/jira/secure/attachment/12505970/4847_all.v8.patch
  against trunk revision .

+1 @author.  The patch does not contain any @author tags.

+1 tests included.  The patch appears to include 12 new or modified tests.

-1 javadoc.  The javadoc tool appears to have generated -160 warning 
messages.

+1 javac.  The applied patch does not increase the total number of javac 
compiler warnings.

-1 findbugs.  The patch appears to introduce 71 new Findbugs (version 
1.3.9) warnings.

+1 release audit.  The applied patch does not increase the total number of 
release audit warnings.

 -1 core tests.  The patch failed these unit tests:
 

Test results: 
https://builds.apache.org/job/PreCommit-HBASE-Build/429//testReport/
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/429//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/429//console

This message is automatically generated.

> Activate single jvm for small tests on jenkins
> --
>
> Key: HBASE-4847
> URL: https://issues.apache.org/jira/browse/HBASE-4847
> Project: HBase
>  Issue Type: Improvement
>  Components: build, test
>Affects Versions: 0.94.0
> Environment: build
>Reporter: nkeywal
>Assignee: nkeywal
>Priority: Minor
> Attachments: 4847_all.v4.patch, 4847_all.v5.patch, 4847_all.v6.patch, 
> 4847_all.v6.patch, 4847_all.v7.patch, 4847_all.v7.patch, 4847_all.v7.patch, 
> 4847_all.v7.patch, 4847_all.v8.patch, 4847_all.v8.patch, 4847_pom.patch, 
> 4847_pom.v2.patch, 4847_pom.v2.patch, 4847_pom.v2.patch, 4847_pom.v3.patch
>
>
> This will not revolutionate performances alone. We will win between 1 to 4 
> minutes.
> But we win as well:
>  - it's a step for parallelizing the tests
>  - new tests are less expensive as they do not create a new jvm: it's a 
> continuous win
>  - it will allow to push it on dev env while having the same env on dev & on 
> build, and 3 minutes are 10% of small + medium tests execution time.
> I will do a few "submit patch" to see if it works well before asking for the 
> real commit.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-4939) [book] expanding architecture overview, added more to FAQ, and some reformatting in troubleshooting

[Hudson (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-4939?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13162084#comment-13162084
 ] 

Hudson commented on HBASE-4939:
---

Integrated in HBase-TRUNK-security #19 (See 
[https://builds.apache.org/job/HBase-TRUNK-security/19/])
hbase-4939 book.xml (architecture/faq), troubleshooting.xml (created 
resources section)


> [book] expanding architecture overview, added more to FAQ, and some 
> reformatting in troubleshooting
> ---
>
> Key: HBASE-4939
> URL: https://issues.apache.org/jira/browse/HBASE-4939
> Project: HBase
>  Issue Type: Improvement
>Reporter: Doug Meil
>Assignee: Doug Meil
>Priority: Minor
> Attachments: docbkx_HBASE_4939.patch
>
>
> book.xml
> * Added an Overview section in Architecture (NoSQL?  When should I use 
> Hadoop?  And the difference between HBase and HDFS)
> * Moved one entry in the FAQ to the Architecture Overview
> * Added a few more links in FAQ
> troubleshooting.xml
> * Created new 'Resources' section in Troubleshooting
> * This points out the mailing lists, search-hadoop, and jira.
> * The mailing lists weren't mentioned in troubleshooting, and the last 2 were 
> in 'external tools' which is not exactly what they are in the context of 
> troubleshooting.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-4899) Region would be assigned twice easily with continually killing server and moving region in testing environment

[Hudson (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-4899?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13162081#comment-13162081
 ] 

Hudson commented on HBASE-4899:
---

Integrated in HBase-TRUNK-security #19 (See 
[https://builds.apache.org/job/HBase-TRUNK-security/19/])
HBASE-4899 Region would be assigned twice easily with continually killing 
server and moving region in testing environment

stack : 
Files : 
* 
/hbase/trunk/src/main/java/org/apache/hadoop/hbase/master/handler/ServerShutdownHandler.java


> Region would be assigned twice easily with continually  killing server and 
> moving region in testing environment
> ---
>
> Key: HBASE-4899
> URL: https://issues.apache.org/jira/browse/HBASE-4899
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 0.92.1
>Reporter: chunhui shen
>Assignee: chunhui shen
>Priority: Critical
> Fix For: 0.92.1
>
> Attachments: hbase-4899.patch, hbase-4899v2.patch, hbase-4899v3.patch
>
>
> Before assigning region in ServerShutdownHandler#process, it will check 
> whether region is in RIT,
> however, this checking doesn't work as the excepted in the following case:
> 1.move region A from server B to server C
> 2.kill server B
> 3.start server B immediately
> Let's see what happen in the code for the above case
> {code}
> for step1:
> 1.1 server B close the region A,
> 1.2 master setOffline for region 
> A,(AssignmentManager#setOffline:this.regions.remove(regionInfo))
> 1.3 server C start to open region A.(Not completed)
> for step3:
> master ServerShutdownHandler#process() for server B
> {
> ..
> splitlog()
> ...
> List regionsInTransition =
> this.services.getAssignmentManager()
> .processServerShutdown(this.serverName);
> ...
> Skip regions that were in transition unless CLOSING or PENDING_CLOSE
> ...
> assign region
> }
> {code}
> In fact, when running 
> ServerShutdownHandler#process()#this.services.getAssignmentManager().processServerShutdown(this.serverName),
>  region A is in RIT (step1.3 not completed), but the return List 
> regionsInTransition doesn't contain it, because region A has removed from 
> AssignmentManager.regions by AssignmentManager#setOffline in step 1.2
> Therefore, region A will be assigned twice.
> Actually, one server killed and started twice will also easily cause region 
> assigned twice.
> Exclude the above reason, another probability : 
> when execute ServerShutdownHandler#process()#MetaReader.getServerUserRegions 
> ,region is included which is in RIT now.
> But after completing MetaReader.getServerUserRegions, the region has been 
> opened in other server and is not in RIT now.
> In our testing environment where balancing,moving and killing are executed 
> periodly, assigning region twice often happens, and it is hateful because it 
> will affect other test cases.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-4930) Reformat HBase home page

[Hudson (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-4930?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13162082#comment-13162082
 ] 

Hudson commented on HBASE-4930:
---

Integrated in HBase-TRUNK-security #19 (See 
[https://builds.apache.org/job/HBase-TRUNK-security/19/])
hbase-4930.  home page (index.html) reformat.


> Reformat HBase home page
> 
>
> Key: HBASE-4930
> URL: https://issues.apache.org/jira/browse/HBASE-4930
> Project: HBase
>  Issue Type: Improvement
>  Components: documentation
>Reporter: Doug Meil
>Assignee: Doug Meil
>Priority: Minor
> Attachments: faq.html, index.html, index_HBASE_4930.xml.patch
>
>
> Reformat the HBase home page.  It's not a radical redesign but breaks the 
> information that is there into clearer sections:
> * Welcome to HBase!  (it's always good to be polite to the reader)
> * When Should I Use HBase?
> * Features
> * How Can I Get More Information?
> * News 
> I also updated the feature-list in this page (added some things, clarified a 
> few things, removed one or two things)
> Note:  I just overhauled the FAQ in the book today as well.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-4903) Return a result from RegionObserver.preIncrement()

[Hudson (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-4903?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13162083#comment-13162083
 ] 

Hudson commented on HBASE-4903:
---

Integrated in HBase-TRUNK-security #19 (See 
[https://builds.apache.org/job/HBase-TRUNK-security/19/])
HBASE-4903 addendum for security build

larsh : 
Files : 
* 
/hbase/trunk/security/src/main/java/org/apache/hadoop/hbase/security/access/AccessController.java


> Return a result from RegionObserver.preIncrement()
> --
>
> Key: HBASE-4903
> URL: https://issues.apache.org/jira/browse/HBASE-4903
> Project: HBase
>  Issue Type: Improvement
>Reporter: Daniel Gómez Ferro
> Fix For: 0.94.0, 0.92.1
>
> Attachments: 4903-add.txt, HBASE-4903-0.92.patch, HBASE-4903.patch, 
> HBASE-4903.patch
>
>
> The only way to return a result from RegionObserver.preIncrement() is to use 
> Result.readFields() after serializing the correct result.
> This can be fixed either returning a Result object from that function or 
> adding setters to Result. Another option is to modify the parameters and 
> receive a List as preGet() does.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-4847) Activate single jvm for small tests on jenkins

[nkeywal (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-4847?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

nkeywal updated HBASE-4847:
---

Status: Open  (was: Patch Available)

> Activate single jvm for small tests on jenkins
> --
>
> Key: HBASE-4847
> URL: https://issues.apache.org/jira/browse/HBASE-4847
> Project: HBase
>  Issue Type: Improvement
>  Components: build, test
>Affects Versions: 0.94.0
> Environment: build
>Reporter: nkeywal
>Assignee: nkeywal
>Priority: Minor
> Attachments: 4847_all.v4.patch, 4847_all.v5.patch, 4847_all.v6.patch, 
> 4847_all.v6.patch, 4847_all.v7.patch, 4847_all.v7.patch, 4847_all.v7.patch, 
> 4847_all.v7.patch, 4847_all.v8.patch, 4847_all.v8.patch, 4847_all.v9.patch, 
> 4847_pom.patch, 4847_pom.v2.patch, 4847_pom.v2.patch, 4847_pom.v2.patch, 
> 4847_pom.v3.patch
>
>
> This will not revolutionate performances alone. We will win between 1 to 4 
> minutes.
> But we win as well:
>  - it's a step for parallelizing the tests
>  - new tests are less expensive as they do not create a new jvm: it's a 
> continuous win
>  - it will allow to push it on dev env while having the same env on dev & on 
> build, and 3 minutes are 10% of small + medium tests execution time.
> I will do a few "submit patch" to see if it works well before asking for the 
> real commit.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-4847) Activate single jvm for small tests on jenkins

[nkeywal (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-4847?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

nkeywal updated HBASE-4847:
---

Attachment: 4847_all.v9.patch

> Activate single jvm for small tests on jenkins
> --
>
> Key: HBASE-4847
> URL: https://issues.apache.org/jira/browse/HBASE-4847
> Project: HBase
>  Issue Type: Improvement
>  Components: build, test
>Affects Versions: 0.94.0
> Environment: build
>Reporter: nkeywal
>Assignee: nkeywal
>Priority: Minor
> Attachments: 4847_all.v4.patch, 4847_all.v5.patch, 4847_all.v6.patch, 
> 4847_all.v6.patch, 4847_all.v7.patch, 4847_all.v7.patch, 4847_all.v7.patch, 
> 4847_all.v7.patch, 4847_all.v8.patch, 4847_all.v8.patch, 4847_all.v9.patch, 
> 4847_pom.patch, 4847_pom.v2.patch, 4847_pom.v2.patch, 4847_pom.v2.patch, 
> 4847_pom.v3.patch
>
>
> This will not revolutionate performances alone. We will win between 1 to 4 
> minutes.
> But we win as well:
>  - it's a step for parallelizing the tests
>  - new tests are less expensive as they do not create a new jvm: it's a 
> continuous win
>  - it will allow to push it on dev env while having the same env on dev & on 
> build, and 3 minutes are 10% of small + medium tests execution time.
> I will do a few "submit patch" to see if it works well before asking for the 
> real commit.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-4847) Activate single jvm for small tests on jenkins

[nkeywal (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-4847?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

nkeywal updated HBASE-4847:
---

Status: Patch Available  (was: Open)

v9 removed the redirectToFile as I want to see the logs and it seems they're 
not available on prebuilds...

> Activate single jvm for small tests on jenkins
> --
>
> Key: HBASE-4847
> URL: https://issues.apache.org/jira/browse/HBASE-4847
> Project: HBase
>  Issue Type: Improvement
>  Components: build, test
>Affects Versions: 0.94.0
> Environment: build
>Reporter: nkeywal
>Assignee: nkeywal
>Priority: Minor
> Attachments: 4847_all.v4.patch, 4847_all.v5.patch, 4847_all.v6.patch, 
> 4847_all.v6.patch, 4847_all.v7.patch, 4847_all.v7.patch, 4847_all.v7.patch, 
> 4847_all.v7.patch, 4847_all.v8.patch, 4847_all.v8.patch, 4847_all.v9.patch, 
> 4847_pom.patch, 4847_pom.v2.patch, 4847_pom.v2.patch, 4847_pom.v2.patch, 
> 4847_pom.v3.patch
>
>
> This will not revolutionate performances alone. We will win between 1 to 4 
> minutes.
> But we win as well:
>  - it's a step for parallelizing the tests
>  - new tests are less expensive as they do not create a new jvm: it's a 
> continuous win
>  - it will allow to push it on dev env while having the same env on dev & on 
> build, and 3 minutes are 10% of small + medium tests execution time.
> I will do a few "submit patch" to see if it works well before asking for the 
> real commit.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-4847) Activate single jvm for small tests on jenkins

[Hadoop QA (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-4847?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13162091#comment-13162091
 ] 

Hadoop QA commented on HBASE-4847:
--

-1 overall.  Here are the results of testing the latest attachment 
  http://issues.apache.org/jira/secure/attachment/12505977/4847_all.v9.patch
  against trunk revision .

+1 @author.  The patch does not contain any @author tags.

+1 tests included.  The patch appears to include 12 new or modified tests.

-1 javadoc.  The javadoc tool appears to have generated -160 warning 
messages.

+1 javac.  The applied patch does not increase the total number of javac 
compiler warnings.

-1 findbugs.  The patch appears to introduce 71 new Findbugs (version 
1.3.9) warnings.

+1 release audit.  The applied patch does not increase the total number of 
release audit warnings.

 -1 core tests.  The patch failed these unit tests:
   org.apache.hadoop.hbase.regionserver.TestScanner
  org.apache.hadoop.hbase.executor.TestExecutorService
  org.apache.hadoop.hbase.regionserver.TestSplitTransaction
  org.apache.hadoop.hbase.util.TestThreads
  org.apache.hadoop.hbase.coprocessor.TestCoprocessorInterface
  
org.apache.hadoop.hbase.regionserver.TestReadWriteConsistencyControl
  org.apache.hadoop.hbase.filter.TestDependentColumnFilter
  org.apache.hadoop.hbase.coprocessor.TestRegionObserverStacking
  org.apache.hadoop.hbase.filter.TestColumnPrefixFilter
  org.apache.hadoop.hbase.filter.TestMultipleColumnPrefixFilter
  org.apache.hadoop.hbase.regionserver.TestMinVersions
  org.apache.hadoop.hbase.regionserver.TestGetClosestAtOrBefore
  org.apache.hadoop.hbase.regionserver.TestWideScanner
  org.apache.hadoop.hbase.filter.TestFilter

Test results: 
https://builds.apache.org/job/PreCommit-HBASE-Build/430//testReport/
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/430//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/430//console

This message is automatically generated.

> Activate single jvm for small tests on jenkins
> --
>
> Key: HBASE-4847
> URL: https://issues.apache.org/jira/browse/HBASE-4847
> Project: HBase
>  Issue Type: Improvement
>  Components: build, test
>Affects Versions: 0.94.0
> Environment: build
>Reporter: nkeywal
>Assignee: nkeywal
>Priority: Minor
> Attachments: 4847_all.v4.patch, 4847_all.v5.patch, 4847_all.v6.patch, 
> 4847_all.v6.patch, 4847_all.v7.patch, 4847_all.v7.patch, 4847_all.v7.patch, 
> 4847_all.v7.patch, 4847_all.v8.patch, 4847_all.v8.patch, 4847_all.v9.patch, 
> 4847_pom.patch, 4847_pom.v2.patch, 4847_pom.v2.patch, 4847_pom.v2.patch, 
> 4847_pom.v3.patch
>
>
> This will not revolutionate performances alone. We will win between 1 to 4 
> minutes.
> But we win as well:
>  - it's a step for parallelizing the tests
>  - new tests are less expensive as they do not create a new jvm: it's a 
> continuous win
>  - it will allow to push it on dev env while having the same env on dev & on 
> build, and 3 minutes are 10% of small + medium tests execution time.
> I will do a few "submit patch" to see if it works well before asking for the 
> real commit.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-4847) Activate single jvm for small tests on jenkins

[nkeywal (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-4847?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

nkeywal updated HBASE-4847:
---

Status: Open  (was: Patch Available)

> Activate single jvm for small tests on jenkins
> --
>
> Key: HBASE-4847
> URL: https://issues.apache.org/jira/browse/HBASE-4847
> Project: HBase
>  Issue Type: Improvement
>  Components: build, test
>Affects Versions: 0.94.0
> Environment: build
>Reporter: nkeywal
>Assignee: nkeywal
>Priority: Minor
> Attachments: 4847_all.v4.patch, 4847_all.v5.patch, 4847_all.v6.patch, 
> 4847_all.v6.patch, 4847_all.v7.patch, 4847_all.v7.patch, 4847_all.v7.patch, 
> 4847_all.v7.patch, 4847_all.v8.patch, 4847_all.v8.patch, 4847_all.v9.patch, 
> 4847_pom.patch, 4847_pom.v2.patch, 4847_pom.v2.patch, 4847_pom.v2.patch, 
> 4847_pom.v3.patch
>
>
> This will not revolutionate performances alone. We will win between 1 to 4 
> minutes.
> But we win as well:
>  - it's a step for parallelizing the tests
>  - new tests are less expensive as they do not create a new jvm: it's a 
> continuous win
>  - it will allow to push it on dev env while having the same env on dev & on 
> build, and 3 minutes are 10% of small + medium tests execution time.
> I will do a few "submit patch" to see if it works well before asking for the 
> real commit.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-4847) Activate single jvm for small tests on jenkins

[nkeywal (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-4847?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

nkeywal updated HBASE-4847:
---

Attachment: 4847_all.v10.patch

> Activate single jvm for small tests on jenkins
> --
>
> Key: HBASE-4847
> URL: https://issues.apache.org/jira/browse/HBASE-4847
> Project: HBase
>  Issue Type: Improvement
>  Components: build, test
>Affects Versions: 0.94.0
> Environment: build
>Reporter: nkeywal
>Assignee: nkeywal
>Priority: Minor
> Attachments: 4847_all.v10.patch, 4847_all.v4.patch, 
> 4847_all.v5.patch, 4847_all.v6.patch, 4847_all.v6.patch, 4847_all.v7.patch, 
> 4847_all.v7.patch, 4847_all.v7.patch, 4847_all.v7.patch, 4847_all.v8.patch, 
> 4847_all.v8.patch, 4847_all.v9.patch, 4847_pom.patch, 4847_pom.v2.patch, 
> 4847_pom.v2.patch, 4847_pom.v2.patch, 4847_pom.v3.patch
>
>
> This will not revolutionate performances alone. We will win between 1 to 4 
> minutes.
> But we win as well:
>  - it's a step for parallelizing the tests
>  - new tests are less expensive as they do not create a new jvm: it's a 
> continuous win
>  - it will allow to push it on dev env while having the same env on dev & on 
> build, and 3 minutes are 10% of small + medium tests execution time.
> I will do a few "submit patch" to see if it works well before asking for the 
> real commit.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-4847) Activate single jvm for small tests on jenkins

[nkeywal (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-4847?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

nkeywal updated HBASE-4847:
---

Status: Patch Available  (was: Open)

> Activate single jvm for small tests on jenkins
> --
>
> Key: HBASE-4847
> URL: https://issues.apache.org/jira/browse/HBASE-4847
> Project: HBase
>  Issue Type: Improvement
>  Components: build, test
>Affects Versions: 0.94.0
> Environment: build
>Reporter: nkeywal
>Assignee: nkeywal
>Priority: Minor
> Attachments: 4847_all.v10.patch, 4847_all.v4.patch, 
> 4847_all.v5.patch, 4847_all.v6.patch, 4847_all.v6.patch, 4847_all.v7.patch, 
> 4847_all.v7.patch, 4847_all.v7.patch, 4847_all.v7.patch, 4847_all.v8.patch, 
> 4847_all.v8.patch, 4847_all.v9.patch, 4847_pom.patch, 4847_pom.v2.patch, 
> 4847_pom.v2.patch, 4847_pom.v2.patch, 4847_pom.v3.patch
>
>
> This will not revolutionate performances alone. We will win between 1 to 4 
> minutes.
> But we win as well:
>  - it's a step for parallelizing the tests
>  - new tests are less expensive as they do not create a new jvm: it's a 
> continuous win
>  - it will allow to push it on dev env while having the same env on dev & on 
> build, and 3 minutes are 10% of small + medium tests execution time.
> I will do a few "submit patch" to see if it works well before asking for the 
> real commit.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-4847) Activate single jvm for small tests on jenkins

[nkeywal (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-4847?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13162099#comment-13162099
 ] 

nkeywal commented on HBASE-4847:


v10 contains a real fix on TestStoreScanner/EnvironmentEdgeManager but the logs 
are still redirected to the console to see the other stuff.


> Activate single jvm for small tests on jenkins
> --
>
> Key: HBASE-4847
> URL: https://issues.apache.org/jira/browse/HBASE-4847
> Project: HBase
>  Issue Type: Improvement
>  Components: build, test
>Affects Versions: 0.94.0
> Environment: build
>Reporter: nkeywal
>Assignee: nkeywal
>Priority: Minor
> Attachments: 4847_all.v10.patch, 4847_all.v4.patch, 
> 4847_all.v5.patch, 4847_all.v6.patch, 4847_all.v6.patch, 4847_all.v7.patch, 
> 4847_all.v7.patch, 4847_all.v7.patch, 4847_all.v7.patch, 4847_all.v8.patch, 
> 4847_all.v8.patch, 4847_all.v9.patch, 4847_pom.patch, 4847_pom.v2.patch, 
> 4847_pom.v2.patch, 4847_pom.v2.patch, 4847_pom.v3.patch
>
>
> This will not revolutionate performances alone. We will win between 1 to 4 
> minutes.
> But we win as well:
>  - it's a step for parallelizing the tests
>  - new tests are less expensive as they do not create a new jvm: it's a 
> continuous win
>  - it will allow to push it on dev env while having the same env on dev & on 
> build, and 3 minutes are 10% of small + medium tests execution time.
> I will do a few "submit patch" to see if it works well before asking for the 
> real commit.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-4847) Activate single jvm for small tests on jenkins

[nkeywal (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-4847?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

nkeywal updated HBASE-4847:
---

Status: Open  (was: Patch Available)

> Activate single jvm for small tests on jenkins
> --
>
> Key: HBASE-4847
> URL: https://issues.apache.org/jira/browse/HBASE-4847
> Project: HBase
>  Issue Type: Improvement
>  Components: build, test
>Affects Versions: 0.94.0
> Environment: build
>Reporter: nkeywal
>Assignee: nkeywal
>Priority: Minor
> Attachments: 4847_all.v10.patch, 4847_all.v10.patch, 
> 4847_all.v4.patch, 4847_all.v5.patch, 4847_all.v6.patch, 4847_all.v6.patch, 
> 4847_all.v7.patch, 4847_all.v7.patch, 4847_all.v7.patch, 4847_all.v7.patch, 
> 4847_all.v8.patch, 4847_all.v8.patch, 4847_all.v9.patch, 4847_pom.patch, 
> 4847_pom.v2.patch, 4847_pom.v2.patch, 4847_pom.v2.patch, 4847_pom.v3.patch
>
>
> This will not revolutionate performances alone. We will win between 1 to 4 
> minutes.
> But we win as well:
>  - it's a step for parallelizing the tests
>  - new tests are less expensive as they do not create a new jvm: it's a 
> continuous win
>  - it will allow to push it on dev env while having the same env on dev & on 
> build, and 3 minutes are 10% of small + medium tests execution time.
> I will do a few "submit patch" to see if it works well before asking for the 
> real commit.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-4847) Activate single jvm for small tests on jenkins

[nkeywal (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-4847?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

nkeywal updated HBASE-4847:
---

Status: Patch Available  (was: Open)

> Activate single jvm for small tests on jenkins
> --
>
> Key: HBASE-4847
> URL: https://issues.apache.org/jira/browse/HBASE-4847
> Project: HBase
>  Issue Type: Improvement
>  Components: build, test
>Affects Versions: 0.94.0
> Environment: build
>Reporter: nkeywal
>Assignee: nkeywal
>Priority: Minor
> Attachments: 4847_all.v10.patch, 4847_all.v10.patch, 
> 4847_all.v4.patch, 4847_all.v5.patch, 4847_all.v6.patch, 4847_all.v6.patch, 
> 4847_all.v7.patch, 4847_all.v7.patch, 4847_all.v7.patch, 4847_all.v7.patch, 
> 4847_all.v8.patch, 4847_all.v8.patch, 4847_all.v9.patch, 4847_pom.patch, 
> 4847_pom.v2.patch, 4847_pom.v2.patch, 4847_pom.v2.patch, 4847_pom.v3.patch
>
>
> This will not revolutionate performances alone. We will win between 1 to 4 
> minutes.
> But we win as well:
>  - it's a step for parallelizing the tests
>  - new tests are less expensive as they do not create a new jvm: it's a 
> continuous win
>  - it will allow to push it on dev env while having the same env on dev & on 
> build, and 3 minutes are 10% of small + medium tests execution time.
> I will do a few "submit patch" to see if it works well before asking for the 
> real commit.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-4847) Activate single jvm for small tests on jenkins

[nkeywal (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-4847?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

nkeywal updated HBASE-4847:
---

Attachment: 4847_all.v10.patch

> Activate single jvm for small tests on jenkins
> --
>
> Key: HBASE-4847
> URL: https://issues.apache.org/jira/browse/HBASE-4847
> Project: HBase
>  Issue Type: Improvement
>  Components: build, test
>Affects Versions: 0.94.0
> Environment: build
>Reporter: nkeywal
>Assignee: nkeywal
>Priority: Minor
> Attachments: 4847_all.v10.patch, 4847_all.v10.patch, 
> 4847_all.v4.patch, 4847_all.v5.patch, 4847_all.v6.patch, 4847_all.v6.patch, 
> 4847_all.v7.patch, 4847_all.v7.patch, 4847_all.v7.patch, 4847_all.v7.patch, 
> 4847_all.v8.patch, 4847_all.v8.patch, 4847_all.v9.patch, 4847_pom.patch, 
> 4847_pom.v2.patch, 4847_pom.v2.patch, 4847_pom.v2.patch, 4847_pom.v3.patch
>
>
> This will not revolutionate performances alone. We will win between 1 to 4 
> minutes.
> But we win as well:
>  - it's a step for parallelizing the tests
>  - new tests are less expensive as they do not create a new jvm: it's a 
> continuous win
>  - it will allow to push it on dev env while having the same env on dev & on 
> build, and 3 minutes are 10% of small + medium tests execution time.
> I will do a few "submit patch" to see if it works well before asking for the 
> real commit.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-4847) Activate single jvm for small tests on jenkins

[Hadoop QA (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-4847?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13162100#comment-13162100
 ] 

Hadoop QA commented on HBASE-4847:
--

-1 overall.  Here are the results of testing the latest attachment 
  http://issues.apache.org/jira/secure/attachment/12505981/4847_all.v10.patch
  against trunk revision .

+1 @author.  The patch does not contain any @author tags.

+1 tests included.  The patch appears to include 15 new or modified tests.

-1 javadoc.  The javadoc tool appears to have generated -160 warning 
messages.

+1 javac.  The applied patch does not increase the total number of javac 
compiler warnings.

-1 findbugs.  The patch appears to introduce 71 new Findbugs (version 
1.3.9) warnings.

+1 release audit.  The applied patch does not increase the total number of 
release audit warnings.

 -1 core tests.  The patch failed these unit tests:
   org.apache.hadoop.hbase.regionserver.TestScanner
  org.apache.hadoop.hbase.executor.TestExecutorService
  org.apache.hadoop.hbase.regionserver.TestSplitTransaction
  org.apache.hadoop.hbase.util.TestThreads
  org.apache.hadoop.hbase.coprocessor.TestCoprocessorInterface
  
org.apache.hadoop.hbase.regionserver.TestReadWriteConsistencyControl
  org.apache.hadoop.hbase.filter.TestDependentColumnFilter
  org.apache.hadoop.hbase.coprocessor.TestRegionObserverStacking
  org.apache.hadoop.hbase.filter.TestColumnPrefixFilter
  org.apache.hadoop.hbase.filter.TestMultipleColumnPrefixFilter
  org.apache.hadoop.hbase.regionserver.TestMinVersions
  org.apache.hadoop.hbase.regionserver.TestGetClosestAtOrBefore
  org.apache.hadoop.hbase.regionserver.TestWideScanner
  org.apache.hadoop.hbase.filter.TestFilter

Test results: 
https://builds.apache.org/job/PreCommit-HBASE-Build/431//testReport/
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/431//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/431//console

This message is automatically generated.

> Activate single jvm for small tests on jenkins
> --
>
> Key: HBASE-4847
> URL: https://issues.apache.org/jira/browse/HBASE-4847
> Project: HBase
>  Issue Type: Improvement
>  Components: build, test
>Affects Versions: 0.94.0
> Environment: build
>Reporter: nkeywal
>Assignee: nkeywal
>Priority: Minor
> Attachments: 4847_all.v10.patch, 4847_all.v10.patch, 
> 4847_all.v4.patch, 4847_all.v5.patch, 4847_all.v6.patch, 4847_all.v6.patch, 
> 4847_all.v7.patch, 4847_all.v7.patch, 4847_all.v7.patch, 4847_all.v7.patch, 
> 4847_all.v8.patch, 4847_all.v8.patch, 4847_all.v9.patch, 4847_pom.patch, 
> 4847_pom.v2.patch, 4847_pom.v2.patch, 4847_pom.v2.patch, 4847_pom.v3.patch
>
>
> This will not revolutionate performances alone. We will win between 1 to 4 
> minutes.
> But we win as well:
>  - it's a step for parallelizing the tests
>  - new tests are less expensive as they do not create a new jvm: it's a 
> continuous win
>  - it will allow to push it on dev env while having the same env on dev & on 
> build, and 3 minutes are 10% of small + medium tests execution time.
> I will do a few "submit patch" to see if it works well before asking for the 
> real commit.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-4847) Activate single jvm for small tests on jenkins

[nkeywal (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-4847?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

nkeywal updated HBASE-4847:
---

Status: Open  (was: Patch Available)

> Activate single jvm for small tests on jenkins
> --
>
> Key: HBASE-4847
> URL: https://issues.apache.org/jira/browse/HBASE-4847
> Project: HBase
>  Issue Type: Improvement
>  Components: build, test
>Affects Versions: 0.94.0
> Environment: build
>Reporter: nkeywal
>Assignee: nkeywal
>Priority: Minor
> Attachments: 4847_all.v10.patch, 4847_all.v10.patch, 
> 4847_all.v10.patch, 4847_all.v4.patch, 4847_all.v5.patch, 4847_all.v6.patch, 
> 4847_all.v6.patch, 4847_all.v7.patch, 4847_all.v7.patch, 4847_all.v7.patch, 
> 4847_all.v7.patch, 4847_all.v8.patch, 4847_all.v8.patch, 4847_all.v9.patch, 
> 4847_pom.patch, 4847_pom.v2.patch, 4847_pom.v2.patch, 4847_pom.v2.patch, 
> 4847_pom.v3.patch
>
>
> This will not revolutionate performances alone. We will win between 1 to 4 
> minutes.
> But we win as well:
>  - it's a step for parallelizing the tests
>  - new tests are less expensive as they do not create a new jvm: it's a 
> continuous win
>  - it will allow to push it on dev env while having the same env on dev & on 
> build, and 3 minutes are 10% of small + medium tests execution time.
> I will do a few "submit patch" to see if it works well before asking for the 
> real commit.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-4847) Activate single jvm for small tests on jenkins

[nkeywal (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-4847?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

nkeywal updated HBASE-4847:
---

Attachment: 4847_all.v10.patch

> Activate single jvm for small tests on jenkins
> --
>
> Key: HBASE-4847
> URL: https://issues.apache.org/jira/browse/HBASE-4847
> Project: HBase
>  Issue Type: Improvement
>  Components: build, test
>Affects Versions: 0.94.0
> Environment: build
>Reporter: nkeywal
>Assignee: nkeywal
>Priority: Minor
> Attachments: 4847_all.v10.patch, 4847_all.v10.patch, 
> 4847_all.v10.patch, 4847_all.v4.patch, 4847_all.v5.patch, 4847_all.v6.patch, 
> 4847_all.v6.patch, 4847_all.v7.patch, 4847_all.v7.patch, 4847_all.v7.patch, 
> 4847_all.v7.patch, 4847_all.v8.patch, 4847_all.v8.patch, 4847_all.v9.patch, 
> 4847_pom.patch, 4847_pom.v2.patch, 4847_pom.v2.patch, 4847_pom.v2.patch, 
> 4847_pom.v3.patch
>
>
> This will not revolutionate performances alone. We will win between 1 to 4 
> minutes.
> But we win as well:
>  - it's a step for parallelizing the tests
>  - new tests are less expensive as they do not create a new jvm: it's a 
> continuous win
>  - it will allow to push it on dev env while having the same env on dev & on 
> build, and 3 minutes are 10% of small + medium tests execution time.
> I will do a few "submit patch" to see if it works well before asking for the 
> real commit.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-4847) Activate single jvm for small tests on jenkins

[nkeywal (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-4847?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

nkeywal updated HBASE-4847:
---

Status: Patch Available  (was: Open)

> Activate single jvm for small tests on jenkins
> --
>
> Key: HBASE-4847
> URL: https://issues.apache.org/jira/browse/HBASE-4847
> Project: HBase
>  Issue Type: Improvement
>  Components: build, test
>Affects Versions: 0.94.0
> Environment: build
>Reporter: nkeywal
>Assignee: nkeywal
>Priority: Minor
> Attachments: 4847_all.v10.patch, 4847_all.v10.patch, 
> 4847_all.v10.patch, 4847_all.v4.patch, 4847_all.v5.patch, 4847_all.v6.patch, 
> 4847_all.v6.patch, 4847_all.v7.patch, 4847_all.v7.patch, 4847_all.v7.patch, 
> 4847_all.v7.patch, 4847_all.v8.patch, 4847_all.v8.patch, 4847_all.v9.patch, 
> 4847_pom.patch, 4847_pom.v2.patch, 4847_pom.v2.patch, 4847_pom.v2.patch, 
> 4847_pom.v3.patch
>
>
> This will not revolutionate performances alone. We will win between 1 to 4 
> minutes.
> But we win as well:
>  - it's a step for parallelizing the tests
>  - new tests are less expensive as they do not create a new jvm: it's a 
> continuous win
>  - it will allow to push it on dev env while having the same env on dev & on 
> build, and 3 minutes are 10% of small + medium tests execution time.
> I will do a few "submit patch" to see if it works well before asking for the 
> real commit.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-4847) Activate single jvm for small tests on jenkins

[Hadoop QA (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-4847?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13162104#comment-13162104
 ] 

Hadoop QA commented on HBASE-4847:
--

-1 overall.  Here are the results of testing the latest attachment 
  http://issues.apache.org/jira/secure/attachment/12505982/4847_all.v10.patch
  against trunk revision .

+1 @author.  The patch does not contain any @author tags.

+1 tests included.  The patch appears to include 15 new or modified tests.

-1 javadoc.  The javadoc tool appears to have generated -160 warning 
messages.

+1 javac.  The applied patch does not increase the total number of javac 
compiler warnings.

-1 findbugs.  The patch appears to introduce 71 new Findbugs (version 
1.3.9) warnings.

+1 release audit.  The applied patch does not increase the total number of 
release audit warnings.

 -1 core tests.  The patch failed these unit tests:
   org.apache.hadoop.hbase.regionserver.TestScanner
  org.apache.hadoop.hbase.executor.TestExecutorService
  org.apache.hadoop.hbase.regionserver.TestSplitTransaction
  org.apache.hadoop.hbase.util.TestThreads
  org.apache.hadoop.hbase.coprocessor.TestCoprocessorInterface
  
org.apache.hadoop.hbase.regionserver.TestReadWriteConsistencyControl
  org.apache.hadoop.hbase.filter.TestDependentColumnFilter
  org.apache.hadoop.hbase.coprocessor.TestRegionObserverStacking
  org.apache.hadoop.hbase.filter.TestColumnPrefixFilter
  org.apache.hadoop.hbase.filter.TestMultipleColumnPrefixFilter
  org.apache.hadoop.hbase.regionserver.TestMinVersions
  org.apache.hadoop.hbase.regionserver.TestGetClosestAtOrBefore
  org.apache.hadoop.hbase.regionserver.TestWideScanner
  org.apache.hadoop.hbase.filter.TestFilter

Test results: 
https://builds.apache.org/job/PreCommit-HBASE-Build/432//testReport/
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/432//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/432//console

This message is automatically generated.

> Activate single jvm for small tests on jenkins
> --
>
> Key: HBASE-4847
> URL: https://issues.apache.org/jira/browse/HBASE-4847
> Project: HBase
>  Issue Type: Improvement
>  Components: build, test
>Affects Versions: 0.94.0
> Environment: build
>Reporter: nkeywal
>Assignee: nkeywal
>Priority: Minor
> Attachments: 4847_all.v10.patch, 4847_all.v10.patch, 
> 4847_all.v10.patch, 4847_all.v4.patch, 4847_all.v5.patch, 4847_all.v6.patch, 
> 4847_all.v6.patch, 4847_all.v7.patch, 4847_all.v7.patch, 4847_all.v7.patch, 
> 4847_all.v7.patch, 4847_all.v8.patch, 4847_all.v8.patch, 4847_all.v9.patch, 
> 4847_pom.patch, 4847_pom.v2.patch, 4847_pom.v2.patch, 4847_pom.v2.patch, 
> 4847_pom.v3.patch
>
>
> This will not revolutionate performances alone. We will win between 1 to 4 
> minutes.
> But we win as well:
>  - it's a step for parallelizing the tests
>  - new tests are less expensive as they do not create a new jvm: it's a 
> continuous win
>  - it will allow to push it on dev env while having the same env on dev & on 
> build, and 3 minutes are 10% of small + medium tests execution time.
> I will do a few "submit patch" to see if it works well before asking for the 
> real commit.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-4847) Activate single jvm for small tests on jenkins

[nkeywal (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-4847?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

nkeywal updated HBASE-4847:
---

Attachment: 4847_all.v11.patch

> Activate single jvm for small tests on jenkins
> --
>
> Key: HBASE-4847
> URL: https://issues.apache.org/jira/browse/HBASE-4847
> Project: HBase
>  Issue Type: Improvement
>  Components: build, test
>Affects Versions: 0.94.0
> Environment: build
>Reporter: nkeywal
>Assignee: nkeywal
>Priority: Minor
> Attachments: 4847_all.v10.patch, 4847_all.v10.patch, 
> 4847_all.v10.patch, 4847_all.v11.patch, 4847_all.v4.patch, 4847_all.v5.patch, 
> 4847_all.v6.patch, 4847_all.v6.patch, 4847_all.v7.patch, 4847_all.v7.patch, 
> 4847_all.v7.patch, 4847_all.v7.patch, 4847_all.v8.patch, 4847_all.v8.patch, 
> 4847_all.v9.patch, 4847_pom.patch, 4847_pom.v2.patch, 4847_pom.v2.patch, 
> 4847_pom.v2.patch, 4847_pom.v3.patch
>
>
> This will not revolutionate performances alone. We will win between 1 to 4 
> minutes.
> But we win as well:
>  - it's a step for parallelizing the tests
>  - new tests are less expensive as they do not create a new jvm: it's a 
> continuous win
>  - it will allow to push it on dev env while having the same env on dev & on 
> build, and 3 minutes are 10% of small + medium tests execution time.
> I will do a few "submit patch" to see if it works well before asking for the 
> real commit.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-4847) Activate single jvm for small tests on jenkins

[nkeywal (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-4847?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

nkeywal updated HBASE-4847:
---

Status: Open  (was: Patch Available)

> Activate single jvm for small tests on jenkins
> --
>
> Key: HBASE-4847
> URL: https://issues.apache.org/jira/browse/HBASE-4847
> Project: HBase
>  Issue Type: Improvement
>  Components: build, test
>Affects Versions: 0.94.0
> Environment: build
>Reporter: nkeywal
>Assignee: nkeywal
>Priority: Minor
> Attachments: 4847_all.v10.patch, 4847_all.v10.patch, 
> 4847_all.v10.patch, 4847_all.v11.patch, 4847_all.v4.patch, 4847_all.v5.patch, 
> 4847_all.v6.patch, 4847_all.v6.patch, 4847_all.v7.patch, 4847_all.v7.patch, 
> 4847_all.v7.patch, 4847_all.v7.patch, 4847_all.v8.patch, 4847_all.v8.patch, 
> 4847_all.v9.patch, 4847_pom.patch, 4847_pom.v2.patch, 4847_pom.v2.patch, 
> 4847_pom.v2.patch, 4847_pom.v3.patch
>
>
> This will not revolutionate performances alone. We will win between 1 to 4 
> minutes.
> But we win as well:
>  - it's a step for parallelizing the tests
>  - new tests are less expensive as they do not create a new jvm: it's a 
> continuous win
>  - it will allow to push it on dev env while having the same env on dev & on 
> build, and 3 minutes are 10% of small + medium tests execution time.
> I will do a few "submit patch" to see if it works well before asking for the 
> real commit.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-4847) Activate single jvm for small tests on jenkins

[nkeywal (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-4847?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

nkeywal updated HBASE-4847:
---

Status: Patch Available  (was: Open)

> Activate single jvm for small tests on jenkins
> --
>
> Key: HBASE-4847
> URL: https://issues.apache.org/jira/browse/HBASE-4847
> Project: HBase
>  Issue Type: Improvement
>  Components: build, test
>Affects Versions: 0.94.0
> Environment: build
>Reporter: nkeywal
>Assignee: nkeywal
>Priority: Minor
> Attachments: 4847_all.v10.patch, 4847_all.v10.patch, 
> 4847_all.v10.patch, 4847_all.v11.patch, 4847_all.v4.patch, 4847_all.v5.patch, 
> 4847_all.v6.patch, 4847_all.v6.patch, 4847_all.v7.patch, 4847_all.v7.patch, 
> 4847_all.v7.patch, 4847_all.v7.patch, 4847_all.v8.patch, 4847_all.v8.patch, 
> 4847_all.v9.patch, 4847_pom.patch, 4847_pom.v2.patch, 4847_pom.v2.patch, 
> 4847_pom.v2.patch, 4847_pom.v3.patch
>
>
> This will not revolutionate performances alone. We will win between 1 to 4 
> minutes.
> But we win as well:
>  - it's a step for parallelizing the tests
>  - new tests are less expensive as they do not create a new jvm: it's a 
> continuous win
>  - it will allow to push it on dev env while having the same env on dev & on 
> build, and 3 minutes are 10% of small + medium tests execution time.
> I will do a few "submit patch" to see if it works well before asking for the 
> real commit.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-4847) Activate single jvm for small tests on jenkins

[Hadoop QA (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-4847?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13162137#comment-13162137
 ] 

Hadoop QA commented on HBASE-4847:
--

-1 overall.  Here are the results of testing the latest attachment 
  http://issues.apache.org/jira/secure/attachment/12505983/4847_all.v11.patch
  against trunk revision .

+1 @author.  The patch does not contain any @author tags.

+1 tests included.  The patch appears to include 21 new or modified tests.

-1 javadoc.  The javadoc tool appears to have generated -160 warning 
messages.

+1 javac.  The applied patch does not increase the total number of javac 
compiler warnings.

-1 findbugs.  The patch appears to introduce 71 new Findbugs (version 
1.3.9) warnings.

+1 release audit.  The applied patch does not increase the total number of 
release audit warnings.

 -1 core tests.  The patch failed these unit tests:
   org.apache.hadoop.hbase.regionserver.TestScanner
  org.apache.hadoop.hbase.executor.TestExecutorService
  org.apache.hadoop.hbase.regionserver.TestSplitTransaction
  org.apache.hadoop.hbase.util.TestThreads
  org.apache.hadoop.hbase.coprocessor.TestCoprocessorInterface
  
org.apache.hadoop.hbase.regionserver.TestReadWriteConsistencyControl
  org.apache.hadoop.hbase.filter.TestDependentColumnFilter
  org.apache.hadoop.hbase.coprocessor.TestRegionObserverStacking
  org.apache.hadoop.hbase.filter.TestColumnPrefixFilter
  org.apache.hadoop.hbase.filter.TestMultipleColumnPrefixFilter
  org.apache.hadoop.hbase.regionserver.TestMinVersions
  org.apache.hadoop.hbase.regionserver.TestGetClosestAtOrBefore
  org.apache.hadoop.hbase.regionserver.TestWideScanner
  org.apache.hadoop.hbase.filter.TestFilter

Test results: 
https://builds.apache.org/job/PreCommit-HBASE-Build/433//testReport/
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/433//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/433//console

This message is automatically generated.

> Activate single jvm for small tests on jenkins
> --
>
> Key: HBASE-4847
> URL: https://issues.apache.org/jira/browse/HBASE-4847
> Project: HBase
>  Issue Type: Improvement
>  Components: build, test
>Affects Versions: 0.94.0
> Environment: build
>Reporter: nkeywal
>Assignee: nkeywal
>Priority: Minor
> Attachments: 4847_all.v10.patch, 4847_all.v10.patch, 
> 4847_all.v10.patch, 4847_all.v11.patch, 4847_all.v4.patch, 4847_all.v5.patch, 
> 4847_all.v6.patch, 4847_all.v6.patch, 4847_all.v7.patch, 4847_all.v7.patch, 
> 4847_all.v7.patch, 4847_all.v7.patch, 4847_all.v8.patch, 4847_all.v8.patch, 
> 4847_all.v9.patch, 4847_pom.patch, 4847_pom.v2.patch, 4847_pom.v2.patch, 
> 4847_pom.v2.patch, 4847_pom.v3.patch
>
>
> This will not revolutionate performances alone. We will win between 1 to 4 
> minutes.
> But we win as well:
>  - it's a step for parallelizing the tests
>  - new tests are less expensive as they do not create a new jvm: it's a 
> continuous win
>  - it will allow to push it on dev env while having the same env on dev & on 
> build, and 3 minutes are 10% of small + medium tests execution time.
> I will do a few "submit patch" to see if it works well before asking for the 
> real commit.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (HBASE-4941) [book] fixed mis-spelling of 'Configuration' in 2 places

[Doug Meil (Created) (JIRA)]

[book] fixed mis-spelling of 'Configuration' in 2 places


 Key: HBASE-4941
 URL: https://issues.apache.org/jira/browse/HBASE-4941
 Project: HBase
  Issue Type: Improvement
Reporter: Doug Meil
Assignee: Doug Meil
Priority: Trivial


Changed "Configuation" to "Configuration"

Fixed in configuration.xml
Fixed in hbase-default.xml  (in the 
"hbase.mapreduce.hfileoutputformat.blocksize" description)

Thanks for Otis for catching this.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-4941) [book] fixed mis-spelling of 'Configuration' in 2 places

[Doug Meil (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-4941?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Doug Meil updated HBASE-4941:
-

Attachment: src_HBASE_4941.patch

> [book] fixed mis-spelling of 'Configuration' in 2 places
> 
>
> Key: HBASE-4941
> URL: https://issues.apache.org/jira/browse/HBASE-4941
> Project: HBase
>  Issue Type: Improvement
>Reporter: Doug Meil
>Assignee: Doug Meil
>Priority: Trivial
> Attachments: src_HBASE_4941.patch
>
>
> Changed "Configuation" to "Configuration"
> Fixed in configuration.xml
> Fixed in hbase-default.xml  (in the 
> "hbase.mapreduce.hfileoutputformat.blocksize" description)
> Thanks for Otis for catching this.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-4941) [book] fixed mis-spelling of 'Configuration' in 2 places

[Doug Meil (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-4941?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Doug Meil updated HBASE-4941:
-

Status: Patch Available  (was: Open)

> [book] fixed mis-spelling of 'Configuration' in 2 places
> 
>
> Key: HBASE-4941
> URL: https://issues.apache.org/jira/browse/HBASE-4941
> Project: HBase
>  Issue Type: Improvement
>Reporter: Doug Meil
>Assignee: Doug Meil
>Priority: Trivial
> Attachments: src_HBASE_4941.patch
>
>
> Changed "Configuation" to "Configuration"
> Fixed in configuration.xml
> Fixed in hbase-default.xml  (in the 
> "hbase.mapreduce.hfileoutputformat.blocksize" description)
> Thanks for Otis for catching this.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-4941) [book] fixed mis-spelling of 'Configuration' in 2 places

[Doug Meil (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-4941?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Doug Meil updated HBASE-4941:
-

Resolution: Fixed
Status: Resolved  (was: Patch Available)

> [book] fixed mis-spelling of 'Configuration' in 2 places
> 
>
> Key: HBASE-4941
> URL: https://issues.apache.org/jira/browse/HBASE-4941
> Project: HBase
>  Issue Type: Improvement
>Reporter: Doug Meil
>Assignee: Doug Meil
>Priority: Trivial
> Attachments: src_HBASE_4941.patch
>
>
> Changed "Configuation" to "Configuration"
> Fixed in configuration.xml
> Fixed in hbase-default.xml  (in the 
> "hbase.mapreduce.hfileoutputformat.blocksize" description)
> Thanks for Otis for catching this.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-4941) [book] fixed mis-spelling of 'Configuration' in 2 places

[Hudson (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-4941?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13162168#comment-13162168
 ] 

Hudson commented on HBASE-4941:
---

Integrated in HBase-TRUNK #2510 (See 
[https://builds.apache.org/job/HBase-TRUNK/2510/])
hbase-4941.  Fixed mis-spelling of 'configuration' in 2 places 
(configuration.xml and hbase-default.xml)


> [book] fixed mis-spelling of 'Configuration' in 2 places
> 
>
> Key: HBASE-4941
> URL: https://issues.apache.org/jira/browse/HBASE-4941
> Project: HBase
>  Issue Type: Improvement
>Reporter: Doug Meil
>Assignee: Doug Meil
>Priority: Trivial
> Attachments: src_HBASE_4941.patch
>
>
> Changed "Configuation" to "Configuration"
> Fixed in configuration.xml
> Fixed in hbase-default.xml  (in the 
> "hbase.mapreduce.hfileoutputformat.blocksize" description)
> Thanks for Otis for catching this.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-4847) Activate single jvm for small tests on jenkins

[nkeywal (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-4847?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

nkeywal updated HBASE-4847:
---

Attachment: 4847_all.v11.patch

> Activate single jvm for small tests on jenkins
> --
>
> Key: HBASE-4847
> URL: https://issues.apache.org/jira/browse/HBASE-4847
> Project: HBase
>  Issue Type: Improvement
>  Components: build, test
>Affects Versions: 0.94.0
> Environment: build
>Reporter: nkeywal
>Assignee: nkeywal
>Priority: Minor
> Attachments: 4847_all.v10.patch, 4847_all.v10.patch, 
> 4847_all.v10.patch, 4847_all.v11.patch, 4847_all.v11.patch, 
> 4847_all.v4.patch, 4847_all.v5.patch, 4847_all.v6.patch, 4847_all.v6.patch, 
> 4847_all.v7.patch, 4847_all.v7.patch, 4847_all.v7.patch, 4847_all.v7.patch, 
> 4847_all.v8.patch, 4847_all.v8.patch, 4847_all.v9.patch, 4847_pom.patch, 
> 4847_pom.v2.patch, 4847_pom.v2.patch, 4847_pom.v2.patch, 4847_pom.v3.patch
>
>
> This will not revolutionate performances alone. We will win between 1 to 4 
> minutes.
> But we win as well:
>  - it's a step for parallelizing the tests
>  - new tests are less expensive as they do not create a new jvm: it's a 
> continuous win
>  - it will allow to push it on dev env while having the same env on dev & on 
> build, and 3 minutes are 10% of small + medium tests execution time.
> I will do a few "submit patch" to see if it works well before asking for the 
> real commit.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-4847) Activate single jvm for small tests on jenkins

[nkeywal (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-4847?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

nkeywal updated HBASE-4847:
---

Status: Open  (was: Patch Available)

> Activate single jvm for small tests on jenkins
> --
>
> Key: HBASE-4847
> URL: https://issues.apache.org/jira/browse/HBASE-4847
> Project: HBase
>  Issue Type: Improvement
>  Components: build, test
>Affects Versions: 0.94.0
> Environment: build
>Reporter: nkeywal
>Assignee: nkeywal
>Priority: Minor
> Attachments: 4847_all.v10.patch, 4847_all.v10.patch, 
> 4847_all.v10.patch, 4847_all.v11.patch, 4847_all.v11.patch, 
> 4847_all.v4.patch, 4847_all.v5.patch, 4847_all.v6.patch, 4847_all.v6.patch, 
> 4847_all.v7.patch, 4847_all.v7.patch, 4847_all.v7.patch, 4847_all.v7.patch, 
> 4847_all.v8.patch, 4847_all.v8.patch, 4847_all.v9.patch, 4847_pom.patch, 
> 4847_pom.v2.patch, 4847_pom.v2.patch, 4847_pom.v2.patch, 4847_pom.v3.patch
>
>
> This will not revolutionate performances alone. We will win between 1 to 4 
> minutes.
> But we win as well:
>  - it's a step for parallelizing the tests
>  - new tests are less expensive as they do not create a new jvm: it's a 
> continuous win
>  - it will allow to push it on dev env while having the same env on dev & on 
> build, and 3 minutes are 10% of small + medium tests execution time.
> I will do a few "submit patch" to see if it works well before asking for the 
> real commit.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-4847) Activate single jvm for small tests on jenkins

[nkeywal (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-4847?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

nkeywal updated HBASE-4847:
---

Status: Patch Available  (was: Open)

> Activate single jvm for small tests on jenkins
> --
>
> Key: HBASE-4847
> URL: https://issues.apache.org/jira/browse/HBASE-4847
> Project: HBase
>  Issue Type: Improvement
>  Components: build, test
>Affects Versions: 0.94.0
> Environment: build
>Reporter: nkeywal
>Assignee: nkeywal
>Priority: Minor
> Attachments: 4847_all.v10.patch, 4847_all.v10.patch, 
> 4847_all.v10.patch, 4847_all.v11.patch, 4847_all.v11.patch, 
> 4847_all.v4.patch, 4847_all.v5.patch, 4847_all.v6.patch, 4847_all.v6.patch, 
> 4847_all.v7.patch, 4847_all.v7.patch, 4847_all.v7.patch, 4847_all.v7.patch, 
> 4847_all.v8.patch, 4847_all.v8.patch, 4847_all.v9.patch, 4847_pom.patch, 
> 4847_pom.v2.patch, 4847_pom.v2.patch, 4847_pom.v2.patch, 4847_pom.v3.patch
>
>
> This will not revolutionate performances alone. We will win between 1 to 4 
> minutes.
> But we win as well:
>  - it's a step for parallelizing the tests
>  - new tests are less expensive as they do not create a new jvm: it's a 
> continuous win
>  - it will allow to push it on dev env while having the same env on dev & on 
> build, and 3 minutes are 10% of small + medium tests execution time.
> I will do a few "submit patch" to see if it works well before asking for the 
> real commit.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-4847) Activate single jvm for small tests on jenkins

[Hadoop QA (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-4847?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13162182#comment-13162182
 ] 

Hadoop QA commented on HBASE-4847:
--

-1 overall.  Here are the results of testing the latest attachment 
  http://issues.apache.org/jira/secure/attachment/12505999/4847_all.v11.patch
  against trunk revision .

+1 @author.  The patch does not contain any @author tags.

+1 tests included.  The patch appears to include 21 new or modified tests.

-1 javadoc.  The javadoc tool appears to have generated -160 warning 
messages.

+1 javac.  The applied patch does not increase the total number of javac 
compiler warnings.

-1 findbugs.  The patch appears to introduce 71 new Findbugs (version 
1.3.9) warnings.

+1 release audit.  The applied patch does not increase the total number of 
release audit warnings.

 -1 core tests.  The patch failed these unit tests:
   org.apache.hadoop.hbase.regionserver.TestScanner
  org.apache.hadoop.hbase.executor.TestExecutorService
  org.apache.hadoop.hbase.regionserver.TestSplitTransaction
  org.apache.hadoop.hbase.util.TestThreads
  org.apache.hadoop.hbase.coprocessor.TestCoprocessorInterface
  
org.apache.hadoop.hbase.regionserver.TestReadWriteConsistencyControl
  org.apache.hadoop.hbase.filter.TestDependentColumnFilter
  org.apache.hadoop.hbase.coprocessor.TestRegionObserverStacking
  org.apache.hadoop.hbase.filter.TestColumnPrefixFilter
  org.apache.hadoop.hbase.filter.TestMultipleColumnPrefixFilter
  org.apache.hadoop.hbase.regionserver.TestMinVersions
  org.apache.hadoop.hbase.regionserver.TestGetClosestAtOrBefore
  org.apache.hadoop.hbase.regionserver.TestWideScanner
  org.apache.hadoop.hbase.filter.TestFilter

Test results: 
https://builds.apache.org/job/PreCommit-HBASE-Build/434//testReport/
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/434//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/434//console

This message is automatically generated.

> Activate single jvm for small tests on jenkins
> --
>
> Key: HBASE-4847
> URL: https://issues.apache.org/jira/browse/HBASE-4847
> Project: HBase
>  Issue Type: Improvement
>  Components: build, test
>Affects Versions: 0.94.0
> Environment: build
>Reporter: nkeywal
>Assignee: nkeywal
>Priority: Minor
> Attachments: 4847_all.v10.patch, 4847_all.v10.patch, 
> 4847_all.v10.patch, 4847_all.v11.patch, 4847_all.v11.patch, 
> 4847_all.v4.patch, 4847_all.v5.patch, 4847_all.v6.patch, 4847_all.v6.patch, 
> 4847_all.v7.patch, 4847_all.v7.patch, 4847_all.v7.patch, 4847_all.v7.patch, 
> 4847_all.v8.patch, 4847_all.v8.patch, 4847_all.v9.patch, 4847_pom.patch, 
> 4847_pom.v2.patch, 4847_pom.v2.patch, 4847_pom.v2.patch, 4847_pom.v3.patch
>
>
> This will not revolutionate performances alone. We will win between 1 to 4 
> minutes.
> But we win as well:
>  - it's a step for parallelizing the tests
>  - new tests are less expensive as they do not create a new jvm: it's a 
> continuous win
>  - it will allow to push it on dev env while having the same env on dev & on 
> build, and 3 minutes are 10% of small + medium tests execution time.
> I will do a few "submit patch" to see if it works well before asking for the 
> real commit.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-4847) Activate single jvm for small tests on jenkins

[nkeywal (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-4847?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13162184#comment-13162184
 ] 

nkeywal commented on HBASE-4847:


"java.lang.OutOfMemoryError: unable to create new native thread" twice in a row.

let's wait.

> Activate single jvm for small tests on jenkins
> --
>
> Key: HBASE-4847
> URL: https://issues.apache.org/jira/browse/HBASE-4847
> Project: HBase
>  Issue Type: Improvement
>  Components: build, test
>Affects Versions: 0.94.0
> Environment: build
>Reporter: nkeywal
>Assignee: nkeywal
>Priority: Minor
> Attachments: 4847_all.v10.patch, 4847_all.v10.patch, 
> 4847_all.v10.patch, 4847_all.v11.patch, 4847_all.v11.patch, 
> 4847_all.v4.patch, 4847_all.v5.patch, 4847_all.v6.patch, 4847_all.v6.patch, 
> 4847_all.v7.patch, 4847_all.v7.patch, 4847_all.v7.patch, 4847_all.v7.patch, 
> 4847_all.v8.patch, 4847_all.v8.patch, 4847_all.v9.patch, 4847_pom.patch, 
> 4847_pom.v2.patch, 4847_pom.v2.patch, 4847_pom.v2.patch, 4847_pom.v3.patch
>
>
> This will not revolutionate performances alone. We will win between 1 to 4 
> minutes.
> But we win as well:
>  - it's a step for parallelizing the tests
>  - new tests are less expensive as they do not create a new jvm: it's a 
> continuous win
>  - it will allow to push it on dev env while having the same env on dev & on 
> build, and 3 minutes are 10% of small + medium tests execution time.
> I will do a few "submit patch" to see if it works well before asking for the 
> real commit.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (HBASE-4942) HMaster is unable to start of HFile V1 is used

[Ted Yu (Created) (JIRA)]

HMaster is unable to start of HFile V1 is used
--

 Key: HBASE-4942
 URL: https://issues.apache.org/jira/browse/HBASE-4942
 Project: HBase
  Issue Type: Bug
  Components: io
Affects Versions: 0.92.0
Reporter: Ted Yu
 Fix For: 0.92.0, 0.94.0


This was reported by HH Zhu (zhh200...@gmail.com)
If the following is specified in hbase-site.xml：
{code}

hfile.format.version
1

{code}
Clear the hdfs directory "hbase.rootdir" so that MasterFileSystem.bootstrap() 
is executed.
You would see:
{code}
java.lang.NullPointerException
at 
org.apache.hadoop.hbase.io.hfile.HFileReaderV1.close(HFileReaderV1.java:358)
at 
org.apache.hadoop.hbase.regionserver.StoreFile$Reader.close(StoreFile.java:1083)
at 
org.apache.hadoop.hbase.regionserver.StoreFile.closeReader(StoreFile.java:570)
at org.apache.hadoop.hbase.regionserver.Store.close(Store.java:441)
at org.apache.hadoop.hbase.regionserver.HRegion.doClose(HRegion.java:782)
at org.apache.hadoop.hbase.regionserver.HRegion.close(HRegion.java:717)
at org.apache.hadoop.hbase.regionserver.HRegion.close(HRegion.java:688)
at 
org.apache.hadoop.hbase.master.MasterFileSystem.bootstrap(MasterFileSystem.java:390)
at 
org.apache.hadoop.hbase.master.MasterFileSystem.checkRootDir(MasterFileSystem.java:356)
at 
org.apache.hadoop.hbase.master.MasterFileSystem.createInitialFileSystemLayout(MasterFileSystem.java:128)
at 
org.apache.hadoop.hbase.master.MasterFileSystem.(MasterFileSystem.java:113)
at 
org.apache.hadoop.hbase.master.HMaster.finishInitialization(HMaster.java:435)
at org.apache.hadoop.hbase.master.HMaster.run(HMaster.java:314)
at java.lang.Thread.run(Thread.java:619)
{code}
The above exception would lead to:
{code}
java.lang.RuntimeException: HMaster Aborted
at 
org.apache.hadoop.hbase.master.HMasterCommandLine.startMaster(HMasterCommandLine.java:152)
at 
org.apache.hadoop.hbase.master.HMasterCommandLine.run(HMasterCommandLine.java:103)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
at 
org.apache.hadoop.hbase.util.ServerCommandLine.doMain(ServerCommandLine.java:76)
at org.apache.hadoop.hbase.master.HMaster.main(HMaster.java:1512)
{code}

In org.apache.hadoop.hbase.master.HMaster.HMaster(Configuration conf), we have:
{code}
this.conf.setFloat(CacheConfig.HFILE_BLOCK_CACHE_SIZE_KEY, 0.0f);
{code}
When CacheConfig is instantiated, the following is called:
{code}
org.apache.hadoop.hbase.io.hfile.CacheConfig.instantiateBlockCache(Configuration
 conf)
{code}
Since "hfile.block.cache.size" is 0.0, instantiateBlockCache() would return 
null, resulting in blockCache field of CacheConfig to be null.
When master closes Root region, 
org.apache.hadoop.hbase.io.hfile.HFileReaderV1.close(boolean evictOnClose) 
would be called. cacheConf.getBlockCache() returns null, leading to master 
abort.

The following should be called in HFileReaderV1.close(), similar to the code in 
HFileReaderV2.close():
{code}
if (evictOnClose && cacheConf.isBlockCacheEnabled())
{code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (HBASE-4943) [book] variety of updates to book 12-3-2011

[Doug Meil (Created) (JIRA)]

[book] variety of updates to book 12-3-2011
---

 Key: HBASE-4943
 URL: https://issues.apache.org/jira/browse/HBASE-4943
 Project: HBase
  Issue Type: Improvement
Reporter: Doug Meil
Assignee: Doug Meil
Priority: Minor
 Attachments: docbkx_HBASE_4943.patch

book.xml
* added FAQ entry for "hbase in action" (link to newly created appendix for 
other resources)
* added new appendix entry for other resources (videos, presentations, etc.)
* moved a build FAQ into the Development chapter.

troubleshooting.xml
* added IRC into the 'resources' section


--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-4943) [book] variety of updates to book 12-3-2011

[Doug Meil (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-4943?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Doug Meil updated HBASE-4943:
-

Attachment: docbkx_HBASE_4943.patch

> [book] variety of updates to book 12-3-2011
> ---
>
> Key: HBASE-4943
> URL: https://issues.apache.org/jira/browse/HBASE-4943
> Project: HBase
>  Issue Type: Improvement
>Reporter: Doug Meil
>Assignee: Doug Meil
>Priority: Minor
> Attachments: docbkx_HBASE_4943.patch
>
>
> book.xml
> * added FAQ entry for "hbase in action" (link to newly created appendix for 
> other resources)
> * added new appendix entry for other resources (videos, presentations, etc.)
> * moved a build FAQ into the Development chapter.
> troubleshooting.xml
> * added IRC into the 'resources' section

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-4943) [book] variety of updates to book 12-3-2011

[Doug Meil (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-4943?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Doug Meil updated HBASE-4943:
-

Status: Patch Available  (was: Open)

> [book] variety of updates to book 12-3-2011
> ---
>
> Key: HBASE-4943
> URL: https://issues.apache.org/jira/browse/HBASE-4943
> Project: HBase
>  Issue Type: Improvement
>Reporter: Doug Meil
>Assignee: Doug Meil
>Priority: Minor
> Attachments: docbkx_HBASE_4943.patch
>
>
> book.xml
> * added FAQ entry for "hbase in action" (link to newly created appendix for 
> other resources)
> * added new appendix entry for other resources (videos, presentations, etc.)
> * moved a build FAQ into the Development chapter.
> troubleshooting.xml
> * added IRC into the 'resources' section

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-4943) [book] variety of updates to book 12-3-2011

[Doug Meil (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-4943?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Doug Meil updated HBASE-4943:
-

Resolution: Fixed
Status: Resolved  (was: Patch Available)

> [book] variety of updates to book 12-3-2011
> ---
>
> Key: HBASE-4943
> URL: https://issues.apache.org/jira/browse/HBASE-4943
> Project: HBase
>  Issue Type: Improvement
>Reporter: Doug Meil
>Assignee: Doug Meil
>Priority: Minor
> Attachments: docbkx_HBASE_4943.patch
>
>
> book.xml
> * added FAQ entry for "hbase in action" (link to newly created appendix for 
> other resources)
> * added new appendix entry for other resources (videos, presentations, etc.)
> * moved a build FAQ into the Development chapter.
> troubleshooting.xml
> * added IRC into the 'resources' section

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-4921) HTable initialization looks for EMPTY_START_ROW

[Ted Yu (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-4921?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13162215#comment-13162215
 ] 

Ted Yu commented on HBASE-4921:
---

@Pritam:
Can you provide a patch and let us know the result of running through test 
suite ?

Thanks

> HTable initialization looks for EMPTY_START_ROW
> ---
>
> Key: HBASE-4921
> URL: https://issues.apache.org/jira/browse/HBASE-4921
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 0.90.4
>Reporter: Pritam Damania
>
> The HTable initialization does something like this : 
> {code}this.connection.locateRegion(tableName, 
> HConstants.EMPTY_START_ROW);{code}
> What is the rationale behind this ? What would happen if this region is in 
> flight ? I ran into a problem where I disabled the first region of the table 
> and now I can't create an HTable instance to this table.
> Disabling the first region is like disabling the entire table from a client 
> perspective. I feel this is not the correct behavior.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-4927) CatalogJanior:SplitParentFirstComparator doesn't sort as expected, for the last region when the endkey is empty

[jirapos...@reviews.apache.org (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-4927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13162223#comment-13162223
 ] 

jirapos...@reviews.apache.org commented on HBASE-4927:
--


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/2992/#review3607
---



src/test/java/org/apache/hadoop/hbase/master/TestCatalogJanitor.java


Would parentWithSpecifiedEndKeyCleanedEvenIfDaughterGoneFirst be a better 
name for this method ?



src/test/java/org/apache/hadoop/hbase/master/TestCatalogJanitor.java


This sentence seems to be inconsistent with the comment on line 361.


- Ted


On 2011-12-02 19:09:19, Jimmy Xiang wrote:
bq.  
bq.  ---
bq.  This is an automatically generated e-mail. To reply, visit:
bq.  https://reviews.apache.org/r/2992/
bq.  ---
bq.  
bq.  (Updated 2011-12-02 19:09:19)
bq.  
bq.  
bq.  Review request for hbase and Michael Stack.
bq.  
bq.  
bq.  Summary
bq.  ---
bq.  
bq.  Some HRegionInfo related comparator doesn't consider empty end key.  
That's why there are some hacks in HRegionInfo's containsRange and containsRow 
method to check empty key.
bq.  However, due to this issue, SplitParentFirstComparator has a bug.
bq.  
bq.  
bq.  This addresses bug HBASE-4927.
bq.  https://issues.apache.org/jira/browse/HBASE-4927
bq.  
bq.  
bq.  Diffs
bq.  -
bq.  
bq.src/main/java/org/apache/hadoop/hbase/HRegionInfo.java 2a24d0b 
bq.src/main/java/org/apache/hadoop/hbase/master/CatalogJanitor.java b356b22 
bq.src/test/java/org/apache/hadoop/hbase/master/TestCatalogJanitor.java 
e047e30 
bq.src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegionInfo.java 
baca241 
bq.  
bq.  Diff: https://reviews.apache.org/r/2992/diff
bq.  
bq.  
bq.  Testing
bq.  ---
bq.  
bq.  TestHRegionInfo and TestCatalogJanitor are geen.
bq.  
bq.  
bq.  Thanks,
bq.  
bq.  Jimmy
bq.  
bq.



> CatalogJanior:SplitParentFirstComparator doesn't sort as expected, for the 
> last region when the endkey is empty
> ---
>
> Key: HBASE-4927
> URL: https://issues.apache.org/jira/browse/HBASE-4927
> Project: HBase
>  Issue Type: Bug
>  Components: master
>Affects Versions: 0.92.0, 0.94.0
>Reporter: Jimmy Xiang
>Assignee: Jimmy Xiang
>Priority: Minor
> Fix For: 0.94.0
>
> Attachments: 
> 0001-HBASE-4927-CatalogJanior-SplitParentFirstComparator-.patch
>
>
> When reviewing HBASE-4238 backporting, Jon found this issue.
> What happens if the split points are  (empty end key is the last key, empty 
> start key is the first key)
> Parent [A,)
> L daughter [A,B), 
> R daughter [B,)
> When sorted, we gets to end key comparision which results in this incorrector 
> order:
> [A,B), [A,), [B,) 
> we wanted:
> [A,), [A,B), [B,)

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-4379) [hbck] Does not complain about tables with no end region [Z,]

[Ted Yu (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-4379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13162224#comment-13162224
 ] 

Ted Yu commented on HBASE-4379:
---

@Jonathan:
You need to re-attach patch because HadoopQA checks the attachment Id.
If the attachment Id corresponds to a patch which was verified earlier, there 
would be no test suite execution.

Thanks

> [hbck] Does not complain about tables with no end region [Z,]
> -
>
> Key: HBASE-4379
> URL: https://issues.apache.org/jira/browse/HBASE-4379
> Project: HBase
>  Issue Type: Bug
>  Components: hbck
>Affects Versions: 0.92.0, 0.90.5
>Reporter: Jonathan Hsieh
>Assignee: Jonathan Hsieh
> Attachments: 
> 0001-HBASE-4379-hbck-does-not-complain-about-tables-with-.patch, 
> hbase-4379.v2.patch
>
>
> hbck does not detect or have an error condition when the last region of a 
> table is missing (end key != '').

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-4943) [book] variety of updates to book 12-3-2011

[Hudson (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-4943?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13162240#comment-13162240
 ] 

Hudson commented on HBASE-4943:
---

Integrated in HBase-TRUNK #2511 (See 
[https://builds.apache.org/job/HBase-TRUNK/2511/])
hbase-4943.  book updates (more FAQ, add to appendix for other resources)


> [book] variety of updates to book 12-3-2011
> ---
>
> Key: HBASE-4943
> URL: https://issues.apache.org/jira/browse/HBASE-4943
> Project: HBase
>  Issue Type: Improvement
>Reporter: Doug Meil
>Assignee: Doug Meil
>Priority: Minor
> Attachments: docbkx_HBASE_4943.patch
>
>
> book.xml
> * added FAQ entry for "hbase in action" (link to newly created appendix for 
> other resources)
> * added new appendix entry for other resources (videos, presentations, etc.)
> * moved a build FAQ into the Development chapter.
> troubleshooting.xml
> * added IRC into the 'resources' section

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (HBASE-4944) Optionally verify bulk loaded HFiles

[Andrew Purtell (Created) (JIRA)]

Optionally verify bulk loaded HFiles


 Key: HBASE-4944
 URL: https://issues.apache.org/jira/browse/HBASE-4944
 Project: HBase
  Issue Type: Improvement
  Components: regionserver
Reporter: Andrew Purtell
Priority: Minor
 Fix For: 0.92.0, 0.94.0, 0.90.5


We rely on users to produce properly formatted HFiles for bulk import. Attached 
patch adds an optional code path, toggled by a configuration property, that 
verifies the HFile under consideration for import is properly sorted. The 
default maintains the current behavior, which does not scan the file for 
correctness.

Patch is against trunk but can apply against all active branches.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-4944) Optionally verify bulk loaded HFiles

[Andrew Purtell (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-4944?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrew Purtell updated HBASE-4944:
--

Fix Version/s: (was: 0.90.5)
   (was: 0.94.0)
   (was: 0.92.0)
Affects Version/s: 0.90.5
   0.94.0
   0.92.0
   Status: Patch Available  (was: Open)

> Optionally verify bulk loaded HFiles
> 
>
> Key: HBASE-4944
> URL: https://issues.apache.org/jira/browse/HBASE-4944
> Project: HBase
>  Issue Type: Improvement
>  Components: regionserver
>Affects Versions: 0.92.0, 0.94.0, 0.90.5
>Reporter: Andrew Purtell
>Priority: Minor
>
> We rely on users to produce properly formatted HFiles for bulk import. 
> Attached patch adds an optional code path, toggled by a configuration 
> property, that verifies the HFile under consideration for import is properly 
> sorted. The default maintains the current behavior, which does not scan the 
> file for correctness.
> Patch is against trunk but can apply against all active branches.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-4944) Optionally verify bulk loaded HFiles

[Andrew Purtell (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-4944?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13162253#comment-13162253
 ] 

Andrew Purtell commented on HBASE-4944:
---

>From JIRA: "Cannot attach file HBASE-4944.patch: Unknown server error (500)."

The patch is pretty small, so here it is:

{code}
Index: src/main/java/org/apache/hadoop/hbase/regionserver/Store.java
===
--- src/main/java/org/apache/hadoop/hbase/regionserver/Store.java   
(revision 1210044)
+++ src/main/java/org/apache/hadoop/hbase/regionserver/Store.java   
(working copy)
@@ -50,6 +50,7 @@
 import org.apache.hadoop.hbase.io.hfile.Compression;
 import org.apache.hadoop.hbase.io.hfile.HFile;
 import org.apache.hadoop.hbase.io.hfile.HFileScanner;
+import org.apache.hadoop.hbase.io.hfile.InvalidHFileException;
 import org.apache.hadoop.hbase.monitoring.MonitoredTask;
 import org.apache.hadoop.hbase.regionserver.StoreScanner.ScanType;
 import org.apache.hadoop.hbase.regionserver.compactions.CompactionProgress;
@@ -123,6 +124,7 @@
   private final String storeNameStr;
   private CompactionProgress progress;
   private final int compactionKVMax;
+  private final boolean verifyBulkLoads;
 
   // not private for testing
   /* package */ScanInfo scanInfo;
@@ -222,6 +224,9 @@
   = conf.getLong("hbase.hstore.compaction.max.size", Long.MAX_VALUE);
 this.compactionKVMax = conf.getInt("hbase.hstore.compaction.kv.max", 10);
 
+this.verifyBulkLoads = conf.getBoolean("hbase.hstore.bulkload.verify",
+true);
+
 if (Store.closeCheckInterval == 0) {
   Store.closeCheckInterval = conf.getInt(
   "hbase.hstore.close.check.interval", 10*1000*1000 /* 10 MB */);
@@ -355,8 +360,8 @@
   }
 
   /**
-   * This throws a WrongRegionException if the bulkHFile does not fit in this
-   * region.
+   * This throws a WrongRegionException if the HFile does not fit in this
+   * region, or an InvalidHFileException if the HFile is not valid.
*
*/
   void assertBulkLoadHFileOk(Path srcPath) throws IOException {
@@ -386,6 +391,34 @@
 "Bulk load file " + srcPath.toString() + " does not fit inside 
region "
 + this.region);
   }
+
+  if (verifyBulkLoads) {
+KeyValue pkv = null;
+HFileScanner scanner = reader.getScanner(false, false, false);
+scanner.seekTo();
+do {
+  KeyValue kv = scanner.getKeyValue();
+  if (pkv != null) {
+if (Bytes.compareTo(pkv.getBuffer(), pkv.getRowOffset(),
+pkv.getRowLength(), kv.getBuffer(), kv.getRowOffset(),
+kv.getRowLength()) > 0) {
+  throw new InvalidHFileException("Previous row is greater then"
+  + " current row: path=" + srcPath + " previous="
+  + Bytes.toStringBinary(pkv.getKey()) + " current="
+  + Bytes.toStringBinary(kv.getKey()));
+}
+if (Bytes.compareTo(pkv.getBuffer(), pkv.getFamilyOffset(),
+pkv.getFamilyLength(), kv.getBuffer(), kv.getFamilyOffset(),
+kv.getFamilyLength()) != 0) {
+  throw new InvalidHFileException("Previous key had different"
+  + " family compared to current key: path=" + srcPath
+  + " previous=" + Bytes.toStringBinary(pkv.getKey())
+  + " current=" + Bytes.toStringBinary(kv.getKey()));
+}
+  }
+  pkv = kv;
+} while (scanner.next());
+  }
 } finally {
   if (reader != null) reader.close();
 }
Index: src/main/java/org/apache/hadoop/hbase/io/hfile/InvalidHFileException.java
===
--- src/main/java/org/apache/hadoop/hbase/io/hfile/InvalidHFileException.java   
(revision 0)
+++ src/main/java/org/apache/hadoop/hbase/io/hfile/InvalidHFileException.java   
(revision 0)
@@ -0,0 +1,40 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hbase.io.hfile;
+
+import java.io.IOException;
+
+/**
+ * Thrown when an invalid HFile format is detec

[jira] [Commented] (HBASE-4936) Cached HRegionInterface connections crash when getting UnknownHost exceptions

[Hadoop QA (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-4936?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13162255#comment-13162255
 ] 

Hadoop QA commented on HBASE-4936:
--

-1 overall.  Here are the results of testing the latest attachment 
  http://issues.apache.org/jira/secure/attachment/12505896/HBASE-4936.patch
  against trunk revision .

+1 @author.  The patch does not contain any @author tags.

-1 tests included.  The patch doesn't appear to include any new or modified 
tests.
Please justify why no new tests are needed for this 
patch.
Also please list what manual steps were performed to 
verify this patch.

-1 patch.  The patch command could not apply the patch.

Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/435//console

This message is automatically generated.

> Cached HRegionInterface connections crash when getting UnknownHost exceptions
> -
>
> Key: HBASE-4936
> URL: https://issues.apache.org/jira/browse/HBASE-4936
> Project: HBase
>  Issue Type: Bug
>  Components: regionserver
>Affects Versions: 0.92.0
>Reporter: Andrei Dragomir
> Attachments: HBASE-4936.patch
>
>
> This isssue is unlikely to come up in a cluster test case. However, for 
> development, the following thing happens: 
> 1. Start the HBase cluster locally, on network A (DNS A, etc)
> 2. The region locations are cached using the hostname 
> (mycomputer.company.com, 211.x.y.z - real ip)
> 3. Change network location (go home)
> 4. Start the HBase cluster locally. My hostname / ips are not different 
> (mycomputer, 192.168.0.130 - new ip)
> If the region locations have been cached using the hostname, there is an 
> UnknownHostException in CatalogTracker.getCachedConnection(ServerName sn), 
> uncaught in the catch statements. The server will crash constantly. 
> The error should be caught and not rethrown, so that the cached connection 
> expires normally. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Assigned] (HBASE-4942) HMaster is unable to start of HFile V1 is used

[Ted Yu (Assigned) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-4942?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ted Yu reassigned HBASE-4942:
-

Assignee: honghua zhu

> HMaster is unable to start of HFile V1 is used
> --
>
> Key: HBASE-4942
> URL: https://issues.apache.org/jira/browse/HBASE-4942
> Project: HBase
>  Issue Type: Bug
>  Components: io
>Affects Versions: 0.92.0
>Reporter: Ted Yu
>Assignee: honghua zhu
> Fix For: 0.92.0, 0.94.0
>
>
> This was reported by HH Zhu (zhh200...@gmail.com)
> If the following is specified in hbase-site.xml：
> {code}
> 
> hfile.format.version
> 1
> 
> {code}
> Clear the hdfs directory "hbase.rootdir" so that MasterFileSystem.bootstrap() 
> is executed.
> You would see:
> {code}
> java.lang.NullPointerException
> at 
> org.apache.hadoop.hbase.io.hfile.HFileReaderV1.close(HFileReaderV1.java:358)
> at 
> org.apache.hadoop.hbase.regionserver.StoreFile$Reader.close(StoreFile.java:1083)
> at 
> org.apache.hadoop.hbase.regionserver.StoreFile.closeReader(StoreFile.java:570)
> at org.apache.hadoop.hbase.regionserver.Store.close(Store.java:441)
> at org.apache.hadoop.hbase.regionserver.HRegion.doClose(HRegion.java:782)
> at org.apache.hadoop.hbase.regionserver.HRegion.close(HRegion.java:717)
> at org.apache.hadoop.hbase.regionserver.HRegion.close(HRegion.java:688)
> at 
> org.apache.hadoop.hbase.master.MasterFileSystem.bootstrap(MasterFileSystem.java:390)
> at 
> org.apache.hadoop.hbase.master.MasterFileSystem.checkRootDir(MasterFileSystem.java:356)
> at 
> org.apache.hadoop.hbase.master.MasterFileSystem.createInitialFileSystemLayout(MasterFileSystem.java:128)
> at 
> org.apache.hadoop.hbase.master.MasterFileSystem.(MasterFileSystem.java:113)
> at 
> org.apache.hadoop.hbase.master.HMaster.finishInitialization(HMaster.java:435)
> at org.apache.hadoop.hbase.master.HMaster.run(HMaster.java:314)
> at java.lang.Thread.run(Thread.java:619)
> {code}
> The above exception would lead to:
> {code}
> java.lang.RuntimeException: HMaster Aborted
> at 
> org.apache.hadoop.hbase.master.HMasterCommandLine.startMaster(HMasterCommandLine.java:152)
> at 
> org.apache.hadoop.hbase.master.HMasterCommandLine.run(HMasterCommandLine.java:103)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
> at 
> org.apache.hadoop.hbase.util.ServerCommandLine.doMain(ServerCommandLine.java:76)
> at org.apache.hadoop.hbase.master.HMaster.main(HMaster.java:1512)
> {code}
> In org.apache.hadoop.hbase.master.HMaster.HMaster(Configuration conf), we 
> have:
> {code}
> this.conf.setFloat(CacheConfig.HFILE_BLOCK_CACHE_SIZE_KEY, 0.0f);
> {code}
> When CacheConfig is instantiated, the following is called:
> {code}
> org.apache.hadoop.hbase.io.hfile.CacheConfig.instantiateBlockCache(Configuration
>  conf)
> {code}
> Since "hfile.block.cache.size" is 0.0, instantiateBlockCache() would return 
> null, resulting in blockCache field of CacheConfig to be null.
> When master closes Root region, 
> org.apache.hadoop.hbase.io.hfile.HFileReaderV1.close(boolean evictOnClose) 
> would be called. cacheConf.getBlockCache() returns null, leading to master 
> abort.
> The following should be called in HFileReaderV1.close(), similar to the code 
> in HFileReaderV2.close():
> {code}
> if (evictOnClose && cacheConf.isBlockCacheEnabled())
> {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-4942) HMaster is unable to start of HFile V1 is used

[Lars Hofhansl (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-4942?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13162269#comment-13162269
 ] 

Lars Hofhansl commented on HBASE-4942:
--

Similar to HBASE-4673 (but for v1)?

> HMaster is unable to start of HFile V1 is used
> --
>
> Key: HBASE-4942
> URL: https://issues.apache.org/jira/browse/HBASE-4942
> Project: HBase
>  Issue Type: Bug
>  Components: io
>Affects Versions: 0.92.0
>Reporter: Ted Yu
>Assignee: honghua zhu
> Fix For: 0.92.0, 0.94.0
>
>
> This was reported by HH Zhu (zhh200...@gmail.com)
> If the following is specified in hbase-site.xml：
> {code}
> 
> hfile.format.version
> 1
> 
> {code}
> Clear the hdfs directory "hbase.rootdir" so that MasterFileSystem.bootstrap() 
> is executed.
> You would see:
> {code}
> java.lang.NullPointerException
> at 
> org.apache.hadoop.hbase.io.hfile.HFileReaderV1.close(HFileReaderV1.java:358)
> at 
> org.apache.hadoop.hbase.regionserver.StoreFile$Reader.close(StoreFile.java:1083)
> at 
> org.apache.hadoop.hbase.regionserver.StoreFile.closeReader(StoreFile.java:570)
> at org.apache.hadoop.hbase.regionserver.Store.close(Store.java:441)
> at org.apache.hadoop.hbase.regionserver.HRegion.doClose(HRegion.java:782)
> at org.apache.hadoop.hbase.regionserver.HRegion.close(HRegion.java:717)
> at org.apache.hadoop.hbase.regionserver.HRegion.close(HRegion.java:688)
> at 
> org.apache.hadoop.hbase.master.MasterFileSystem.bootstrap(MasterFileSystem.java:390)
> at 
> org.apache.hadoop.hbase.master.MasterFileSystem.checkRootDir(MasterFileSystem.java:356)
> at 
> org.apache.hadoop.hbase.master.MasterFileSystem.createInitialFileSystemLayout(MasterFileSystem.java:128)
> at 
> org.apache.hadoop.hbase.master.MasterFileSystem.(MasterFileSystem.java:113)
> at 
> org.apache.hadoop.hbase.master.HMaster.finishInitialization(HMaster.java:435)
> at org.apache.hadoop.hbase.master.HMaster.run(HMaster.java:314)
> at java.lang.Thread.run(Thread.java:619)
> {code}
> The above exception would lead to:
> {code}
> java.lang.RuntimeException: HMaster Aborted
> at 
> org.apache.hadoop.hbase.master.HMasterCommandLine.startMaster(HMasterCommandLine.java:152)
> at 
> org.apache.hadoop.hbase.master.HMasterCommandLine.run(HMasterCommandLine.java:103)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
> at 
> org.apache.hadoop.hbase.util.ServerCommandLine.doMain(ServerCommandLine.java:76)
> at org.apache.hadoop.hbase.master.HMaster.main(HMaster.java:1512)
> {code}
> In org.apache.hadoop.hbase.master.HMaster.HMaster(Configuration conf), we 
> have:
> {code}
> this.conf.setFloat(CacheConfig.HFILE_BLOCK_CACHE_SIZE_KEY, 0.0f);
> {code}
> When CacheConfig is instantiated, the following is called:
> {code}
> org.apache.hadoop.hbase.io.hfile.CacheConfig.instantiateBlockCache(Configuration
>  conf)
> {code}
> Since "hfile.block.cache.size" is 0.0, instantiateBlockCache() would return 
> null, resulting in blockCache field of CacheConfig to be null.
> When master closes Root region, 
> org.apache.hadoop.hbase.io.hfile.HFileReaderV1.close(boolean evictOnClose) 
> would be called. cacheConf.getBlockCache() returns null, leading to master 
> abort.
> The following should be called in HFileReaderV1.close(), similar to the code 
> in HFileReaderV2.close():
> {code}
> if (evictOnClose && cacheConf.isBlockCacheEnabled())
> {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-4942) HMaster is unable to start of HFile V1 is used

[Ted Yu (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-4942?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13162270#comment-13162270
 ] 

Ted Yu commented on HBASE-4942:
---

You're right Lars.

> HMaster is unable to start of HFile V1 is used
> --
>
> Key: HBASE-4942
> URL: https://issues.apache.org/jira/browse/HBASE-4942
> Project: HBase
>  Issue Type: Bug
>  Components: io
>Affects Versions: 0.92.0
>Reporter: Ted Yu
>Assignee: honghua zhu
> Fix For: 0.92.0, 0.94.0
>
>
> This was reported by HH Zhu (zhh200...@gmail.com)
> If the following is specified in hbase-site.xml：
> {code}
> 
> hfile.format.version
> 1
> 
> {code}
> Clear the hdfs directory "hbase.rootdir" so that MasterFileSystem.bootstrap() 
> is executed.
> You would see:
> {code}
> java.lang.NullPointerException
> at 
> org.apache.hadoop.hbase.io.hfile.HFileReaderV1.close(HFileReaderV1.java:358)
> at 
> org.apache.hadoop.hbase.regionserver.StoreFile$Reader.close(StoreFile.java:1083)
> at 
> org.apache.hadoop.hbase.regionserver.StoreFile.closeReader(StoreFile.java:570)
> at org.apache.hadoop.hbase.regionserver.Store.close(Store.java:441)
> at org.apache.hadoop.hbase.regionserver.HRegion.doClose(HRegion.java:782)
> at org.apache.hadoop.hbase.regionserver.HRegion.close(HRegion.java:717)
> at org.apache.hadoop.hbase.regionserver.HRegion.close(HRegion.java:688)
> at 
> org.apache.hadoop.hbase.master.MasterFileSystem.bootstrap(MasterFileSystem.java:390)
> at 
> org.apache.hadoop.hbase.master.MasterFileSystem.checkRootDir(MasterFileSystem.java:356)
> at 
> org.apache.hadoop.hbase.master.MasterFileSystem.createInitialFileSystemLayout(MasterFileSystem.java:128)
> at 
> org.apache.hadoop.hbase.master.MasterFileSystem.(MasterFileSystem.java:113)
> at 
> org.apache.hadoop.hbase.master.HMaster.finishInitialization(HMaster.java:435)
> at org.apache.hadoop.hbase.master.HMaster.run(HMaster.java:314)
> at java.lang.Thread.run(Thread.java:619)
> {code}
> The above exception would lead to:
> {code}
> java.lang.RuntimeException: HMaster Aborted
> at 
> org.apache.hadoop.hbase.master.HMasterCommandLine.startMaster(HMasterCommandLine.java:152)
> at 
> org.apache.hadoop.hbase.master.HMasterCommandLine.run(HMasterCommandLine.java:103)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
> at 
> org.apache.hadoop.hbase.util.ServerCommandLine.doMain(ServerCommandLine.java:76)
> at org.apache.hadoop.hbase.master.HMaster.main(HMaster.java:1512)
> {code}
> In org.apache.hadoop.hbase.master.HMaster.HMaster(Configuration conf), we 
> have:
> {code}
> this.conf.setFloat(CacheConfig.HFILE_BLOCK_CACHE_SIZE_KEY, 0.0f);
> {code}
> When CacheConfig is instantiated, the following is called:
> {code}
> org.apache.hadoop.hbase.io.hfile.CacheConfig.instantiateBlockCache(Configuration
>  conf)
> {code}
> Since "hfile.block.cache.size" is 0.0, instantiateBlockCache() would return 
> null, resulting in blockCache field of CacheConfig to be null.
> When master closes Root region, 
> org.apache.hadoop.hbase.io.hfile.HFileReaderV1.close(boolean evictOnClose) 
> would be called. cacheConf.getBlockCache() returns null, leading to master 
> abort.
> The following should be called in HFileReaderV1.close(), similar to the code 
> in HFileReaderV2.close():
> {code}
> if (evictOnClose && cacheConf.isBlockCacheEnabled())
> {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-4936) Cached HRegionInterface connections crash when getting UnknownHost exceptions

[Ted Yu (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-4936?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13162272#comment-13162272
 ] 

Ted Yu commented on HBASE-4936:
---

+1 on patch.

@Andrei:
Please re-attach patch using --no-prefix option.
HadoopQA uses -p0 to apply patches.

> Cached HRegionInterface connections crash when getting UnknownHost exceptions
> -
>
> Key: HBASE-4936
> URL: https://issues.apache.org/jira/browse/HBASE-4936
> Project: HBase
>  Issue Type: Bug
>  Components: regionserver
>Affects Versions: 0.92.0
>Reporter: Andrei Dragomir
> Attachments: HBASE-4936.patch
>
>
> This isssue is unlikely to come up in a cluster test case. However, for 
> development, the following thing happens: 
> 1. Start the HBase cluster locally, on network A (DNS A, etc)
> 2. The region locations are cached using the hostname 
> (mycomputer.company.com, 211.x.y.z - real ip)
> 3. Change network location (go home)
> 4. Start the HBase cluster locally. My hostname / ips are not different 
> (mycomputer, 192.168.0.130 - new ip)
> If the region locations have been cached using the hostname, there is an 
> UnknownHostException in CatalogTracker.getCachedConnection(ServerName sn), 
> uncaught in the catch statements. The server will crash constantly. 
> The error should be caught and not rethrown, so that the cached connection 
> expires normally. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-4945) NPE in HRegion.bulkLoadHFiles(...)

[Lars Hofhansl (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-4945?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lars Hofhansl updated HBASE-4945:
-

Description: 
Was playing with "completebulkload", and ran into an NPE.
The problem is here (HRegion.bulkLoadHFiles(...)).

{code}
Store store = getStore(familyName);
if (store == null) {
  IOException ioe = new DoNotRetryIOException(
  "No such column family " + Bytes.toStringBinary(familyName));
  ioes.add(ioe);
  failures.add(p);
}

try {
  store.assertBulkLoadHFileOk(new Path(path));
} catch (WrongRegionException wre) {
  // recoverable (file doesn't fit in region)
  failures.add(p);
} catch (IOException ioe) {
  // unrecoverable (hdfs problem)
  ioes.add(ioe);
}
{code}

This should be 
{code}
Store store = getStore(familyName);
if (store == null) {
...
} else {
  try {
store.assertBulkLoadHFileOk(new Path(path));
...
}
{code}


  was:
Was playing with "completebulkload", and ran into an NPE.
The problem is here.

{code}
Store store = getStore(familyName);
if (store == null) {
  IOException ioe = new DoNotRetryIOException(
  "No such column family " + Bytes.toStringBinary(familyName));
  ioes.add(ioe);
  failures.add(p);
}

try {
  store.assertBulkLoadHFileOk(new Path(path));
} catch (WrongRegionException wre) {
  // recoverable (file doesn't fit in region)
  failures.add(p);
} catch (IOException ioe) {
  // unrecoverable (hdfs problem)
  ioes.add(ioe);
}
{code}

This should be 
{code}
Store store = getStore(familyName);
if (store == null) {
...
} else {
  try {
store.assertBulkLoadHFileOk(new Path(path));
...
}
{code}



> NPE in HRegion.bulkLoadHFiles(...)
> --
>
> Key: HBASE-4945
> URL: https://issues.apache.org/jira/browse/HBASE-4945
> Project: HBase
>  Issue Type: Bug
>  Components: mapreduce
>Affects Versions: 0.94.0
>Reporter: Lars Hofhansl
>Priority: Minor
>
> Was playing with "completebulkload", and ran into an NPE.
> The problem is here (HRegion.bulkLoadHFiles(...)).
> {code}
> Store store = getStore(familyName);
> if (store == null) {
>   IOException ioe = new DoNotRetryIOException(
>   "No such column family " + Bytes.toStringBinary(familyName));
>   ioes.add(ioe);
>   failures.add(p);
> }
> try {
>   store.assertBulkLoadHFileOk(new Path(path));
> } catch (WrongRegionException wre) {
>   // recoverable (file doesn't fit in region)
>   failures.add(p);
> } catch (IOException ioe) {
>   // unrecoverable (hdfs problem)
>   ioes.add(ioe);
> }
> {code}
> This should be 
> {code}
> Store store = getStore(familyName);
> if (store == null) {
> ...
> } else {
>   try {
> store.assertBulkLoadHFileOk(new Path(path));
> ...
> }
> {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (HBASE-4945) NPE in HRegion.bulkLoadHFiles(...)

[Lars Hofhansl (Created) (JIRA)]

NPE in HRegion.bulkLoadHFiles(...)
--

 Key: HBASE-4945
 URL: https://issues.apache.org/jira/browse/HBASE-4945
 Project: HBase
  Issue Type: Bug
  Components: mapreduce
Affects Versions: 0.94.0
Reporter: Lars Hofhansl
Priority: Minor


Was playing with "completebulkload", and ran into an NPE.
The problem is here.

{code}
Store store = getStore(familyName);
if (store == null) {
  IOException ioe = new DoNotRetryIOException(
  "No such column family " + Bytes.toStringBinary(familyName));
  ioes.add(ioe);
  failures.add(p);
}

try {
  store.assertBulkLoadHFileOk(new Path(path));
} catch (WrongRegionException wre) {
  // recoverable (file doesn't fit in region)
  failures.add(p);
} catch (IOException ioe) {
  // unrecoverable (hdfs problem)
  ioes.add(ioe);
}
{code}

This should be 
{code}
Store store = getStore(familyName);
if (store == null) {
...
} else {
  try {
store.assertBulkLoadHFileOk(new Path(path));
...
}
{code}


--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Assigned] (HBASE-4936) Cached HRegionInterface connections crash when getting UnknownHost exceptions

[Ted Yu (Assigned) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-4936?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ted Yu reassigned HBASE-4936:
-

Assignee: Andrei Dragomir

> Cached HRegionInterface connections crash when getting UnknownHost exceptions
> -
>
> Key: HBASE-4936
> URL: https://issues.apache.org/jira/browse/HBASE-4936
> Project: HBase
>  Issue Type: Bug
>  Components: regionserver
>Affects Versions: 0.92.0
>Reporter: Andrei Dragomir
>Assignee: Andrei Dragomir
> Attachments: HBASE-4936.patch
>
>
> This isssue is unlikely to come up in a cluster test case. However, for 
> development, the following thing happens: 
> 1. Start the HBase cluster locally, on network A (DNS A, etc)
> 2. The region locations are cached using the hostname 
> (mycomputer.company.com, 211.x.y.z - real ip)
> 3. Change network location (go home)
> 4. Start the HBase cluster locally. My hostname / ips are not different 
> (mycomputer, 192.168.0.130 - new ip)
> If the region locations have been cached using the hostname, there is an 
> UnknownHostException in CatalogTracker.getCachedConnection(ServerName sn), 
> uncaught in the catch statements. The server will crash constantly. 
> The error should be caught and not rethrown, so that the cached connection 
> expires normally. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-4945) NPE in HRegion.bulkLoadHFiles(...)

[Ted Yu (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-4945?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13162275#comment-13162275
 ] 

Ted Yu commented on HBASE-4945:
---

Good catch, Lars.

I think the fix should go to 0.92

> NPE in HRegion.bulkLoadHFiles(...)
> --
>
> Key: HBASE-4945
> URL: https://issues.apache.org/jira/browse/HBASE-4945
> Project: HBase
>  Issue Type: Bug
>  Components: mapreduce
>Affects Versions: 0.94.0
>Reporter: Lars Hofhansl
>Priority: Minor
>
> Was playing with "completebulkload", and ran into an NPE.
> The problem is here (HRegion.bulkLoadHFiles(...)).
> {code}
> Store store = getStore(familyName);
> if (store == null) {
>   IOException ioe = new DoNotRetryIOException(
>   "No such column family " + Bytes.toStringBinary(familyName));
>   ioes.add(ioe);
>   failures.add(p);
> }
> try {
>   store.assertBulkLoadHFileOk(new Path(path));
> } catch (WrongRegionException wre) {
>   // recoverable (file doesn't fit in region)
>   failures.add(p);
> } catch (IOException ioe) {
>   // unrecoverable (hdfs problem)
>   ioes.add(ioe);
> }
> {code}
> This should be 
> {code}
> Store store = getStore(familyName);
> if (store == null) {
> ...
> } else {
>   try {
> store.assertBulkLoadHFileOk(new Path(path));
> ...
> }
> {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-4944) Optionally verify bulk loaded HFiles

[Ted Yu (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-4944?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ted Yu updated HBASE-4944:
--

Attachment: 4944.txt

Patch from Andy.

> Optionally verify bulk loaded HFiles
> 
>
> Key: HBASE-4944
> URL: https://issues.apache.org/jira/browse/HBASE-4944
> Project: HBase
>  Issue Type: Improvement
>  Components: regionserver
>Affects Versions: 0.92.0, 0.94.0, 0.90.5
>Reporter: Andrew Purtell
>Priority: Minor
> Attachments: 4944.txt
>
>
> We rely on users to produce properly formatted HFiles for bulk import. 
> Attached patch adds an optional code path, toggled by a configuration 
> property, that verifies the HFile under consideration for import is properly 
> sorted. The default maintains the current behavior, which does not scan the 
> file for correctness.
> Patch is against trunk but can apply against all active branches.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-4880) Region is on service before completing openRegionHanlder, may cause data loss

[Ted Yu (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-4880?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13162278#comment-13162278
 ] 

Ted Yu commented on HBASE-4880:
---

I think the patch makes sense.

I ran TestHCM with patch applied. It passed.

> Region is on service before completing openRegionHanlder, may cause data loss
> -
>
> Key: HBASE-4880
> URL: https://issues.apache.org/jira/browse/HBASE-4880
> Project: HBase
>  Issue Type: Bug
>Reporter: chunhui shen
>Assignee: chunhui shen
> Attachments: hbase-4880.patch
>
>
> OpenRegionHandler in regionserver is processed as the following steps:
> {code}
> 1.openregion()(Through it, closed = false, closing = false)
> 2.addToOnlineRegions(region)
> 3.update .meta. table 
> 4.update ZK's node state to RS_ZK_REGION_OPEND
> {code}
> We can find that region is on service before Step 4.
> It means client could put data to this region after step 3.
> What will happen if step 4 is failed processing?
> It will execute OpenRegionHandler#cleanupFailedOpen which will do closing 
> region, and master assign this region to another regionserver.
> If closing region is failed, the data which is put between step 3 and step 4 
> may loss, because the region has been opend on another regionserver and be 
> put new data. Therefore, it may not be recoverd through replayRecoveredEdit() 
> because the edit's LogSeqId is smaller than current region SeqId.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-4944) Optionally verify bulk loaded HFiles

[Hadoop QA (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-4944?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13162279#comment-13162279
 ] 

Hadoop QA commented on HBASE-4944:
--

-1 overall.  Here are the results of testing the latest attachment 
  http://issues.apache.org/jira/secure/attachment/12506016/4944.txt
  against trunk revision .

+1 @author.  The patch does not contain any @author tags.

-1 tests included.  The patch doesn't appear to include any new or modified 
tests.
Please justify why no new tests are needed for this 
patch.
Also please list what manual steps were performed to 
verify this patch.

-1 patch.  The patch command could not apply the patch.

Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/436//console

This message is automatically generated.

> Optionally verify bulk loaded HFiles
> 
>
> Key: HBASE-4944
> URL: https://issues.apache.org/jira/browse/HBASE-4944
> Project: HBase
>  Issue Type: Improvement
>  Components: regionserver
>Affects Versions: 0.92.0, 0.94.0, 0.90.5
>Reporter: Andrew Purtell
>Priority: Minor
> Attachments: 4944.txt
>
>
> We rely on users to produce properly formatted HFiles for bulk import. 
> Attached patch adds an optional code path, toggled by a configuration 
> property, that verifies the HFile under consideration for import is properly 
> sorted. The default maintains the current behavior, which does not scan the 
> file for correctness.
> Patch is against trunk but can apply against all active branches.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-4944) Optionally verify bulk loaded HFiles

[Ted Yu (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-4944?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13162280#comment-13162280
 ] 

Ted Yu commented on HBASE-4944:
---

Minor comments:
{code}
+KeyValue pkv = null;
{code}
The variable can be named prevKV which is clearer.
{code}
+  throw new InvalidHFileException("Previous row is greater then"
{code}
Typo above, should be 'greater than'.



> Optionally verify bulk loaded HFiles
> 
>
> Key: HBASE-4944
> URL: https://issues.apache.org/jira/browse/HBASE-4944
> Project: HBase
>  Issue Type: Improvement
>  Components: regionserver
>Affects Versions: 0.92.0, 0.94.0, 0.90.5
>Reporter: Andrew Purtell
>Priority: Minor
> Attachments: 4944.txt
>
>
> We rely on users to produce properly formatted HFiles for bulk import. 
> Attached patch adds an optional code path, toggled by a configuration 
> property, that verifies the HFile under consideration for import is properly 
> sorted. The default maintains the current behavior, which does not scan the 
> file for correctness.
> Patch is against trunk but can apply against all active branches.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-4944) Optionally verify bulk loaded HFiles

[Ted Yu (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-4944?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13162281#comment-13162281
 ] 

Ted Yu commented on HBASE-4944:
---

Looks like the patch should be rebased:
{code}
4 out of 5 hunks FAILED -- saving rejects to file 
src/main/java/org/apache/hadoop/hbase/regionserver/Store.java.rej
{code}

> Optionally verify bulk loaded HFiles
> 
>
> Key: HBASE-4944
> URL: https://issues.apache.org/jira/browse/HBASE-4944
> Project: HBase
>  Issue Type: Improvement
>  Components: regionserver
>Affects Versions: 0.92.0, 0.94.0, 0.90.5
>Reporter: Andrew Purtell
>Priority: Minor
> Attachments: 4944.txt
>
>
> We rely on users to produce properly formatted HFiles for bulk import. 
> Attached patch adds an optional code path, toggled by a configuration 
> property, that verifies the HFile under consideration for import is properly 
> sorted. The default maintains the current behavior, which does not scan the 
> file for correctness.
> Patch is against trunk but can apply against all active branches.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-4944) Optionally verify bulk loaded HFiles

[Andrew Purtell (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-4944?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrew Purtell updated HBASE-4944:
--

Attachment: HBASE-4944-v2.patch

Rebased patch addressing Ted's comments.

> Optionally verify bulk loaded HFiles
> 
>
> Key: HBASE-4944
> URL: https://issues.apache.org/jira/browse/HBASE-4944
> Project: HBase
>  Issue Type: Improvement
>  Components: regionserver
>Affects Versions: 0.92.0, 0.94.0, 0.90.5
>Reporter: Andrew Purtell
>Priority: Minor
> Attachments: 4944.txt, HBASE-4944-v2.patch
>
>
> We rely on users to produce properly formatted HFiles for bulk import. 
> Attached patch adds an optional code path, toggled by a configuration 
> property, that verifies the HFile under consideration for import is properly 
> sorted. The default maintains the current behavior, which does not scan the 
> file for correctness.
> Patch is against trunk but can apply against all active branches.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-4944) Optionally verify bulk loaded HFiles

[Andrew Purtell (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-4944?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrew Purtell updated HBASE-4944:
--

Attachment: (was: 4944.txt)

> Optionally verify bulk loaded HFiles
> 
>
> Key: HBASE-4944
> URL: https://issues.apache.org/jira/browse/HBASE-4944
> Project: HBase
>  Issue Type: Improvement
>  Components: regionserver
>Affects Versions: 0.92.0, 0.94.0, 0.90.5
>Reporter: Andrew Purtell
>Priority: Minor
> Attachments: HBASE-4944-v2.patch, HBASE-4944-v3.patch
>
>
> We rely on users to produce properly formatted HFiles for bulk import. 
> Attached patch adds an optional code path, toggled by a configuration 
> property, that verifies the HFile under consideration for import is properly 
> sorted. The default maintains the current behavior, which does not scan the 
> file for correctness.
> Patch is against trunk but can apply against all active branches.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-4944) Optionally verify bulk loaded HFiles

[Andrew Purtell (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-4944?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrew Purtell updated HBASE-4944:
--

Attachment: HBASE-4944-v3.patch

Sorry, v3 patch restores the default to false (current behavior)

> Optionally verify bulk loaded HFiles
> 
>
> Key: HBASE-4944
> URL: https://issues.apache.org/jira/browse/HBASE-4944
> Project: HBase
>  Issue Type: Improvement
>  Components: regionserver
>Affects Versions: 0.92.0, 0.94.0, 0.90.5
>Reporter: Andrew Purtell
>Priority: Minor
> Attachments: HBASE-4944-v2.patch, HBASE-4944-v3.patch
>
>
> We rely on users to produce properly formatted HFiles for bulk import. 
> Attached patch adds an optional code path, toggled by a configuration 
> property, that verifies the HFile under consideration for import is properly 
> sorted. The default maintains the current behavior, which does not scan the 
> file for correctness.
> Patch is against trunk but can apply against all active branches.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-4945) NPE in HRegion.bulkLoadHFiles(...)

[Andrew Purtell (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-4945?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrew Purtell updated HBASE-4945:
--

Attachment: HBASE-4945.patch

> NPE in HRegion.bulkLoadHFiles(...)
> --
>
> Key: HBASE-4945
> URL: https://issues.apache.org/jira/browse/HBASE-4945
> Project: HBase
>  Issue Type: Bug
>  Components: mapreduce
>Affects Versions: 0.92.0, 0.94.0
>Reporter: Lars Hofhansl
>Priority: Minor
> Attachments: HBASE-4945.patch
>
>
> Was playing with "completebulkload", and ran into an NPE.
> The problem is here (HRegion.bulkLoadHFiles(...)).
> {code}
> Store store = getStore(familyName);
> if (store == null) {
>   IOException ioe = new DoNotRetryIOException(
>   "No such column family " + Bytes.toStringBinary(familyName));
>   ioes.add(ioe);
>   failures.add(p);
> }
> try {
>   store.assertBulkLoadHFileOk(new Path(path));
> } catch (WrongRegionException wre) {
>   // recoverable (file doesn't fit in region)
>   failures.add(p);
> } catch (IOException ioe) {
>   // unrecoverable (hdfs problem)
>   ioes.add(ioe);
> }
> {code}
> This should be 
> {code}
> Store store = getStore(familyName);
> if (store == null) {
> ...
> } else {
>   try {
> store.assertBulkLoadHFileOk(new Path(path));
> ...
> }
> {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-4945) NPE in HRegion.bulkLoadHFiles(...)

[Andrew Purtell (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-4945?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrew Purtell updated HBASE-4945:
--

Affects Version/s: 0.92.0
   Status: Patch Available  (was: Open)

> NPE in HRegion.bulkLoadHFiles(...)
> --
>
> Key: HBASE-4945
> URL: https://issues.apache.org/jira/browse/HBASE-4945
> Project: HBase
>  Issue Type: Bug
>  Components: mapreduce
>Affects Versions: 0.92.0, 0.94.0
>Reporter: Lars Hofhansl
>Priority: Minor
> Attachments: HBASE-4945.patch
>
>
> Was playing with "completebulkload", and ran into an NPE.
> The problem is here (HRegion.bulkLoadHFiles(...)).
> {code}
> Store store = getStore(familyName);
> if (store == null) {
>   IOException ioe = new DoNotRetryIOException(
>   "No such column family " + Bytes.toStringBinary(familyName));
>   ioes.add(ioe);
>   failures.add(p);
> }
> try {
>   store.assertBulkLoadHFileOk(new Path(path));
> } catch (WrongRegionException wre) {
>   // recoverable (file doesn't fit in region)
>   failures.add(p);
> } catch (IOException ioe) {
>   // unrecoverable (hdfs problem)
>   ioes.add(ioe);
> }
> {code}
> This should be 
> {code}
> Store store = getStore(familyName);
> if (store == null) {
> ...
> } else {
>   try {
> store.assertBulkLoadHFileOk(new Path(path));
> ...
> }
> {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-4945) NPE in HRegion.bulkLoadHFiles(...)

[Lars Hofhansl (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-4945?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13162287#comment-13162287
 ] 

Lars Hofhansl commented on HBASE-4945:
--

+1 that's exactly the patch I had in mind.

> NPE in HRegion.bulkLoadHFiles(...)
> --
>
> Key: HBASE-4945
> URL: https://issues.apache.org/jira/browse/HBASE-4945
> Project: HBase
>  Issue Type: Bug
>  Components: mapreduce
>Affects Versions: 0.92.0, 0.94.0
>Reporter: Lars Hofhansl
>Priority: Minor
> Attachments: HBASE-4945.patch
>
>
> Was playing with "completebulkload", and ran into an NPE.
> The problem is here (HRegion.bulkLoadHFiles(...)).
> {code}
> Store store = getStore(familyName);
> if (store == null) {
>   IOException ioe = new DoNotRetryIOException(
>   "No such column family " + Bytes.toStringBinary(familyName));
>   ioes.add(ioe);
>   failures.add(p);
> }
> try {
>   store.assertBulkLoadHFileOk(new Path(path));
> } catch (WrongRegionException wre) {
>   // recoverable (file doesn't fit in region)
>   failures.add(p);
> } catch (IOException ioe) {
>   // unrecoverable (hdfs problem)
>   ioes.add(ioe);
> }
> {code}
> This should be 
> {code}
> Store store = getStore(familyName);
> if (store == null) {
> ...
> } else {
>   try {
> store.assertBulkLoadHFileOk(new Path(path));
> ...
> }
> {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Work started] (HBASE-4942) HMaster is unable to start of HFile V1 is used

[honghua zhu (Work started) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-4942?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Work on HBASE-4942 started by honghua zhu.

> HMaster is unable to start of HFile V1 is used
> --
>
> Key: HBASE-4942
> URL: https://issues.apache.org/jira/browse/HBASE-4942
> Project: HBase
>  Issue Type: Bug
>  Components: io
>Affects Versions: 0.92.0
>Reporter: Ted Yu
>Assignee: honghua zhu
> Fix For: 0.92.0, 0.94.0
>
>
> This was reported by HH Zhu (zhh200...@gmail.com)
> If the following is specified in hbase-site.xml：
> {code}
> 
> hfile.format.version
> 1
> 
> {code}
> Clear the hdfs directory "hbase.rootdir" so that MasterFileSystem.bootstrap() 
> is executed.
> You would see:
> {code}
> java.lang.NullPointerException
> at 
> org.apache.hadoop.hbase.io.hfile.HFileReaderV1.close(HFileReaderV1.java:358)
> at 
> org.apache.hadoop.hbase.regionserver.StoreFile$Reader.close(StoreFile.java:1083)
> at 
> org.apache.hadoop.hbase.regionserver.StoreFile.closeReader(StoreFile.java:570)
> at org.apache.hadoop.hbase.regionserver.Store.close(Store.java:441)
> at org.apache.hadoop.hbase.regionserver.HRegion.doClose(HRegion.java:782)
> at org.apache.hadoop.hbase.regionserver.HRegion.close(HRegion.java:717)
> at org.apache.hadoop.hbase.regionserver.HRegion.close(HRegion.java:688)
> at 
> org.apache.hadoop.hbase.master.MasterFileSystem.bootstrap(MasterFileSystem.java:390)
> at 
> org.apache.hadoop.hbase.master.MasterFileSystem.checkRootDir(MasterFileSystem.java:356)
> at 
> org.apache.hadoop.hbase.master.MasterFileSystem.createInitialFileSystemLayout(MasterFileSystem.java:128)
> at 
> org.apache.hadoop.hbase.master.MasterFileSystem.(MasterFileSystem.java:113)
> at 
> org.apache.hadoop.hbase.master.HMaster.finishInitialization(HMaster.java:435)
> at org.apache.hadoop.hbase.master.HMaster.run(HMaster.java:314)
> at java.lang.Thread.run(Thread.java:619)
> {code}
> The above exception would lead to:
> {code}
> java.lang.RuntimeException: HMaster Aborted
> at 
> org.apache.hadoop.hbase.master.HMasterCommandLine.startMaster(HMasterCommandLine.java:152)
> at 
> org.apache.hadoop.hbase.master.HMasterCommandLine.run(HMasterCommandLine.java:103)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
> at 
> org.apache.hadoop.hbase.util.ServerCommandLine.doMain(ServerCommandLine.java:76)
> at org.apache.hadoop.hbase.master.HMaster.main(HMaster.java:1512)
> {code}
> In org.apache.hadoop.hbase.master.HMaster.HMaster(Configuration conf), we 
> have:
> {code}
> this.conf.setFloat(CacheConfig.HFILE_BLOCK_CACHE_SIZE_KEY, 0.0f);
> {code}
> When CacheConfig is instantiated, the following is called:
> {code}
> org.apache.hadoop.hbase.io.hfile.CacheConfig.instantiateBlockCache(Configuration
>  conf)
> {code}
> Since "hfile.block.cache.size" is 0.0, instantiateBlockCache() would return 
> null, resulting in blockCache field of CacheConfig to be null.
> When master closes Root region, 
> org.apache.hadoop.hbase.io.hfile.HFileReaderV1.close(boolean evictOnClose) 
> would be called. cacheConf.getBlockCache() returns null, leading to master 
> abort.
> The following should be called in HFileReaderV1.close(), similar to the code 
> in HFileReaderV2.close():
> {code}
> if (evictOnClose && cacheConf.isBlockCacheEnabled())
> {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-4942) HMaster is unable to start of HFile V1 is used

[honghua zhu (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-4942?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

honghua zhu updated HBASE-4942:
---

Attachment: HBase_0.92.0_HBASE-4942

HBASE-4942 patch for hbase-0.92.0

> HMaster is unable to start of HFile V1 is used
> --
>
> Key: HBASE-4942
> URL: https://issues.apache.org/jira/browse/HBASE-4942
> Project: HBase
>  Issue Type: Bug
>  Components: io
>Affects Versions: 0.92.0
>Reporter: Ted Yu
>Assignee: honghua zhu
> Fix For: 0.92.0, 0.94.0
>
> Attachments: HBase_0.92.0_HBASE-4942
>
>
> This was reported by HH Zhu (zhh200...@gmail.com)
> If the following is specified in hbase-site.xml：
> {code}
> 
> hfile.format.version
> 1
> 
> {code}
> Clear the hdfs directory "hbase.rootdir" so that MasterFileSystem.bootstrap() 
> is executed.
> You would see:
> {code}
> java.lang.NullPointerException
> at 
> org.apache.hadoop.hbase.io.hfile.HFileReaderV1.close(HFileReaderV1.java:358)
> at 
> org.apache.hadoop.hbase.regionserver.StoreFile$Reader.close(StoreFile.java:1083)
> at 
> org.apache.hadoop.hbase.regionserver.StoreFile.closeReader(StoreFile.java:570)
> at org.apache.hadoop.hbase.regionserver.Store.close(Store.java:441)
> at org.apache.hadoop.hbase.regionserver.HRegion.doClose(HRegion.java:782)
> at org.apache.hadoop.hbase.regionserver.HRegion.close(HRegion.java:717)
> at org.apache.hadoop.hbase.regionserver.HRegion.close(HRegion.java:688)
> at 
> org.apache.hadoop.hbase.master.MasterFileSystem.bootstrap(MasterFileSystem.java:390)
> at 
> org.apache.hadoop.hbase.master.MasterFileSystem.checkRootDir(MasterFileSystem.java:356)
> at 
> org.apache.hadoop.hbase.master.MasterFileSystem.createInitialFileSystemLayout(MasterFileSystem.java:128)
> at 
> org.apache.hadoop.hbase.master.MasterFileSystem.(MasterFileSystem.java:113)
> at 
> org.apache.hadoop.hbase.master.HMaster.finishInitialization(HMaster.java:435)
> at org.apache.hadoop.hbase.master.HMaster.run(HMaster.java:314)
> at java.lang.Thread.run(Thread.java:619)
> {code}
> The above exception would lead to:
> {code}
> java.lang.RuntimeException: HMaster Aborted
> at 
> org.apache.hadoop.hbase.master.HMasterCommandLine.startMaster(HMasterCommandLine.java:152)
> at 
> org.apache.hadoop.hbase.master.HMasterCommandLine.run(HMasterCommandLine.java:103)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
> at 
> org.apache.hadoop.hbase.util.ServerCommandLine.doMain(ServerCommandLine.java:76)
> at org.apache.hadoop.hbase.master.HMaster.main(HMaster.java:1512)
> {code}
> In org.apache.hadoop.hbase.master.HMaster.HMaster(Configuration conf), we 
> have:
> {code}
> this.conf.setFloat(CacheConfig.HFILE_BLOCK_CACHE_SIZE_KEY, 0.0f);
> {code}
> When CacheConfig is instantiated, the following is called:
> {code}
> org.apache.hadoop.hbase.io.hfile.CacheConfig.instantiateBlockCache(Configuration
>  conf)
> {code}
> Since "hfile.block.cache.size" is 0.0, instantiateBlockCache() would return 
> null, resulting in blockCache field of CacheConfig to be null.
> When master closes Root region, 
> org.apache.hadoop.hbase.io.hfile.HFileReaderV1.close(boolean evictOnClose) 
> would be called. cacheConf.getBlockCache() returns null, leading to master 
> abort.
> The following should be called in HFileReaderV1.close(), similar to the code 
> in HFileReaderV2.close():
> {code}
> if (evictOnClose && cacheConf.isBlockCacheEnabled())
> {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-4944) Optionally verify bulk loaded HFiles

[Hadoop QA (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-4944?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13162291#comment-13162291
 ] 

Hadoop QA commented on HBASE-4944:
--

-1 overall.  Here are the results of testing the latest attachment 
  http://issues.apache.org/jira/secure/attachment/12506018/HBASE-4944-v3.patch
  against trunk revision .

+1 @author.  The patch does not contain any @author tags.

-1 tests included.  The patch doesn't appear to include any new or modified 
tests.
Please justify why no new tests are needed for this 
patch.
Also please list what manual steps were performed to 
verify this patch.

-1 javadoc.  The javadoc tool appears to have generated -160 warning 
messages.

+1 javac.  The applied patch does not increase the total number of javac 
compiler warnings.

-1 findbugs.  The patch appears to introduce 71 new Findbugs (version 
1.3.9) warnings.

+1 release audit.  The applied patch does not increase the total number of 
release audit warnings.

 -1 core tests.  The patch failed these unit tests:
   org.apache.hadoop.hbase.mapreduce.TestTimeRangeMapRed
  
org.apache.hadoop.hbase.mapreduce.TestLoadIncrementalHFilesSplitRecovery
  org.apache.hadoop.hbase.TestDrainingServer
  org.apache.hadoop.hbase.TestFullLogReconstruction
  org.apache.hadoop.hbase.mapreduce.TestImportTsv
  org.apache.hadoop.hbase.mapreduce.TestTableMapReduce

Test results: 
https://builds.apache.org/job/PreCommit-HBASE-Build/437//testReport/
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/437//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/437//console

This message is automatically generated.

> Optionally verify bulk loaded HFiles
> 
>
> Key: HBASE-4944
> URL: https://issues.apache.org/jira/browse/HBASE-4944
> Project: HBase
>  Issue Type: Improvement
>  Components: regionserver
>Affects Versions: 0.92.0, 0.94.0, 0.90.5
>Reporter: Andrew Purtell
>Priority: Minor
> Attachments: HBASE-4944-v2.patch, HBASE-4944-v3.patch
>
>
> We rely on users to produce properly formatted HFiles for bulk import. 
> Attached patch adds an optional code path, toggled by a configuration 
> property, that verifies the HFile under consideration for import is properly 
> sorted. The default maintains the current behavior, which does not scan the 
> file for correctness.
> Patch is against trunk but can apply against all active branches.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-4942) HMaster is unable to start of HFile V1 is used

[honghua zhu (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-4942?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

honghua zhu updated HBASE-4942:
---

Attachment: HBase_0.94.0_HBASE-4942

HBASE-4942 patch for hbase-0.94.0(trunk)

> HMaster is unable to start of HFile V1 is used
> --
>
> Key: HBASE-4942
> URL: https://issues.apache.org/jira/browse/HBASE-4942
> Project: HBase
>  Issue Type: Bug
>  Components: io
>Affects Versions: 0.92.0
>Reporter: Ted Yu
>Assignee: honghua zhu
> Fix For: 0.92.0, 0.94.0
>
> Attachments: HBase_0.92.0_HBASE-4942, HBase_0.94.0_HBASE-4942
>
>
> This was reported by HH Zhu (zhh200...@gmail.com)
> If the following is specified in hbase-site.xml：
> {code}
> 
> hfile.format.version
> 1
> 
> {code}
> Clear the hdfs directory "hbase.rootdir" so that MasterFileSystem.bootstrap() 
> is executed.
> You would see:
> {code}
> java.lang.NullPointerException
> at 
> org.apache.hadoop.hbase.io.hfile.HFileReaderV1.close(HFileReaderV1.java:358)
> at 
> org.apache.hadoop.hbase.regionserver.StoreFile$Reader.close(StoreFile.java:1083)
> at 
> org.apache.hadoop.hbase.regionserver.StoreFile.closeReader(StoreFile.java:570)
> at org.apache.hadoop.hbase.regionserver.Store.close(Store.java:441)
> at org.apache.hadoop.hbase.regionserver.HRegion.doClose(HRegion.java:782)
> at org.apache.hadoop.hbase.regionserver.HRegion.close(HRegion.java:717)
> at org.apache.hadoop.hbase.regionserver.HRegion.close(HRegion.java:688)
> at 
> org.apache.hadoop.hbase.master.MasterFileSystem.bootstrap(MasterFileSystem.java:390)
> at 
> org.apache.hadoop.hbase.master.MasterFileSystem.checkRootDir(MasterFileSystem.java:356)
> at 
> org.apache.hadoop.hbase.master.MasterFileSystem.createInitialFileSystemLayout(MasterFileSystem.java:128)
> at 
> org.apache.hadoop.hbase.master.MasterFileSystem.(MasterFileSystem.java:113)
> at 
> org.apache.hadoop.hbase.master.HMaster.finishInitialization(HMaster.java:435)
> at org.apache.hadoop.hbase.master.HMaster.run(HMaster.java:314)
> at java.lang.Thread.run(Thread.java:619)
> {code}
> The above exception would lead to:
> {code}
> java.lang.RuntimeException: HMaster Aborted
> at 
> org.apache.hadoop.hbase.master.HMasterCommandLine.startMaster(HMasterCommandLine.java:152)
> at 
> org.apache.hadoop.hbase.master.HMasterCommandLine.run(HMasterCommandLine.java:103)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
> at 
> org.apache.hadoop.hbase.util.ServerCommandLine.doMain(ServerCommandLine.java:76)
> at org.apache.hadoop.hbase.master.HMaster.main(HMaster.java:1512)
> {code}
> In org.apache.hadoop.hbase.master.HMaster.HMaster(Configuration conf), we 
> have:
> {code}
> this.conf.setFloat(CacheConfig.HFILE_BLOCK_CACHE_SIZE_KEY, 0.0f);
> {code}
> When CacheConfig is instantiated, the following is called:
> {code}
> org.apache.hadoop.hbase.io.hfile.CacheConfig.instantiateBlockCache(Configuration
>  conf)
> {code}
> Since "hfile.block.cache.size" is 0.0, instantiateBlockCache() would return 
> null, resulting in blockCache field of CacheConfig to be null.
> When master closes Root region, 
> org.apache.hadoop.hbase.io.hfile.HFileReaderV1.close(boolean evictOnClose) 
> would be called. cacheConf.getBlockCache() returns null, leading to master 
> abort.
> The following should be called in HFileReaderV1.close(), similar to the code 
> in HFileReaderV2.close():
> {code}
> if (evictOnClose && cacheConf.isBlockCacheEnabled())
> {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-4942) HMaster is unable to start of HFile V1 is used

[honghua zhu (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-4942?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

honghua zhu updated HBASE-4942:
---

Status: Patch Available  (was: In Progress)

> HMaster is unable to start of HFile V1 is used
> --
>
> Key: HBASE-4942
> URL: https://issues.apache.org/jira/browse/HBASE-4942
> Project: HBase
>  Issue Type: Bug
>  Components: io
>Affects Versions: 0.92.0
>Reporter: Ted Yu
>Assignee: honghua zhu
> Fix For: 0.92.0, 0.94.0
>
> Attachments: HBase_0.92.0_HBASE-4942, HBase_0.94.0_HBASE-4942
>
>
> This was reported by HH Zhu (zhh200...@gmail.com)
> If the following is specified in hbase-site.xml：
> {code}
> 
> hfile.format.version
> 1
> 
> {code}
> Clear the hdfs directory "hbase.rootdir" so that MasterFileSystem.bootstrap() 
> is executed.
> You would see:
> {code}
> java.lang.NullPointerException
> at 
> org.apache.hadoop.hbase.io.hfile.HFileReaderV1.close(HFileReaderV1.java:358)
> at 
> org.apache.hadoop.hbase.regionserver.StoreFile$Reader.close(StoreFile.java:1083)
> at 
> org.apache.hadoop.hbase.regionserver.StoreFile.closeReader(StoreFile.java:570)
> at org.apache.hadoop.hbase.regionserver.Store.close(Store.java:441)
> at org.apache.hadoop.hbase.regionserver.HRegion.doClose(HRegion.java:782)
> at org.apache.hadoop.hbase.regionserver.HRegion.close(HRegion.java:717)
> at org.apache.hadoop.hbase.regionserver.HRegion.close(HRegion.java:688)
> at 
> org.apache.hadoop.hbase.master.MasterFileSystem.bootstrap(MasterFileSystem.java:390)
> at 
> org.apache.hadoop.hbase.master.MasterFileSystem.checkRootDir(MasterFileSystem.java:356)
> at 
> org.apache.hadoop.hbase.master.MasterFileSystem.createInitialFileSystemLayout(MasterFileSystem.java:128)
> at 
> org.apache.hadoop.hbase.master.MasterFileSystem.(MasterFileSystem.java:113)
> at 
> org.apache.hadoop.hbase.master.HMaster.finishInitialization(HMaster.java:435)
> at org.apache.hadoop.hbase.master.HMaster.run(HMaster.java:314)
> at java.lang.Thread.run(Thread.java:619)
> {code}
> The above exception would lead to:
> {code}
> java.lang.RuntimeException: HMaster Aborted
> at 
> org.apache.hadoop.hbase.master.HMasterCommandLine.startMaster(HMasterCommandLine.java:152)
> at 
> org.apache.hadoop.hbase.master.HMasterCommandLine.run(HMasterCommandLine.java:103)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
> at 
> org.apache.hadoop.hbase.util.ServerCommandLine.doMain(ServerCommandLine.java:76)
> at org.apache.hadoop.hbase.master.HMaster.main(HMaster.java:1512)
> {code}
> In org.apache.hadoop.hbase.master.HMaster.HMaster(Configuration conf), we 
> have:
> {code}
> this.conf.setFloat(CacheConfig.HFILE_BLOCK_CACHE_SIZE_KEY, 0.0f);
> {code}
> When CacheConfig is instantiated, the following is called:
> {code}
> org.apache.hadoop.hbase.io.hfile.CacheConfig.instantiateBlockCache(Configuration
>  conf)
> {code}
> Since "hfile.block.cache.size" is 0.0, instantiateBlockCache() would return 
> null, resulting in blockCache field of CacheConfig to be null.
> When master closes Root region, 
> org.apache.hadoop.hbase.io.hfile.HFileReaderV1.close(boolean evictOnClose) 
> would be called. cacheConf.getBlockCache() returns null, leading to master 
> abort.
> The following should be called in HFileReaderV1.close(), similar to the code 
> in HFileReaderV2.close():
> {code}
> if (evictOnClose && cacheConf.isBlockCacheEnabled())
> {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-4942) HMaster is unable to start of HFile V1 is used

[Andrew Purtell (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-4942?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13162296#comment-13162296
 ] 

Andrew Purtell commented on HBASE-4942:
---

+1

> HMaster is unable to start of HFile V1 is used
> --
>
> Key: HBASE-4942
> URL: https://issues.apache.org/jira/browse/HBASE-4942
> Project: HBase
>  Issue Type: Bug
>  Components: io
>Affects Versions: 0.92.0
>Reporter: Ted Yu
>Assignee: honghua zhu
> Fix For: 0.92.0, 0.94.0
>
> Attachments: HBase_0.92.0_HBASE-4942, HBase_0.94.0_HBASE-4942
>
>
> This was reported by HH Zhu (zhh200...@gmail.com)
> If the following is specified in hbase-site.xml：
> {code}
> 
> hfile.format.version
> 1
> 
> {code}
> Clear the hdfs directory "hbase.rootdir" so that MasterFileSystem.bootstrap() 
> is executed.
> You would see:
> {code}
> java.lang.NullPointerException
> at 
> org.apache.hadoop.hbase.io.hfile.HFileReaderV1.close(HFileReaderV1.java:358)
> at 
> org.apache.hadoop.hbase.regionserver.StoreFile$Reader.close(StoreFile.java:1083)
> at 
> org.apache.hadoop.hbase.regionserver.StoreFile.closeReader(StoreFile.java:570)
> at org.apache.hadoop.hbase.regionserver.Store.close(Store.java:441)
> at org.apache.hadoop.hbase.regionserver.HRegion.doClose(HRegion.java:782)
> at org.apache.hadoop.hbase.regionserver.HRegion.close(HRegion.java:717)
> at org.apache.hadoop.hbase.regionserver.HRegion.close(HRegion.java:688)
> at 
> org.apache.hadoop.hbase.master.MasterFileSystem.bootstrap(MasterFileSystem.java:390)
> at 
> org.apache.hadoop.hbase.master.MasterFileSystem.checkRootDir(MasterFileSystem.java:356)
> at 
> org.apache.hadoop.hbase.master.MasterFileSystem.createInitialFileSystemLayout(MasterFileSystem.java:128)
> at 
> org.apache.hadoop.hbase.master.MasterFileSystem.(MasterFileSystem.java:113)
> at 
> org.apache.hadoop.hbase.master.HMaster.finishInitialization(HMaster.java:435)
> at org.apache.hadoop.hbase.master.HMaster.run(HMaster.java:314)
> at java.lang.Thread.run(Thread.java:619)
> {code}
> The above exception would lead to:
> {code}
> java.lang.RuntimeException: HMaster Aborted
> at 
> org.apache.hadoop.hbase.master.HMasterCommandLine.startMaster(HMasterCommandLine.java:152)
> at 
> org.apache.hadoop.hbase.master.HMasterCommandLine.run(HMasterCommandLine.java:103)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
> at 
> org.apache.hadoop.hbase.util.ServerCommandLine.doMain(ServerCommandLine.java:76)
> at org.apache.hadoop.hbase.master.HMaster.main(HMaster.java:1512)
> {code}
> In org.apache.hadoop.hbase.master.HMaster.HMaster(Configuration conf), we 
> have:
> {code}
> this.conf.setFloat(CacheConfig.HFILE_BLOCK_CACHE_SIZE_KEY, 0.0f);
> {code}
> When CacheConfig is instantiated, the following is called:
> {code}
> org.apache.hadoop.hbase.io.hfile.CacheConfig.instantiateBlockCache(Configuration
>  conf)
> {code}
> Since "hfile.block.cache.size" is 0.0, instantiateBlockCache() would return 
> null, resulting in blockCache field of CacheConfig to be null.
> When master closes Root region, 
> org.apache.hadoop.hbase.io.hfile.HFileReaderV1.close(boolean evictOnClose) 
> would be called. cacheConf.getBlockCache() returns null, leading to master 
> abort.
> The following should be called in HFileReaderV1.close(), similar to the code 
> in HFileReaderV2.close():
> {code}
> if (evictOnClose && cacheConf.isBlockCacheEnabled())
> {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-4942) HMaster is unable to start of HFile V1 is used

[Hadoop QA (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-4942?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13162297#comment-13162297
 ] 

Hadoop QA commented on HBASE-4942:
--

-1 overall.  Here are the results of testing the latest attachment 
  
http://issues.apache.org/jira/secure/attachment/12506021/HBase_0.94.0_HBASE-4942
  against trunk revision .

+1 @author.  The patch does not contain any @author tags.

-1 tests included.  The patch doesn't appear to include any new or modified 
tests.
Please justify why no new tests are needed for this 
patch.
Also please list what manual steps were performed to 
verify this patch.

-1 javadoc.  The javadoc tool appears to have generated -160 warning 
messages.

+1 javac.  The applied patch does not increase the total number of javac 
compiler warnings.

-1 findbugs.  The patch appears to introduce 71 new Findbugs (version 
1.3.9) warnings.

+1 release audit.  The applied patch does not increase the total number of 
release audit warnings.

 -1 core tests.  The patch failed these unit tests:
   org.apache.hadoop.hbase.TestDrainingServer
  org.apache.hadoop.hbase.TestFullLogReconstruction

Test results: 
https://builds.apache.org/job/PreCommit-HBASE-Build/439//testReport/
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/439//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/439//console

This message is automatically generated.

> HMaster is unable to start of HFile V1 is used
> --
>
> Key: HBASE-4942
> URL: https://issues.apache.org/jira/browse/HBASE-4942
> Project: HBase
>  Issue Type: Bug
>  Components: io
>Affects Versions: 0.92.0
>Reporter: Ted Yu
>Assignee: honghua zhu
> Fix For: 0.92.0, 0.94.0
>
> Attachments: HBase_0.92.0_HBASE-4942, HBase_0.94.0_HBASE-4942
>
>
> This was reported by HH Zhu (zhh200...@gmail.com)
> If the following is specified in hbase-site.xml：
> {code}
> 
> hfile.format.version
> 1
> 
> {code}
> Clear the hdfs directory "hbase.rootdir" so that MasterFileSystem.bootstrap() 
> is executed.
> You would see:
> {code}
> java.lang.NullPointerException
> at 
> org.apache.hadoop.hbase.io.hfile.HFileReaderV1.close(HFileReaderV1.java:358)
> at 
> org.apache.hadoop.hbase.regionserver.StoreFile$Reader.close(StoreFile.java:1083)
> at 
> org.apache.hadoop.hbase.regionserver.StoreFile.closeReader(StoreFile.java:570)
> at org.apache.hadoop.hbase.regionserver.Store.close(Store.java:441)
> at org.apache.hadoop.hbase.regionserver.HRegion.doClose(HRegion.java:782)
> at org.apache.hadoop.hbase.regionserver.HRegion.close(HRegion.java:717)
> at org.apache.hadoop.hbase.regionserver.HRegion.close(HRegion.java:688)
> at 
> org.apache.hadoop.hbase.master.MasterFileSystem.bootstrap(MasterFileSystem.java:390)
> at 
> org.apache.hadoop.hbase.master.MasterFileSystem.checkRootDir(MasterFileSystem.java:356)
> at 
> org.apache.hadoop.hbase.master.MasterFileSystem.createInitialFileSystemLayout(MasterFileSystem.java:128)
> at 
> org.apache.hadoop.hbase.master.MasterFileSystem.(MasterFileSystem.java:113)
> at 
> org.apache.hadoop.hbase.master.HMaster.finishInitialization(HMaster.java:435)
> at org.apache.hadoop.hbase.master.HMaster.run(HMaster.java:314)
> at java.lang.Thread.run(Thread.java:619)
> {code}
> The above exception would lead to:
> {code}
> java.lang.RuntimeException: HMaster Aborted
> at 
> org.apache.hadoop.hbase.master.HMasterCommandLine.startMaster(HMasterCommandLine.java:152)
> at 
> org.apache.hadoop.hbase.master.HMasterCommandLine.run(HMasterCommandLine.java:103)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
> at 
> org.apache.hadoop.hbase.util.ServerCommandLine.doMain(ServerCommandLine.java:76)
> at org.apache.hadoop.hbase.master.HMaster.main(HMaster.java:1512)
> {code}
> In org.apache.hadoop.hbase.master.HMaster.HMaster(Configuration conf), we 
> have:
> {code}
> this.conf.setFloat(CacheConfig.HFILE_BLOCK_CACHE_SIZE_KEY, 0.0f);
> {code}
> When CacheConfig is instantiated, the following is called:
> {code}
> org.apache.hadoop.hbase.io.hfile.CacheConfig.instantiateBlockCache(Configuration
>  conf)
> {code}
> Since "hfile.block.cache.size" is 0.0, instantiateBlockCache() would return 
> null, resulting in blockCache field of CacheConfig to be null.
> When master closes Root region, 
> org.apache.hadoop.hbase.io.hfile.HFileReaderV1.close(boolean evictOnClose) 
> would be called. cacheConf.getBlockCache() returns null, leading to master 
> abort.
> The following should be called in HFileReaderV1.close(), 

[jira] [Commented] (HBASE-4704) A JRuby script for identifying active master

[Phabricator (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-4704?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13162301#comment-13162301
 ] 

Phabricator commented on HBASE-4704:


mbautin has abandoned the revision "HBASE-4704 [jira] A JRuby script for 
identifying active master".

  This was committed into trunk, closing the Differential revision.

REVISION DETAIL
  https://reviews.facebook.net/D117


> A JRuby script for identifying active master
> 
>
> Key: HBASE-4704
> URL: https://issues.apache.org/jira/browse/HBASE-4704
> Project: HBase
>  Issue Type: Improvement
>Reporter: Mikhail Bautin
>Assignee: Mikhail Bautin
>Priority: Trivial
> Fix For: 0.94.0
>
> Attachments: D117.1.patch, HBASE-4704.D117.2.patch, 
> HBASE-4704.D117.3.patch
>
>
> This simple script reads the HBase master ZK node and outputs the hostname of 
> the active master. This is needed so that operational scripts can decide 
> where the primary master is running. I am also including a one-line 
> hbase-jruby script so we can make our jruby scripts proper UNIX executables 
> by including an "#!/usr/bin/env hbase-jruby" at the top.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-4936) Cached HRegionInterface connections crash when getting UnknownHost exceptions

[Andrei Dragomir (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-4936?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrei Dragomir updated HBASE-4936:
---

Attachment: HBASE-4936-v2.patch

Re-attached with --no-prefix. Thanks Ted !

> Cached HRegionInterface connections crash when getting UnknownHost exceptions
> -
>
> Key: HBASE-4936
> URL: https://issues.apache.org/jira/browse/HBASE-4936
> Project: HBase
>  Issue Type: Bug
>  Components: regionserver
>Affects Versions: 0.92.0
>Reporter: Andrei Dragomir
>Assignee: Andrei Dragomir
> Attachments: HBASE-4936-v2.patch, HBASE-4936.patch
>
>
> This isssue is unlikely to come up in a cluster test case. However, for 
> development, the following thing happens: 
> 1. Start the HBase cluster locally, on network A (DNS A, etc)
> 2. The region locations are cached using the hostname 
> (mycomputer.company.com, 211.x.y.z - real ip)
> 3. Change network location (go home)
> 4. Start the HBase cluster locally. My hostname / ips are not different 
> (mycomputer, 192.168.0.130 - new ip)
> If the region locations have been cached using the hostname, there is an 
> UnknownHostException in CatalogTracker.getCachedConnection(ServerName sn), 
> uncaught in the catch statements. The server will crash constantly. 
> The error should be caught and not rethrown, so that the cached connection 
> expires normally. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-4945) NPE in HRegion.bulkLoadHFiles(...)

[Hadoop QA (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-4945?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13162305#comment-13162305
 ] 

Hadoop QA commented on HBASE-4945:
--

-1 overall.  Here are the results of testing the latest attachment 
  http://issues.apache.org/jira/secure/attachment/12506019/HBASE-4945.patch
  against trunk revision .

+1 @author.  The patch does not contain any @author tags.

-1 tests included.  The patch doesn't appear to include any new or modified 
tests.
Please justify why no new tests are needed for this 
patch.
Also please list what manual steps were performed to 
verify this patch.

-1 javadoc.  The javadoc tool appears to have generated -160 warning 
messages.

+1 javac.  The applied patch does not increase the total number of javac 
compiler warnings.

-1 findbugs.  The patch appears to introduce 71 new Findbugs (version 
1.3.9) warnings.

+1 release audit.  The applied patch does not increase the total number of 
release audit warnings.

 -1 core tests.  The patch failed these unit tests:
   org.apache.hadoop.hbase.client.TestAdmin
  org.apache.hadoop.hbase.client.TestInstantSchemaChange

Test results: 
https://builds.apache.org/job/PreCommit-HBASE-Build/438//testReport/
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/438//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/438//console

This message is automatically generated.

> NPE in HRegion.bulkLoadHFiles(...)
> --
>
> Key: HBASE-4945
> URL: https://issues.apache.org/jira/browse/HBASE-4945
> Project: HBase
>  Issue Type: Bug
>  Components: mapreduce
>Affects Versions: 0.92.0, 0.94.0
>Reporter: Lars Hofhansl
>Priority: Minor
> Attachments: HBASE-4945.patch
>
>
> Was playing with "completebulkload", and ran into an NPE.
> The problem is here (HRegion.bulkLoadHFiles(...)).
> {code}
> Store store = getStore(familyName);
> if (store == null) {
>   IOException ioe = new DoNotRetryIOException(
>   "No such column family " + Bytes.toStringBinary(familyName));
>   ioes.add(ioe);
>   failures.add(p);
> }
> try {
>   store.assertBulkLoadHFileOk(new Path(path));
> } catch (WrongRegionException wre) {
>   // recoverable (file doesn't fit in region)
>   failures.add(p);
> } catch (IOException ioe) {
>   // unrecoverable (hdfs problem)
>   ioes.add(ioe);
> }
> {code}
> This should be 
> {code}
> Store store = getStore(familyName);
> if (store == null) {
> ...
> } else {
>   try {
> store.assertBulkLoadHFileOk(new Path(path));
> ...
> }
> {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (HBASE-4946) HTable.coprocessorExec (and possibly coprocessorProxy) does not work with dynamically loaded coprocessors (from hdfs or local system), because the RPC system tries to des

[Andrei Dragomir (Created) (JIRA)]

HTable.coprocessorExec (and possibly coprocessorProxy) does not work with 
dynamically loaded coprocessors (from hdfs or local system), because the RPC 
system tries to deserialize an unknown class. 
-

 Key: HBASE-4946
 URL: https://issues.apache.org/jira/browse/HBASE-4946
 Project: HBase
  Issue Type: Bug
  Components: coprocessors
Affects Versions: 0.92.0
Reporter: Andrei Dragomir


Loading coprocessors jars from hdfs works fine. I load it from the shell, after 
setting the attribute, and it gets loaded:

{noformat}
INFO org.apache.hadoop.hbase.regionserver.HRegion: Setting up tabledescriptor 
config now ...
INFO org.apache.hadoop.hbase.coprocessor.CoprocessorHost: Class 
com.MyCoprocessorClass needs to be loaded from a file - 
hdfs://localhost:9000/coproc/rt-  >0.0.1-SNAPSHOT.jar.
INFO org.apache.hadoop.hbase.coprocessor.CoprocessorHost: loadInstance: 
com.MyCoprocessorClass
INFO org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost: 
RegionEnvironment createEnvironment
DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Registered protocol 
handler: region=t1,,1322572939753.6409aee1726d31f5e5671a59fe6e384f. 
protocol=com.MyCoprocessorClassProtocol
INFO org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost: Load 
coprocessor com.MyCoprocessorClass from HTD of t1 successfully.
{noformat}

The problem is that this coprocessors simply extends BaseEndpointCoprocessor, 
with a dynamic method. When calling this method from the client with 
HTable.coprocessorExec, I get errors on the HRegionServer, because the call 
cannot be deserialized from writables. 

The problem is that Exec tries to do an "early" resolve of the coprocessor 
class. The coprocessor class is loaded, but it is in the context of the 
HRegionServer / HRegion. So, the call fails:

{noformat}
2011-12-02 00:34:17,348 ERROR org.apache.hadoop.hbase.io.HbaseObjectWritable: 
Error in readFields
java.io.IOException: Protocol class com.MyCoprocessorClassProtocol not found
  at org.apache.hadoop.hbase.client.coprocessor.Exec.readFields(Exec.java:125)
  at 
org.apache.hadoop.hbase.io.HbaseObjectWritable.readObject(HbaseObjectWritable.java:575)
  at org.apache.hadoop.hbase.ipc.Invocation.readFields(Invocation.java:105)
  at 
org.apache.hadoop.hbase.ipc.HBaseServer$Connection.processData(HBaseServer.java:1237)
  at 
org.apache.hadoop.hbase.ipc.HBaseServer$Connection.readAndProcess(HBaseServer.java:1167)
  at 
org.apache.hadoop.hbase.ipc.HBaseServer$Listener.doRead(HBaseServer.java:703)
  at 
org.apache.hadoop.hbase.ipc.HBaseServer$Listener$Reader.doRunLoop(HBaseServer.java:495)
  at 
org.apache.hadoop.hbase.ipc.HBaseServer$Listener$Reader.run(HBaseServer.java:470)
  at 
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
  at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
  at java.lang.Thread.run(Thread.java:680)
Caused by: java.lang.ClassNotFoundException: com.MyCoprocessorClassProtocol
  at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
  at java.security.AccessController.doPrivileged(Native Method)
  at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
  at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
  at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
  at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
  at java.lang.Class.forName0(Native Method)
  at java.lang.Class.forName(Class.java:247)
  at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:943)
  at org.apache.hadoop.hbase.client.coprocessor.Exec.readFields(Exec.java:122)
  ... 10 more
{noformat}

Probably the correct way to fix this is to make Exec really smart, so that it 
knows all the class definitions loaded in CoprocessorHost(s).

I created a small patch that simply doesn't resolve the class definition in the 
Exec, instead passing it as string down to the HRegion layer. This layer knows 
all the definitions, and simply loads it by name. 



--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-4946) HTable.coprocessorExec (and possibly coprocessorProxy) does not work with dynamically loaded coprocessors (from hdfs or local system), because the RPC system tries to des

[Andrei Dragomir (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-4946?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrei Dragomir updated HBASE-4946:
---

Attachment: HBASE-4946.patch

There is no test for now, because TestDynamicClassLoading fails on my machie 
locally (will investigate). Also, protocolHandlerNames could be a premature 
optimization. 

> HTable.coprocessorExec (and possibly coprocessorProxy) does not work with 
> dynamically loaded coprocessors (from hdfs or local system), because the RPC 
> system tries to deserialize an unknown class. 
> -
>
> Key: HBASE-4946
> URL: https://issues.apache.org/jira/browse/HBASE-4946
> Project: HBase
>  Issue Type: Bug
>  Components: coprocessors
>Affects Versions: 0.92.0
>Reporter: Andrei Dragomir
> Attachments: HBASE-4946.patch
>
>
> Loading coprocessors jars from hdfs works fine. I load it from the shell, 
> after setting the attribute, and it gets loaded:
> {noformat}
> INFO org.apache.hadoop.hbase.regionserver.HRegion: Setting up tabledescriptor 
> config now ...
> INFO org.apache.hadoop.hbase.coprocessor.CoprocessorHost: Class 
> com.MyCoprocessorClass needs to be loaded from a file - 
> hdfs://localhost:9000/coproc/rt-  >0.0.1-SNAPSHOT.jar.
> INFO org.apache.hadoop.hbase.coprocessor.CoprocessorHost: loadInstance: 
> com.MyCoprocessorClass
> INFO org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost: 
> RegionEnvironment createEnvironment
> DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Registered protocol 
> handler: region=t1,,1322572939753.6409aee1726d31f5e5671a59fe6e384f. 
> protocol=com.MyCoprocessorClassProtocol
> INFO org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost: Load 
> coprocessor com.MyCoprocessorClass from HTD of t1 successfully.
> {noformat}
> The problem is that this coprocessors simply extends BaseEndpointCoprocessor, 
> with a dynamic method. When calling this method from the client with 
> HTable.coprocessorExec, I get errors on the HRegionServer, because the call 
> cannot be deserialized from writables. 
> The problem is that Exec tries to do an "early" resolve of the coprocessor 
> class. The coprocessor class is loaded, but it is in the context of the 
> HRegionServer / HRegion. So, the call fails:
> {noformat}
> 2011-12-02 00:34:17,348 ERROR org.apache.hadoop.hbase.io.HbaseObjectWritable: 
> Error in readFields
> java.io.IOException: Protocol class com.MyCoprocessorClassProtocol not found
>   at org.apache.hadoop.hbase.client.coprocessor.Exec.readFields(Exec.java:125)
>   at 
> org.apache.hadoop.hbase.io.HbaseObjectWritable.readObject(HbaseObjectWritable.java:575)
>   at org.apache.hadoop.hbase.ipc.Invocation.readFields(Invocation.java:105)
>   at 
> org.apache.hadoop.hbase.ipc.HBaseServer$Connection.processData(HBaseServer.java:1237)
>   at 
> org.apache.hadoop.hbase.ipc.HBaseServer$Connection.readAndProcess(HBaseServer.java:1167)
>   at 
> org.apache.hadoop.hbase.ipc.HBaseServer$Listener.doRead(HBaseServer.java:703)
>   at 
> org.apache.hadoop.hbase.ipc.HBaseServer$Listener$Reader.doRunLoop(HBaseServer.java:495)
>   at 
> org.apache.hadoop.hbase.ipc.HBaseServer$Listener$Reader.run(HBaseServer.java:470)
>   at 
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>   at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>   at java.lang.Thread.run(Thread.java:680)
> Caused by: java.lang.ClassNotFoundException: com.MyCoprocessorClassProtocol
>   at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
>   at java.security.AccessController.doPrivileged(Native Method)
>   at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
>   at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
>   at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
>   at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
>   at java.lang.Class.forName0(Native Method)
>   at java.lang.Class.forName(Class.java:247)
>   at 
> org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:943)
>   at org.apache.hadoop.hbase.client.coprocessor.Exec.readFields(Exec.java:122)
>   ... 10 more
> {noformat}
> Probably the correct way to fix this is to make Exec really smart, so that it 
> knows all the class definitions loaded in CoprocessorHost(s).
> I created a small patch that simply doesn't resolve the class definition in 
> the Exec, instead passing it as string down to the HRegion layer. This layer 
> knows all the definitions, and simply loads it by name. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://i

[jira] [Updated] (HBASE-4946) HTable.coprocessorExec (and possibly coprocessorProxy) does not work with dynamically loaded coprocessors (from hdfs or local system), because the RPC system tries to des

[Andrei Dragomir (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-4946?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrei Dragomir updated HBASE-4946:
---

Status: Patch Available  (was: Open)

> HTable.coprocessorExec (and possibly coprocessorProxy) does not work with 
> dynamically loaded coprocessors (from hdfs or local system), because the RPC 
> system tries to deserialize an unknown class. 
> -
>
> Key: HBASE-4946
> URL: https://issues.apache.org/jira/browse/HBASE-4946
> Project: HBase
>  Issue Type: Bug
>  Components: coprocessors
>Affects Versions: 0.92.0
>Reporter: Andrei Dragomir
> Attachments: HBASE-4946.patch
>
>
> Loading coprocessors jars from hdfs works fine. I load it from the shell, 
> after setting the attribute, and it gets loaded:
> {noformat}
> INFO org.apache.hadoop.hbase.regionserver.HRegion: Setting up tabledescriptor 
> config now ...
> INFO org.apache.hadoop.hbase.coprocessor.CoprocessorHost: Class 
> com.MyCoprocessorClass needs to be loaded from a file - 
> hdfs://localhost:9000/coproc/rt-  >0.0.1-SNAPSHOT.jar.
> INFO org.apache.hadoop.hbase.coprocessor.CoprocessorHost: loadInstance: 
> com.MyCoprocessorClass
> INFO org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost: 
> RegionEnvironment createEnvironment
> DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Registered protocol 
> handler: region=t1,,1322572939753.6409aee1726d31f5e5671a59fe6e384f. 
> protocol=com.MyCoprocessorClassProtocol
> INFO org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost: Load 
> coprocessor com.MyCoprocessorClass from HTD of t1 successfully.
> {noformat}
> The problem is that this coprocessors simply extends BaseEndpointCoprocessor, 
> with a dynamic method. When calling this method from the client with 
> HTable.coprocessorExec, I get errors on the HRegionServer, because the call 
> cannot be deserialized from writables. 
> The problem is that Exec tries to do an "early" resolve of the coprocessor 
> class. The coprocessor class is loaded, but it is in the context of the 
> HRegionServer / HRegion. So, the call fails:
> {noformat}
> 2011-12-02 00:34:17,348 ERROR org.apache.hadoop.hbase.io.HbaseObjectWritable: 
> Error in readFields
> java.io.IOException: Protocol class com.MyCoprocessorClassProtocol not found
>   at org.apache.hadoop.hbase.client.coprocessor.Exec.readFields(Exec.java:125)
>   at 
> org.apache.hadoop.hbase.io.HbaseObjectWritable.readObject(HbaseObjectWritable.java:575)
>   at org.apache.hadoop.hbase.ipc.Invocation.readFields(Invocation.java:105)
>   at 
> org.apache.hadoop.hbase.ipc.HBaseServer$Connection.processData(HBaseServer.java:1237)
>   at 
> org.apache.hadoop.hbase.ipc.HBaseServer$Connection.readAndProcess(HBaseServer.java:1167)
>   at 
> org.apache.hadoop.hbase.ipc.HBaseServer$Listener.doRead(HBaseServer.java:703)
>   at 
> org.apache.hadoop.hbase.ipc.HBaseServer$Listener$Reader.doRunLoop(HBaseServer.java:495)
>   at 
> org.apache.hadoop.hbase.ipc.HBaseServer$Listener$Reader.run(HBaseServer.java:470)
>   at 
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>   at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>   at java.lang.Thread.run(Thread.java:680)
> Caused by: java.lang.ClassNotFoundException: com.MyCoprocessorClassProtocol
>   at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
>   at java.security.AccessController.doPrivileged(Native Method)
>   at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
>   at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
>   at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
>   at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
>   at java.lang.Class.forName0(Native Method)
>   at java.lang.Class.forName(Class.java:247)
>   at 
> org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:943)
>   at org.apache.hadoop.hbase.client.coprocessor.Exec.readFields(Exec.java:122)
>   ... 10 more
> {noformat}
> Probably the correct way to fix this is to make Exec really smart, so that it 
> knows all the class definitions loaded in CoprocessorHost(s).
> I created a small patch that simply doesn't resolve the class definition in 
> the Exec, instead passing it as string down to the HRegion layer. This layer 
> knows all the definitions, and simply loads it by name. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-4941) [book] fixed mis-spelling of 'Configuration' in 2 places

[Hudson (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-4941?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13162308#comment-13162308
 ] 

Hudson commented on HBASE-4941:
---

Integrated in HBase-TRUNK-security #20 (See 
[https://builds.apache.org/job/HBase-TRUNK-security/20/])
hbase-4941.  Fixed mis-spelling of 'configuration' in 2 places 
(configuration.xml and hbase-default.xml)


> [book] fixed mis-spelling of 'Configuration' in 2 places
> 
>
> Key: HBASE-4941
> URL: https://issues.apache.org/jira/browse/HBASE-4941
> Project: HBase
>  Issue Type: Improvement
>Reporter: Doug Meil
>Assignee: Doug Meil
>Priority: Trivial
> Attachments: src_HBASE_4941.patch
>
>
> Changed "Configuation" to "Configuration"
> Fixed in configuration.xml
> Fixed in hbase-default.xml  (in the 
> "hbase.mapreduce.hfileoutputformat.blocksize" description)
> Thanks for Otis for catching this.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-4943) [book] variety of updates to book 12-3-2011

[Hudson (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-4943?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13162309#comment-13162309
 ] 

Hudson commented on HBASE-4943:
---

Integrated in HBase-TRUNK-security #20 (See 
[https://builds.apache.org/job/HBase-TRUNK-security/20/])
hbase-4943.  book updates (more FAQ, add to appendix for other resources)


> [book] variety of updates to book 12-3-2011
> ---
>
> Key: HBASE-4943
> URL: https://issues.apache.org/jira/browse/HBASE-4943
> Project: HBase
>  Issue Type: Improvement
>Reporter: Doug Meil
>Assignee: Doug Meil
>Priority: Minor
> Attachments: docbkx_HBASE_4943.patch
>
>
> book.xml
> * added FAQ entry for "hbase in action" (link to newly created appendix for 
> other resources)
> * added new appendix entry for other resources (videos, presentations, etc.)
> * moved a build FAQ into the Development chapter.
> troubleshooting.xml
> * added IRC into the 'resources' section

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-4946) HTable.coprocessorExec (and possibly coprocessorProxy) does not work with dynamically loaded coprocessors (from hdfs or local system), because the RPC system tries to d

[Hadoop QA (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-4946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13162311#comment-13162311
 ] 

Hadoop QA commented on HBASE-4946:
--

-1 overall.  Here are the results of testing the latest attachment 
  http://issues.apache.org/jira/secure/attachment/12506025/HBASE-4946.patch
  against trunk revision .

+1 @author.  The patch does not contain any @author tags.

-1 tests included.  The patch doesn't appear to include any new or modified 
tests.
Please justify why no new tests are needed for this 
patch.
Also please list what manual steps were performed to 
verify this patch.

-1 javadoc.  The javadoc tool appears to have generated -160 warning 
messages.

+1 javac.  The applied patch does not increase the total number of javac 
compiler warnings.

-1 findbugs.  The patch appears to introduce 72 new Findbugs (version 
1.3.9) warnings.

+1 release audit.  The applied patch does not increase the total number of 
release audit warnings.

 -1 core tests.  The patch failed these unit tests:
   org.apache.hadoop.hbase.TestDrainingServer
  org.apache.hadoop.hbase.TestFullLogReconstruction

Test results: 
https://builds.apache.org/job/PreCommit-HBASE-Build/441//testReport/
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/441//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/441//console

This message is automatically generated.

> HTable.coprocessorExec (and possibly coprocessorProxy) does not work with 
> dynamically loaded coprocessors (from hdfs or local system), because the RPC 
> system tries to deserialize an unknown class. 
> -
>
> Key: HBASE-4946
> URL: https://issues.apache.org/jira/browse/HBASE-4946
> Project: HBase
>  Issue Type: Bug
>  Components: coprocessors
>Affects Versions: 0.92.0
>Reporter: Andrei Dragomir
> Attachments: HBASE-4946.patch
>
>
> Loading coprocessors jars from hdfs works fine. I load it from the shell, 
> after setting the attribute, and it gets loaded:
> {noformat}
> INFO org.apache.hadoop.hbase.regionserver.HRegion: Setting up tabledescriptor 
> config now ...
> INFO org.apache.hadoop.hbase.coprocessor.CoprocessorHost: Class 
> com.MyCoprocessorClass needs to be loaded from a file - 
> hdfs://localhost:9000/coproc/rt-  >0.0.1-SNAPSHOT.jar.
> INFO org.apache.hadoop.hbase.coprocessor.CoprocessorHost: loadInstance: 
> com.MyCoprocessorClass
> INFO org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost: 
> RegionEnvironment createEnvironment
> DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Registered protocol 
> handler: region=t1,,1322572939753.6409aee1726d31f5e5671a59fe6e384f. 
> protocol=com.MyCoprocessorClassProtocol
> INFO org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost: Load 
> coprocessor com.MyCoprocessorClass from HTD of t1 successfully.
> {noformat}
> The problem is that this coprocessors simply extends BaseEndpointCoprocessor, 
> with a dynamic method. When calling this method from the client with 
> HTable.coprocessorExec, I get errors on the HRegionServer, because the call 
> cannot be deserialized from writables. 
> The problem is that Exec tries to do an "early" resolve of the coprocessor 
> class. The coprocessor class is loaded, but it is in the context of the 
> HRegionServer / HRegion. So, the call fails:
> {noformat}
> 2011-12-02 00:34:17,348 ERROR org.apache.hadoop.hbase.io.HbaseObjectWritable: 
> Error in readFields
> java.io.IOException: Protocol class com.MyCoprocessorClassProtocol not found
>   at org.apache.hadoop.hbase.client.coprocessor.Exec.readFields(Exec.java:125)
>   at 
> org.apache.hadoop.hbase.io.HbaseObjectWritable.readObject(HbaseObjectWritable.java:575)
>   at org.apache.hadoop.hbase.ipc.Invocation.readFields(Invocation.java:105)
>   at 
> org.apache.hadoop.hbase.ipc.HBaseServer$Connection.processData(HBaseServer.java:1237)
>   at 
> org.apache.hadoop.hbase.ipc.HBaseServer$Connection.readAndProcess(HBaseServer.java:1167)
>   at 
> org.apache.hadoop.hbase.ipc.HBaseServer$Listener.doRead(HBaseServer.java:703)
>   at 
> org.apache.hadoop.hbase.ipc.HBaseServer$Listener$Reader.doRunLoop(HBaseServer.java:495)
>   at 
> org.apache.hadoop.hbase.ipc.HBaseServer$Listener$Reader.run(HBaseServer.java:470)
>   at 
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>   at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>   at java.lang.Thread.run(Thread.java:680