date:20140209


[ 
https://issues.apache.org/jira/browse/HBASE-10487?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13895963#comment-13895963
 ] 

Lars Hofhansl commented on HBASE-10487:
---

Looks good. +1

Is that a common scenario, though. Append is not very useful (compared to just 
Put) when you don't update an existing value.


 Avoid allocating new KeyValue and according bytes-copying for appended kvs 
 which don't have existing(old) values
 

 Key: HBASE-10487
 URL: https://issues.apache.org/jira/browse/HBASE-10487
 Project: HBase
  Issue Type: Improvement
  Components: regionserver
Reporter: Feng Honghua
Assignee: Feng Honghua
 Attachments: HBASE-10487-trunk_v1.patch


 in HRegion.append, new KeyValues will be allocated and do according 
 bytes-copying no matter whether there are existing kv for the appended cells, 
 we can improve here by avoiding the allocating of new KeyValue and according 
 bytes-copying for kv which don't have existing(old) values by reusing the 
 passed-in kv and only updating its timestamp to 'now'(its original timestamp 
 is latest, so can be updated)



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

[jira] [Commented] (HBASE-10485) PrefixFilter#filterKeyValue() should perform filtering on row key


[ 
https://issues.apache.org/jira/browse/HBASE-10485?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13895980#comment-13895980
 ] 

Hudson commented on HBASE-10485:


SUCCESS: Integrated in HBase-TRUNK #4901 (See 
[https://builds.apache.org/job/HBase-TRUNK/4901/])
HBASE-10485 PrefixFilter#filterKeyValue() should perform filtering on row key 
(tedyu: rev 1566307)
* 
/hbase/trunk/hbase-client/src/main/java/org/apache/hadoop/hbase/filter/PrefixFilter.java
* 
/hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/filter/TestFilterListAdditional.java


 PrefixFilter#filterKeyValue() should perform filtering on row key
 -

 Key: HBASE-10485
 URL: https://issues.apache.org/jira/browse/HBASE-10485
 Project: HBase
  Issue Type: Bug
Reporter: Ted Yu
Assignee: Ted Yu
 Attachments: 10485-0.94.txt, 10485-v1.txt


 Niels reported an issue under the thread 'Trouble writing custom filter for 
 use in FilterList' where his custom filter used in FilterList along with 
 PrefixFilter produced an unexpected results.
 His test can be found here:
 https://github.com/nielsbasjes/HBase-filter-problem
 This is due to PrefixFilter#filterKeyValue() using 
 FilterBase#filterKeyValue() which returns ReturnCode.INCLUDE
 When FilterList.Operator.MUST_PASS_ONE is specified, 
 FilterList#filterKeyValue() would return ReturnCode.INCLUDE even when row key 
 prefix doesn't match meanwhile the other filter's filterKeyValue() returns 
 ReturnCode.NEXT_COL



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

[jira] [Commented] (HBASE-10487) Avoid allocating new KeyValue and according bytes-copying for appended kvs which don't have existing(old) values


[ 
https://issues.apache.org/jira/browse/HBASE-10487?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13896003#comment-13896003
 ] 

Andrew Purtell commented on HBASE-10487:


+1

 Avoid allocating new KeyValue and according bytes-copying for appended kvs 
 which don't have existing(old) values
 

 Key: HBASE-10487
 URL: https://issues.apache.org/jira/browse/HBASE-10487
 Project: HBase
  Issue Type: Improvement
  Components: regionserver
Reporter: Feng Honghua
Assignee: Feng Honghua
 Attachments: HBASE-10487-trunk_v1.patch


 in HRegion.append, new KeyValues will be allocated and do according 
 bytes-copying no matter whether there are existing kv for the appended cells, 
 we can improve here by avoiding the allocating of new KeyValue and according 
 bytes-copying for kv which don't have existing(old) values by reusing the 
 passed-in kv and only updating its timestamp to 'now'(its original timestamp 
 is latest, so can be updated)



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

[jira] [Commented] (HBASE-10485) PrefixFilter#filterKeyValue() should perform filtering on row key


[ 
https://issues.apache.org/jira/browse/HBASE-10485?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13896004#comment-13896004
 ] 

Andrew Purtell commented on HBASE-10485:


+1 for 0.98 thanks Ted

 PrefixFilter#filterKeyValue() should perform filtering on row key
 -

 Key: HBASE-10485
 URL: https://issues.apache.org/jira/browse/HBASE-10485
 Project: HBase
  Issue Type: Bug
Reporter: Ted Yu
Assignee: Ted Yu
 Attachments: 10485-0.94.txt, 10485-v1.txt


 Niels reported an issue under the thread 'Trouble writing custom filter for 
 use in FilterList' where his custom filter used in FilterList along with 
 PrefixFilter produced an unexpected results.
 His test can be found here:
 https://github.com/nielsbasjes/HBase-filter-problem
 This is due to PrefixFilter#filterKeyValue() using 
 FilterBase#filterKeyValue() which returns ReturnCode.INCLUDE
 When FilterList.Operator.MUST_PASS_ONE is specified, 
 FilterList#filterKeyValue() would return ReturnCode.INCLUDE even when row key 
 prefix doesn't match meanwhile the other filter's filterKeyValue() returns 
 ReturnCode.NEXT_COL



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

[jira] [Commented] (HBASE-10389) Add namespace help info in table related shell commands


[ 
https://issues.apache.org/jira/browse/HBASE-10389?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13896006#comment-13896006
 ] 

Andrew Purtell commented on HBASE-10389:


+1 for 0.98 thanks Enis

 Add namespace help info in table related shell commands
 ---

 Key: HBASE-10389
 URL: https://issues.apache.org/jira/browse/HBASE-10389
 Project: HBase
  Issue Type: Improvement
  Components: shell
Affects Versions: 0.96.0, 0.96.1
Reporter: Jerry He
Assignee: Jerry He
 Attachments: HBASE-10389-trunk.patch


 Currently in the help info of table related shell command, we don't mention 
 or give namespace as part of the table name.  
 For example, to create table:
 {code}
 hbase(main):001:0 help 'create'
 Creates a table. Pass a table name, and a set of column family
 specifications (at least one), and, optionally, table configuration.
 Column specification can be a simple string (name), or a dictionary
 (dictionaries are described below in main help output), necessarily
 including NAME attribute.
 Examples:
   hbase create 't1', {NAME = 'f1', VERSIONS = 5}
   hbase create 't1', {NAME = 'f1'}, {NAME = 'f2'}, {NAME = 'f3'}
   hbase # The above in shorthand would be the following:
   hbase create 't1', 'f1', 'f2', 'f3'
   hbase create 't1', {NAME = 'f1', VERSIONS = 1, TTL = 2592000, 
 BLOCKCACHE = true}
   hbase create 't1', {NAME = 'f1', CONFIGURATION = 
 {'hbase.hstore.blockingStoreFiles' = '10'}}
 Table configuration options can be put at the end.
 Examples:
   hbase create 't1', 'f1', SPLITS = ['10', '20', '30', '40']
   hbase create 't1', 'f1', SPLITS_FILE = 'splits.txt', OWNER = 'johndoe'
   hbase create 't1', {NAME = 'f1', VERSIONS = 5}, METADATA = { 'mykey' = 
 'myvalue' }
   hbase # Optionally pre-split the table into NUMREGIONS, using
   hbase # SPLITALGO (HexStringSplit, UniformSplit or classname)
   hbase create 't1', 'f1', {NUMREGIONS = 15, SPLITALGO = 'HexStringSplit'}
   hbase create 't1', 'f1', {NUMREGIONS = 15, SPLITALGO = 'HexStringSplit', 
 CONFIGURATION = {'hbase.hregion.scan.loadColumnFamiliesOnDemand' = 'true'}}
 You can also keep around a reference to the created table:
   hbase t1 = create 't1', 'f1'
 Which gives you a reference to the table named 't1', on which you can then
 call methods.
 {code}
 We should document the usage of namespace in these commands.
 For example:
 #namespace=foo and table qualifier=bar
 create 'foo:bar', 'fam'
 #namespace=default and table qualifier=bar
 create 'bar', 'fam'



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

[jira] [Commented] (HBASE-10486) ProtobufUtil Append Increment deserialization lost cell level timestamp


[ 
https://issues.apache.org/jira/browse/HBASE-10486?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13896007#comment-13896007
 ] 

Andrew Purtell commented on HBASE-10486:


+1 for 0.98

 ProtobufUtil Append  Increment deserialization lost cell level timestamp
 -

 Key: HBASE-10486
 URL: https://issues.apache.org/jira/browse/HBASE-10486
 Project: HBase
  Issue Type: Bug
Affects Versions: 0.98.0, 0.96.1
Reporter: Jeffrey Zhong
Assignee: Jeffrey Zhong
 Fix For: 0.98.1

 Attachments: hbase-10486-v2.patch, hbase-10486.patch


 When we deserialized Append  Increment, we uses wrong timestamp value during 
 deserialization in trunk  0.98 code and discard the value in 0.96 code base. 



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

[jira] [Updated] (HBASE-10485) PrefixFilter#filterKeyValue() should perform filtering on row key


 [ 
https://issues.apache.org/jira/browse/HBASE-10485?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ted Yu updated HBASE-10485:
---

Fix Version/s: 0.99.0
   0.98.0
 Hadoop Flags: Reviewed

 PrefixFilter#filterKeyValue() should perform filtering on row key
 -

 Key: HBASE-10485
 URL: https://issues.apache.org/jira/browse/HBASE-10485
 Project: HBase
  Issue Type: Bug
Reporter: Ted Yu
Assignee: Ted Yu
 Fix For: 0.98.0, 0.99.0

 Attachments: 10485-0.94.txt, 10485-v1.txt


 Niels reported an issue under the thread 'Trouble writing custom filter for 
 use in FilterList' where his custom filter used in FilterList along with 
 PrefixFilter produced an unexpected results.
 His test can be found here:
 https://github.com/nielsbasjes/HBase-filter-problem
 This is due to PrefixFilter#filterKeyValue() using 
 FilterBase#filterKeyValue() which returns ReturnCode.INCLUDE
 When FilterList.Operator.MUST_PASS_ONE is specified, 
 FilterList#filterKeyValue() would return ReturnCode.INCLUDE even when row key 
 prefix doesn't match meanwhile the other filter's filterKeyValue() returns 
 ReturnCode.NEXT_COL



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

[jira] [Created] (HBASE-10488) 'mvn site' is broken due to org.apache.jasper.JspC not found

Ted Yu created HBASE-10488:
--

 Summary: 'mvn site' is broken due to org.apache.jasper.JspC not 
found
 Key: HBASE-10488
 URL: https://issues.apache.org/jira/browse/HBASE-10488
 Project: HBase
  Issue Type: Bug
Reporter: Ted Yu


From 
https://builds.apache.org/job/PreCommit-HBASE-Build/8642/artifact/trunk/patchprocess/patchSiteOutput.txt
 :
{code}
[WARNING] The POM for 
org.apache.hbase:hbase-server:jar:0.99.0-20140127.165302-1 is invalid, 
transitive dependencies (if any) will not be available, enable debug logging 
for more details
[WARNING] The POM for 
org.apache.hbase:hbase-server:jar:tests:0.99.0-20140127.165302-1 is invalid, 
transitive dependencies (if any) will not be available, enable debug logging 
for more details
...
[ERROR] Failed to execute goal 
org.apache.maven.plugins:maven-site-plugin:3.3:site (default-site) on project 
hbase: failed to get report for org.apache.maven.plugins:maven-javadoc-plugin: 
Failed to execute goal org.apache.maven.plugins:maven-antrun-plugin:1.6:run 
(generate) on project hbase-thrift: An Ant BuildException has occured: taskdef 
class org.apache.jasper.JspC cannot be found
{code}



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

[jira] [Commented] (HBASE-10485) PrefixFilter#filterKeyValue() should perform filtering on row key


[ 
https://issues.apache.org/jira/browse/HBASE-10485?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13896041#comment-13896041
 ] 

Lars Hofhansl commented on HBASE-10485:
---

Why not 0.94 and 0.96?

 PrefixFilter#filterKeyValue() should perform filtering on row key
 -

 Key: HBASE-10485
 URL: https://issues.apache.org/jira/browse/HBASE-10485
 Project: HBase
  Issue Type: Bug
Reporter: Ted Yu
Assignee: Ted Yu
 Fix For: 0.98.0, 0.99.0

 Attachments: 10485-0.94.txt, 10485-v1.txt


 Niels reported an issue under the thread 'Trouble writing custom filter for 
 use in FilterList' where his custom filter used in FilterList along with 
 PrefixFilter produced an unexpected results.
 His test can be found here:
 https://github.com/nielsbasjes/HBase-filter-problem
 This is due to PrefixFilter#filterKeyValue() using 
 FilterBase#filterKeyValue() which returns ReturnCode.INCLUDE
 When FilterList.Operator.MUST_PASS_ONE is specified, 
 FilterList#filterKeyValue() would return ReturnCode.INCLUDE even when row key 
 prefix doesn't match meanwhile the other filter's filterKeyValue() returns 
 ReturnCode.NEXT_COL



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

[jira] [Updated] (HBASE-10363) [0.94] TestInputSampler and TestInputSamplerTool fail under hadoop 2.0/23 profiles.


 [ 
https://issues.apache.org/jira/browse/HBASE-10363?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lars Hofhansl updated HBASE-10363:
--

Hadoop Flags: Reviewed

 [0.94] TestInputSampler and TestInputSamplerTool fail under hadoop 2.0/23 
 profiles.
 ---

 Key: HBASE-10363
 URL: https://issues.apache.org/jira/browse/HBASE-10363
 Project: HBase
  Issue Type: Bug
Affects Versions: 0.94.15
Reporter: Jonathan Hsieh
Assignee: Lars Hofhansl
Priority: Critical
 Fix For: 0.94.17

 Attachments: 10363.txt


 From tip of 0.94 and from 0.94.15.
 {code}
 jon@swoop:~/proj/hbase-0.94$ mvn clean test -Dhadoop.profile=2.0 
 -Dtest=TestInputSampler,TestInputSamplerTool -PlocalTests
 ...
 Running org.apache.hadoop.hbase.mapreduce.hadoopbackport.TestInputSamplerTool
 Tests run: 4, Failures: 0, Errors: 3, Skipped: 0, Time elapsed: 3.718 sec  
 FAILURE!
 Running org.apache.hadoop.hbase.mapreduce.hadoopbackport.TestInputSampler
 Tests run: 2, Failures: 0, Errors: 2, Skipped: 0, Time elapsed: 0.666 sec  
 FAILURE!
 Results :
 Tests in error: 
   
 testSplitInterval(org.apache.hadoop.hbase.mapreduce.hadoopbackport.TestInputSamplerTool):
  Failed getting constructor
   
 testSplitRamdom(org.apache.hadoop.hbase.mapreduce.hadoopbackport.TestInputSamplerTool):
  Failed getting constructor
   
 testSplitSample(org.apache.hadoop.hbase.mapreduce.hadoopbackport.TestInputSamplerTool):
  Failed getting constructor
   
 testSplitSampler(org.apache.hadoop.hbase.mapreduce.hadoopbackport.TestInputSampler):
  Failed getting constructor
   
 testIntervalSampler(org.apache.hadoop.hbase.mapreduce.hadoopbackport.TestInputSampler):
  Failed getting constructor
 Tests run: 6, Failures: 0, Errors: 5, Skipped: 0
 {code}



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

[jira] [Created] (HBASE-10489) TestImportExport fails in 0.94 with Hadoop2

Lars Hofhansl created HBASE-10489:
-

 Summary: TestImportExport fails in 0.94 with Hadoop2
 Key: HBASE-10489
 URL: https://issues.apache.org/jira/browse/HBASE-10489
 Project: HBase
  Issue Type: Bug
Reporter: Lars Hofhansl
 Fix For: 0.94.17


With HBASE-10363 fixed, we're now seeing other M/R tests failing.
TestImportExport is one of them.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

[jira] [Updated] (HBASE-10363) [0.94] TestInputSampler and TestInputSamplerTool fail under hadoop 2.0/23 profiles.


 [ 
https://issues.apache.org/jira/browse/HBASE-10363?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lars Hofhansl updated HBASE-10363:
--

Hadoop Flags:   (was: Reviewed)

 [0.94] TestInputSampler and TestInputSamplerTool fail under hadoop 2.0/23 
 profiles.
 ---

 Key: HBASE-10363
 URL: https://issues.apache.org/jira/browse/HBASE-10363
 Project: HBase
  Issue Type: Bug
Affects Versions: 0.94.15
Reporter: Jonathan Hsieh
Assignee: Lars Hofhansl
Priority: Critical
 Fix For: 0.94.17

 Attachments: 10363.txt


 From tip of 0.94 and from 0.94.15.
 {code}
 jon@swoop:~/proj/hbase-0.94$ mvn clean test -Dhadoop.profile=2.0 
 -Dtest=TestInputSampler,TestInputSamplerTool -PlocalTests
 ...
 Running org.apache.hadoop.hbase.mapreduce.hadoopbackport.TestInputSamplerTool
 Tests run: 4, Failures: 0, Errors: 3, Skipped: 0, Time elapsed: 3.718 sec  
 FAILURE!
 Running org.apache.hadoop.hbase.mapreduce.hadoopbackport.TestInputSampler
 Tests run: 2, Failures: 0, Errors: 2, Skipped: 0, Time elapsed: 0.666 sec  
 FAILURE!
 Results :
 Tests in error: 
   
 testSplitInterval(org.apache.hadoop.hbase.mapreduce.hadoopbackport.TestInputSamplerTool):
  Failed getting constructor
   
 testSplitRamdom(org.apache.hadoop.hbase.mapreduce.hadoopbackport.TestInputSamplerTool):
  Failed getting constructor
   
 testSplitSample(org.apache.hadoop.hbase.mapreduce.hadoopbackport.TestInputSamplerTool):
  Failed getting constructor
   
 testSplitSampler(org.apache.hadoop.hbase.mapreduce.hadoopbackport.TestInputSampler):
  Failed getting constructor
   
 testIntervalSampler(org.apache.hadoop.hbase.mapreduce.hadoopbackport.TestInputSampler):
  Failed getting constructor
 Tests run: 6, Failures: 0, Errors: 5, Skipped: 0
 {code}



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

[jira] [Assigned] (HBASE-10489) TestImportExport fails in 0.94 with Hadoop2


 [ 
https://issues.apache.org/jira/browse/HBASE-10489?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lars Hofhansl reassigned HBASE-10489:
-

Assignee: Lars Hofhansl

 TestImportExport fails in 0.94 with Hadoop2
 ---

 Key: HBASE-10489
 URL: https://issues.apache.org/jira/browse/HBASE-10489
 Project: HBase
  Issue Type: Bug
  Components: test
Reporter: Lars Hofhansl
Assignee: Lars Hofhansl
 Fix For: 0.94.17


 With HBASE-10363 fixed, we're now seeing other M/R tests failing.
 TestImportExport is one of them.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

[jira] [Updated] (HBASE-10489) TestImportExport fails in 0.94 with Hadoop2


 [ 
https://issues.apache.org/jira/browse/HBASE-10489?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lars Hofhansl updated HBASE-10489:
--

Component/s: test

 TestImportExport fails in 0.94 with Hadoop2
 ---

 Key: HBASE-10489
 URL: https://issues.apache.org/jira/browse/HBASE-10489
 Project: HBase
  Issue Type: Bug
  Components: test
Reporter: Lars Hofhansl
 Fix For: 0.94.17


 With HBASE-10363 fixed, we're now seeing other M/R tests failing.
 TestImportExport is one of them.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

[jira] [Updated] (HBASE-10489) TestImportExport fails in 0.94 with Hadoop2


 [ 
https://issues.apache.org/jira/browse/HBASE-10489?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lars Hofhansl updated HBASE-10489:
--

Attachment: 10489.txt

Here's a simple fix. The issue was to make it work for both Hadoop1 and Hadoop2.
The test used to get the configuration from the cluster. The various 
Mini*Clusters mess around with the configuration enough to have this fail with 
Hadoop2.
Hadoop1 failed with UTIL configuration. After diff'ing the configurations I 
found the issue was that mapred.job.tracker was set to localhost:40462 un UTIL.
Forcing that to local makes it work again for Hadoop1.

So in total this patch works Hadoop1 and Hadoop2.

Planning to commit tomorrow (it's an uncontroversial test change, I think)


 TestImportExport fails in 0.94 with Hadoop2
 ---

 Key: HBASE-10489
 URL: https://issues.apache.org/jira/browse/HBASE-10489
 Project: HBase
  Issue Type: Bug
  Components: test
Reporter: Lars Hofhansl
Assignee: Lars Hofhansl
 Fix For: 0.94.17

 Attachments: 10489.txt


 With HBASE-10363 fixed, we're now seeing other M/R tests failing.
 TestImportExport is one of them.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

[jira] [Commented] (HBASE-10485) PrefixFilter#filterKeyValue() should perform filtering on row key


[ 
https://issues.apache.org/jira/browse/HBASE-10485?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13896084#comment-13896084
 ] 

Hudson commented on HBASE-10485:


FAILURE: Integrated in HBase-0.98-on-Hadoop-1.1 #131 (See 
[https://builds.apache.org/job/HBase-0.98-on-Hadoop-1.1/131/])
HBASE-10485 PrefixFilter#filterKeyValue() should perform filtering on row key 
(tedyu: rev 1566386)
* 
/hbase/branches/0.98/hbase-client/src/main/java/org/apache/hadoop/hbase/filter/PrefixFilter.java
* 
/hbase/branches/0.98/hbase-server/src/test/java/org/apache/hadoop/hbase/filter/TestFilterListAdditional.java


 PrefixFilter#filterKeyValue() should perform filtering on row key
 -

 Key: HBASE-10485
 URL: https://issues.apache.org/jira/browse/HBASE-10485
 Project: HBase
  Issue Type: Bug
Reporter: Ted Yu
Assignee: Ted Yu
 Fix For: 0.98.0, 0.99.0

 Attachments: 10485-0.94.txt, 10485-v1.txt


 Niels reported an issue under the thread 'Trouble writing custom filter for 
 use in FilterList' where his custom filter used in FilterList along with 
 PrefixFilter produced an unexpected results.
 His test can be found here:
 https://github.com/nielsbasjes/HBase-filter-problem
 This is due to PrefixFilter#filterKeyValue() using 
 FilterBase#filterKeyValue() which returns ReturnCode.INCLUDE
 When FilterList.Operator.MUST_PASS_ONE is specified, 
 FilterList#filterKeyValue() would return ReturnCode.INCLUDE even when row key 
 prefix doesn't match meanwhile the other filter's filterKeyValue() returns 
 ReturnCode.NEXT_COL



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

[jira] [Commented] (HBASE-10489) TestImportExport fails in 0.94 with Hadoop2


[ 
https://issues.apache.org/jira/browse/HBASE-10489?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13896111#comment-13896111
 ] 

stack commented on HBASE-10489:
---

lgtm

 TestImportExport fails in 0.94 with Hadoop2
 ---

 Key: HBASE-10489
 URL: https://issues.apache.org/jira/browse/HBASE-10489
 Project: HBase
  Issue Type: Bug
  Components: test
Reporter: Lars Hofhansl
Assignee: Lars Hofhansl
 Fix For: 0.94.17

 Attachments: 10489.txt


 With HBASE-10363 fixed, we're now seeing other M/R tests failing.
 TestImportExport is one of them.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

[jira] [Commented] (HBASE-10489) TestImportExport fails in 0.94 with Hadoop2


[ 
https://issues.apache.org/jira/browse/HBASE-10489?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13896126#comment-13896126
 ] 

Lars Hofhansl commented on HBASE-10489:
---

Thanks. Lemme commit then.

 TestImportExport fails in 0.94 with Hadoop2
 ---

 Key: HBASE-10489
 URL: https://issues.apache.org/jira/browse/HBASE-10489
 Project: HBase
  Issue Type: Bug
  Components: test
Reporter: Lars Hofhansl
Assignee: Lars Hofhansl
 Fix For: 0.94.17

 Attachments: 10489.txt


 With HBASE-10363 fixed, we're now seeing other M/R tests failing.
 TestImportExport is one of them.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

[jira] [Resolved] (HBASE-10489) TestImportExport fails in 0.94 with Hadoop2


 [ 
https://issues.apache.org/jira/browse/HBASE-10489?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lars Hofhansl resolved HBASE-10489.
---

  Resolution: Fixed
Hadoop Flags: Reviewed

Committed to 0.94 only. Thanks for looking [~stack]

 TestImportExport fails in 0.94 with Hadoop2
 ---

 Key: HBASE-10489
 URL: https://issues.apache.org/jira/browse/HBASE-10489
 Project: HBase
  Issue Type: Bug
  Components: test
Reporter: Lars Hofhansl
Assignee: Lars Hofhansl
 Fix For: 0.94.17

 Attachments: 10489.txt


 With HBASE-10363 fixed, we're now seeing other M/R tests failing.
 TestImportExport is one of them.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

[jira] [Updated] (HBASE-10436) hbase 0.96+ jmx does not have regionserver info any more.

2014-02-09 Thread Jonathan Hsieh (JIRA)

[
https://issues.apache.org/jira/browse/HBASE-10436?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Jonathan Hsieh updated HBASE-10436:
---

Attachment: hbase-10436.notbean.patch

[~eclark] still waiting for responses here.

I've atttached another version, hbase-10436.notbean.patch, includes a version
that goes through the newer metrics system. It takes the exact same locks that
the current numRegionServers metric takes, and presents data in a less friendly
manner than the original patches.

hbase 0.96+ jmx does not have regionserver info any more.
-

Key: HBASE-10436
URL: https://issues.apache.org/jira/browse/HBASE-10436
Project: HBase
Issue Type: Bug
Affects Versions: 0.98.0, 0.96.0, 0.99.0
Reporter: Jonathan Hsieh
Assignee: Jonathan Hsieh
Priority: Critical
Attachments: hbase-10436.notbean.patch, hbase-10436.patch,
hbase-10436.v2.patch

HBase 0.96's refactored jmx beans do not contain the master's list of dead
region servers and live regionservers with load info. HBase 0.94 did (though
in a single monolithic blob).
This JMX interface should be considered as much of an API as the the normal
wire or java api. Dropping values from this was done without deprecation and
the removal of this information is a functional regression.
We should provide the information in the 0.96+ JMX. HBase 0.94 had a
monolithic JMX blob (hadoop:service=Master,name=Master) that contained a
lot of information, including the regionserver list and the cached
regionserver load for each region found on the master webpage. 0.96+
refactored jmx this into several jmx beans which could be selectively
retrieved. These include:
* hadoop:service=HBase,name=Master,sub=AssignmentManager
* hadoop:service=HBase,name=Master,sub=Balancer
* hadoop:service=HBase,name=Master,sub=Server
* hadoop:service=HBase,name=Master,sub=FileSystem
Specifically the (Hadoop:service=HBase,name=Master,sub=Server) listing that
used to contain regionservers and deadregionservers in jmx were replaced in
with numRegionServers and numDeadRegionservers which only contain counts.
I propose just adding another mbean called RegionServers under the bean:
hadoop:service=HBase,name=Master,sub=RegionServers

--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

[jira] [Commented] (HBASE-10485) PrefixFilter#filterKeyValue() should perform filtering on row key


[ 
https://issues.apache.org/jira/browse/HBASE-10485?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13896138#comment-13896138
 ] 

Hudson commented on HBASE-10485:


FAILURE: Integrated in HBase-TRUNK-on-Hadoop-1.1 #85 (See 
[https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-1.1/85/])
HBASE-10485 PrefixFilter#filterKeyValue() should perform filtering on row key 
(tedyu: rev 1566307)
* 
/hbase/trunk/hbase-client/src/main/java/org/apache/hadoop/hbase/filter/PrefixFilter.java
* 
/hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/filter/TestFilterListAdditional.java


 PrefixFilter#filterKeyValue() should perform filtering on row key
 -

 Key: HBASE-10485
 URL: https://issues.apache.org/jira/browse/HBASE-10485
 Project: HBase
  Issue Type: Bug
Reporter: Ted Yu
Assignee: Ted Yu
 Fix For: 0.98.0, 0.99.0

 Attachments: 10485-0.94.txt, 10485-v1.txt


 Niels reported an issue under the thread 'Trouble writing custom filter for 
 use in FilterList' where his custom filter used in FilterList along with 
 PrefixFilter produced an unexpected results.
 His test can be found here:
 https://github.com/nielsbasjes/HBase-filter-problem
 This is due to PrefixFilter#filterKeyValue() using 
 FilterBase#filterKeyValue() which returns ReturnCode.INCLUDE
 When FilterList.Operator.MUST_PASS_ONE is specified, 
 FilterList#filterKeyValue() would return ReturnCode.INCLUDE even when row key 
 prefix doesn't match meanwhile the other filter's filterKeyValue() returns 
 ReturnCode.NEXT_COL



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

[jira] [Commented] (HBASE-10487) Avoid allocating new KeyValue and according bytes-copying for appended kvs which don't have existing(old) values

2014-02-09 Thread Feng Honghua (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-10487?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13896142#comment-13896142
 ] 

Feng Honghua commented on HBASE-10487:
--

Thanks all
bq.Is that a common scenario, though. Append is not very useful (compared to 
just Put) when you don't update an existing value.
Agree that Append on non-existing value isn't a common scenario, it occurs only 
once on cells for which Append is used to update the value. But the 
first-time-update can't be replaced as a Put though, so the first-time-update 
on those cells are always the scenario where Append is used to update a 
non-existing value

 Avoid allocating new KeyValue and according bytes-copying for appended kvs 
 which don't have existing(old) values
 

 Key: HBASE-10487
 URL: https://issues.apache.org/jira/browse/HBASE-10487
 Project: HBase
  Issue Type: Improvement
  Components: regionserver
Reporter: Feng Honghua
Assignee: Feng Honghua
 Attachments: HBASE-10487-trunk_v1.patch


 in HRegion.append, new KeyValues will be allocated and do according 
 bytes-copying no matter whether there are existing kv for the appended cells, 
 we can improve here by avoiding the allocating of new KeyValue and according 
 bytes-copying for kv which don't have existing(old) values by reusing the 
 passed-in kv and only updating its timestamp to 'now'(its original timestamp 
 is latest, so can be updated)



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

[jira] [Commented] (HBASE-9740) A corrupt HFile could cause endless attempts to assign the region without a chance of success

2014-02-09 Thread Ping (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-9740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13896147#comment-13896147
 ] 

Ping commented on HBASE-9740:
-

[~lhofhansl], yes

 A corrupt HFile could cause endless attempts to assign the region without a 
 chance of success
 -

 Key: HBASE-9740
 URL: https://issues.apache.org/jira/browse/HBASE-9740
 Project: HBase
  Issue Type: Bug
Affects Versions: 0.94.16
Reporter: Aditya Kishore
Assignee: Aditya Kishore
 Fix For: 0.94.18

 Attachments: HBase-9749_0.94_v2.patch, HBase-9749_0.94_v3.patch, 
 patch-9740_0.94.txt


 As described in HBASE-9737, a corrupt HFile in a region could lead to an 
 assignment storm in the cluster since the Master will keep trying to assign 
 the region to each region server one after another and obviously none will 
 succeed.
 The region server, upon detecting such a scenario should mark the region as 
 RS_ZK_REGION_FAILED_ERROR (or something to the effect) in the Zookeeper 
 which should indicate the Master to stop assigning the region until the error 
 has been resolved (via an HBase shell command, probably assign?)



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

[jira] [Commented] (HBASE-10436) hbase 0.96+ jmx does not have regionserver info any more.

2014-02-09 Thread Jonathan Hsieh (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-10436?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13896158#comment-13896158
 ] 

Jonathan Hsieh commented on HBASE-10436:


here's how it looks in the jmx

{code}
 {
name : Hadoop:service=HBase,name=Master,sub=Server,
modelerType : Master,sub=Server,
tag.Context : master,
tag.liveRegionServers : localhost,56196,1391992019130,
tag.deadRegionServers : 
localhost,40010,1391035309673;localhost,41408,1391990380724;localhost,38682,1390950017735,
tag.zookeeperQuorum : localhost:2181,
tag.serverName : localhost,37288,1391992016212,
tag.clusterId : 3b5b95ba-315f-495d-9e08-08a9626ccb37,
tag.isActiveMaster : true,
tag.Context : master,
tag.Hostname : swoop,
masterActiveTime : 1391992021835,
masterStartTime : 1391992020960,
averageLoad : 3.0,
numRegionServers : 1,
numDeadRegionServers : 3,
clusterRequests : 103248
  }, 
{code}

Note that numRegionServers and numDeadRegionsrevers both take the exact same 
locks [1][2] as the previous implementation I posted in the core metrics path.  
Not sure why it isn't an issue with those values and why it is an issue with 
the new ones.

[1] 
https://github.com/apache/hbase/blob/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/master/MetricsMasterWrapperImpl.java#L76
[2] 
https://github.com/apache/hbase/blob/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/master/ServerManager.java#L419

 hbase 0.96+ jmx does not have regionserver info any more.
 -

 Key: HBASE-10436
 URL: https://issues.apache.org/jira/browse/HBASE-10436
 Project: HBase
  Issue Type: Bug
Affects Versions: 0.98.0, 0.96.0, 0.99.0
Reporter: Jonathan Hsieh
Assignee: Jonathan Hsieh
Priority: Critical
 Attachments: hbase-10436.notbean.patch, hbase-10436.patch, 
 hbase-10436.v2.patch


 HBase 0.96's refactored jmx beans do not contain the master's list of dead 
 region servers and live regionservers with load info.  HBase 0.94 did (though 
 in a single monolithic blob).  
 This JMX interface should be considered as much of an API as the the normal 
 wire or java api.  Dropping values from this was done without deprecation and 
 the removal of this information is a functional regression.
 We should provide the information in the 0.96+ JMX.  HBase 0.94 had a  
 monolithic JMX blob (hadoop:service=Master,name=Master)  that contained a 
 lot of information, including the regionserver list and the cached 
 regionserver load for each region  found on the master webpage.  0.96+ 
 refactored jmx this into several jmx beans which could be selectively 
 retrieved.  These include:
 * hadoop:service=HBase,name=Master,sub=AssignmentManager
 * hadoop:service=HBase,name=Master,sub=Balancer
 * hadoop:service=HBase,name=Master,sub=Server
 * hadoop:service=HBase,name=Master,sub=FileSystem
 Specifically the (Hadoop:service=HBase,name=Master,sub=Server) listing that 
 used to contain regionservers and deadregionservers in jmx were replaced in   
 with numRegionServers and numDeadRegionservers which only contain counts.  
 I propose just adding another mbean called RegionServers under the bean: 
 hadoop:service=HBase,name=Master,sub=RegionServers



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

[jira] [Commented] (HBASE-10489) TestImportExport fails in 0.94 with Hadoop2


[ 
https://issues.apache.org/jira/browse/HBASE-10489?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13896167#comment-13896167
 ] 

Lars Hofhansl commented on HBASE-10489:
---

Hmm... One of the tests still failed in the latest build (but for a different 
reason, which does not happen locally).

 TestImportExport fails in 0.94 with Hadoop2
 ---

 Key: HBASE-10489
 URL: https://issues.apache.org/jira/browse/HBASE-10489
 Project: HBase
  Issue Type: Bug
  Components: test
Reporter: Lars Hofhansl
Assignee: Lars Hofhansl
 Fix For: 0.94.17

 Attachments: 10489.txt


 With HBASE-10363 fixed, we're now seeing other M/R tests failing.
 TestImportExport is one of them.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

[jira] [Commented] (HBASE-10489) TestImportExport fails in 0.94 with Hadoop2


[ 
https://issues.apache.org/jira/browse/HBASE-10489?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13896170#comment-13896170
 ] 

Hudson commented on HBASE-10489:


FAILURE: Integrated in HBase-0.94-on-Hadoop-2 #14 (See 
[https://builds.apache.org/job/HBase-0.94-on-Hadoop-2/14/])
HBASE-10489 TestImportExport fails in 0.94 with Hadoop2. (larsh: rev 1566484)
* 
/hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/mapreduce/TestImportExport.java


 TestImportExport fails in 0.94 with Hadoop2
 ---

 Key: HBASE-10489
 URL: https://issues.apache.org/jira/browse/HBASE-10489
 Project: HBase
  Issue Type: Bug
  Components: test
Reporter: Lars Hofhansl
Assignee: Lars Hofhansl
 Fix For: 0.94.17

 Attachments: 10489.txt


 With HBASE-10363 fixed, we're now seeing other M/R tests failing.
 TestImportExport is one of them.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

[jira] [Commented] (HBASE-10489) TestImportExport fails in 0.94 with Hadoop2


[ 
https://issues.apache.org/jira/browse/HBASE-10489?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13896172#comment-13896172
 ] 

Hudson commented on HBASE-10489:


SUCCESS: Integrated in HBase-0.94-security #405 (See 
[https://builds.apache.org/job/HBase-0.94-security/405/])
HBASE-10489 TestImportExport fails in 0.94 with Hadoop2. (larsh: rev 1566484)
* 
/hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/mapreduce/TestImportExport.java


 TestImportExport fails in 0.94 with Hadoop2
 ---

 Key: HBASE-10489
 URL: https://issues.apache.org/jira/browse/HBASE-10489
 Project: HBase
  Issue Type: Bug
  Components: test
Reporter: Lars Hofhansl
Assignee: Lars Hofhansl
 Fix For: 0.94.17

 Attachments: 10489.txt


 With HBASE-10363 fixed, we're now seeing other M/R tests failing.
 TestImportExport is one of them.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

[jira] [Commented] (HBASE-10436) hbase 0.96+ jmx does not have regionserver info any more.

2014-02-09 Thread Hadoop QA (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-10436?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13896176#comment-13896176
 ] 

Hadoop QA commented on HBASE-10436:
---

{color:red}-1 overall{color}.  Here are the results of testing the latest 
attachment 
  
http://issues.apache.org/jira/secure/attachment/12627906/hbase-10436.notbean.patch
  against trunk revision .
  ATTACHMENT ID: 12627906

{color:green}+1 @author{color}.  The patch does not contain any @author 
tags.

{color:green}+1 tests included{color}.  The patch appears to include 3 new 
or modified tests.

{color:green}+1 hadoop1.0{color}.  The patch compiles against the hadoop 
1.0 profile.

{color:green}+1 hadoop1.1{color}.  The patch compiles against the hadoop 
1.1 profile.

{color:green}+1 javadoc{color}.  The javadoc tool did not generate any 
warning messages.

{color:green}+1 javac{color}.  The applied patch does not increase the 
total number of javac compiler warnings.

{color:green}+1 findbugs{color}.  The patch does not introduce any new 
Findbugs (version 1.3.9) warnings.

{color:green}+1 release audit{color}.  The applied patch does not increase 
the total number of release audit warnings.

{color:green}+1 lineLengths{color}.  The patch does not introduce lines 
longer than 100

{color:red}-1 site{color}.  The patch appears to cause mvn site goal to 
fail.

{color:green}+1 core tests{color}.  The patch passed unit tests in .

Test results: 
https://builds.apache.org/job/PreCommit-HBASE-Build/8646//testReport/
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/8646//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/8646//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/8646//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/8646//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/8646//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/8646//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/8646//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/8646//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/8646//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output: 
https://builds.apache.org/job/PreCommit-HBASE-Build/8646//console

This message is automatically generated.

 hbase 0.96+ jmx does not have regionserver info any more.
 -

 Key: HBASE-10436
 URL: https://issues.apache.org/jira/browse/HBASE-10436
 Project: HBase
  Issue Type: Bug
Affects Versions: 0.98.0, 0.96.0, 0.99.0
Reporter: Jonathan Hsieh
Assignee: Jonathan Hsieh
Priority: Critical
 Attachments: hbase-10436.notbean.patch, hbase-10436.patch, 
 hbase-10436.v2.patch


 HBase 0.96's refactored jmx beans do not contain the master's list of dead 
 region servers and live regionservers with load info.  HBase 0.94 did (though 
 in a single monolithic blob).  
 This JMX interface should be considered as much of an API as the the normal 
 wire or java api.  Dropping values from this was done without deprecation and 
 the removal of this information is a functional regression.
 We should provide the information in the 0.96+ JMX.  HBase 0.94 had a  
 monolithic JMX blob (hadoop:service=Master,name=Master)  that contained a 
 lot of information, including the regionserver list and the cached 
 regionserver load for each region  found on the master webpage.  0.96+ 
 refactored jmx this into several jmx beans which could be selectively 
 retrieved.  These include:
 * hadoop:service=HBase,name=Master,sub=AssignmentManager
 * hadoop:service=HBase,name=Master,sub=Balancer
 * hadoop:service=HBase,name=Master,sub=Server
 * hadoop:service=HBase,name=Master,sub=FileSystem
 Specifically the (Hadoop:service=HBase,name=Master,sub=Server) listing that 
 used to contain regionservers and deadregionservers in jmx were replaced in   
 with numRegionServers and numDeadRegionservers which only contain counts.  
 I propose just adding another mbean called RegionServers under the bean:

[jira] [Commented] (HBASE-10489) TestImportExport fails in 0.94 with Hadoop2


[ 
https://issues.apache.org/jira/browse/HBASE-10489?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13896178#comment-13896178
 ] 

Hudson commented on HBASE-10489:


ABORTED: Integrated in HBase-0.94 #1279 (See 
[https://builds.apache.org/job/HBase-0.94/1279/])
HBASE-10489 TestImportExport fails in 0.94 with Hadoop2. (larsh: rev 1566484)
* 
/hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/mapreduce/TestImportExport.java


 TestImportExport fails in 0.94 with Hadoop2
 ---

 Key: HBASE-10489
 URL: https://issues.apache.org/jira/browse/HBASE-10489
 Project: HBase
  Issue Type: Bug
  Components: test
Reporter: Lars Hofhansl
Assignee: Lars Hofhansl
 Fix For: 0.94.17

 Attachments: 10489.txt


 With HBASE-10363 fixed, we're now seeing other M/R tests failing.
 TestImportExport is one of them.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

[jira] [Commented] (HBASE-10489) TestImportExport fails in 0.94 with Hadoop2


[ 
https://issues.apache.org/jira/browse/HBASE-10489?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13896181#comment-13896181
 ] 

Hudson commented on HBASE-10489:


ABORTED: Integrated in HBase-0.94-JDK7 #43 (See 
[https://builds.apache.org/job/HBase-0.94-JDK7/43/])
HBASE-10489 TestImportExport fails in 0.94 with Hadoop2. (larsh: rev 1566484)
* 
/hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/mapreduce/TestImportExport.java


 TestImportExport fails in 0.94 with Hadoop2
 ---

 Key: HBASE-10489
 URL: https://issues.apache.org/jira/browse/HBASE-10489
 Project: HBase
  Issue Type: Bug
  Components: test
Reporter: Lars Hofhansl
Assignee: Lars Hofhansl
 Fix For: 0.94.17

 Attachments: 10489.txt


 With HBASE-10363 fixed, we're now seeing other M/R tests failing.
 TestImportExport is one of them.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

[jira] [Commented] (HBASE-10479) HConnection interface is public but is used internally, and contains a bunch of methods

2014-02-09 Thread Sergey Shelukhin (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-10479?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13896196#comment-13896196
 ] 

Sergey Shelukhin commented on HBASE-10479:
--

TestFlushSnapshotFromClient appears to be flaky, will fix missing Apache header 
and non-javadoc long lines

 HConnection interface is public but is used internally, and contains a bunch 
 of methods
 ---

 Key: HBASE-10479
 URL: https://issues.apache.org/jira/browse/HBASE-10479
 Project: HBase
  Issue Type: Improvement
Reporter: Sergey Shelukhin
Assignee: Sergey Shelukhin
 Attachments: 45751591.jpg, HBASE-10479.01.patch, 
 HBASE-10479.02.patch, HBASE-10479.03.patch, HBASE-10479.04.patch, 
 HBASE-10479.patch


 HConnection has too many methods for a public interface, and some of these 
 should not be public.
 It is used extensively for internal purposes, so we keep adding methods to it 
 that may not make sense for public interface.
 The idea is to create a separate internal interface inheriting HConnection, 
 copy some methods to it and deprecate them on HConnection. New methods for 
 internal use would be added to new interface; the deprecated methods would 
 eventually be removed from public interface.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

[jira] [Updated] (HBASE-10479) HConnection interface is public but is used internally, and contains a bunch of methods

2014-02-09 Thread Sergey Shelukhin (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-10479?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sergey Shelukhin updated HBASE-10479:
-

Attachment: HBASE-10479.final.patch

Final patch, appeasing the QA

 HConnection interface is public but is used internally, and contains a bunch 
 of methods
 ---

 Key: HBASE-10479
 URL: https://issues.apache.org/jira/browse/HBASE-10479
 Project: HBase
  Issue Type: Improvement
Reporter: Sergey Shelukhin
Assignee: Sergey Shelukhin
 Attachments: 45751591.jpg, HBASE-10479.01.patch, 
 HBASE-10479.02.patch, HBASE-10479.03.patch, HBASE-10479.04.patch, 
 HBASE-10479.final.patch, HBASE-10479.patch


 HConnection has too many methods for a public interface, and some of these 
 should not be public.
 It is used extensively for internal purposes, so we keep adding methods to it 
 that may not make sense for public interface.
 The idea is to create a separate internal interface inheriting HConnection, 
 copy some methods to it and deprecate them on HConnection. New methods for 
 internal use would be added to new interface; the deprecated methods would 
 eventually be removed from public interface.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

[jira] [Commented] (HBASE-10479) HConnection interface is public but is used internally, and contains a bunch of methods

2014-02-09 Thread Hadoop QA (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-10479?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13896207#comment-13896207
 ] 

Hadoop QA commented on HBASE-10479:
---

{color:red}-1 overall{color}.  Here are the results of testing the latest 
attachment 
  
http://issues.apache.org/jira/secure/attachment/12627919/HBASE-10479.final.patch
  against trunk revision .
  ATTACHMENT ID: 12627919

{color:green}+1 @author{color}.  The patch does not contain any @author 
tags.

{color:green}+1 tests included{color}.  The patch appears to include 30 new 
or modified tests.

{color:red}-1 patch{color}.  The patch command could not apply the patch.

Console output: 
https://builds.apache.org/job/PreCommit-HBASE-Build/8647//console

This message is automatically generated.

 HConnection interface is public but is used internally, and contains a bunch 
 of methods
 ---

 Key: HBASE-10479
 URL: https://issues.apache.org/jira/browse/HBASE-10479
 Project: HBase
  Issue Type: Improvement
Reporter: Sergey Shelukhin
Assignee: Sergey Shelukhin
 Attachments: 45751591.jpg, HBASE-10479.01.patch, 
 HBASE-10479.02.patch, HBASE-10479.03.patch, HBASE-10479.04.patch, 
 HBASE-10479.final.patch, HBASE-10479.patch


 HConnection has too many methods for a public interface, and some of these 
 should not be public.
 It is used extensively for internal purposes, so we keep adding methods to it 
 that may not make sense for public interface.
 The idea is to create a separate internal interface inheriting HConnection, 
 copy some methods to it and deprecate them on HConnection. New methods for 
 internal use would be added to new interface; the deprecated methods would 
 eventually be removed from public interface.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

[jira] [Commented] (HBASE-10389) Add namespace help info in table related shell commands


[ 
https://issues.apache.org/jira/browse/HBASE-10389?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13896218#comment-13896218
 ] 

stack commented on HBASE-10389:
---

+1 for 0.96.


 Add namespace help info in table related shell commands
 ---

 Key: HBASE-10389
 URL: https://issues.apache.org/jira/browse/HBASE-10389
 Project: HBase
  Issue Type: Improvement
  Components: shell
Affects Versions: 0.96.0, 0.96.1
Reporter: Jerry He
Assignee: Jerry He
 Attachments: HBASE-10389-trunk.patch


 Currently in the help info of table related shell command, we don't mention 
 or give namespace as part of the table name.  
 For example, to create table:
 {code}
 hbase(main):001:0 help 'create'
 Creates a table. Pass a table name, and a set of column family
 specifications (at least one), and, optionally, table configuration.
 Column specification can be a simple string (name), or a dictionary
 (dictionaries are described below in main help output), necessarily
 including NAME attribute.
 Examples:
   hbase create 't1', {NAME = 'f1', VERSIONS = 5}
   hbase create 't1', {NAME = 'f1'}, {NAME = 'f2'}, {NAME = 'f3'}
   hbase # The above in shorthand would be the following:
   hbase create 't1', 'f1', 'f2', 'f3'
   hbase create 't1', {NAME = 'f1', VERSIONS = 1, TTL = 2592000, 
 BLOCKCACHE = true}
   hbase create 't1', {NAME = 'f1', CONFIGURATION = 
 {'hbase.hstore.blockingStoreFiles' = '10'}}
 Table configuration options can be put at the end.
 Examples:
   hbase create 't1', 'f1', SPLITS = ['10', '20', '30', '40']
   hbase create 't1', 'f1', SPLITS_FILE = 'splits.txt', OWNER = 'johndoe'
   hbase create 't1', {NAME = 'f1', VERSIONS = 5}, METADATA = { 'mykey' = 
 'myvalue' }
   hbase # Optionally pre-split the table into NUMREGIONS, using
   hbase # SPLITALGO (HexStringSplit, UniformSplit or classname)
   hbase create 't1', 'f1', {NUMREGIONS = 15, SPLITALGO = 'HexStringSplit'}
   hbase create 't1', 'f1', {NUMREGIONS = 15, SPLITALGO = 'HexStringSplit', 
 CONFIGURATION = {'hbase.hregion.scan.loadColumnFamiliesOnDemand' = 'true'}}
 You can also keep around a reference to the created table:
   hbase t1 = create 't1', 'f1'
 Which gives you a reference to the table named 't1', on which you can then
 call methods.
 {code}
 We should document the usage of namespace in these commands.
 For example:
 #namespace=foo and table qualifier=bar
 create 'foo:bar', 'fam'
 #namespace=default and table qualifier=bar
 create 'bar', 'fam'



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

[jira] [Commented] (HBASE-10485) PrefixFilter#filterKeyValue() should perform filtering on row key


[ 
https://issues.apache.org/jira/browse/HBASE-10485?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13896221#comment-13896221
 ] 

ramkrishna.s.vasudevan commented on HBASE-10485:


Is it need to be fixed in the Prefixfilter.  This fix would mean that for every 
KV we would evaluate the row part of it.  Ideally prefixfilters are for acting 
on the entire row (and not on KV).
What I would think is the filter AlwaysNextColFilter should say filterRow() as 
true?
The filter's idea is not to include any KV and tries to do next_col.  Then 
finally saying filterRow() as false would mean that though i filtered out, i 
need that row.  
Our FilterBase gives a default impl but still it makes sense to make 
filterRow() as true rather than fixing it in Prefixfilter.
Thoughts?

 PrefixFilter#filterKeyValue() should perform filtering on row key
 -

 Key: HBASE-10485
 URL: https://issues.apache.org/jira/browse/HBASE-10485
 Project: HBase
  Issue Type: Bug
Reporter: Ted Yu
Assignee: Ted Yu
 Fix For: 0.98.0, 0.99.0

 Attachments: 10485-0.94.txt, 10485-v1.txt


 Niels reported an issue under the thread 'Trouble writing custom filter for 
 use in FilterList' where his custom filter used in FilterList along with 
 PrefixFilter produced an unexpected results.
 His test can be found here:
 https://github.com/nielsbasjes/HBase-filter-problem
 This is due to PrefixFilter#filterKeyValue() using 
 FilterBase#filterKeyValue() which returns ReturnCode.INCLUDE
 When FilterList.Operator.MUST_PASS_ONE is specified, 
 FilterList#filterKeyValue() would return ReturnCode.INCLUDE even when row key 
 prefix doesn't match meanwhile the other filter's filterKeyValue() returns 
 ReturnCode.NEXT_COL



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

[jira] [Commented] (HBASE-10485) PrefixFilter#filterKeyValue() should perform filtering on row key


[ 
https://issues.apache.org/jira/browse/HBASE-10485?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13896222#comment-13896222
 ] 

ramkrishna.s.vasudevan commented on HBASE-10485:


What i felt is that fixing this in PrefixFilter would ideally slow down scans 
with this filter alone added because we are evaluating per KV.

 PrefixFilter#filterKeyValue() should perform filtering on row key
 -

 Key: HBASE-10485
 URL: https://issues.apache.org/jira/browse/HBASE-10485
 Project: HBase
  Issue Type: Bug
Reporter: Ted Yu
Assignee: Ted Yu
 Fix For: 0.98.0, 0.99.0

 Attachments: 10485-0.94.txt, 10485-v1.txt


 Niels reported an issue under the thread 'Trouble writing custom filter for 
 use in FilterList' where his custom filter used in FilterList along with 
 PrefixFilter produced an unexpected results.
 His test can be found here:
 https://github.com/nielsbasjes/HBase-filter-problem
 This is due to PrefixFilter#filterKeyValue() using 
 FilterBase#filterKeyValue() which returns ReturnCode.INCLUDE
 When FilterList.Operator.MUST_PASS_ONE is specified, 
 FilterList#filterKeyValue() would return ReturnCode.INCLUDE even when row key 
 prefix doesn't match meanwhile the other filter's filterKeyValue() returns 
 ReturnCode.NEXT_COL



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

[jira] [Comment Edited] (HBASE-10485) PrefixFilter#filterKeyValue() should perform filtering on row key


[ 
https://issues.apache.org/jira/browse/HBASE-10485?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13896222#comment-13896222
 ] 

ramkrishna.s.vasudevan edited comment on HBASE-10485 at 2/10/14 5:20 AM:
-

What i felt is that fixing this in PrefixFilter would ideally slow down scans 
with this filter alone added because we are evaluating per KV.
If others are fine with the fix, then am also fine +1.  
{Edited the comment}


was (Author: ram_krish):
What i felt is that fixing this in PrefixFilter would ideally slow down scans 
with this filter alone added because we are evaluating per KV.

 PrefixFilter#filterKeyValue() should perform filtering on row key
 -

 Key: HBASE-10485
 URL: https://issues.apache.org/jira/browse/HBASE-10485
 Project: HBase
  Issue Type: Bug
Reporter: Ted Yu
Assignee: Ted Yu
 Fix For: 0.98.0, 0.99.0

 Attachments: 10485-0.94.txt, 10485-v1.txt


 Niels reported an issue under the thread 'Trouble writing custom filter for 
 use in FilterList' where his custom filter used in FilterList along with 
 PrefixFilter produced an unexpected results.
 His test can be found here:
 https://github.com/nielsbasjes/HBase-filter-problem
 This is due to PrefixFilter#filterKeyValue() using 
 FilterBase#filterKeyValue() which returns ReturnCode.INCLUDE
 When FilterList.Operator.MUST_PASS_ONE is specified, 
 FilterList#filterKeyValue() would return ReturnCode.INCLUDE even when row key 
 prefix doesn't match meanwhile the other filter's filterKeyValue() returns 
 ReturnCode.NEXT_COL



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

[jira] [Commented] (HBASE-10485) PrefixFilter#filterKeyValue() should perform filtering on row key


[ 
https://issues.apache.org/jira/browse/HBASE-10485?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13896224#comment-13896224
 ] 

Ted Yu commented on HBASE-10485:


bq. scans with this filter alone
In RegionScannerImpl#nextInternal():
{code}
  if (filterRowKey(currentRow, offset, length)) {
boolean moreRows = nextRow(currentRow, offset, length);
if (!moreRows) return false;
results.clear();
continue;
  }
{code}
If PrefixFilter is the sole filter, filterKeyValue() won't be called.

 PrefixFilter#filterKeyValue() should perform filtering on row key
 -

 Key: HBASE-10485
 URL: https://issues.apache.org/jira/browse/HBASE-10485
 Project: HBase
  Issue Type: Bug
Reporter: Ted Yu
Assignee: Ted Yu
 Fix For: 0.98.0, 0.99.0

 Attachments: 10485-0.94.txt, 10485-v1.txt


 Niels reported an issue under the thread 'Trouble writing custom filter for 
 use in FilterList' where his custom filter used in FilterList along with 
 PrefixFilter produced an unexpected results.
 His test can be found here:
 https://github.com/nielsbasjes/HBase-filter-problem
 This is due to PrefixFilter#filterKeyValue() using 
 FilterBase#filterKeyValue() which returns ReturnCode.INCLUDE
 When FilterList.Operator.MUST_PASS_ONE is specified, 
 FilterList#filterKeyValue() would return ReturnCode.INCLUDE even when row key 
 prefix doesn't match meanwhile the other filter's filterKeyValue() returns 
 ReturnCode.NEXT_COL



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

[jira] [Updated] (HBASE-10486) ProtobufUtil Append Increment deserialization lost cell level timestamp

2014-02-09 Thread Jeffrey Zhong (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-10486?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jeffrey Zhong updated HBASE-10486:
--

  Resolution: Fixed
Hadoop Flags: Reviewed
  Status: Resolved  (was: Patch Available)

Thanks [~saint@gmail.com]  [~apurtell] for the reviews! I've integrated 
the fix into trunk and 0.98 branch. 

[~saint@gmail.com] Let me know if you want this also in 0.96 branch. Thanks.

 ProtobufUtil Append  Increment deserialization lost cell level timestamp
 -

 Key: HBASE-10486
 URL: https://issues.apache.org/jira/browse/HBASE-10486
 Project: HBase
  Issue Type: Bug
Affects Versions: 0.98.0, 0.96.1
Reporter: Jeffrey Zhong
Assignee: Jeffrey Zhong
 Fix For: 0.98.1

 Attachments: hbase-10486-v2.patch, hbase-10486.patch


 When we deserialized Append  Increment, we uses wrong timestamp value during 
 deserialization in trunk  0.98 code and discard the value in 0.96 code base. 



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

[jira] [Comment Edited] (HBASE-10485) PrefixFilter#filterKeyValue() should perform filtering on row key


[ 
https://issues.apache.org/jira/browse/HBASE-10485?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13896224#comment-13896224
 ] 

Ted Yu edited comment on HBASE-10485 at 2/10/14 5:29 AM:
-

bq. scans with this filter alone
In RegionScannerImpl#nextInternal():
{code}
  if (filterRowKey(currentRow, offset, length)) {
boolean moreRows = nextRow(currentRow, offset, length);
if (!moreRows) return false;
results.clear();
continue;
  }
{code}
If PrefixFilter is the sole filter, filterKeyValue() won't be called if the 
underlying row is filtered.


was (Author: yuzhih...@gmail.com):
bq. scans with this filter alone
In RegionScannerImpl#nextInternal():
{code}
  if (filterRowKey(currentRow, offset, length)) {
boolean moreRows = nextRow(currentRow, offset, length);
if (!moreRows) return false;
results.clear();
continue;
  }
{code}
If PrefixFilter is the sole filter, filterKeyValue() won't be called.

 PrefixFilter#filterKeyValue() should perform filtering on row key
 -

 Key: HBASE-10485
 URL: https://issues.apache.org/jira/browse/HBASE-10485
 Project: HBase
  Issue Type: Bug
Reporter: Ted Yu
Assignee: Ted Yu
 Fix For: 0.98.0, 0.99.0

 Attachments: 10485-0.94.txt, 10485-v1.txt


 Niels reported an issue under the thread 'Trouble writing custom filter for 
 use in FilterList' where his custom filter used in FilterList along with 
 PrefixFilter produced an unexpected results.
 His test can be found here:
 https://github.com/nielsbasjes/HBase-filter-problem
 This is due to PrefixFilter#filterKeyValue() using 
 FilterBase#filterKeyValue() which returns ReturnCode.INCLUDE
 When FilterList.Operator.MUST_PASS_ONE is specified, 
 FilterList#filterKeyValue() would return ReturnCode.INCLUDE even when row key 
 prefix doesn't match meanwhile the other filter's filterKeyValue() returns 
 ReturnCode.NEXT_COL



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

[jira] [Commented] (HBASE-10452) Potential bugs in exception handlers


[ 
https://issues.apache.org/jira/browse/HBASE-10452?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13896228#comment-13896228
 ] 

Andrew Purtell commented on HBASE-10452:


Adding error logging is fine.

Throwing RuntimeExceptions is borderline. I would prefer you just add more 
error logging in those places.

The changes to RegionMergeRequest and SplitRequest are not good. Don't retry 
with a hardcoded and arbitrary strategy. If it is important enough to abort, 
just abort.

 Potential bugs in exception handlers
 

 Key: HBASE-10452
 URL: https://issues.apache.org/jira/browse/HBASE-10452
 Project: HBase
  Issue Type: Bug
  Components: Client, master, regionserver, util
Affects Versions: 0.96.1
Reporter: Ding Yuan
 Attachments: HBase-10452-trunk-v2.patch, HBase-10452-trunk.patch


 Hi HBase developers,
 We are a group of researchers on software reliability. Recently we did a 
 study and found that majority of the most severe failures in HBase are caused 
 by bugs in exception handling logic -- that it is hard to anticipate all the 
 possible real-world error scenarios. Therefore we built a simple checking 
 tool that automatically detects some bug patterns that have caused some very 
 severe real-world failures. I am reporting some of the results here. Any 
 feedback is much appreciated!
 Ding
 =
 Case 1:
   Line: 134, File: 
 org/apache/hadoop/hbase/regionserver/RegionMergeRequest.java
 {noformat}
   protected void releaseTableLock() {
 if (this.tableLock != null) {
   try {
 this.tableLock.release();
   } catch (IOException ex) {
 LOG.warn(Could not release the table lock, ex);
 //TODO: if we get here, and not abort RS, this lock will never be 
 released
   }
 }
 {noformat}
 The lock is not released if the exception occurs, causing potential deadlock 
 or starvation.
 Similar code pattern can be found at:
   Line: 135, File: org/apache/hadoop/hbase/regionserver/SplitRequest.java
 ==
 =
 Case 2:
   Line: 252, File: 
 org/apache/hadoop/hbase/regionserver/wal/SequenceFileLogReader.java
 {noformat}
 try {
   Field fEnd = SequenceFile.Reader.class.getDeclaredField(end);
   fEnd.setAccessible(true);
   end = fEnd.getLong(this.reader);
 } catch(Exception e) { /* reflection fail. keep going */ }
 {noformat}
 The caught Exception seems to be too general.
 While reflection-related errors might be harmless, the try block can throw
 other exceptions including SecurityException, IllegalAccessException, 
 etc. Currently
 all those exceptions are ignored. Maybe
 the safe way is to ignore the specific reflection-related errors while 
 logging and
 handling other types of unexpected exceptions.
 ==
 =
 Case 3:
   Line: 148, File: org/apache/hadoop/hbase/HBaseConfiguration.java
 {noformat}
 try {
   if (Class.forName(org.apache.hadoop.conf.ConfServlet) != null) {
 isShowConf = true;
   }
 } catch (Exception e) {
 }
 {noformat}
 Similar to the previous case, the exception handling is too general. While 
 ClassNotFound error might be the normal case and ignored, Class.forName can 
 also throw other exceptions (e.g., LinkageError) under some unexpected and 
 rare error cases. If that happens, the error will be lost. So maybe change it 
 to below:
 {noformat}
 try {
   if (Class.forName(org.apache.hadoop.conf.ConfServlet) != null) {
 isShowConf = true;
   }
 } catch (LinkageError e) {
   LOG.warn(..);
   // handle linkage error
 } catch (ExceptionInInitializerError e) {
   LOG.warn(..);
   // handle Initializer error
 } catch (ClassNotFoundException e) {
  LOG.debug(..);
  // ignore
 }
 {noformat}
 ==
 =
 Case 4:
   Line: 163, File: org/apache/hadoop/hbase/client/Get.java
 {noformat}
   public Get setTimeStamp(long timestamp) {
 try {
   tr = new TimeRange(timestamp, timestamp+1);
 } catch(IOException e) {
   // Will never happen
 }
 return this;
   }
 {noformat}
 Even if the IOException never happens right now, is it possible to happen in 
 the future due to code change?
 At least there should be a log message. The current behavior is dangerous 
 since if the exception ever happens
 in any unexpected scenario, it will be silently swallowed.
 Similar code pattern can be found at:
   Line: 300, File: org/apache/hadoop/hbase/client/Scan.java
 ==
 =
 Case 5:
   Line: 207, File: org/apache/hadoop/hbase/util/JVM.java
 {noformat}
if (input != null){
 try {

[jira] [Commented] (HBASE-10485) PrefixFilter#filterKeyValue() should perform filtering on row key

[
https://issues.apache.org/jira/browse/HBASE-10485?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13896229#comment-13896229
]

ramkrishna.s.vasudevan commented on HBASE-10485:

bq. filterKeyValue() won't be called if the underlying row is filtered.
Right. I missed that. :)
Anyway still my argument applies in cases where there are more than one filter
along with the PrefixFilter. Doing perKV on this is not right when we are
saying that prefix is applied on the row.
As I said if other are fine with this then am ok with the patch.

PrefixFilter#filterKeyValue() should perform filtering on row key
-

Key: HBASE-10485
URL: https://issues.apache.org/jira/browse/HBASE-10485
Project: HBase
Issue Type: Bug
Reporter: Ted Yu
Assignee: Ted Yu
Fix For: 0.98.0, 0.99.0

Attachments: 10485-0.94.txt, 10485-v1.txt

Niels reported an issue under the thread 'Trouble writing custom filter for
use in FilterList' where his custom filter used in FilterList along with
PrefixFilter produced an unexpected results.
His test can be found here:
https://github.com/nielsbasjes/HBase-filter-problem
This is due to PrefixFilter#filterKeyValue() using
FilterBase#filterKeyValue() which returns ReturnCode.INCLUDE
When FilterList.Operator.MUST_PASS_ONE is specified,
FilterList#filterKeyValue() would return ReturnCode.INCLUDE even when row key
prefix doesn't match meanwhile the other filter's filterKeyValue() returns
ReturnCode.NEXT_COL

--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

[jira] [Commented] (HBASE-10488) 'mvn site' is broken due to org.apache.jasper.JspC not found


[ 
https://issues.apache.org/jira/browse/HBASE-10488?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13896231#comment-13896231
 ] 

stack commented on HBASE-10488:
---

[~ted_yu] You fixing it?  Otherwise lets close.  This passes for me locally.  
You have other input?

 'mvn site' is broken due to org.apache.jasper.JspC not found
 

 Key: HBASE-10488
 URL: https://issues.apache.org/jira/browse/HBASE-10488
 Project: HBase
  Issue Type: Bug
Reporter: Ted Yu

 From 
 https://builds.apache.org/job/PreCommit-HBASE-Build/8642/artifact/trunk/patchprocess/patchSiteOutput.txt
  :
 {code}
 [WARNING] The POM for 
 org.apache.hbase:hbase-server:jar:0.99.0-20140127.165302-1 is invalid, 
 transitive dependencies (if any) will not be available, enable debug logging 
 for more details
 [WARNING] The POM for 
 org.apache.hbase:hbase-server:jar:tests:0.99.0-20140127.165302-1 is invalid, 
 transitive dependencies (if any) will not be available, enable debug logging 
 for more details
 ...
 [ERROR] Failed to execute goal 
 org.apache.maven.plugins:maven-site-plugin:3.3:site (default-site) on project 
 hbase: failed to get report for 
 org.apache.maven.plugins:maven-javadoc-plugin: Failed to execute goal 
 org.apache.maven.plugins:maven-antrun-plugin:1.6:run (generate) on project 
 hbase-thrift: An Ant BuildException has occured: taskdef class 
 org.apache.jasper.JspC cannot be found
 {code}



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

[jira] [Commented] (HBASE-10485) PrefixFilter#filterKeyValue() should perform filtering on row key


[ 
https://issues.apache.org/jira/browse/HBASE-10485?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13896233#comment-13896233
 ] 

stack commented on HBASE-10485:
---

Can you sketch code on what you mean [~ramkrishna.s.vasude...@gmail.com]?

[~ted_yu] you have numbers that show this does not slow scanning w/ this filter 
enabled?

 PrefixFilter#filterKeyValue() should perform filtering on row key
 -

 Key: HBASE-10485
 URL: https://issues.apache.org/jira/browse/HBASE-10485
 Project: HBase
  Issue Type: Bug
Reporter: Ted Yu
Assignee: Ted Yu
 Fix For: 0.98.0, 0.99.0

 Attachments: 10485-0.94.txt, 10485-v1.txt


 Niels reported an issue under the thread 'Trouble writing custom filter for 
 use in FilterList' where his custom filter used in FilterList along with 
 PrefixFilter produced an unexpected results.
 His test can be found here:
 https://github.com/nielsbasjes/HBase-filter-problem
 This is due to PrefixFilter#filterKeyValue() using 
 FilterBase#filterKeyValue() which returns ReturnCode.INCLUDE
 When FilterList.Operator.MUST_PASS_ONE is specified, 
 FilterList#filterKeyValue() would return ReturnCode.INCLUDE even when row key 
 prefix doesn't match meanwhile the other filter's filterKeyValue() returns 
 ReturnCode.NEXT_COL



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

[jira] [Commented] (HBASE-10488) 'mvn site' is broken due to org.apache.jasper.JspC not found


[ 
https://issues.apache.org/jira/browse/HBASE-10488?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13896234#comment-13896234
 ] 

Ted Yu commented on HBASE-10488:


QA build has shown 'mvn site' failure since #8633

I can reproduce the failure locally.

 'mvn site' is broken due to org.apache.jasper.JspC not found
 

 Key: HBASE-10488
 URL: https://issues.apache.org/jira/browse/HBASE-10488
 Project: HBase
  Issue Type: Bug
Reporter: Ted Yu

 From 
 https://builds.apache.org/job/PreCommit-HBASE-Build/8642/artifact/trunk/patchprocess/patchSiteOutput.txt
  :
 {code}
 [WARNING] The POM for 
 org.apache.hbase:hbase-server:jar:0.99.0-20140127.165302-1 is invalid, 
 transitive dependencies (if any) will not be available, enable debug logging 
 for more details
 [WARNING] The POM for 
 org.apache.hbase:hbase-server:jar:tests:0.99.0-20140127.165302-1 is invalid, 
 transitive dependencies (if any) will not be available, enable debug logging 
 for more details
 ...
 [ERROR] Failed to execute goal 
 org.apache.maven.plugins:maven-site-plugin:3.3:site (default-site) on project 
 hbase: failed to get report for 
 org.apache.maven.plugins:maven-javadoc-plugin: Failed to execute goal 
 org.apache.maven.plugins:maven-antrun-plugin:1.6:run (generate) on project 
 hbase-thrift: An Ant BuildException has occured: taskdef class 
 org.apache.jasper.JspC cannot be found
 {code}



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

[jira] [Commented] (HBASE-10452) Potential bugs in exception handlers


[ 
https://issues.apache.org/jira/browse/HBASE-10452?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13896236#comment-13896236
 ] 

stack commented on HBASE-10452:
---

Thanks for looking into this [~d.yuan] All looks good to me caveat what Andy 
says -- just abort if you can't release the lock (something is really wrong if 
you can't release a lock).

 Potential bugs in exception handlers
 

 Key: HBASE-10452
 URL: https://issues.apache.org/jira/browse/HBASE-10452
 Project: HBase
  Issue Type: Bug
  Components: Client, master, regionserver, util
Affects Versions: 0.96.1
Reporter: Ding Yuan
 Attachments: HBase-10452-trunk-v2.patch, HBase-10452-trunk.patch


 Hi HBase developers,
 We are a group of researchers on software reliability. Recently we did a 
 study and found that majority of the most severe failures in HBase are caused 
 by bugs in exception handling logic -- that it is hard to anticipate all the 
 possible real-world error scenarios. Therefore we built a simple checking 
 tool that automatically detects some bug patterns that have caused some very 
 severe real-world failures. I am reporting some of the results here. Any 
 feedback is much appreciated!
 Ding
 =
 Case 1:
   Line: 134, File: 
 org/apache/hadoop/hbase/regionserver/RegionMergeRequest.java
 {noformat}
   protected void releaseTableLock() {
 if (this.tableLock != null) {
   try {
 this.tableLock.release();
   } catch (IOException ex) {
 LOG.warn(Could not release the table lock, ex);
 //TODO: if we get here, and not abort RS, this lock will never be 
 released
   }
 }
 {noformat}
 The lock is not released if the exception occurs, causing potential deadlock 
 or starvation.
 Similar code pattern can be found at:
   Line: 135, File: org/apache/hadoop/hbase/regionserver/SplitRequest.java
 ==
 =
 Case 2:
   Line: 252, File: 
 org/apache/hadoop/hbase/regionserver/wal/SequenceFileLogReader.java
 {noformat}
 try {
   Field fEnd = SequenceFile.Reader.class.getDeclaredField(end);
   fEnd.setAccessible(true);
   end = fEnd.getLong(this.reader);
 } catch(Exception e) { /* reflection fail. keep going */ }
 {noformat}
 The caught Exception seems to be too general.
 While reflection-related errors might be harmless, the try block can throw
 other exceptions including SecurityException, IllegalAccessException, 
 etc. Currently
 all those exceptions are ignored. Maybe
 the safe way is to ignore the specific reflection-related errors while 
 logging and
 handling other types of unexpected exceptions.
 ==
 =
 Case 3:
   Line: 148, File: org/apache/hadoop/hbase/HBaseConfiguration.java
 {noformat}
 try {
   if (Class.forName(org.apache.hadoop.conf.ConfServlet) != null) {
 isShowConf = true;
   }
 } catch (Exception e) {
 }
 {noformat}
 Similar to the previous case, the exception handling is too general. While 
 ClassNotFound error might be the normal case and ignored, Class.forName can 
 also throw other exceptions (e.g., LinkageError) under some unexpected and 
 rare error cases. If that happens, the error will be lost. So maybe change it 
 to below:
 {noformat}
 try {
   if (Class.forName(org.apache.hadoop.conf.ConfServlet) != null) {
 isShowConf = true;
   }
 } catch (LinkageError e) {
   LOG.warn(..);
   // handle linkage error
 } catch (ExceptionInInitializerError e) {
   LOG.warn(..);
   // handle Initializer error
 } catch (ClassNotFoundException e) {
  LOG.debug(..);
  // ignore
 }
 {noformat}
 ==
 =
 Case 4:
   Line: 163, File: org/apache/hadoop/hbase/client/Get.java
 {noformat}
   public Get setTimeStamp(long timestamp) {
 try {
   tr = new TimeRange(timestamp, timestamp+1);
 } catch(IOException e) {
   // Will never happen
 }
 return this;
   }
 {noformat}
 Even if the IOException never happens right now, is it possible to happen in 
 the future due to code change?
 At least there should be a log message. The current behavior is dangerous 
 since if the exception ever happens
 in any unexpected scenario, it will be silently swallowed.
 Similar code pattern can be found at:
   Line: 300, File: org/apache/hadoop/hbase/client/Scan.java
 ==
 =
 Case 5:
   Line: 207, File: org/apache/hadoop/hbase/util/JVM.java
 {noformat}
if (input != null){
 try {
   input.close();
 } catch (IOException ignored) {
 }
   }
 {noformat}
 Any exception encountered in close is completely

[jira] [Commented] (HBASE-9740) A corrupt HFile could cause endless attempts to assign the region without a chance of success

2014-02-09 Thread Aditya Kishore (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-9740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13896235#comment-13896235
 ] 

Aditya Kishore commented on HBASE-9740:
---

[~lhofhansl], could you please add [~Tom_play] to contributor's list and assign 
this JIRA to him.

 A corrupt HFile could cause endless attempts to assign the region without a 
 chance of success
 -

 Key: HBASE-9740
 URL: https://issues.apache.org/jira/browse/HBASE-9740
 Project: HBase
  Issue Type: Bug
Affects Versions: 0.94.16
Reporter: Aditya Kishore
Assignee: Aditya Kishore
 Fix For: 0.94.18

 Attachments: HBase-9749_0.94_v2.patch, HBase-9749_0.94_v3.patch, 
 patch-9740_0.94.txt


 As described in HBASE-9737, a corrupt HFile in a region could lead to an 
 assignment storm in the cluster since the Master will keep trying to assign 
 the region to each region server one after another and obviously none will 
 succeed.
 The region server, upon detecting such a scenario should mark the region as 
 RS_ZK_REGION_FAILED_ERROR (or something to the effect) in the Zookeeper 
 which should indicate the Master to stop assigning the region until the error 
 has been resolved (via an HBase shell command, probably assign?)



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

[jira] [Resolved] (HBASE-10488) 'mvn site' is broken due to org.apache.jasper.JspC not found


 [ 
https://issues.apache.org/jira/browse/HBASE-10488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stack resolved HBASE-10488.
---

Resolution: Invalid

I asked for other info and if filer was going to fix it.  Got neither. Closing 
as invalid.

 'mvn site' is broken due to org.apache.jasper.JspC not found
 

 Key: HBASE-10488
 URL: https://issues.apache.org/jira/browse/HBASE-10488
 Project: HBase
  Issue Type: Bug
Reporter: Ted Yu

 From 
 https://builds.apache.org/job/PreCommit-HBASE-Build/8642/artifact/trunk/patchprocess/patchSiteOutput.txt
  :
 {code}
 [WARNING] The POM for 
 org.apache.hbase:hbase-server:jar:0.99.0-20140127.165302-1 is invalid, 
 transitive dependencies (if any) will not be available, enable debug logging 
 for more details
 [WARNING] The POM for 
 org.apache.hbase:hbase-server:jar:tests:0.99.0-20140127.165302-1 is invalid, 
 transitive dependencies (if any) will not be available, enable debug logging 
 for more details
 ...
 [ERROR] Failed to execute goal 
 org.apache.maven.plugins:maven-site-plugin:3.3:site (default-site) on project 
 hbase: failed to get report for 
 org.apache.maven.plugins:maven-javadoc-plugin: Failed to execute goal 
 org.apache.maven.plugins:maven-antrun-plugin:1.6:run (generate) on project 
 hbase-thrift: An Ant BuildException has occured: taskdef class 
 org.apache.jasper.JspC cannot be found
 {code}



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

[jira] [Commented] (HBASE-10485) PrefixFilter#filterKeyValue() should perform filtering on row key


[ 
https://issues.apache.org/jira/browse/HBASE-10485?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13896237#comment-13896237
 ] 

ramkrishna.s.vasudevan commented on HBASE-10485:


{code}
if (filterRowKey(currentRow, offset, length)) {
boolean moreRows = nextRow(currentRow, offset, length);
if (!moreRows) return false;
results.clear();
continue;
  }
{code}
Here filterRowKey for prefixfilter knows that we have already sure this row is 
not needed.  But we are evaluating the same row again in the filterKV by 
checking the row part alone.  That is what i mean.  
May be an easier way would  be to in PrefixFilter.filterKeyValue()
{code}
if(filterRow()) {
return skip;
} else {
return include;
}
{code}
This won't do by comparison once again.  Because the variable filterRow would 
have been already set to true or false at this point. Correct?

 PrefixFilter#filterKeyValue() should perform filtering on row key
 -

 Key: HBASE-10485
 URL: https://issues.apache.org/jira/browse/HBASE-10485
 Project: HBase
  Issue Type: Bug
Reporter: Ted Yu
Assignee: Ted Yu
 Fix For: 0.98.0, 0.99.0

 Attachments: 10485-0.94.txt, 10485-v1.txt


 Niels reported an issue under the thread 'Trouble writing custom filter for 
 use in FilterList' where his custom filter used in FilterList along with 
 PrefixFilter produced an unexpected results.
 His test can be found here:
 https://github.com/nielsbasjes/HBase-filter-problem
 This is due to PrefixFilter#filterKeyValue() using 
 FilterBase#filterKeyValue() which returns ReturnCode.INCLUDE
 When FilterList.Operator.MUST_PASS_ONE is specified, 
 FilterList#filterKeyValue() would return ReturnCode.INCLUDE even when row key 
 prefix doesn't match meanwhile the other filter's filterKeyValue() returns 
 ReturnCode.NEXT_COL



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

[jira] [Commented] (HBASE-10486) ProtobufUtil Append Increment deserialization lost cell level timestamp


[ 
https://issues.apache.org/jira/browse/HBASE-10486?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13896241#comment-13896241
 ] 

stack commented on HBASE-10486:
---

Yes please [~jeffreyz]

 ProtobufUtil Append  Increment deserialization lost cell level timestamp
 -

 Key: HBASE-10486
 URL: https://issues.apache.org/jira/browse/HBASE-10486
 Project: HBase
  Issue Type: Bug
Affects Versions: 0.98.0, 0.96.1
Reporter: Jeffrey Zhong
Assignee: Jeffrey Zhong
 Fix For: 0.98.1

 Attachments: hbase-10486-v2.patch, hbase-10486.patch


 When we deserialized Append  Increment, we uses wrong timestamp value during 
 deserialization in trunk  0.98 code and discard the value in 0.96 code base. 



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

[jira] [Updated] (HBASE-10485) PrefixFilter#filterKeyValue() should perform filtering on row key


 [ 
https://issues.apache.org/jira/browse/HBASE-10485?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ted Yu updated HBASE-10485:
---

Attachment: 10485-trunk.addendum

All Filter related tests pass.

 PrefixFilter#filterKeyValue() should perform filtering on row key
 -

 Key: HBASE-10485
 URL: https://issues.apache.org/jira/browse/HBASE-10485
 Project: HBase
  Issue Type: Bug
Reporter: Ted Yu
Assignee: Ted Yu
 Fix For: 0.98.0, 0.99.0

 Attachments: 10485-0.94.txt, 10485-trunk.addendum, 10485-v1.txt


 Niels reported an issue under the thread 'Trouble writing custom filter for 
 use in FilterList' where his custom filter used in FilterList along with 
 PrefixFilter produced an unexpected results.
 His test can be found here:
 https://github.com/nielsbasjes/HBase-filter-problem
 This is due to PrefixFilter#filterKeyValue() using 
 FilterBase#filterKeyValue() which returns ReturnCode.INCLUDE
 When FilterList.Operator.MUST_PASS_ONE is specified, 
 FilterList#filterKeyValue() would return ReturnCode.INCLUDE even when row key 
 prefix doesn't match meanwhile the other filter's filterKeyValue() returns 
 ReturnCode.NEXT_COL



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

[jira] [Reopened] (HBASE-10488) 'mvn site' is broken due to org.apache.jasper.JspC not found


 [ 
https://issues.apache.org/jira/browse/HBASE-10488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ted Yu reopened HBASE-10488:



 'mvn site' is broken due to org.apache.jasper.JspC not found
 

 Key: HBASE-10488
 URL: https://issues.apache.org/jira/browse/HBASE-10488
 Project: HBase
  Issue Type: Bug
Reporter: Ted Yu

 From 
 https://builds.apache.org/job/PreCommit-HBASE-Build/8642/artifact/trunk/patchprocess/patchSiteOutput.txt
  :
 {code}
 [WARNING] The POM for 
 org.apache.hbase:hbase-server:jar:0.99.0-20140127.165302-1 is invalid, 
 transitive dependencies (if any) will not be available, enable debug logging 
 for more details
 [WARNING] The POM for 
 org.apache.hbase:hbase-server:jar:tests:0.99.0-20140127.165302-1 is invalid, 
 transitive dependencies (if any) will not be available, enable debug logging 
 for more details
 ...
 [ERROR] Failed to execute goal 
 org.apache.maven.plugins:maven-site-plugin:3.3:site (default-site) on project 
 hbase: failed to get report for 
 org.apache.maven.plugins:maven-javadoc-plugin: Failed to execute goal 
 org.apache.maven.plugins:maven-antrun-plugin:1.6:run (generate) on project 
 hbase-thrift: An Ant BuildException has occured: taskdef class 
 org.apache.jasper.JspC cannot be found
 {code}



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

[jira] [Resolved] (HBASE-10488) 'mvn site' is broken due to org.apache.jasper.JspC not found


 [ 
https://issues.apache.org/jira/browse/HBASE-10488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stack resolved HBASE-10488.
---

Resolution: Invalid

Resolving again.  Was reopened but nothing filed to justify reopening.

 'mvn site' is broken due to org.apache.jasper.JspC not found
 

 Key: HBASE-10488
 URL: https://issues.apache.org/jira/browse/HBASE-10488
 Project: HBase
  Issue Type: Bug
Reporter: Ted Yu

 From 
 https://builds.apache.org/job/PreCommit-HBASE-Build/8642/artifact/trunk/patchprocess/patchSiteOutput.txt
  :
 {code}
 [WARNING] The POM for 
 org.apache.hbase:hbase-server:jar:0.99.0-20140127.165302-1 is invalid, 
 transitive dependencies (if any) will not be available, enable debug logging 
 for more details
 [WARNING] The POM for 
 org.apache.hbase:hbase-server:jar:tests:0.99.0-20140127.165302-1 is invalid, 
 transitive dependencies (if any) will not be available, enable debug logging 
 for more details
 ...
 [ERROR] Failed to execute goal 
 org.apache.maven.plugins:maven-site-plugin:3.3:site (default-site) on project 
 hbase: failed to get report for 
 org.apache.maven.plugins:maven-javadoc-plugin: Failed to execute goal 
 org.apache.maven.plugins:maven-antrun-plugin:1.6:run (generate) on project 
 hbase-thrift: An Ant BuildException has occured: taskdef class 
 org.apache.jasper.JspC cannot be found
 {code}



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

[jira] [Commented] (HBASE-10485) PrefixFilter#filterKeyValue() should perform filtering on row key


[ 
https://issues.apache.org/jira/browse/HBASE-10485?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13896249#comment-13896249
 ] 

ramkrishna.s.vasudevan commented on HBASE-10485:


+1 on addendum.  Thanks Ted.  We can commit this. 0.94 and 0.96 also if needed 
you can commit.

 PrefixFilter#filterKeyValue() should perform filtering on row key
 -

 Key: HBASE-10485
 URL: https://issues.apache.org/jira/browse/HBASE-10485
 Project: HBase
  Issue Type: Bug
Reporter: Ted Yu
Assignee: Ted Yu
 Fix For: 0.98.0, 0.99.0

 Attachments: 10485-0.94.txt, 10485-trunk.addendum, 10485-v1.txt


 Niels reported an issue under the thread 'Trouble writing custom filter for 
 use in FilterList' where his custom filter used in FilterList along with 
 PrefixFilter produced an unexpected results.
 His test can be found here:
 https://github.com/nielsbasjes/HBase-filter-problem
 This is due to PrefixFilter#filterKeyValue() using 
 FilterBase#filterKeyValue() which returns ReturnCode.INCLUDE
 When FilterList.Operator.MUST_PASS_ONE is specified, 
 FilterList#filterKeyValue() would return ReturnCode.INCLUDE even when row key 
 prefix doesn't match meanwhile the other filter's filterKeyValue() returns 
 ReturnCode.NEXT_COL



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

[jira] [Commented] (HBASE-10485) PrefixFilter#filterKeyValue() should perform filtering on row key